What Makes a Good Questionnaire: A Market Researcher’s Guide
Is it just me, or do market research fundamentals like questionnaire design not get enough attention in our fast-paced work environment?
Channeling what my seniors taught me a decade ago, I’m writing this guide hoping it can help market researchers create effective questionnaires. I’m writing this for myself—as a refresher and a reference (because in the age of AI, it’s helpful to have a clear description of good questionnaire design that I can quickly feed to an LLM when I want it to check any survey I’m drafting). I’m also writing this for my current and future juniors in the industry, so I have an easy way to share these lessons on the job.
The Three Core Characteristics of a Good Questionnaire
1. Logical Flow
Good questionnaires follow a natural, intuitive progression from one question to the next. When respondents move through your survey, the journey should make sense — topics should be grouped together rather than jumping back and forth between different subjects.
For example, if you’re evaluating both brand performance and campaign effectiveness, you should:
- Complete all brand performance questions first, because the topic is broader with X number of brands
- Then move to campaign-specific questions, which just focus on one brand they might not even care about
- Avoid switching between the broad and campaign-specific questions as much as possible. Sometimes, if there’s no choice, you may need to do this. If so, make sure there are enough instructions to help reframe the respondent’s mindset. (e.g., use phrases like “thinking about the time when….” or “you have mentioned…” if you absolutely need to break flow.)
This isn’t just about convenience. Proper flow serves two critical purposes:
- It reduces cognitive dissonance. When questions flow logically, respondents can maintain their train of thought and provide more accurate answers.
- It prevents bias. If you ask narrow questions first, you will have primed respondents with brand associations and claims before they answer brand performance questions. You’ll inevitably color their subsequent answers about overall brand perception, which makes the market research unreliable.
A well-structured flow protects the integrity of your data by ensuring respondents encounter questions in the order that minimizes cross-contamination between sections.
2. Brevity
Every question in your questionnaire must serve a clear purpose. If a question doesn’t directly contribute to answering your research objectives, ruthlessly cut it.
This discipline is more important now than ever because:
- Online surveys receive significantly less respondent attention than face-to-face interviews
- You’re competing with every other tab, notification, and distraction on a respondent’s screen
- Respondent fatigue dramatically reduces data quality
Industry best practice: Maximum 15 minutes for survey completion.
Beyond this threshold, you’ll see dramatic drops in response quality. Shorter questionnaires lead to better engagement and more thoughtful responses. Every minute you can shave off improves your data quality, so be merciless in your editing.
3. Unbiased Question Design
Questions must be neutral and not lead respondents toward a particular answer. This is especially important for researchers who are new to the field, as there’s a natural tendency to craft questions that guide respondents toward the answers you hope to find.
We’ll talk more about this in the next section.
Seven Principles for Building Questionnaires
Principle 1: Ask Difficult and Most Important Questions Early
Place cognitively demanding questions at the beginning of your questionnaire when respondents are fresh and engaged.
Why this matters:
- Open-ended questions require significantly more mental effort than multiple-choice questions.
- Respondents must formulate thoughts and type out answers.
- By the end of a 20-minute survey, people are rushing to finish; attention has waned, and answers become superficial.
Protect your most important data by collecting it when attention is highest. If you have open-ended questions that are central to your research objectives, position them early. Your respondents will be more willing to put in effort in the first few minutes of your survey.
Principle 2: Start Broad, Then Narrow
Structure your questionnaire to move from general topics to specific ones. This principle operates at multiple levels throughout your survey.
For example:
- Start with general sections (e.g., category behavior)
- Before you move on to specific sections (e.g., brand awareness and usage patterns)
- Before you get to even more focused sections (e.g., brand-specific campaign evaluation)
This progression from broad to narrow creates a natural funnel that guides respondents through increasingly focused topics, making the survey feel conversational rather than disjointed.
Principle 3: Eliminate Leading Language and Assumptions
Never assume a positive or negative stance in your question wording. This is where inexperienced researchers most commonly stumble, often under pressure from stakeholders who have strong hypotheses they want to confirm.
Biased example: “If brand X communicates claim A, how much more likely are you to purchase this product?”
This assumes:
- The claim is attractive to the respondent
- The claim will increase purchase intent
The wording:
- eliminates the possibility of capturing neutral or negative reactions
- sounds like it’s trying too hard since Brand A hasn’t even made Claim X
Unbiased version: “Imagine brand X claimed A, how would this affect your likelihood to purchase brand X?”
Get KK’s stories in your inbox
Join Medium for free to get updates from this writer.
Notice the subtle but crucial difference — “how would this affect your likelihood to purchase” instead of “how much more likely are you to purchase.” Then provide a balanced scale:
- Would increase my likelihood to purchase
- Would not affect my likelihood to purchase
- Would decrease my likelihood to purchase
This seems like a small distinction, but it makes an enormous difference in data validity. By eliminating leading language and assumptions, we capture the full range of possible responses. You might discover that a claim your client loves actually turns off a significant portion of the target audience — invaluable information that would be completely hidden by biased wording.
Remember: An efficient but biased question is worse than no question at all, because it gives you false confidence in misleading data.
One more thing: notice how using “imagine” clearly frames this as a hypothetical scenario. This subtle word choice signals to respondents that you’re testing a concept, not making promises on behalf of the brand. It keeps the question grounded and credible.
Principle 4: Ensure Options are Mutually Exclusive
Response options must be mutually exclusive so respondents can clearly select one answer without confusion, and the researcher can clearly pinpoint what resonates with the respondent.
Each statement or response option should test only one thing at a time.
Bad example: “Which of these makes a brand more attractive to you?” a. Brand X is affordable and easy to find; b. …
The problem: If a respondent answered A, you can’t pinpoint if Brand X is perceived as:
- Affordable only
- Easy to find only
- Both affordable AND easy to find
If respondents only agree with one part of the statement, they might still choose the statement, but you won’t know which part resonated. You don’t want that.
Good example: Split into two separate statements:
- “Brand X is affordable”
- “Brand X is easy to find”
Now you have clear, actionable data on each attribute.
Principle 5: Give Respondents a Way Out
When there’s no appropriate choice available, you force respondents into a false position.
Key guidelines:
- Include options like “I don’t know” or “None of the above” when appropriate
- These aren’t throwaway options — they’re essential escape valves that preserve data integrity
- If you deliberately exclude these options, document this decision and its implications
Principle 6: Combat Response Position Bias through Rotation
When your response options are non-categorical (like brand names, product variants, or image options), rotation is crucial for unbiased data collection. Without rotation, position bias significantly skews your results.
The problem with fixed order:
Imagine you have 20 brands you’re assessing for brand perception. If the response options don’t rotate:
- The first three brands (in positions 1–3) will consistently receive more positive ratings
- Respondents tend to pay more attention to items at the top of a list
- They may also experience fatigue as they move down the list
- This creates systemic bias that makes the first few options artificially better
The solution: Rotate response options randomly across respondents so that each brand gets the “good seat” equally. This means:
- Brand A appears in position 1 for some respondents, position 5 for others, position 15 for others
- Over your full sample, each brand appears in every position roughly the same number of times
- This rotation practice averages out position bias and gives you clean, comparable data
When to rotate vs. when not to:
- DO rotate: Brand names, product variants, attribute lists, image options
- DON’T rotate: Categorical scales (Strongly Agree to Strongly Disagree), ordinal lists (Age ranges, income brackets), naturally ordered options (frequencies like “Daily” to “Never”)
Principle 7: Use Clear, Simple Language
Avoid business jargon and technical terms that respondents may not understand. This seems obvious, but it’s remarkable how often researchers default to industry language or corporate speak when writing questions.
The test:
- Read your questions aloud as if you were having a normal conversation
- If it sounds stilted or overly formal, rewrite it
- Your questionnaire should feel like a natural dialogue, not a corporate memo
Conclusion
In an era where survey platforms make it trivially easy to launch a study, the fundamentals matter more than ever. Anyone can create a questionnaire, but building one that yields reliable, unbiased data requires discipline and adherence to design principles proven over decades of practice.
Whether you’re feeding this article to an AI to serve as a checklist for reviewing your work, sharing it with junior team members, or just reminding yourself of the basics before a big project, I hope this guide serves as a practical reference.
If you have additional principles, examples, or corrections based on your own experience, I’d love to hear them too!
