Day 6: Concept Testing
Stage: Early design · Method: Mixed · Who: Design teams
What It Is
Concept testing sits in the gap between having an idea and building it. It’s the method you use when you have a concept — a new feature, a new flow, a new product direction — and need to know whether it’s worth building before you commit the resources. It’s different from competitor research (looks outward), discovery (explores needs), and usability testing (evaluates a working product).
Why It Matters
The most expensive mistake a product team makes is building something users don’t understand or don’t want. Concept testing is cheap relative to a full build — a 15-session study takes days, not months. The information it gives you — do users understand this? Do they want it? Would they use it? — directly determines whether the investment in building is justified.
When to Use It
- You have a concept, wireframe, or early prototype and want to validate it before building
- You’re choosing between two or three design directions
- Stakeholders need evidence before approving development resources
- You’ve pivoted and want to test whether the new direction resonates
When to Skip It
- You’ve already built a working prototype and want to test if it’s usable (use Usability Testing)
- You need to understand what problem to solve (do Discovery Research first)
- Your concept is too vague to show to participants
The Process (9 Steps)
Step 1: Concept Definition
What exactly are you testing? “Users like it” isn’t validation. “Users understand what it does and say they’d use it” is. Define your hypothesis and set specific, measurable thresholds before you run the study.
Step 2: Stimuli Preparation
Keep fidelity low enough that feedback is about the concept, not the visual polish. If participants start commenting on font choices or colours, your fidelity is too high. A sketch or wireframe is often better than a high-fidelity mockup for concept testing.
Get Navid Semi’s stories in your inbox
Join Medium for free to get updates from this writer.
Step 3: Evaluation Criteria
Define what you’ll measure before running a single session. Core metrics: Comprehension (can participants describe what it does without prompting?), Desirability (would they use it?), Perceived effort (does it feel worth it?), Preference (which concept won, if testing multiple?).
Step 4: Participant Recruitment
Test with your actual target audience. 8–15 participants per segment for moderated qualitative testing. Screen for actual relevance to the concept — don’t recruit “people who use computers at work” when you need B2B finance professionals.
Step 5: Test Script
The order matters enormously. Get first impressions before you explain anything — once you’ve explained the concept, you can’t un-explain it. Show concept → 30 seconds of silence → “What do you think this is?” → then explore comprehension and desirability.
Step 6: Concept Testing Sessions
The first 30 seconds of a participant’s reaction is the most valuable thing in the study. Capture it verbatim, before they’ve rationalised or been influenced by your explanation. Note what they try to click — intent signals are revealing.
Step 7: Synthesis — Comprehension & Desirability
Count results against your predefined thresholds. Then go deeper: what specifically was misunderstood? What drove desirability or killed it? What did the participants who didn’t understand get wrong?
Step 8: Preference Results
Only fill this in if you tested multiple concepts. The why behind preference is more useful than the ranking — it tells you what to carry into the winning concept.
Step 9: Recommendation & Next Steps
Proceed / pivot / kill — be direct. “Proceed with Concept A. Change required: rename the intro copy — the word ‘survey’ caused misunderstanding in 4/15 sessions.” Vague recommendations get ignored.
Tips for Success
- The “Five Second Test” is a useful pre-screen: show the concept for 5 seconds, ask what they saw
- Treat participants who misunderstand as design feedback, not participant failure
- Run at least two sessions before doing any analysis
- Document the “kill criteria” before you start — know the threshold in advance
- Two-person sessions work well: one moderator, one note-taker
Common Mistakes to Avoid
- Explaining the concept before testing comprehension — you’ve contaminated the data
- Asking “would you use this?” without probing for behaviour
- Testing with internal team or friends too familiar with the problem space
- Using high-fidelity visuals when testing comprehension
- Treating concept testing as a way to get approval rather than to learn
