Day 7: Usability Testing
Stage: Validate · Method: Mixed · Who: Everyone
What It Is
Usability testing is the most widely practised form of user research, and with good reason: it directly answers the most important question a product team can ask — can real users actually use this? Not “do they like it?” or “would they use it?” — but can they complete a specific task in your product without help?
Why It Matters
Every team believes their product is intuitive until they watch someone use it for the first time. Usability testing breaks the curse of knowledge — the cognitive bias that makes it impossible to see your own product through a new user’s eyes. Five sessions of usability testing will reveal more actionable issues than months of analytics review, because you see not just what happens but why.
When to Use It
- You have a working prototype or live product and want to identify usability issues
- You’ve made significant changes to a core user flow
- You’re about to launch a new feature and want a final check
- Your analytics show users struggling and you need to understand why
When to Skip It
- You don’t have anything to test yet (use Discovery or Concept Testing first)
- You’re testing whether users want the product (use Concept Testing)
- You need large-scale quantitative data (use surveys or analytics)
The Process (10 Steps)
Step 1: Test Objectives
Define what you want to learn. Good objectives are specific and tied to a decision: “Determine whether new users can complete sign-up without assistance” — not “check if the product is usable.”
Step 2: What You’re Testing
Describe exactly what you’re putting in front of participants. Live product, working prototype, paper prototype, or competitor product — each reveals different kinds of issues. Always have a backup. Prototypes break.
Get Navid Semi’s stories in your inbox
Join Medium for free to get updates from this writer.
Step 3: Task Scenarios
Tasks are the heart of usability testing. They must be realistic, specific, and avoid giving away the answer. Not “click the Settings button” — but “you’ve just joined a new company and want to invite your colleague.” Based on real user goals, written in the user’s language, with a clear success state.
Step 4: Participant Recruitment
5 participants per user segment reveals 85% of major usability issues. Screen for realistic usage patterns. Avoid recruiting exclusively happy, engaged users — you’ll miss the issues that affect everyone else.
Step 5: Success Metrics
Define in advance how you’ll measure success. Task completion rate, time on task, error rate, post-task satisfaction (1–5), and System Usability Scale (SUS) at the end. Set your success criteria before running sessions.
Step 6: Moderator Script
A consistent script ensures you’re testing the product, not the participant’s experience of you. Introduction, warm-up, task delivery, probing (“what are you thinking?”), post-task questions, closing. Never say “good job” during tasks — it creates demand characteristics.
Step 7: Session Logistics
Pre-session checklist: test environment set up, consent form ready, prototype on the correct version, task cards printed, note-taking template ready, timer accessible. Last-minute scrambling is visible to participants.
Step 8: Notes & Observation Log
Capture per participant per task: completed (Y/N, assisted/unassisted), time taken, specific errors, direct quotes verbatim, observations, body language notes. Verbatim quotes are more useful than your interpretation of what was said.
Step 9: Findings & Severity Rating
Not all usability issues are equal. Severity scale: Critical (4) — prevents task completion, fix before launch. High (3) — significantly slows users, fix soon. Medium (2) — causes confusion but users recover, fix in next sprint. Low (1) — minor friction, fix when convenient.
Step 10: Recommendations & Retest Plan
For each issue: “Users couldn’t find X because Y. Recommended fix: [specific design change]. Confidence: high (seen in 4/5 sessions).” Retest critical issues before launch. Document baseline metrics for future comparison.
Tips for Success
- Run a pilot session with a colleague before real participants
- Invite your team to observe live sessions — first-hand observation is 10x more persuasive than a report
- Coach think-aloud in the introduction: “Tell me everything you’re thinking as you work through this”
- Don’t intervene when participants struggle. 60 seconds of silence produces more insight than stepping in at 20 seconds
- Debrief with observers immediately after each session while observations are fresh
Common Mistakes to Avoid
- Testing with colleagues or team members who know the product
- Writing tasks that use the exact words from the UI
- Moderating without a script
- Reporting findings without severity ratings
- Running one round and treating results as definitive
