Usability Testing

Usability testing is watching real people try to use your product. Not asking them what they think. Not having them fill out a survey. Watching them attempt actual tasks and observing where they struggle, succeed, and give up. It's the highest-impact research method and the one most teams skip.

The 5-User Rule

Nielsen Norman Group's research shows that 5 users find approximately 85% of usability problems. You don't need a massive study. You need a small study done often.

Usability problems found vs. number of test participants:

100% ─┤
      │                    ●─────●─────●─────●
 85% ─┤              ●
      │         ●
 75% ─┤
      │    ●
      │
 50% ─┤
      │
      │●
   0% ─┼────┼────┼────┼────┼────┼────┼────┼────
       0    1    2    3    4    5    6    7    8
                   Participants

5 users → ~85% of problems found
Beyond 5 → diminishing returns

Better approach: Test with 5 users, fix the problems, test with 5 more users. Two rounds of 5 finds more than one round of 10.

Test Types

Choosing the Right Type

Moderated vs. Unmoderated: Tradeoffs

Planning a Usability Test

Define Your Goals

Before writing tasks, answer these questions:

Recruit the Right Participants

Screener question examples:

• "How often do you shop online?" (Filter for frequency)
• "Which of these tools have you used in the past 6 months?" (Filter for experience)
• "What is your primary role at work?" (Filter for job relevance)

Writing Good Tasks

Tasks are the heart of usability testing. Bad tasks produce useless data. Good tasks reveal real usability problems.

Task Structure

Every task should include:

1. Context/scenario: Why the user is doing this
2. Goal: What they need to accomplish
3. No instructions: Don't tell them how to do it

Good vs. Bad Tasks

Task Difficulty Spectrum

Include a mix of task difficulties:

How Many Tasks?

Facilitating a Session

Before the Session

• Test your recording software
• Prepare a printed/digital test script
• Have the prototype/product ready
• Remove any personal data from test accounts
• Silence your phone

During the Session

The Think-Aloud Protocol

Ask participants to narrate their thoughts as they work:

"Please think out loud as you go through this. Tell me what you're looking for, what you expect to happen, and what you're thinking as you make decisions."

If they go silent:

• "What are you thinking right now?"
• "What are you looking for?"
• "What do you expect to happen next?"

Don't: Ask "Why did you do that?" during the task. It feels like being tested. Save "why" questions for after the task is complete.

Metrics to Track

Quantitative Metrics

Qualitative Metrics

Post-Task Questionnaires

Capture perception after each task:

Single Ease Question (SEQ):

"Overall, how easy or difficult was this task?" 1 (Very Difficult) to 7 (Very Easy) Average: 5.5. Below 5 indicates a problem.

After all tasks, System Usability Scale (SUS):

10 alternating positive/negative statements, rated 1-5:

1. I think I would like to use this system frequently
2. I found the system unnecessarily complex
3. I thought the system was easy to use
4. I think I would need tech support to use this
5. I found the functions well integrated
6. I thought there was too much inconsistency
7. I imagine most people would learn this quickly
8. I found the system cumbersome to use
9. I felt confident using the system
10. I needed to learn a lot before using this

SUS Scoring:

• Odd questions: score - 1
• Even questions: 5 - score
• Sum all scores × 2.5
• Result: 0-100 scale

Analyzing Results

Step 1: Compile Findings

For each task, document:

• Success/failure for each participant
• Time taken
• Errors observed
• Quotes and reactions
• Observations about behavior

Step 2: Categorize Severity

Step 3: Prioritize

Plot issues on an impact/frequency matrix:

         High Impact
              │
    FIX NOW   │   FIX SOON
    (Critical)│   (Major)
              │
──────────────┼──────────────
              │
    BACKLOG   │   MONITOR
    (Rare but │   (Common but
     severe)  │    minor)
              │
         Low Impact
    Low Frequency    High Frequency

Step 4: Create Recommendations

For each issue, document:

ISSUE: Users can't find the "Apply Coupon" field during checkout
SEVERITY: Major
OBSERVED: 4 of 5 participants missed it
EVIDENCE: "I have a coupon code but I don't see where to enter it"
          Participants scrolled past it, looked in cart summary
RECOMMENDATION: Move coupon field above the order summary,
                add a visible "Have a promo code?" link
BEFORE: [screenshot]
AFTER: [mockup]

Step 5: Share Results

Running Unmoderated Tests

Tools

Unmoderated Test Structure

1. WELCOME SCREEN
   "Thanks for participating! This test takes about 10 minutes.
   We're testing a design, not you. There are no wrong answers."

2. SCREENER QUESTIONS (2-3)
   Filter out non-qualifying participants

3. TASKS (3-5)
   Task description → Participant completes task → Post-task question (SEQ)

4. POST-TEST QUESTIONS (3-5)
   SUS or overall impressions

5. THANK YOU
   "Thanks! Your $X gift card will arrive within 24 hours."

Specialized Test Types

First-Click Testing

Show a design and ask: "Where would you click first to [accomplish goal]?" If the first click is correct, users succeed 87% of the time. If the first click is wrong, success drops to 46%.

5-Second Test

Show a design for 5 seconds, then ask:

• "What is this page about?"
• "What do you remember?"
• "What would you do on this page?"

Tests first impressions and visual hierarchy.

A/B Testing

Common Mistakes

Key Takeaways

• Test with 5 users, fix, test again. Two small rounds beats one large study.
• Write scenario-based tasks that give context and goals, not step-by-step instructions.
• Stay silent during tasks. The urge to help is strong. Resist it.
• Watch what users do, not what they say. Behavior reveals truth.
• Categorize findings by severity (critical/major/minor/cosmetic) and prioritize accordingly.
• Share findings with video clips. A 30-second clip of a user struggling is worth more than a 20-page report.
• SUS score of 68 is average. Below that, you have usability problems to fix.
• Test early with wireframes. Don't wait for a polished product.