UX Design: The most comprehensive guide to usability testing

User research is an essential part of the design thinking and user experience design process.

The application of user research is to

  • Unearth new consumer behaviors (Design Thinking)

Discover innovative ideas by observing what users do

  • Usability testing (UX Design)

Close the gap between designers’ and users’ mental models

Why do we do usability testing?

By getting feedback from those who will use the product regularly, we want to make sure that the design will help them to get things done faster, and in doing so, it also delights them.

In this article, I offer you my hard-earned experiences doing user research and usability testing for years. I categorized the insights into four segments

  • A: Before testing
  • B: During the testing
  • C: After testing
  • D: Interview questions

TIPS FOR USABILITY TESTING

PART A: BEFORE TESTING

— How to recruit users

  • Cheap Option — Fast but NOT reliable

Tap into communities across social media channels such as Facebook groups, Linkedin groups, etc. You can source testers within a few hours.

This is a great option for testing cases that deal with consumer products. The only caveat is that testers could cancel on you easily. And it happened to me (a lot!)

TIP: If you need to source 5 testers, aim to book at least 10.

In this case, you have to create the NDA document, book the time with the testers, and guide them into the facility. All on your own. It’s resource-consuming for sure.

  • Expensive Option — Slower BUT reliable

Through platforms such as Testing time, you can book testers. Platforms as such offer options to filter your search. This is a great option for testing cases that deal with Business to Business (B2B) products.

Bear in mind that, by adding a few filters, the total price of sourcing 5 testers may easily go over 1000€. Use this option when you deal with a niche market in which finding the right person is strenuous.

The booking time takes longer. You have to contact them at least a week before the testing session so they have enough time to source testers for you.

In this case, you don’t need to do anything. The platform books the testers and makes sure they find their way to the doorstep of your office. You just need to open the door and say hello.

— Prepare your interview script (testing plan) carefully

Your interview script is the guideline by which the team aligns on the purpose of the project and the interviewer(s) performs her job.

Sometimes you have to bring a professional from the outside of the organization to interview testers. Hence you want to make sure the interview script is clear and includes all the sections below:

  • Intro about the project

You don’t need to go in-depth explaining the project but it should offer enough information so the interviewer understands the purpose of the project.

  • Testing objectives

You want to explain the objectives of the testing. The clearer the objectives are, the more promising the results of the testing.

The next chapters are about the questions you want to ask. Try to be brief and ask only questions you need to answer. Your time is limited; hence your questions.

  • Background questions (mostly demographics)

This is the part to write down the general questions you have about the users including where they live, what they do and their age etc.

  • Open-ended questions about their habits

Here you get more specific about their habits when it comes to the topic of the testing. For example, if you’re developing an app to track users’ diets, then the open-ended questions relate to how users prepare their food daily.

  • Tasks to perform

Here write down the tasks you’d like the testers to perform in case you develop a prototype.

— Understand the kinds of task you want testers to perform

  • Get their Impression

For example, you’re developing a landing page for a new product, and you’d like to know how people feel about the content, design, and branding.

To get the testers’ impression, try the Five Second Test exercise first. Show the design for only five seconds. And then ask them to share their impression. And then show the design again but this time with more time, and record what’s changed in their impression.

  • Perform a task

This is for when you’re developing a prototype of an app that performs certain functions. Then you want to test those functions with users like signing up or booking an appointment.

IMPORTANT: Often we only design tasks to test the usability of the application. Seldom do we design tasks to test whether users are willing to contribute to the growth of the business.

A user can contribute to the growth of the business in two ways

1) By subscribing to the application

2) By inviting their friends to sign up for the application

When you design a testing plan, it has to include a few tasks to test whether a user is willing to contribute to the the user or revenue growth. How?

Solution 1: Pull up an overlay at the end of the testing which asks testers to subscribe to the app. It should feel sudden and random. Try to surprise them here with a solid number like $9.99 / month.

Solution 2: Pull up an overlay screen and ask testers to invite their friends to the app.

In both scenarios, you want to get testers off-guard so their reaction would be genuine and honest.

— Allocate someone unbiased to lead the usability testing

Do not let the designers, product managers, or developers, who are invested in the project, to be the ones who interview testers. This is especially important if you have a prototype. Find someone outside of the team.

— Double-check the wireframes again with the designers the midday (not evening) before the testing day

Midday is important! In case there are things to improve, you’d have enough time to implement changes.

— Try to meet some of the users in their own space

Meet users where they are. Especially important if you’re conducting user research. For example, I was given this task to redesign an application for insurance brokers. I sat with brokers for three weeks and only shadowed them and observed how they worked.

For usability testing, it’s okay to invite them to your office.

— Only show the final designs to the testers

Don’t show testers low-fidelity prototypes. Testers won’t take the testing session seriously. When you have low-fidelity prototypes, then test them internally so colleagues can spot the ice-berg-type usability problems. Then finalize the design and test them with real users.

— For a proper user interview, you need 45 minutes

I’ve tried 15, 30, 45 and 60-minute formats, none of them worked better than a 45-minute format. It’s just the sweet spot.

— You have to at least interview 5 testers

Study shows that interviewing five testers reveals about 80% of all the potential issues with your design. Also, our experience proves this claim right. On top of that, it’s only possible to conduct five usability tests per day.

— Use Rainbow Sheet to capture interview outputs

To capture interview observations, notes, and scores for each task, use the rainbow sheet. Read about it here and look at it here.

— For a proper interview, you need two rooms

Try to have two rooms. One room for the interview. The other room for your team to observe the interview via a TV and take notes. The Interviewer does only lead the session; the team does the note-taking.

For remote settings: Use Zoom. Ask all your team members to turn off their cameras. It’d be only the cameras of interviewer and interviewee on. Then tick the “hide non-camera participants” option. This way you can mimic the two-room situation.

— Schedule the interviews properly

Only do user testing in the morning and early afternoon. Leave the rest of the afternoon for the reflection and sorting insights

PART B: DURING THE USER TESTING

— Look in their eyes, and introduce yourself and your role clearly. For example,

Hi there! My name is Claire and today I’ll be guiding you through the testing process. On behalf of my team, I’d like to thank you for being here with us. Your feedback means so much to our team.

— We don’t test you; there’s no right or wrong

Remind testers that you don’t test them and you’d like to get their reactions to certain things you’re working on. Remind them that their critical eyes could help so much to improve the experience.

— Don’t jump into the conversation right away

Make testers feel comfortable first. Use an ice-breaker. Find a relevant topic to talk about e.g. today weather.

— Observe the reactions to the task

Your task as a testing team is to observe user behaviors. As the interviewer, stay in the background and don’t disturb the tester flow.

— Ask testers to sign the consent form before hitting on the record button

You want to record the testing session, but before that, testers need to give their consent. Just google user testing consent form, and you get what you need.

— Press the record button before you begin the interview

Don’t forget.

— Record how long it takes to perform each task

Don’t help testers during the task performance. Let them deal with the task themselves. If they’re getting frustrated, don’t interrupt. Observe their situation and encourage them to talk about their frustration.

— Remain unbiased towards design

Be unbiased towards each screen and try not to point their attention to certain parts of the design.

— Encourage testers to think out loud

Encourage testers to think out loud constantly. Ask them questions when they’re silent for a while. Silence yields more silence.

Be careful; It’s a fine line though. Don’t keep interrupting the testers constantly. Give them enough time to express their thoughts seamlessly before you ask them a question.

— Try to criticize the design yourself

I said you should be unbiased. However, here’s a trick. Early on, try to make a light-hearted joke about the design yourself e.g. oops our designers forget to change this font again!

Doing that, it encourages users to express their judgment without the fear of being judged, wrong, or impolite. They see you as their friend. And that’s important.

— Answer questions with questions

Do not help testers to figure out things. Period. If they ask you something, throw the ball back in their court again by asking back a question.

— Wrap up and reflection

IMPORTANT: Allow testers to reflect and summarize their experience and add the last points if they desire. Do not skip this part. Sometimes testers need time to conclude their thoughts and give their final feedback.

PART C: AFTER THE TESTING

— What if the testing results were super negative?

Great. Then iterate. Don’t be jaded by the results of the test. There’s no failure. Feedbacks are there for you to improve your work. Happy iterating!

— What if the testing results were super positive?

Great. Then continue with the detailed design and preparing the assets for the development.

— What if the testing results were somewhat positive?

Reflect. Derive clear action points from studying the results of the tests, improve the prototype, and TEST again.

IMPORTANT: Do not continue without testing the improved version with the testers again. You want to proceed only if you got super positive results from the follow-up user testing session.

PART D: TYPES OF INTERVIEW QUESTIONS

This section is taken from the design thinking playbook. It’s a fantastic book and you should have it on your bookshelf.

Tracing behaviors

  • Who taught you that and how do you know it works?
  • Why do you smile when you say that?
  • If this was a difficult task, why was it difficult for you?

Gaining clarity

  • What exactly do you mean by this?
  • How would you describe it in your own words?

Asking about a sequence

  • Tell me more about the first memory of…
  • and what happened before or after the time you…?
  • How did you do this before? How did you improve?

Asking for examples

  • What software are you using?
  • What was the last app you downloaded? With whom did you discuss it?

Exploring exceptions

  • Did you have problems with this before?
  • When and why didn’t this work, then?

Understanding connections and relations

  • How do you communicate with your boss?
  • From whom did you hear that? And how did you hear of it?
  • Who helped you with that?

Informing outsiders

  • If you had to explain it to an exchange student, what would you say?
  • How would you explain this to your grandparents, or even describe it to a small child?

Comparing processes

  • What is the difference between your home and that of your friend?
  • What is the difference when you do this on the road instead of at home?

I hope that you find this article helpful. IN case you have feedback to share, please reach out to me at contact@bonanza.design


UX Design: The most comprehensive guide to usability testing was originally published in UX Planet on Medium, where people are continuing the conversation by highlighting and responding to this story.

Leave a Reply

Your email address will not be published. Required fields are marked *