Weirdly enough, it’s surprisingly common for UX researchers to feel that usability testing should be reserved for rare design and product development projects. This often stems from the belief that usability studies require a large pool of users which will increase the budget for such a project, take more time, and create a prolonged period of user interview analysis.
What a load of rubbish!
If you’re recruiting users for usability testing, try not to listen to the naysayers. Sure, usability tests can be complicated depending on the topic of your project – but so can everything! And if there’s one thing it definitely doesn’t have to be, it’s expensive. Recruiting user testing participants can be remarkably affordable – if you know what you’re doing…
So, how many participants are recommended for usability testing? Let’s get down to it.
How Many Users Do You Need For Usability Testing?
When deciding how many users to have for usability testing, it’s important to remember that every project is different and will require a different amount of research. However, there are some clever minds over at the Nielsen Norman Group that have worked out an average number of users to test: 5.
According to the NN Group, five participants will reveal around 85% of the problems with your product (so long as you are asking the right interview questions). Once you get onto 6 users or more, you start to get less new information, meaning you are burning through your budget to get information that you mostly already have.
How Do Usability Tests Work?
Generally, you will interview several users about their experience with your product, and more specifically, the area in which you are aiming to improve.
You can also use some free UX research tools to help you with your usability tests. While you may want to have heatmaps to track what your users do, you would also greatly benefit from downloading tl;dv, the best virtual meeting tool for recording, transcribing and timestamping video calls – including for user interviews!
Typically, the user interview stage will go as follows: the first user will offer the most insights on usability. The second will repeat much of what was observed by the first, but with some additional insights. The third will produce a lesser amount of fresh insights than the second, and so on. By the time you test a sixth user, most of what you’ll hear from them will be repetition. Read: useless.
@tldv.io If my mom would like it, then it passes. Right? // @been.ian skipped one of our meetings to finish this #product #tech #momtest ♬ original sound – tldv.io – Meeting Recorder
It’s also worth noting that you don’t want to interview all your users back-to-back. Give yourself some time to digest what they’re saying. You’re not a robot!
One of the best ways to do this is with the online meeting recorder we mentioned earlier. tl;dv comes with a built-in notetaker feature for both Google Meet and Zoom. You can make notes during the meeting, and even highlight and timestamp certain parts so that you (or your colleagues) can easily jump back to the specific moment and hear the feedback from the voice of the customer, instead of your hastily scrawled notes.
Whichever way you choose to make sense of your data, ensure it’s contextualized and thematically grouped. You can look into learnability for more info about how to improve your usability.
How to Find Users for Usability Testing?
It’s one thing knowing what to ask a user and how to present yourself in a way that makes them feel comfortable and open, it’s an entirely different thing to know how to find users for usability testing in the first place.
If you have five users lined up for interviews and you get a no-show, you’ll quickly find yourself in deep water. How are you supposed to get those delicious insights when you’re a man down?
Like with everything, recruiting user testing participants doesn’t have to be difficult. In fact, with a bit of preparation, recruiting users for usability testing can be pretty easy!
The first step is to use a thorough screening process: especially if you’re getting users from social media.
There are agencies that can be used for such a thing, but if you want to get real customer insights, you’re better off reaching out to your own customers. In a B2C business, this should be fairly easy. Emails are a standard method. If you operate on a B2B model, then things could be a little trickier, but still feasible!
Most agencies also allow you to screen your potential users. If you’re testing an airline website, for example, you can get people who have recently booked a flight, i.e potential customers.
While it may be tempting, it’s best to avoid friends, family, and colleagues for usability tests. They’ll already be familiar with your product and company, so they’re likely to give one-sided answers or say things that they think you want to hear. Using them is akin to asking them to read your novel manuscript: they don’t want to be as harsh as you need them to be.
Check out UX Design Institute’s coverage of how to find users for more details.
Diversity is KEY!
Another pro tip for recruiting user testing participants: be diverse!
Having a broad variety of different people in your interviews can benefit you more than you could ever imagine. People with different experiences, skills, and backgrounds, will all bring something new to the table. They see your product through different eyes.
While you obviously want the user-in-question to be a potential customer, it’s important to get views from a diverse range of people.
Diversity and inclusion is about valuing the contributions and opinions of others, regardless of their race, ethnicity, gender, sex, age, location, income, education, disability, or profession. Having the widest range of people possible leads to better decision-making.
Keep diversity in mind when recruiting users for usability testing.
How to Run Usability Tests?
As we’ve already said, testing five people over two days is both cost-effective and practical. This gives you enough users to get all the insights you need for the first batch of updates (it’s wise to run another group of usability tests after the updates from your first group are implemented), and enough time to really digest what was said and contextualize your notes.
A usability test in Dublin, like those found in other European tech cities, may cost around $100 per user and include a bonus of $50 per user employed by a recruitment firm. The average agency cost in the US is $107, but this doesn’t account for other related expenses such as equipment and software.
@tldv.io • Instagram photos and videos
It’s recommended to always record your sessions when running usability tests. After all, the main goal of them is to convince stakeholders or senior designers to implement a change of some kind. With quotes directly from the horse’s mouth, tl;dv empowers you to easily persuade your stakeholders to take action. You can show them what the users actually said, what expression they pulled when saying it, and how their body language changed. It’s safe to say that tl;dv has revolutionized user research.
The Work Day
For a full day, usability testing may cost upwards of $900. Testing can reveal critical user concerns and insights that you couldn’t otherwise discover. It’s also an excellent investment since it helps you identify important user issues and observations.
With just five participants, you can conduct a whole round of user testing in one day. However, it’s advised to break them up over a two day period so that you can discuss, digest, and deeply understand what each user said and meant. If you want to go with more than 5 participants, that means more days and more resources spent on your usability testing.
To make the most of your working day, set aside an hour time block for each user with at least a thirty-minute break between sessions. You may use that time to collect your thoughts and make notes on the previous session. You may need longer to organize and analyze the early data, not to mention that you’ll need to take breaks, so, again, it’s a good idea to do 3-4 interviews per day, maximum.
@tldv.io We love constructive criticsm #productmanager #product #tech #productmanagement #corporatehumor #startup ♬ original sound – tldv.io – Meeting Recorder
How to Avoid Recency Bias?
Gathering your insights over a period of two days (maximum) should be enough to limit recency bias. You don’t want to leave a big gap in between your user interviews unless you absolutely have to.
It’s inevitable that bias will creep into your user research no matter what you do, but it’s possible to prevent it from bending you over and slapping you silly.
In fact, tl;dv’s meeting recorder enables you to refresh yourself by watching the meeting’s highlights again whenever you want. It’s no longer necessary to rely on your notes for interviews that are fading from memory. Let every user interview hold as much weight as the last one.
When the 5 User Rule Breaks Down
There are a few exceptions to the five-user rule that should be noted. If you’re testing a website or app with a lot of features, remember to test five users for each function. If you ask them to cover every feature, you’ll get vague and sporadic information.
- Multiple features
Imagine testing the purchase and check-in processes of an airline website. Rather than attempting to cover both features in one test, it’s better to distribute them across two separate tests. Here, ten people are suggested; five should be allocated to the purchase process and five to the check-in procedure.
- Demands of the stakeholders
When stakeholders demand statistical evidence from usability testing, the rule of five may have to be violated too. While this isn’t an ideal approach to a qualitative study, it may be unavoidable.
A senior manager may request quantitative proof that the company’s registration and login are ineffective, claiming that they don’t believe them to be. To demonstrate that 12 out of 20 users had difficulty generating a password or 17 out of 20 users had difficulty logging in, you might have to test 20 users. However, these tests may be a lot more specific, thus shorter in duration and less expensive than traditional usability tests.
Ideally, seek to persuade stakeholders not to embark down this road ahead of time! Whenever feasible, aim for a minimum of five users. In the long run, it’s more beneficial for your user experience and the company’s overall goals.
- More testing rounds
You will have to run more tests when you’ve launched a new feature or updated something anyway. Don’t waste resources by interviewing too many people early on.
For more info about how many user interviews to conduct, check out our full guide.
Use tl;dv for Your Usability Tests
We’ve already touched upon the power of this meeting management software, but it’s necessary to comprehend just how much it can help.
With tl;dv you can record all your user interviews, get transcripts in more than twenty languages, and timestamp, letting you link every insight to the unbiased evidence of the exact moment it happened from the voice of the user. Simply send a timestamp on Slack, or your work chat of choice, and your colleagues will be able to view the exact moment for themselves.
Oh, and all of this is available for free…
Let’s Get Testing!
So, how many participants are enough for usability testing? 5 should do it, but it may vary from business to business.
In general, it’s not a good idea to bring on a boatload of users for your next usability test. You can actually get good insights with just a handful of participants if you have the right test setup and are ready to analyze and synthesize your data properly.
And if you don’t want to go through all that hassle, we at tl;dv have got your back. Our platform is designed to help you quickly record all your research tests so you can make informed design decisions without breaking the bank or wasting time. So what are you waiting for? Download tl;dv now and start testing today!
