You can be the hottest stuff UX researcher in all the land, but no matter what, no matter how hard you try, bias in user research is an inevitability.
It’s simply going to happen every single time. Sorry!
You can try to be as impartial, neutral, and super chill as possible, but humans are literally built to make snap judgments and decisions. Our biases are there to keep us “safe” and are there on a pre-cognitive level.
Yes, it sounds dull repeating the old trope about saber tooth tigers and whatnot, but it’s true. And product managers, well, yeah, they have their own special sauce of judgment. 😉
@tldv.io Why do they keep asking me to onboard people #workfromhome #onboarding #corporate #productmanager
♬ original sound - tldv.io - Meeting Recorder
Even when we are consciously trying to remain impartial, our subconscious is still there, whirring away with everything we have ever thought, seen, felt, or experienced, skewing our view. It’ll seep into every little thought we have.
That means UX research biases are as much a part of the process as the research itself.
But fear not! This doesn’t mean your user research is doomed to be biased and inaccurate. Instead, it means you should reduce the amount of bias as much as possible. While you’re never going to eradicate and prevent research bias fully, you can strive to minimize it.
Forewarned is forearmed!
What is Bias in User Research?
Bias in the context of UX means assumptions or preconceived beliefs held by those conducting the research that can lead to invalid or inaccurate data.
It can also be the bias and assumptions that the research subjects have.
Essentially, we are all just a big ol’ bag of massively judgey people. No hate, we’re all in the same boat here.
Take this for example, a researcher interacts with participants and assumes they know an answer to particular questions when they don’t. This can influence the way they phrase their questions and how they interpret participants’ responses, leading to potentially false conclusions.
Another example could be if a researcher has a particular goal in mind when conducting user research. Unwittingly, the researcher could manipulate their questioning to push for a preconceived outcome instead of letting participants provide honest opinions and feedback. The whole thing is doomed from the offset with leading questions, tweaks in language, and even the delivery which causes a bias in the answers given. Even the way that the study is structured before the very first question is even asked, can be biased.
These are just a couple of examples, but there are many more biases that can creep into user research.
All biases must be acknowledged to ensure accurate data is being collected in a way that makes for an effective UX research process.
So Why Does Bias in User Research Occur?
Bias can creep into user research through many different sneaky avenues, even when we are being super vigilant.
Some of the most common causes can be researcher fatigue, which is when researchers become bored or lose focus after asking too many questions in interviews or surveys. This often leads to hasty conclusions being drawn and data misinterpreted due to a lack of attention.
Another cause could be confirmation bias which is when research leads to results that confirm the researcher’s prior beliefs. This can be a particularly dangerous form of bias as it undermines the entire process and leads to false assumptions about user behavior or expectations.
Scariest bit? You don’t even necessarily know that it’s happening! It can happen at any point during the research process.
Why is Bias in User Research Bad for Product Management?
Bias in user research can be INCREDIBLY harmful when it comes to product management.
If inaccurate data is collected, it can lead to incorrect assumptions being made about what users actually want or need from a product. This wastes time and resources, as developers and product teams spend time building something that isn’t backed up by actual user needs.
It’s really important to make sure research results can be trusted and acted upon, as the success of a product often depends on it.
All I need to say here is New Coke…
If you’re too young to know about it (or maybe just hate sugar!), then the monolith of carbonated beverages, nearly bankrupted itself in the 1980s when it decided to change its formula.
Coca-Cola looked at the market and even spent millions on product research, user research, and more, but ultimately it made assumptions.
Rather than creating a NEW product, they replaced a beloved one, and the backlash was like nothing they’d ever seen.
It took them an astonishing 77 days to backtrack from their mistake and bring back a version of its “Original Formula”, but in that time, it had become an object lesson for every marketer and product manager out there about the dangers of bad user research. Silver linings for us I guess!
The lesson here? Bias in user research is bad for product management as it can lead to costly mistakes.
Can You Totally Get Rid of User Bias in Research?
So we have identified that user bias is a big problem. Surely now we know that we can just eradicate it?
While we LOVE your vibe, unfortunately, it’s nearly impossible to totally get rid of user research bias. However, there are steps that can be taken to answer the question “How to avoid bias in user research?”
Make a record of your own assumptions and expectations
Literally, write it down on a piece of paper and seal it in an envelope. This ensures that your own preconceived notions don’t end up influencing the user research process.
Be mindful of language and phrasing
The way you phrase questions to users can dramatically influence how they respond, so being aware of this is key to avoiding bias in user research.
Ensure that participants are comfortable and relaxed
When conducting interviews or surveys, it’s important to create a relaxed atmosphere that encourages participants to share honest feedback. If a subject feels uncomfortable, they may start to give answers they THINK they should give, rather than what they really feel. Hawthorn Effect babyyyy!
Use multiple researchers
You’re not alone! When possible, use multiple researchers who may be able to spot and guard against biases that could go unnoticed by one researcher on their own. This is where a tool like tl;dv can REALLY come into its own. A remote collaboration tool at its very core, tl;dv allows researchers to share their insights with others and allows them to catch up in minutes to help challenge the bias.
Ask open-ended questions
Avoid asking questions with a right or wrong answer. Instead, ask open-ended questions that allow the respondent to provide more detailed feedback on their experiences and thoughts.
Use validated surveys
If you’re using a survey as part of your research process, use one that has been validated such as System Usability Scale. Or alternatively, create your own survey questions that don’t lead the respondent.
NEVER assume
As the old adage says, “if you assume you make an ass out of u and me”!
If a researcher has an idea about what could be causing a problem, test it out! But don’t just assume it is correct without collecting any data first. Similarly, don’t assume that every respondent has the exact same level of knowledge or the same experiences.
Get feedback from as many people as possible
Even if you only have access to a limited pool of subjects try to reach out and obtain feedback from a wide array of individuals to gain an in-depth understanding of the subject. Moreover, don’t forget that one thing always remains true: the more responses you receive, the better picture it paints.
Using tl;dv as a remote user research tool allows you to supercharge your interview flow, easily taking and recording online meetings in a flash.
Diversify your subjects
Also, don’t pin all your responses onto one type, or a couple of types, of demographic. In the age of online research, being able to outreach to as diverse a group of users as possible is the key to real success. It will not only give you a wider range of views to work with, but it’ll also get rid of, or at least dilute, any cultural, psychological, and socio-economic preconceptions you would get from a single group. To gain a better understanding of the participants, try to obtain as much information about them as possible. Analyzing their demographic and psychographic traits may help explain why they’re using the product differently.
Have SUPER accurate documentation methods
To combat bias and ensure accuracy, document user feedback as accurately as possible. This prevents misinterpretation by the researcher or other stakeholders. Recording user behavior and research results in a scientific manner can completely eliminate any potential bias from creeping into your product decisions.
One amazing way to do this (if we say so ourselves!) is by using tl;dv to record user research interviews. Our remote UX research tool captures insights in the users’ own words to minimize misinterpretation!
The tl;dv library serves both as a UX repository and documentation tool: no longer relying on one or two individuals to manually record user input. Data can be shared throughout an organization so that everyone has access to the original form of communication allowing for more informed decisions based on the actual words and non-verbal communication of the participants themselves.
HOT TAKE ALERT! Bad UX Research Is WORSE Than No UX Research
Did your mother ever tell you if a job’s worth doing, it’s worth doing right? The same goes for UX research, and many people agree.
It’s vital to ensure that the research conducted is actually providing useful results. Poorly conducted UX research can lead to faulty assumptions and less-than-accurate product decisions. Rather than relying on guesswork or anecdotes, take the time and effort to gather accurate data that reflects user behavior. Doing so will save your team time and resources, and will also give you the chance to really screen out that bias.
UX research is an essential part of the product development journey. Through a proper and accurate collection of data, assumptions can be debunked, design decisions can be made more confidently, and products will become better with every iteration.
But it’s important to remember that poorly conducted research, skewed with preconceptions is worse than no research at all! So do yourself a favor and take the time to document user feedback accurately. And if you’re looking for the perfect tool to do it, try tl;dv! We promise you won’t regret it. 🙂
Now, who wants a New Coke?
