Consider these variables before you choose a UX research method

Do any of these statements sound familiar?

• “We have a new product idea; let’s run a focus group to find out what people will think!”
• “We’re not sure if people understand how to navigate our site. We should set up a moderated usability test.”
• “Let’s do a 5-month-long longitudinal study to learn how people really interact with our product.”

If so, you’re not alone. Many UX researchers get pressure from stakeholders to conduct a particular type of research in order to answer questions. The problem is that sometimes the recommended research method simply isn’t the most efficient way to get those answers. UX research isn’t just about deliverables; it’s also about strategy. To produce the most helpful results, you’ll need to determine your research objective and then choose a method that supports it. As you evaluate your options, consider these different dimensions of research to help you arrive at the right decision.

Quantitative vs. qualitative research

As you conduct your research, you’ll probably be using a combination of quantitative and qualitative methods to answer your questions. Quantitative research methods rely on using large sample sizes to establish trends and conclusions. Qualitative research, on the other hand, is appropriate for getting a more in-depth, contextual understanding of why those trends occur. Some objectives are best suited for one type of research over the other. However, if you’re looking get a full picture of your user experience, you need to understand both what’s happening and why. If you only have quantitative data, you may be missing out on key insights that could make all the difference in your understanding of the user experience—and make sure you’re solving a problem that actually needs solving. And if you only do qualitative research, you won’t be able to tell whether your findings are representative of a larger population. That’s why the best research strategies incorporate both approaches.

Attitudinal vs. behavioral research

Though sometimes misconstrued as being the same thing, attitudinal and behavioral research are not synonymous. However, as with quantitative and qualitative research, the two can be useful when assessed concurrently.

What people say is often different from what they do.

Attitudinal research involves assessment of users' preconceived attitudes or feelings toward an experience. For example, this could involve asking a user why they like or dislike a feature on your site prior to using it. In contrast, behavioral research is focused on what the user does. Drawing another parallel to the distinction between quantitative and qualitative methods, behavioral research will tell you what’s happening, while attitudinal research helps to provide the reason why it’s happening. 

Stages of the product life cycle

The type of research you do will depend on the stage of development of the product you’re interested in. Each stage of development has different research objectives—and different questions that need to be answered.

Before development

Goal: Exploration Questions: What do people need? What are they already using? What’s working and what’s not? Where are we starting from? Methods: Interviews, focus groups, longitudinal studies, 1:1 user interviews, competitive analysis, benchmark studies

During development

Goal: Evaluation & validation Questions: Are we on the right track? Does the product look and feel right? Do users get how to use it? Can they find what they’re looking for? Methods: Card sorting, tree testing, moderated and unmoderated usability testing, preference testing, prototype testing

Live design

Goal: Measurement Questions: How are we performing? What can we optimize? Does our actual user experience meet users’ expectations? Methods: Moderated and unmoderated usability testing, A/B or multivariate testing, benchmark studies, longitudinal studies, diary studies, multichannel and omnichannel studies

Context and location

Many UX research teams use office space in-house to conduct in-person studies, which can be great for longer usability tests, interviews, and focus groups. But not all research needs to be conducted in person to be useful! Many methods can be done remotely, which means that your pool of potential study participants isn’t limited to a location. You can often conduct more research sessions in a shorter amount of time (and with less recruiting hassle) if you do remote research.

Remote testing can save time and take the hassle out of your research.

In-the-wild research (research that happens in the participant’s environment) can give you more realistic insights than lab research. Since participants aren’t in an artificial and unfamiliar setting, they’re more likely to give honest reactions and unbiased opinions. You also gather contextual findings about how your product is used naturally. Plus, some research just can’t be done in a lab. If you’re trying to answer questions about how users interact with a product in a specific context (using a fitness app at the gym, for example), then you’ll need to make sure your research happens in that location. We call that destination-based testing. For research that isn’t dependent on a location, study participants can be in their homes, offices, or anywhere that is convenient. Finally, there are times when you might need to observe something other than a digital experience on a computer or mobile device. For example, maybe you’re researching how customers find their way around a theme park, install a thermostat at home, or choose which brand of laundry detergent to buy at the store. These studies are called beyond-the-device tests, whereas traditional app and website studies conducted on computers and mobile devices are simply device tests.

Choosing a UX research method

Once you’ve zeroed in on the strategic requirements for your research, you can narrow down your choices of methods. A one-size-fits-all approach is likely to limit the usefulness of your research, so make sure to consider all of your options when it’s time to pick a method. If it turns out the best method is something out of the ordinary, then talk with your stakeholders about why you’re recommending it. Share what results you can expect—and how they support the research objective.