Research Opportunities Throughout Your Product's Development Cycle

When we hear about companies not conducting user research and ask them why, a common response is that they don’t have a live site or app to test yet. We love hearing this because we get the opportunity to show them there’s always a good time to conduct research. So, if you’re in the “we only test when we’re live” camp, put on your research hat and get comfy. Today we’re going to dig into the research opportunities you can take advantage of at every stage of the product development cycle. Research can (and should) happen on a continuum, and to make your planning easier, it can be helpful to break your research down into distinct phases of the product lifecycle. At UserTesting, we break those phases down into four distinct groups, which we’ll cover below. Each stage has its own unique purpose and challenges, and we’ve found certain types of research methodologies work better at some stages than at others.

Phase 1: Discovery

At this phase in the development cycle, your main goal is to discover what your users really want and need. Your goal should be defining the problem you’re trying to solve, and ultimately, determining the product you’re going to develop. The types of research that are most helpful at this stage are those that help you explore your user’s behaviors or needs, as well as vetting general product design ideas. Research at this stage is most useful before you’ve designed or coded anything. Here are a few types of research methodologies we recommend during the discovery phase:

Interviews, focus groups & surveys

These types of research methods are a great place to start the discovery phase. They enable you to ask direct questions to test participants to help guide your understanding of their preferences, attitudes, needs, and goals. While all three methods involve direct questions to participants, there are a few differences between them:

• Focus groups: Participants interact and converse with each other while discussing a topic or question asked by a moderator. The brainstorming leads to people building on each other’s ideas, which allows a group to really dig into an idea or issue. The moderator then gathers in-depth details about the group’s actions, thoughts, and feelings.
• One-on-one interviews: Participants are asked series of questions while a moderator takes note of their feedback and facilitates further discussion accordingly.
• Surveys: Participants answer series of questions without a moderator. Responses are open-ended or multiple choice and are typically about preferences, attitudes, and experiences on a topic or item of interest. In order to make a claim about a user group, the sample size must be large enough to capture statistically significant results.

Ethnographic research

This type of study will help you to better understand how people interact with your product by observing them using it in their natural environment. What people say and what they actually do are often quite different. For example, if you were researching the coffee drinking habits of your target audience, a survey might indicate that the majority of respondents prefer to drink their coffee black. However, when observing those same respondents actually preparing their morning cuppa, you may find a much larger portion snuck in a dash of cream and sugar. Observing users in their natural environment can help bridge the gap between what you’ve uncovered in focus groups, interviews, and surveys and observing users’ natural behaviors.

Comparison studies

Another great way to better understand the needs of your users is to research how your target audience responds to your competitors. A comparison study (sometimes referred to as a competitor study) is when you ask users to interact with the product or experience of your top competitors, and share their thoughts out loud as they go. This kind of study provides insight into what your competitors are doing well and where the opportunities are. It also helps you determine what’s working—and what’s not—with their design.

Phase 2 & 3: Design and build

Once you’ve determined what your users want and need, it’s time to get started building (or improving) it. We group these phases together because of their iterative relationship. As you design and build, you should constantly be testing to determine if you’re on the right track with your design, and whether or not users understand what it’s for and how to use it. The types of tests you conduct at this stage will be geared toward usability, infrastructure flow, and overall user preferences. And the insights you gain at this stage will not only help guide the direction of your design but help address major issues before you have a fully-developed and coded product.

Card sorting

Card sorting allows participants to sort topics like pages, categories, or pieces of content into related groups. Card sorting can provide insight on how people group and label content and information in their own minds, and will help in designing your information architecture in a way that makes sense to your users.

Tree testing

Tree testing can give you insight into how easy (or difficult!) it is for users to find important items within a website or app. Similar to card sorting, this information can provide guidance when designing or assessing an information structure.

Preference testing

In preference testing, participants interact with and review multiple designs to determine which one they prefer and why. Participants can compare aesthetics, interactions, and even content. This type of test can be added to a usability test, or even incorporated with competitor tests. Preference testing differs from A/B testing (which we discuss in the next section) in that it provides more qualitative feedback on why participants prefer one design over another, which can be helpful during this stage of development.

Usability testing

If users can’t achieve their goals with your product, they’re not going to use it. That’s why usability testing is such an important part of your product’s development. The primary goal with this type of research is to find out if users can actually use your product and gather feedback about how they felt about the process. In usability studies, test participants are asked to complete predetermined activities using a design or interface. The participant typically thinks out loud as they complete the tasks, and the researcher will observe their experience. These kinds of studies can be conducted remotely using specialized tools (like UserTesting) to record the session or conducted on-site with a researcher present. This type of testing will require multiple iterations as you should re-test after each change to your design until your users are achieving their goals with ease and delight.

Phase 4: Optimize

One of the biggest mistakes you can make when conducting user research is to quit once your product has launched. Your users will change and so will the market, so the only way to stay ahead of the game is to continually improve and optimize your user experience.

Benchmark testing

Benchmark testing measures your progress over time, by tracking how your site or app performs using the same benchmarks. This type of study not only gives you an indication of how successful your iterations have been, it also provides a quantitative metric that can make analyzing and presenting your findings easier and more compelling. You can run a baseline study on an existing site or product, and then run additional benchmarking studies on a regular basis as the new design becomes available. This will allow you to track changes in usability metrics over time, as well as compare usability metrics from the old design to the new design. Typically you would run a benchmark study in the same way that you'd run a usability study, but you conduct the study with a larger sample size (usually with more than 20 test participants), and would continue to run the same study at regular intervals (such as monthly, quarterly, or yearly).

Longitudinal studies

Sometimes you can’t capture all the feedback you need with one session with test participants. Longitudinal studies collect data on multiple occasions from the same participants over a period of time. Studies can be run over the course of a few weeks, months, or even years. A longitudinal study will provide valuable insight into how users behave and interact with their surroundings, an experience, or a brand over time in their everyday lives. This type of study is a great tool when you want to understand how users interact with your product or service over time, rather than just a moment in time.

A/B testing

A/B testing presents each unique user with one of two versions of a design—the A version or the B version—and conversion rates for each version are compared to see which one performs best. A/B testing is most valuable when only one element is different in each version. For example, using a different call to action for each design approach, while keeping everything else the same, allows you to draw strong conclusions about why one design performed better than another.

Conclusion

Gathering great user feedback is the cornerstone for building a great product or service. But getting started on a new research project—even if you’ve conducted hundreds of studies before—can be complicated. At UserTesting, we’ve conducted hundreds of thousands of tests, and we’ve (almost) seen it all. You can research just about anything. Seriously. The insights you gain from your research will help guide your strategy and the development of your product—with your users in mind. But this was just the beginning. If you want to learn all of the various types of research methodologies, when to use them, and how to organize and share your results, check out The UX Research Methodology Guidebook. It’s for anyone who’s ready to get started testing, but would like a little guidance on how to approach a study, when to run it, and how to interpret the results.