Generative vs. evaluation research

Behind every solid idea that’s primed for success is a slew of research. From who it’s for to your “why,” every question you ask paves the way for actionable steps. The goals of generative and evaluation research (sometimes referred to as evaluative research) are different but both essential. Generative research helps you define the problem you’d like to design a solution for. Evaluation research, on the other hand, helps you evaluate an existing design (in prototype, final, or some other form).

We’ll tease apart these two research design types with detailed differences between the two research methods, why they’re valuable, and the common research approaches associated with each.

Generative research

What is generative research?

Generative research is defined as a method of research that helps researchers develop a deeper understanding of users in order to find opportunities for solutions and innovation. Sometimes referred to as discovery or exploratory research, the goal is always the same.

The goal of generative research is to look to the world around you to find opportunities for solutions and innovation. These solutions could be new products or experiences or they could be an update or improvement to an existing one.

In order to identify new and innovative solutions, you must define the problem you are trying to solve. This requires you to truly understand how people live, including their environments, behaviors, attitudes/opinions, and perceptions.

When conducting generative research, the most important thing to do is to keep an open mind—you don’t actually know what the problem you’re trying to solve is yet.

Why is it valuable?

Quite frankly, if you don’t conduct generative research, you could create something that no one actually needs or uses. This happens all too often—from products that were discontinued after a few months to TV show cancellations that only made it through one season, many failed because they were addressing a (sometimes fictitious) problem the creators didn’t truly understand.

You don’t develop the best solution by doing lots of evaluative research and refining your design—you develop the best solution by properly identifying the problem with generative research.

How UserTesting can help you conduct generative research

At UserTesting, we’re able to conduct generative research with our platform by collecting data and human insight that reveal people’s behaviors, needs, and opinions. Common generative research methods include: ethnography, contextual interviews, focus groups, and data mining.

Our customers often:

1. Ask people to record their everyday activities, behaviors, and thoughts. Previously, this type of research was only possible when a researcher followed a contributor around for a given period of time. For example, in one test, a contributor was previously asked to shop for a television in-store. While shopping, he noted that the price display is not as intuitive as he would have expected. This type of research provides context and insight into customers’ shopping behaviors and habits, and may help teams define a problem they’d like to create a solution for.

2. Explore people’s attitudes, preferences, and opinions. This often helps companies understand their target audience’s point of view so they can create better solutions and experiences. In another test, a contributor who was shopping for shoes described how it’s extremely beneficial for her to see a picture of someone actually wearing the shoe. This information offers DSW’s digital team the perfect opportunity to update their product pages.

3. Understand people’s actions, thoughts, and feelings by conducting focus groups. This allows contributors to interact and converse with others while discussing a topic and allows the researcher to gather in-depth details about a given topic. Focus groups are valuable for understanding people’s motivations, but they should never be a replacement for behavioral research, such as usability testing or ethnography.

Evaluation research

What is evaluation research?

Evaluation research, also referred to as evaluative research, can be defined as a research method used for assessing a specific problem to ensure usability and ground it in the wants, needs, and desires of real people.

The goal of the evaluative research methodology is to test your existing solution to see if it meets people’s needs, is easy to access and use, and is hopefully even enjoyable. This type of research should be conducted throughout the development lifecycle, from early concept design (think rough sketches or prototypes) to the final site, app, or product.

Why is it valuable?

Evaluative research should always be a part of the iterative design process. Getting designs in the hands of our users as soon—and as often—as possible ensures that the experience will be shaped and refined to truly meet customer needs and expectations.

How UserTesting can help you conduct evaluation research

We can get concepts and designs in front of your target audience quickly and easily. If you have something for your contributors to hold or view, whether it’s live or still a prototype, we can get feedback on it.

Our customers commonly:

1. Ensure the design meets users’ expectations as early as possible by getting feedback on working designs or prototypes. Contributors can fully interact with the screen and speak aloud about how they interpret what they see and what could be improved. If they happen to run into issues with semi-functional prototypes, which is totally normal, they can even be asked to speak on what they expected to happen.

2. Optimize existing experiences by asking contributors to complete key tasks on live designs. For example, a contributor was previously asked to filter her flight options as she would normally expect. When searching for a flight, she only wanted to view direct flight options, but was unable to find where to filter. While it may seem like a small issue to some, removing that obstacle could increase customer satisfaction, decrease site abandonment, and ultimately increase the bottom line.

In another example, Purina came to UserTesting half a year after launching the My Pup app, looking for contributors to offer feedback on their initial design and suggest improvements. The result? After implementing changes, Purina saw an increase in engagement by 365% and a boost in active users by 48%.

3. Test “real life” experiences. Customers may ask contributors to record themselves using or unboxing a specific product or taking a test from a particular location. In a previous test, UserTesting’s contributors unboxed and set up a Nest thermostat. This provides valuable insight and context into the experience after customers have completed a purchase, including getting the item, opening the package, installing it, and using it for the first time.

Which is better, generative or evaluation research?

As you can see, there is no “best” approach. The two research methods have different goals and should be done at different times in the development and design process. However, both are extremely valuable. The bottom line is this: companies that create great experiences are more than likely frequently conducting both generative and evaluative research to maximize their potential for success.