Longitudinal studies

Longitudinal studies are tests that collect data on multiple occasions from the same participants, over a period of time. With the ability to be run over the course of a few weeks, months, or even years, a longitudinal study will provide valuable insight into how users behave and interact with their surroundings, an experience, or a brand over time in their everyday lives. This type of study is a great tool when you want to understand how users interact with your product or service in the long haul, rather than just a moment in time.

When should longitudinal studies be conducted? 

Longitudinal studies can be done at any time; however, we recommend running them before product development as a way to gather requirements and understand users. You can also run them after development, which will give you insight into how, when, and why people naturally use your product or service.

Longitudinal studies can capture the experience after the onboarding stage to better understand how that first experience influences future usage. This type of study is also especially helpful if new features are pushed during the first days of using the product. Insights can also be captured around how well product features align with needs. 

UserTesting customers observe these experiences by using the desktop and mobile screen recorders to capture product usage on multiple days and elicit user feedback on their experience over time.

In the pre-development stages, when your focus is exploration, consider asking: 

• How well does the product address a need?
• What are they already using? 
• What’s working and what’s not? 
• Where are we starting from? 
• What are the areas of opportunity that can be used to increase usage and retention?
• Are the right message and value proposition communicated in the first-time user experience?
• Are the next steps to using the product clear? Are they attractive?
• How difficult or easy is it to learn the product or experience?
• What is the learning curve like? Is a steep learning curve deterring first-time users, or making the product less attractive?

After your product has gone live and your goal is measurement, consider asking: 

• How are we performing? 
• What can we optimize? 
• Does our actual user experience meet users’ expectations?

What are the different types of longitudinal studies? 

1. Diary studies

A diary study is a type of longitudinal study where participants create “diary entries” as points of data. These diary entries can be audio, text, images, videos, or a combination of all, and are typically combined with other studies to gauge participants’ expectations at the start and conclusion of a larger study.

Diary studies are ideal for collecting participants’ behavior and reflections. Participants are provided a means to regularly share their experiences in response to a prompt and questions. UserTesting customers can select from a variety of mediums to gather these “diary” entries, from video footage of their activities to video entries of their responses.

Typically, users self-report their activities at regular intervals to create a log of their activities, thoughts, and frustrations. It’s a useful approach for capturing organic feedback on activities that are repetitive, long, or unpredictable. Research questions that involve habits, prolonged processes, or perception change over time (likelihood to use, learnability, etc.) are prime candidates for diary studies.

As with all longitudinal studies, the amount of time required for the study is important. If you can get a reasonable snapshot of behavior in an hour or a day, for example, diary studies may not be right for your questions—try an unmoderated study instead. And while diary studies may be more complex than typical usability studies, they can be done using the UserTesting platform. 

2. Omnichannel studies 

An omnichannel study is when participants complete an activity that spans more than one channel or device. For example, if a customer finds a recipe on their desktop and adds the ingredients to a shopping list on their smartphone, refers to the list on their phone while at the grocery store, and then follows the recipe at home on their tablet, they’ve completed an omnichannel experience.

While usability studies can help you optimize each of your channels, omnichannel studies help validate the connective experience between subsequent channels. It’s important to understand how well the product fits into the landscape for that experience. 

UserTesting customers observe these experiences by using a combination of the platform’s screen recorders and the participant’s front-facing camera to capture participants’ usage and impressions of channels throughout their experience, from desktop to mobile, to beyond-the-device.

Best practices in conducting a longitudinal study 

1. Be sure to let your participants know that they’ll be conducting a series of tests, and be clear about the amount of time that will be required to complete the study. 

2. Include a requirement in your instructions that notes participants will be expected to take a specific number of follow-up tests, and that the test will last a specific amount of time. As an additional incentive, you can offer to pay participants a bonus for completing multiple tests. We’ve found that this can dramatically reduce the participant drop-off rate for this type of study.