Moderated testing 101: when and why you should use it

The best way to maximize the chance of a successful product is to lead with a customer-centric mindset. When you have conversations with your customers to learn more about their expectations, you can use that knowledge to design impactful products. Usability testing helps you collect qualitative data to understand your user better, enabling you to create an ideal user experience for your end-user. When product teams identify how users interact with their products, they can synthesize this data to discern how their designs best match their users' needs. 

Specifically, moderated usability testing enhances feedback loops that help you design more intuitive and effective products. Here’s everything you need to know about moderated testing—from its benefits, when to use it, and how UserTesting can help. 

Table of contents 

1. What is moderated testing? 

2. Which is better—remote or in-person moderated testing?

3. What are the key benefits of moderated testing? 

4. When should you conduct moderated testing? 

5. How to know when moderated testing isn’t the best fit 

6. How UserTesting can help you with moderated testing

What is moderated testing?

During moderated testing, known as Live Conversations at UserTesting, an individual helps facilitate, or moderate, the study. The moderator works directly with test participants, guides them through the study, and answers questions in real-time if the participant encounters any challenges while completing their tasks. When you moderate a test, you're right in the thick of things with the people who use your product. 

Moderated studies offer plenty of flexibility because they can be conducted either remotely or in person, and you can open the space for dynamic discussions. Depending on your organization's goals and budget, consider both the benefits and drawbacks when deciding whether to conduct remote or in-person moderated testing.  

Which is better—remote or in-person moderated testing?

With work-from-home setups becoming more and more widespread, remote moderated testing may be the better and more comfortable option for some. It tends to be more cost-effective and less time-consuming because there are limited or no travel costs or logistics involved. Meanwhile, test participants can engage in studies from the comfort of their homes, which means you can receive results quicker. 

Conversely, in-person testing offers fewer environmental distractions—due to a controlled test environment—ensuring a focused study with little to no interruptions that may affect the study's outcome or the concentration level between moderators and participants. With remote testing, a contributors’ background noise may be an issue thanks to distractions, pets, or other people present. 

Another advantage is that in-person test environments tend to come without technical and internet connectivity issues. During remote testing, it’s all too common for participants to encounter technical difficulties. Video conferencing software—like Zoom or Google Meet—may glitch, or a test participants’ Wi-Fi may not be running at maximum speed. These malfunctions could delay or postpone the study, costing researchers in time and expenses.

Both testing options allow you to observe your users' body language and cues, so you can collect deeper insights about your users. Moderators can have a natural conversation with participants, and there's a higher chance of participant engagement. Not only does this establish trust between the moderator and the participant, but participants are more motivated to engage and reveal their thoughts and feelings when there's a foundation of camaraderie. 

It's important to note that you don't have to choose between remote and in-person moderated tests. You can use both interchangeably to receive effective user feedback and insights. You can start off by running a moderated test to find the nitty-gritty issues of your prototype, then run an unmoderated test when checking to see how the post-prototype site is going. The methods are meant to complement, not compete.

What are the key benefits of moderated testing? 

• Moderators can help participants easily navigate tasks: Moderated testing works best when you need a high level of interaction between you and your participant, like guiding a participant through a complex process or concept—such as a prototype with limited functionality. By answering any questions participants have in the moment, you can guide the tester right past any missing or non-functioning features and refocus the session on the features of interest—while minimizing any confusion and ensuring more successful task completion. 
• Moderators can lead participants back in the right direction: If a participant has a difficult task, moderators can direct them back to the right path. Moderators can provide a deeper explanation of the task to alleviate confusion and to clarify any misunderstandings. While the moderator shouldn’t interfere with or influence a participant's instincts or feedback, having a moderator available can salvage a test that would otherwise have flopped.  
• Teams can observe body language, facial expressions, and pick up on subtle behaviors or responses: With your participant in front of you, you’ll have more opportunity for probing and can minimize the risk of a participant speeding through the tasks or questions. Some tasks may evoke joy, confusion, or some other emotion worth noting. Observing and developing a rapport with your test participant can help establish trust—and leads to candid feedback that might not be possible with other qualitative research methods.
• You’ll have control over privacy and security: Even if you're testing a product with strict security concerns, don’t let this stop you from turning it into a test subject. Moderators can maintain complete control over how the site or app is accessed. Let's say you're testing one element of your product, but the other features are highly confidential, or you simply don’t need participants’ feedback about that particular element. Moderators can allow participants access to specific parts of your software to conduct usability testing while ensuring private data isn't leaked or seen.

The payoff can be significant if you have the time and resources available for a moderated study. While an unmoderated study has a set test plan, a moderated study is flexible and highly interactive. Moderating allows you to probe as you see fit, delving as deep into the user experience as you desire. The human judgment involved in moderated testing creates an unparalleled depth of feedback.

When should you conduct moderated testing?

Moderated tests are best used early and often in the development process. In the prototyping phase, you can use moderated testing for exploratory research. You may only have sketches and wireframes but are interested in gathering user feedback on prospective design ideas. Ask participants for their thoughts on your prototype, so you can gain valuable information that can save you from spending a lot of design and development time on a product that users don't understand. 

How to know when moderated testing isn’t the best fit 

• Moderated testing may be more costly: Typically, participants are paid a higher rate for moderated testing versus unmoderated, which needs to be accounted for in your budget. Contributor payment isn’t the only charge, there’s also the cost of time. Both in-person and remote moderated testing requires more planning and scheduling, and potential cancellations on both sides can lead to project delays. 
• Moderated testing may influence results: Participants may behave differently if a moderator is present because they’re aware of being observed and may not wish to reveal their genuine opinions. Similarly, focus groups are also subject to groupthink and sampling bias, yielding feedback that may lead you down the wrong path. To combat this, employ active listening techniques to make participants feel heard and comfortable. By actively listening and empathizing with your participants, they're more likely to give you their open and honest detailed feedback. 
• Moderated testing takes longer: It requires thorough planning and searching to find target participants and a qualified moderator, and individuals need room in their schedules to conduct the study. Due to the extra time moderated testing requires, researchers may receive results back slower.

How UserTesting can help you with moderated testing

With UserTesting, you get access to a team of research experts who will help moderate sessions—either for you or with you! Global travel technology organization Expedia Group came to UserTesting to improve efficiency and scale throughout their teams. They found in-lab testing labor-intensive and slow, so they looked to develop a faster, more agile approach to augment the small but expanding team. 

After conducting Live Conversations, with the help of the Invite Network, Expedia Group reduced the time and costs for customer interviews and expanded their audience to other markets like Japan. UserTesting’s Live Conversations enabled Expedia Group to create and deploy tests in minutes and receive customer feedback the same day, often within two hours of launching a test. The organization increased its research team from six to 28 researchers, boosting the customer insights they learned. This customer success story proved just how beneficial Live Conversations are—and the importance of a customer-centric mindset.

A final word on moderated testing

A moderated study, regardless of whether it’s remote or in-person, allows you to gain a better understanding of your users and their needs. When you employ user feedback in your design decisions and beyond, you create meaningful products that your users will have a higher chance of enjoying.