How to prepare a think-out-loud study

So you’ve decided that you’re going to collect think-out-loud feedback from your participants on your next UserZoom study. Exciting! There are few things as powerful as verbatim feedback from a participant as they experience your site or product.

If you haven’t created a think out loud study yet with Userzoom, or you just need a quick refresher, we’re here to help! Follow this article as you create your study and you’ll be listening to your results in no time.

And while we’ll be focusing on how to set up your study in UserZoom in this article, you can also see my previous article on how you can get better think out loud feedback from participants for tips and tricks that will help you prepare outside of the platform.

Let’s dive right in.

Creating your study

First things first, you’re going to need to create a study. It’s important to point out that you won’t be able to collect think out loud feedback from every project  type, however. Depending on whether you’re planning on getting feedback on a live mobile app, or on a website/prototype, you have the following options:

If you want to quickly measure what your users think about a concept or live user experience, for example about a new prototype you mocked up in InVision, Agile Usability Test (Think-Out-Loud) is the best project type.

The Agile Usability Test Think-Out-Loud, or TOL for short, uses a task based structure to record a user’s screen and their audio. You can quickly build your project using a simplified task builder, and launching your Agile TOL study sends automatic invites to UserZoom’s panel. This allows you to begin collecting feedback almost instantly.

Keep in mind that this method is designed for a very fast and easy launch, and while this can be created in minutes and launched to our panel with a click of a button, this study type doesn’t offer all of UserZoom’s advanced capabilities.  

If you want to use mixed methods to uncover all of the usability issues and struggles on your prototype or live websites, the Advanced UX Testing project type will serve you well. With Advanced UX Measurement you can test a variety of methods all-in-one!

For example, you can start the study with a handful of survey questions to better understand your users, then ask your users to perform a series of tasks and track things like time to complete and ease of completion. You can turn on screen recording so later you can observe where users went during each task to see if their natural journeys match your design’s intended flow.

To do so you’ll want to go into the Task Settings & Validations section (for each task if you’re not copying the task), click on the Video Recording tab, and click the Record Task Session box. This will turn on screen capture for participants. Clicking Record Task Sessions allows you to click Record Audio (for think out loud) as well as Record Camera (to capture users’ faces via their camera) if you’d like to capture that feedback as well.

After you’ve clicked the Record Audio box your project is setup to record participant’s audio. From here on out, it’s business as usual as you complete your questions, choose a validation method, and so on.

Best practices to enable your participants

While that’s all you need to do from a technical standpoint to create a think out loud study in UserZoom, there are a few more things you can do to elicit better feedback from your participants within the UserZoom platform.

Depending on where you’re sourcing your participants they will likely have a different comfort level with giving a stream of constant verbal feedback. UserZoom’s panel members are trained to give this type of feedback and are used to it, but if you’re sending out an invite to your customer base, for example, they will probably be far less comfortable or even aware of what it is you’re after.

After all – the only time people usually talk to their screens is during sports or trivia shows.

That’s why we offer these tips on educating and enabling your participants to know what you’re after, and how they can give it.

Templates

UserZoom has study templates that our expert research team has created. You can also copy questions, tasks, and welcome pages from other studies into your own. That’s why we suggest copying over the welcome page and initial questionnaire survey questions from the Think-Out-Loud Desktop template and editing it for that studies particular needs.

Simply click on Copy a Welcome page from another project and then select [Template] Think-Out-Loud Desktop from the list.

In the initial questionnaire, simply select Add Copied Question, and again, select [Template] Think-Out-Loud Desktop from the list.

Whether you decide to copy this content over or not, the most important thing is that you want to inform participants what it is you’re expecting of them from the beginning.

Pre-Study Checklist

At UserZoom we are also taking additional steps to improve video quality by ensuring a participant’s device is properly configured prior to entering a study.

To start, we’ve created a pre-study step by step guided checklist for mobile web studies (Agile Usability and Advanced Testing project types) that allows participants to confirm audio and/or camera permissions are enabled depending upon how you’ve configured your study (e.g. you only wish to capture audio, therefore only the audio permissions need to be enabled).

Only participants who provide audio and camera permissions will be able to continue forward to the Welcome Page. We will continue to improve this checklist to include other checks, as well as expand it to desktop devices.