UX testing: how to find the right target audience with screener questions

Users are at the heart of what we do here at UserTesting and the UX industry. They tell us the needs that require filling, inform our designs, and identify strengths and weaknesses within our products.

In a perfect world, usability testing involves users who wholeheartedly represent your product’s target market. And ideally, they’ve adopted the right mindset, speak the right lingo, and have the best insight into an ideal experience.

Though this may not exist, the good news is that whether or not you need to recruit exactly the right target user depends on the nature of your test. Let’s dive in.

Table of contents 

1. When should you test with your exact target market?
2. What are screener questions and why are they important? 
3. How many questions should be in a screener?
4. How many contributors are needed for a usability test?
5. How to write a screener question: best practices for recruiting users for UX testing
6. 4 examples of audience recruitment for UX testing

When should you test with your exact target market?

Many UX thought leaders encourage researchers not to be too specific when recruiting user testing participants. After all, a vast majority of products should be clear and intuitive enough that anyone can figure them out. For example, someone who’s less technologically inclined should be able to make a purchase online without getting confused, and a young child should be able to easily check a weather app to make sure their soccer game won’t get rained out.

Of course, there are circumstances in which UX researchers need to capture insights and collect data from a target audience, whether it’s a specific age group, occupation, or customer. If you’re working on a Human Resources tool, you need to conduct usability testing with HR professionals because they're the only ones who would know whether the tool could be helpful to them as they work. If you’re collecting customer feedback on the usability of a subscription app, you’ll need to target current subscribers and not just any smartphone user. 

If you’re in one of those circumstances, and you’re testing remotely, then you can leverage screener questions.

What are screener questions and why are they important? 

Screener questions, or screeners, are multiple-choice questions that either eliminate the wrong participants from taking part in your study or give access to the right ones. 

Although it would be great to test all participants, it’s not always ideal, especially if your study has specific needs. It’s best to recruit participants who have attributes similar to your existing, or potential users of your product. Asking the right screener questions ensures you’re enlisting the intended participants for your test. 

Here’s an example of a screener question designed to target plus-sized users:

In the above example, only participants who select the third or fourth option will be allowed to proceed to the test, and the rest will be rejected. 

Screeners may seem easy, but it’s often quite challenging to craft screeners properly. If a user misunderstands the questions, you could end up with test participants who don’t fit the needs of your study, negatively impacting your results.

How many questions should be in a screener? 

It’s important to remember that there’s no one-size-fits-all approach. Some teams may only use one screener question, as they could be doing an initial test for preliminary research that won’t count towards their final results. Meanwhile, other teams may need multiple rounds of screener questions to properly recruit for an ethnography study. This entails finding customers of a tangible product—who also have it nearby and are willing to show how they use it for the study. 

Here at UserTesting, we recommend setting up between three and five screener questions. If there are any more, contributors can get discouraged or even exit before completing all of the screeners because they assume they won’t qualify or find it too time-consuming. 

UserTesting’s Research team knows firsthand how important (and challenging) it is to write solid screener questions, and we have a wealth of experience in capturing the correct user group for your needs. Below are some guidelines and examples that can help you get exactly the right participant for your next remote usability test.

How many contributors are needed for a usability test?

Recruiting the ideal number of participants for your usability testing contributes to how efficient the data collected from your study will be.

The number of users you need depends on the usability testing you conduct. Here at UserTesting, five to eight participants are recommended for qualitative research while over 30 contributors can be enlisted for quantitative research (think card sorting or tree testing). 

If you’re testing a design where various users can use the platform, it may make more sense to test a larger sample of users. Consider starting off a study with five or fewer participants, and based on the results, you can improve your design and test further until your design is as effective as it can be. For instance, after reviewing the first few test results, you may have realized that contributors weren’t spending as much time as you expected discussing their feedback—leaving some insight on the table. This requires an edit of the task instructions to be more detailed. 

Keep in mind that there is no universally right number and that it’ll be influenced by the margin of error you find acceptable. However, by starting small, you’re giving yourself leeway to make adjustments and address mistakes, saving your time and resources in the long run. 

How to write a screener question: best practices for recruiting users for UX testing

Many of the guidelines for writing good screener questions are the same as the recommendations for writing great multiple-choice questions:

• Always provide a “None of the above,” “I don’t know,” or “Other” option just in case you’ve forgotten to include an answer that applies to the user or the user is confused by the question. This is especially important to include in screeners because if users don’t have this option and pick an answer at random, they might end up in your study accidentally, even if they don’t qualify. 
• Provide clear and distinct answers that don’t overlap each other so that (for example) a size 6 doesn’t have to decide if they want to be grouped with the 0-6 sizes, or the 6-12 sizes.
• Avoid asking leading questions or yes/no questions because users will be inclined to give you the answer they think you want instead of the one that really applies to them. We find that instead of asking direct questions, instructing users to select the option that most closely applies to them, followed by a list of statements, is the most neutral way to phrase most screeners. This method ensures that users will answer honestly because the desired answer is less obvious, ensuring you get accurate results. 

How to check that your screener is capturing the right users:

Depending on your needs and what you’re studying, you may need someone with a particular background (like a medical degree) or someone who is going through a particular experience (like shopping for a new car). If that’s the case, we recommend that in addition to screeners, you use the first task of your test to verify this:

“You indicated in the screener questions that you are currently shopping for a new car. Please describe what kind of car you are looking for, where you have looked so far, etc.”

Sometimes, just listening to a user describe their experience can let you know if they’re really the right fit—and can give them the opportunity to end the test without submitting.

4 examples of audience recruitment for UX testing 

While your recruitment requirements can be customized, the following examples are four common types of screeners. 

1. Screeners based on familiarity with a product

One of the most common kinds of screener questions that researchers use is capturing users’ level of familiarity with a product or a brand. Sometimes they need fresh users to test out a new tutorial for their app. Other times they are looking for insight from their most frequent users.

Whatever the case, you don’t want to ask point-blank if users fit the mold; people are naturally inclined to say yes just to please you! Instead, ask users to indicate their familiarity, and then define the different levels of familiarity, like in this example:

This screener makes it easy for users to tell which category they belong in, and it isn’t obvious which answer the researcher is looking for. 

As you can see in the above example, it’s helpful to define the Novice, Intermediate, and Expert levels by a few elements: overall content levels, usage levels, and consideration levels. This keeps a user from overestimating their experience level with your product and accidentally skewing your results.

2. Screeners based on a product's frequency of use

Similar rules apply to the related—and equally popular—frequency-of-use screener. A researcher may be interested in finding users who regularly check the weather on their phones, or users who used to play a game but have given up on it over time.

As with experience levels, it’s important to define frequency in solid terms, not just “rarely,” “sometimes,” or “often.” 

Another common screener related to the frequency of use might have to do with how recently a user has participated in a certain activity. For example, many ecommerce product researchers prefer to hear from users who purchase items online fairly often. Meanwhile, many travel product researchers want to hear from those who are planning a trip within the next year.

In these cases, it may be wise to create two screeners: one to confirm that they purchase items online or have an upcoming trip, and then a follow-up screener to determine timeframes.

The following question confirms that the user is planning on going on a vacation soon.

This question filters for users who are planning their trip within four months to a year, and ensures that users’ timelines align with what you’re looking for. 

3. Screeners based on industry or occupation

When you need users within a particular occupation, multiple screeners are a helpful way to reveal a single characteristic. 

For example, a massage therapy retailer might want to hear from people in the massage therapy industry.

Massage therapy is a very specific profession, and it would be hard to come up with an exhaustive list of options inside of one screener question. But you also want to avoid asking a yes or no question. 

Therefore, you might start by listing broader professional categories, including health (which would encompass massage therapy). Then in a follow-up screener, you may have users indicate the role that they occupy within the Health industry. Consider the below example: 

The first screener question gives broad categories to keep from overwhelming the user.

The second question narrows it down to specific industries. As you can see, with each screener question, you’re able to weed out more and more of the contributors that don’t qualify to narrow it down to the ones who do. 

4. Screeners that deal with personal information

The last type of screener that the UserTesting Research Team relies on frequently involves users providing sensitive information, including their income, race, Facebook profile, or body type.

For example, there are apps that help couples who’re attempting to conceive a baby determine peaks in fertility. Obviously, reproductive health isn’t a topic that everyone is comfortable discussing in a user test. Therefore, it’s important to give users a disclaimer, and only accept contributors who’re willing to be open about this personal information. 

Here’s an example of how this screener can look:

By being transparent with users, you’re giving them an opportunity to opt out of the study if they’re not comfortable sharing personal information—and now you’re that much closer to finding those who are. 

For further information on UserTesting’s data protection guidelines and HIPAA, read our help article here. 

Get started and recruit your product’s right target audience

The above examples only scratch the surface of ways that researchers capture their ideal users! Often these are used in some kind of combination, like screening for males who frequently track their fantasy football data, but have never heard of Rotowire, for example. 

Shaping screeners can be challenging, so we encourage you to always perform a dry run to ensure that your screeners are effective—and give yourself room for adjustments. When you use your imagination (or enlist the help of the UserTesting team) and leverage the best practices, the recruiting possibilities are endless.