Five ways to make your research data watertight

This is the final article in our series on UX research data quality. Part one is here: How do you ensure the best-quality participants for user research? and part two is here: Five best practices for ensuring quality user research data.

Here’s the scenario: you’re about to head into the boardroom to present your UX findings to your stakeholders. You’ve got a lot riding on this project. You’re excited about the results. More importantly, you’re keen to convince your stakeholders about the value of UX so you can get an additional budget and develop more projects.

There’s just one problem. 

Your stakeholders are, to put it mildly, ambivalent. They’ve heard about industry issues like fraudulent participants, duplicate results, and professional survey takers. They’re worried that your results are inaccurate.  If your stakeholders are going to move the needle based on your insights, they need to know that your findings are watertight.

Here are five ways to give stakeholders trust

1. Work with a trusted partner

There’s no one way to conduct a research project. You could gather feedback via guerrilla testing, emailing your customer database, or even recruitment through social media. However, it’s hard to control data quality when using these methods. For example, it isn’t easy to vet what people are who they say they are on social media. 

This, in turn, can undermine your efforts when you head into the boardroom, which is why it can be beneficial to work with a trusted partner. Whether you choose a panel vendor or a UX platform that gives you access to your target audience, working with a specialist can boost stakeholder confidence in your project. 

2. Verify your participants

For executives to trust your results, you need both high quality and a high quantity of participants – but fraudulent/invalid participants can undermine your research findings. The good news is that a trusted partner will manage the process of finding participants for you. 

They’ll be able to take out bots, screen out professional testers and review potential participants based on historical data to ensure your sample is up to scratch.

3. Take out the people who are only in it for the pot of gold at the end

While a UX partner can go a long way in ensuring the validity of your results, you still need to keep an eye out for participants that became dis-engaged while taking your survey. By doing your due diligence, you can instill confidence in your stakeholders that your results are entirely accurate. 

So, here’s how to spot inattentive research participants: 

• Use a red herring: A red herring is a way of catching out inattentive participants by placing bizarre answers within your survey. For example, say you’re a car manufacturer, you could ask participants what car they are most likely to buy. One of the answers you include could be 'Duck.' If someone selects this answer, chances are they are just speeding through the survey to get to the reward at the end.
• Check the answers to any open-ended questions: If someone puts gibberish or a single-word answer like ‘no’ to your open-ended question(s), then it’s likely they are not paying attention to your survey and can be removed from the sample. 
• Watch out for consistent errors and nonsense selections: Many mistakes indicate that a participant is rushing and disengaged. We recommend setting a limit on the number of errors you’ll allow a participant to make before discounting their results. You should also ignore unrealistic results, such as someone straight-lining throughout the whole survey.
• Monitor completion time: You should have an idea in your head of the average time it takes to complete your survey. If a participant’s time is dramatically below this, then they will need further investigation. While some people are quicker than others, a speedy completion time indicates that the participant hasn’t put much thought into their answers, which could impact your results. 

4. Design your survey to be as engaging and accessible as possible

If your research questions are long-winded, confusing, or complicated, then your participants are more likely to disengage. 

You also need to make sure your questions aren’t leading so that you get truly accurate results. Design is imperative here. Make sure that your questions are impartial and easy to understand. Plus, keep your survey short and limit the number of answer options. 

5. Ensure your sample is representative of the target population 

Ideally, all of your target respondents would respond to your online survey. However, the online world holds its own biases. For example, you may often see younger individuals within specific income brackets reply to mobile-first surveys. This can lead to sampling bias. 

To prevent this, ensure that you distribute your survey across multiple channels so it’s visible to different demographics. Tactics to consider include survey websites, email blasts, QR codes, and text messages. 

Communication is critical

With these five measures in place, you’ll be able to walk into your stakeholder meeting brimming with confidence that your data quality is watertight. But, remember, your stakeholders aren’t UX researchers. They have other demands on their time, and they’re business-impact focused. 

So before we sign off, our last tip is this: condense your findings and keep them relevant. Carefully choose and focus on the critical results, then show your stakeholders why they should care and what needs to be done next. 

This will make you come across not only as a great UX researcher but a great leader – someone who is well-placed to spearhead future UX projects. 

How UserZoom can boost data quality 

Find out how UserZoom can help enhance data quality and sell your findings to stakeholders. Get in touch today.