When to Test: Incorporating User Testing into Product Design

In this three-part series Jessica DuVerneay, Los Angeles based information architect and user researcher at The Understanding Group, will share usability testing tips on When to Test, What to Test, and How to Test. 

One of the first questions clients ask when starting to incorporate user research into their product design cycles is “When should we test?” The simple answer is this – early and often.

The more elaborate answer would need to consider budget for time and money, staffing availability, internal culture, and goals for each stage of a project to ensure realistic and optimal results. While creating a company or product specific testing plan can sometimes be complex, what isn’t complex is understanding that unmoderated user testing can bring value at any stage of design, even if you aren’t starting early, or testing as often as you might like.

To understand how unmoderated user testing can be leveraged at any stage of your project lifecycle, let’s start by looking at a typical project lifecycle.

Life Cycle of a Project

In any well-executed digital product launch, variations on the following ideal process are common:

• • Discovery: The “What & Why.” Whether a client project or internal product launch, teams should agree on organizational and business goals and establish success metrics. In the discovery phase, experts analyze existing product strengths and weaknesses. We also recommend exploring the competition.
  • User Research: The “Who & Why.” Understanding end user needs and expectations for the same product or service.
  • Strategy & Structural Design: The “How.” Creating a solid strategy & structure for the product or product improvements.
  • Launch & Assessment: Release and Optimize. This is the continuous process when the efficacy of product changes can be measured, and organizations can begin gathering feedback and creating roadmaps for further releases. Ideally this phase feeds into another turn of the entire cycle, beginning once again with the discovery phase.

The good news is given that one phase will ultimately run into the next phase, and no matter where you are in your product cycle, it is advantageous to consider user testing. User testing can add value by providing perspective and unique data sets at each stage of the project lifecycle.

Testing During Launch & Assessment

If there is one place to start incorporating user testing, this may be the most popular. Getting positive metrics and user feedback is a great way for teams to congratulate themselves and prove the value of their most recent efforts. Smart teams will begin testing immediately to collect data that can inform road mapping for the next product improvements. User testing at this phase can:

• • Validate improvement over previous product iterations
  • Validate visual fit as a lightweight QA process
  • Supplement quantitative A/B testing to investigate user preference from a qualitative perspective
  • Identify missed usability issues or awkward product flows
  • Show use cases of a product in multiple environments (Desktop vs. Mobile)
  • Collect user feedback and hear their opinions, suggestions, and desired features lists (especially valuable in the startup space)

Testing During Discovery

Many people erroneously think of user testing as something to do with users only after a product (or, even an MVP) is launched. User testing up front in the discovery phase can help lean UX teams determine where effort should be focused. For startups, user testing can help a company understand how their product might fill existing gaps in the competitive space and start off with a competitive advantage by avoiding or improving on issues found in existing competitive products.

For teams with an existing product, running some lightweight tests with a service like UserTesting during discovery can help:

• • Benchmark your existing product so that at the end of your project lifecycle, you have data against which to compare your improved product.
  • Establish successful existing user flows that should be protected during upcoming versions.
  • Identify show stopping usability issues that may have been missed by a heuristic review.

Testing During User Research

User testing can also add tangible value to user research activities. Though I would not champion user testing as a comprehensive approach to user research, if research budget for a project is light, usability testing can provide a quick and easy way to place an existing product in front of a wide range of possible user types. Furthermore, it’s a form of user research that doesn’t require allocating exorbitant time and money on the most laborious of user research tasks – recruiting.

UserTesting in particular collects useful browser, OS, monitor, location, age and computer proficiency information as well as providing an area where qualitative questions can be asked. I’ve often used user testing data at this phase in conjunction with analytics and survey data to flesh out personas and scenarios that guide strategic and structural decisions.

Testing During Strategy & Structural Design

If you’ve gotten this close to launch and have not yet been able to incorporate user testing into the project lifecycle, don’t worry. This phase is not too late, and in fact can be an optimal place to start. When first impressions or launches are important, this “measure twice, cut once” approach may save an embarrassing first launch that could prove unrecoverable for a new product.

Allowing users who participate in testing at this phase access to a password protected test environment or a lightweight prototype can validate or point out key issues with a strategic or structural direction. The Strategy & Structural Design phase is a good time to test out complex flows and content models. Identifying any issues prior to investing time and money into design and development of a structurally compromised approach can be incredibly advantageous.

It’s Never the Wrong Time to Test

If you’re not testing now, start planning to do so during your next project phase, no matter where you are in the project lifecycle. Each phase of a successful project lifecycle can be positively informed by user testing. If immediate implementation of user testing seems insurmountable in your current project, try to build user testing in to inform your next product iteration to get into the habit of user testing early and often.

Up Next – What to Test

In the next installment of this series, I will explain What to Test. I’ll cover a few primary types of tests that focus on IA and UX issues, what project lifecycle phase they align best with, and what to look for when testing.