11 ways to generate evidence-based growth experiment ideas

Today’s post is an excerpt from our eBook, the product manager’s guide to customer-centered growth. Enjoy!

There are two approaches to growing a product.

1. You A/B test a handful of tactics you think will work based on your opinion, and hope they improve your growth curve.
2. You start by looking at your analytics to identify the steps of your product flow where you’re seeing the biggest dropoffs. Once you know where the problems are, you gather qualitative data to get insight into what’s causing them. Then, you turn those insights into hypotheses and design experiments to test them empirically. And finally, you feed the lessons you learn about your product, customers, and channels back into the process to guide which tests you run next.

It’s a no-brainer. Your growth efforts will be much more effective if you establish a continuous, consistent cadence of experimentation to identify the leaks in your funnel and come up with test ideas based on multiple sources of data.

Peep Laja uses a great metaphor: “Would you rather have a doctor operate on you based on an opinion, or careful examination and tests?” You’re like a growth doctor, and your product is your patient.

Why you need an experimentation process

When you focus on tactics first, you run into three core problems:

• What works for other products won’t necessarily work for yours. Your business model, product, and audience are fundamentally different from anyone else’s.

• Distribution channels are changing faster than ever. Channels that work for you today will likely be saturated and far less effective 12 months from now. To stay ahead of the curve, you have to always be experimenting to discover emerging channels.

• There are no silver bullets. No one-off tactic is going to solve all your growth problems. The tactics that do produce explosive growth are usually the result of a combination of lessons learned from previous experiments.

Tactics are important. But instead of focusing on them first, focus on establishing a systematic and repeatable experimentation process. This will lead you to discover the combination of tactics that result in sustainable growth for your product.

Your growth efforts will be more productive and successful once you move away from testing random tactics and start approaching it as the scientific process it is:

Step 1: generate data-driven experiment ideas

The first step in the growth process is about collecting quantitative behavioral data and qualitative user insights to identify the biggest issues your customers and potential customers are experiencing—and then using those findings to guide your experiments.

Fixing these problems will help you meet your customer’s needs and expectations, craft an experience they’ll love, and improve your growth curve along the way.

Start by gathering quantitative data that shows you where the problem is—where the biggest dropoffs are and where your funnel is leaking. Once you know where the issues are happening, you can gather qualitative insight to understand why. What’s causing people to have doubts, uncertainties, and hesitations?

And once you know the source of the problem, you can come up with hypotheses to test about what you think would remove the friction.

11 data sources to help you identify growth opportunities

Here are 11 data sources you can use to generate data-driven growth experiments. You don’t have to use every single one—choose a combination of three to five quantitative and qualitative sources that make the most sense for what you’re trying to learn. As you do your research, document all the issues you identify and test ideas you come up with in your backlog.

1. Web analytics

Identify specific pages and steps in your product flow where you’re losing customers and potential customers.

2. Technical analysis

Identify annoying cross-device and cross-browser issues that are hurting your conversion rates. Fix issues if the added value is greater than the value of the time it will take your developers to identify and fix them.

3. Form analytics

Know exactly which form fields people hesitate the most to fill out, which fields they leave blank even though they are required, and which ones cause the most error messages.

4. User session replays

Anonymously record video of everything people do as they interact with your product. Watch how people respond to your product while risking their real money or personal information.

5. On-page surveys & live chat

Place surveys on low-performing pages and ask people why they aren’t converting. Chat transcripts give you insight into common frustrations and where people get confused.

6. User testing

Observe people while they use your product and speak their thoughts out loud. Hear and see where people get stuck, confused, and frustrated.

7. Mouse tracking

Use click maps, attention maps, and scroll maps to record what people do on your product pages with their mouse or fingers.

8. Product walkthrough

Hour-long team exercise. Review your product page by page with your team and identify where you can make the experience more relevant, clear, and value-adding—and where you can reduce friction and distraction.

9. Product comparison

Twenty-minute team exercise. Everyone picks one or two products from your competitive and noncompetitive space that they use actively. Walk through their product flow and discuss what they do well, what they could do better, and how you can adapt it to your own product and audience.

10. Growth network

Talk to other growth professionals on the phone on a regular basis. Exchange ideas, share experiments, and discuss what worked, what didn’t, and why you think you got the results you did. Adapt ideas to your own product.

11. Question brainstorm

Hour-long team exercise. Spend the first 20 minutes asking nothing but “why,” “what if,” and “what about,” questions. No discussion or answers. Write your question on a sticky note, announce it to the team, and put it on the whiteboard. This reveals what you don’t know and generates discussion.

What do you do once you’ve collected all this data?

As you gather and analyze the data you collect, you should be dumping all the issues you identify and experiment ideas you come up with into your backlog. The next step in the process is to translate those findings into a prioritized pipeline of hypotheses to test.

If you want to learn how to:

• Implement a proven five-step growth experimentation process
• Use customer insights to discover growth opportunities
• Increase the accuracy of your tests over time

Then click the button below to download the product manager’s guide to customer-centered growth for free.