Walmart + UserTesting

Since its beginnings, Walmart has always been focused on the needs of its customers. Founder Sam Walton built and ran his business on this philosophy, insisting, “Customers are not cold statistics. They are flesh and blood human beings with emotions and feelings just like our own.” Yet, in order to improve the customer experience for the more than 600,000 daily visits to Walmart.ca—to increase traffic to the website as well as drive on-site conversions— the Walmart Canada Site Optimization team was largely relying on quantitative data, such as transactional data from analytics and A/B testing.

“For some departments, user research was seen as inferior to ‘real data,’” noted David Raine, Manager, Site Optimization, Walmart.ca. “It was tough to paint a story that, even without numbers behind your insights, user research is still data and valuable towards driving a business decision.” By focusing on quantitative data, the team found that it was running lots of tests, but in a scattershot manner. Lacking an overlying framework or strategy, they just kept moving from test to test according to perceived areas of opportunity and pushing out incremental site enhancements without a sense of where to go to next.

As a result, the team was burning out, perplexed as to why all of their efforts were failing to yield longer term outcomes. And with more tests being run and more results to sift through, there were more opinions and conflicting expectations. There wasn’t a clear answer to the question, “What should we do next?” and—more importantly—no context as to why

The Site Optimization team turned to UserTesting, using the platform to more frequently gather qualitative customer insights. In addition to the transactional data and analytics they had previously focused on, they relied on this human data to better understand and secure more meaningful information about their customers.

By pairing the quantitative data (what was happening) with qualitative data (why, as spoken from the mouths of their customers), the team was able to achieve a more streamlined, data-driven decision-making process to focus on making the website updates with the greatest impact. This included improved resource allocation, internal alignment towards customer-focused decisions, and results-driven activities.

By understanding the reasons customers were engaging with their digital experiences in a certain way, the team was able to make the right choices to drive the business forward.

The shift has not only resulted in better workflows and outcomes, it’s also helped the entire Walmart.ca team get back to its roots and adhere to founder Sam Walton’s expectations of being truly customer focused. Raine summarized this renewed motivation, saying, “Numbers are insightful, numbers are trackable, and they help you trend over time. And there’s stories you can tell from numbers. But that isn’t the end of the story. You need to be able to blend those numbers with customer data…. If we know what our customers purchase, but not the why, how are we going to make them more loyal?”