What can I do to persuade more people to buy your product online? I tackled this question for 5 years as I ran A/B tests for diverse clients.
I remember one test idea that everyone on the team loved. The client said “That’s the one. That one’s totally going to win.” Well, it didn’t.
The fact is, most A/B test ideas don’t win.
In fact, interpretation is tough, because there are so many sources of uncertainty: What do we want to improve first? Which of a hundred implementations is a valid test of our hypothesis about the problem? If our implementation does better, how statistically reliable is the result?
Is our hypothesis about the users actually true? Did our idea lose, because our hypothesis is false or because of our implementation? If the idea wins, does that support our hypothesis, or did it win for some completely unrelated reason?
Even if we accept everything about the result in the most optimistic way, is there a bigger problem we don’t even know about? Are we inflating the tires while the car is on fire?
If you take anything away from this, take this analogy: inflating your car tires while the car is on fire will not solve your real problem.
I believe the most effective means of selling a product and building a reputable brand is to show how the product meets the customer’s needs. This means we have to know what the customer’s problem is. We have to talk to them.
Then if we run an A/B test and lose, we won’t be back to square one. We’ll know our hypothesis is based in reality and keep trying to solve the problem.
Emulating Competitors
“I heard lots of people found gold in this area. I say we start digging there!”
That actually is a smart strategy: knowing about others’ successes helps define the opportunity. That’s how a gold rush happens.
This is why A/B testing blogs are dominated by patterns and best practices. So-and-so gained 50% in sales by removing a form field… that sort of thing. Now don’t get me wrong: you should be doing a lot of those things. Improve your value proposition. Ensure your buttons are noticed. Don’t use tiny fonts that are hard to read. You don’t need to test anything to improve, especially if you focus on obvious usability issues.
So what’s the problem? Well, let’s go back to the gold analogy. Lots of people went broke. They didn’t find any gold where others had or they didn’t find enough:
“The actual reason that so many people walked away from the rush penniless is that they couldn’t find enough gold to stay ahead of their costs.” ~ Tyler Crowe Sept. 27, 2014 in USAToday
You could be doing a lot of great things, just not doing the RIGHT things.
The good thing is many people do some research. The problem is not enough of it or directly enough. They are still digging in the wrong place.
“If I had only one hour to solve a problem, I would spend up to two-thirds of that hour in attempting to define what the problem is.” ~ An unknown Yale professor, wrongly attributed to Einstein.
Think about this for a moment: How can you sell something to anyone when you’ve never talked to them or listened to what they have to say?
Product owners often believe they know their customers, but assumptions usually outnumber verifiable facts. Watching session playback can hint at problems. Google Analytics gives a funnel breakdown, but it doesn’t give much insight into a customer’s mind. It’s like trying to diagnose the cause of indigestion without being able to ask the patient what they had for dinner or if they have other more serious health complaints.
The problem is it’s all impersonal, there’s no empathy. There’s no “Oh man, that sucks, I see how that is a problem for you”. It’s more like “Maybe people would like a screenshot there. I guess that might be helpful to somebody”.
Real empathy spurs action. When you can place yourself in your customer’s situation, you know how to go about helping them. If your solution doesn’t work, you can try again, because you know the problem is real rather than a figment of your imagination.
A Pattern Is A Solution To A Problem
Therapist: “Wait, don’t tell me your problem. Let me just list all the advice that has helped my other patients.”
Let’s say some type of visual change has worked on 10 different sites. Let’s call it a pattern.
A pattern works, because it solves some problem. So choosing from a library of patterns is choosing the problem you have. You don’t chose Tylenol unless you have a headache or fever. You don’t chose Maalox unless you have indigestion.
If you know what YOUR problem is, you can choose the right patterns to solve it.
If you don’t know the problem, you won’t get far choosing a pattern because it’s popular, because of how strongly it worked or how many people it has worked for. That’s like taking a medication you’ve never heard of and seeing what it does for you.
Pattern libraries are great for when you have a problem and want a quick, time-tested way to solve it:
Research Uncovers The Problem: A Short Story
Say you’re a shoe brand. You decide to reach out to people who are on your mailing list but haven’t purchased yet.
So you send out a survey. Within the first day, it becomes clear that many people are avoiding buying your shoes, because they’re not sure about sizing.
You’re shocked, but you shouldn’t be. User research insights are often surprising.
It’s just that you thought you anticipated this by posting precise measurements, a great return policy, and glowing testimonials. If anything, you thought people would mention the price, but no one so far mentioned price.
That’s a big deal for your product strategy. You need to build trust. So you set aside your plans for a full redesign (those fancy carousels on your competitor’s site sure are tempting). You set aside A/B test ideas about the font size of prices, removing fields, and so on.
You tackle the big problem. You do some research and come up with solutions:
- match sizing to a set of well known brands
- provide a printable foot template
- allow people to order two sizes and return one
- mail out a mock plastic “shoe” free of charge, and so on…
You ask a couple of people to come to the office and try some of your solutions.
Your user testing methodology is simple: First people pick their size based on either the sizing chart or template. Then they see if the real shoe fits.
Result? The matched sizing and the foot template were very effective in predicting fit. In user testing, the initial template didn’t work so well, because it’s hard to place a 3D foot in perfect position on a 2D printout. So, you come up with a template that folds up at the back and front, simulating a shoe. The users liked that much better. In fact, you start working on a cardboard model you can mail cheaply to anyone who requests it.
Now you’re off to testing it in the real world!
You design 2 different foot sizing comparisons, one pretty one with photos of top 3 brands and one long, plain table with 20 different brands. You also create an alternative page that links to the downloadable foot template.
You A/B test these variants over 2 weeks and pick the one that works.
(Then you go back to your research and find the next problem.)
You may also like this post about patterns: Compact Navigation Patterns .
If you want to uncover the biggest problems for your customers, I’m happy to help.