July 10th, 2020
March 1, 2017
This originally appeared as the very first post on Afterpay's Tech Blog.
“Do you like this?”
“Would you use this?”
If you’re a product manager and you’ve been asking customers these questions, keep reading. We all know how important it is to talk to your customers, but it's only half the story. Asking them the right questions could mean the difference between a successful product and a tough lesson best told through a blog post.
Let’s imagine you’ve just opened up a new restaurant, and you invite some friends to come in on opening night. You’ve spent months planning the menu, layout, and spent a ton of money. They come in and you ask them if they liked it when they’re done.
You could get one of these three responses:
See the problem? In response 1 and 2, you’re out of business by the end of your first year.
In response 3, you’re franchising and expanding across the country. I had a similar experience early on in my career. I’d just joined a new company and they were finishing up a 12 month journey to build a first-to-market product. The team had spent the last 3 months pitching customers on the idea and asking them “Would you use it?”. Eventually, they got 6 customers that said they’d give it a go. We got them on-boarded, and then watched their usage over the next couple of weeks.
5 out of 6 weren’t using it at all.
When we went to visit these customers, they all told us that they “liked it” but followed up with “we’ll try it when we have more time”. Now, we’d built a product for small businesses, with the promise that we’d save them time. They were telling us that they didn’t have time to use a product that was supposedly going to save them time!
When we finally dug into our customer’s behavior and understood what they were doing, we found that they never needed our product in the first place. Eventually, we pivoted into something more inline with their actual needs, and only then saw an increase in usage.
I now lead a team focused on shopper engagement at Afterpay, and because a large proportion of our user base are Millennial females (for those keeping track, I am male), eliminating my bias and assumptions about their behavior by talking to them is a huge part of my role. At Afterpay we generally split our product research into 3 main categories:
1. Problem discovery
Ask these questions when you’re trying to understand your problem better and build something worthwhile. For example, if you’re trying to understand how your customers shop, you could try asking:
The key to these questions is to dig deep into a customer’s behavior to truly understand why they do and think that way. This will give you a really great insight into the problem areas you should focus your roadmap on.
2. Solution validation
Ask these when you feel like you understand your customer's problems well enough, you think you have an idea on how to solve it, and you want to know if you’re on the right path. If you’re testing an idea on helping customers find something new to buy, try asking:
You can also look out for:
Solution testing a quick prototype with customers is critical to validate ideas with as little engineering effort as possible, but will still only get you 50% of the way there. The other 50% of validation is only going to come from shipping something and measuring their real behavior.While asking the right questions is critical (so critical that it’s the point of this entire blog post), shipping something is equally important. Use this behavior to shape the next steps with your product.
3. Solution usability
Ask these task based questions when you’ve validated your problem AND your solution, when the idea is coming together but you need to make sure people can use it easily. For example, let’s imagine you’ve designed a way for customers to save something for later:
You may find that your customers get stumped and ask how to complete the task. Don’t help them! Watch them struggle and understand where they’re looking to give you guidance on how to adjust your idea. If your customers can’t complete the task you’ve given them, there’s a usability problem and it may get in the way of making sure your product is successful.
There you have it - 3 types of questions that will give you a head start on understanding your customer’s problems and validating your own ideas.
Let’s hope they like your cooking too.
James Taylor - Product Manager and Designer in San Francisco, previously Sydney.