This originally appeared as the very first post on Afterpay's Tech Blog.

“Do you like this?”

“Would you use this?”

If you’re a product manager and you’ve been asking customers these questions, keep reading. We all know how important it is to talk to your customers, but it's only half the story. Asking them the right questions could mean the difference between a successful product and a tough lesson best told through a blog post.

Let’s imagine you’ve just opened up a new restaurant, and you invite some friends to come in on opening night. You’ve spent months planning the menu, layout, and spent a ton of money. They come in and you ask them if they liked it when they’re done.

You could get one of these three responses:

• They said they liked it (but really, they hated it and didn’t want to hurt your feelings after you’ve spent so much time & money)
• They said they liked it (but really, they thought it was OK. Not bad all considering, but they wouldn’t come back if they didn’t know you)
• You don’t even have to ask if they like it, they’ve already told their friends and have booked to come back next week 

See the problem? In response 1 and 2, you’re out of business by the end of your first year.

In response 3, you’re franchising and expanding across the country. I had a similar experience early on in my career. I’d just joined a new company and they were finishing up a 12 month journey to build a first-to-market product. The team had spent the last 3 months pitching customers on the idea and asking them “Would you use it?”. Eventually, they got 6 customers that said they’d give it a go. We got them on-boarded, and then watched their usage over the next couple of weeks.  

5 out of 6 weren’t using it at all.

When we went to visit these customers, they all told us that they “liked it” but followed up with “we’ll try it when we have more time”. Now, we’d built a product for small businesses, with the promise that we’d save them time. They were telling us that they didn’t have time to use a product that was supposedly going to save them time!

When we finally dug into our customer’s behavior and understood what they were doing, we found that they never needed our product in the first place. Eventually, we pivoted into something more inline with their actual needs, and only then saw an increase in usage.

I now lead a team focused on shopper engagement at Afterpay, and because a large proportion of our user base are Millennial females (for those keeping track, I am male), eliminating my bias and assumptions about their behavior by talking to them is a huge part of my role. At Afterpay we generally split our product research into 3 main categories:

1. Problem discovery

Ask these questions when you’re trying to understand your problem better and build something worthwhile. For example, if you’re trying to understand how your customers shop, you could try asking:

• “Can you tell me about the last time you bought something?”
• “Can you show me where you went?”
  
• “Why did you go there?”
• “What were you looking for?”
• “Why?”
• “How did you feel about that?”
• “What was most frustrating about this?”

The key to these questions is to dig deep into a customer’s behavior to truly understand why they do and think that way. This will give you a really great insight into the problem areas you should focus your roadmap on.

2. Solution validation

Ask these when you feel like you understand your customer's problems well enough, you think you have an idea on how to solve it, and you want to know if you’re on the right path. If you’re testing an idea on helping customers find something new to buy, try asking:

• “What's the most important part of what you’ve seen today?”
• “What's the least important part of what you’ve seen today?”
• “How do you feel about this?”

You can also look out for:

• Facial cues and unprompted reactions to your prototype - look for things like smiles or frowns, excited or frustrated tones in their voice so you can follow up and ask them why they feel this way.
• Telling you stories about how this product either does or doesn’t solve a recent problem they had
• Excitedly asking when it will be released
• If they’re willing to pay you right now for a product you haven’t yet given them (If it worked for Microsoft, then it can work for you!)

Solution testing a quick prototype with customers is critical to validate ideas with as little engineering effort as possible, but will still only get you 50% of the way there. The other 50% of validation is only going to come from shipping something and measuring their real behavior.While asking the right questions is critical (so critical that it’s the point of this entire blog post), shipping something is equally important. Use this behavior to shape the next steps with your product.

3. Solution usability

Ask these task based questions when you’ve validated your problem AND your solution, when the idea is coming together but you need to make sure people can use it easily. For example, let’s imagine you’ve designed a way for customers to save something for later:

• “How would you save a store? Could you show me?”
• “Where would you find your saved stores?”
• “Could you please try and remove the store from your list?”

You may find that your customers get stumped and ask how to complete the task. Don’t help them! Watch them struggle and understand where they’re looking to give you guidance on how to adjust your idea. If your customers can’t complete the task you’ve given them, there’s a usability problem and it may get in the way of making sure your product is successful.

___

There you have it - 3 types of questions that will give you a head start on understanding your  customer’s problems and validating your own ideas.

Let’s hope they like your cooking too.

James Taylor - Design, product & data leader based in Sydney.