Don’t ask your users what they want – a guide to User Testing

The whole user centred design philosophy is based around involving users in the design process. So why do some of the ‘brightest minds’ say we shouldn’t ask our users what they want? Steve Jobs once said;

Why do we want to ask what our audience thinks? We don’t care what they think. How can people tell you what they want if they haven’t seen it before? If we ask them what they want, we’ll end up doing Swan Lake every year!”

Note: there is a lot of debate about this quote being true or not. There is a lot of evidence that supports both sides of this argument.

And Henry Ford also said;

If I had asked people what they wanted, they would have said faster horses.

Note: there is a lot of debate about this quote being true or not. There is a lot of evidence that supports both sides of this argument.

Steve and Henry are right, we shouldn’t ask people what they want because they simply can’t tell you.

Asking this can actually do harm. People’s responses are only guesses of what they think they want. Also, their thoughts are shaped around what they currently know, which is why people in the 1800’s would ask for a faster horse.

Instead, we need to show people what we think they want. Doing this allows them to react to real experiences, which gives us information that improves our designs.

Here’s an example;

Imagine that it’s 2004. You’ve never heard of the iPhone  before, let alone a smartphone. You’re called into a meeting at Apple and Steve Jobs puts the first iteration of the iPhone in your hands. Can you imagine what your reaction would be? You’re used to using a Nokia 2280 and now you’re seeing a product that will change the world.

Steve gives you a task – “Imagine you want to find the score for the latest Lakers game. How would you do that?”

You see an icon called internet. You tap on it. A web browser opens. You type in nba.com and see that the Lakers beat the Celtics 110-98.

This is exactly the type of process that could, or should, have been used to create the iPhone. Steve and his team of designers would have watched your every action and reaction as you completed your task. This information would have been used to refine their designs and create the iPhone we know today.

Steve and his team of designers could have used this information to make improvements to their design. For example, it may have taken you minutes to find the Laker’s score through the web browser. Steve could have thought, ‘there’s got to be an easier way!’ So they decide to test an easier idea with the next person. This could have been the way they created apps or push notifications.

In order to get this type of information you need to facilitate user testing sessions with the right way.

User Testing

User Testing is the moment of truth for your designs. Here you will discover how they perform in a realistic scenario. You’ll learn valuable lessons about how your users interact with your product or service and how you can improve it.

I’ve never had less than 20 key findings the first time I’ve tested a design, so prepare yourself for feedback.

User Testing allows you to evaluate the two things that create a great experience; how much value your product or service creates and how easy your product or service is to use.

This is the moment of truth for your designs. Here you will discover how they perform in a realistic scenario. You’ll recruit participants who are, or could be, the people who actually use your product or service. Then you’ll give your participants tasks that represent how your product or service will actually be used. You’ll be particularly interested in what people think about each part of your designs and why. This information is vital to make improvements.

For example, imagine your user testing one of the first versions of the iPhone. You notice that the participants arrange specific sets of apps on each screen. You ask, “Why have you arranged those apps in this way?” The participant says, “Because they’re all similar types of apps; I have my games on this screen, my productivity apps on the next screen, and all my social media ones on the last screen.”

You can now use this information to create an ‘app grouping’ feature for arranging apps into folders. If you didn’t ask why you may not have discovered that this feature was needed.

Who to test with

The most important part of testing is finding who to test with. You can easily get misleading data by testing with the wrong people. To avoid this, only talk to people who match your personas.

Personas are used to represent different groups of people who use your product or service. They aren’t separated by things like age, gender, or geographical location. Useful personas are always separated by behaviours.

For example, a group of people who exercise three times a week and eat low-carb diets could be considered a persona, even if they were different ages and genders.

This contrasts with the way marketing teams define customer segments. These are based on factors like age, gender, location, and income. Customer segments often find their way into design. However, it doesn’t matter what age you are, you can still be passionate about exercising three times a week and eating a low-carb diet.

Find where your personas hang out and ask them to talk to you about your product. This is called  ‘guerrilla’ testing. Guerrilla testing is when you don’t formally arrange any participants. Instead, you go to where people are and ask them for 5-10 minutes of their time. This can be a bit awkward, but it helps to be clear that you’re not trying to sell them anything. Instead, you’re offering them an incentive, like a chocolate bar or bottled drink, for their time.

Aim to talk to five participants from each persona group. The Nielsen Norman Group conducted a comprehensive study and found that “elaborate usability tests are a waste of resources” and “the best results come from testing no more than five users.”

nn-5 users

As the number of users tested with increases, the percentage of usability findings discovered decreases.


There are two roles for your team in a user testing session;


The facilitator is the person who gives the participant tasks and asks them questions. It’s the facilitator’s job is to uncover exactly what the participant is thinking while making the test feel realistic.

Note taker

The note taker is someone who records precise notes of what’s happening during the testing session. This role is very important – if you don’t document your findings well, you won’t remember what happened and why. This information is vital when iterating your designs.

It’s important for other members of the team to observe the testing sessions. This helps them understand what works and what doesn’t, which is more effective than being told what happened.

Team members can observe the sessions by watching video recordings of them. If what you’re testing is digital, then there are multiple apps that record all the interactions on an Android, iPhone, or laptop screen. An alternative is to use a smartphone, GoPro, or other type of camera to film the session.

If you want to live stream your session then use Google Hangouts, X-Split, Skype or something similar. However, this may be expensive when there’s no WiFi.

Get the team to view the sessions together. While watching they should write down their own findings on post-it notes:

  • Green for positive
  • Pink for negative,
  • Blue for observations.

Remember to number each finding so it’s easy to identify personas and trace findings back to the participant they came from.

Write findings with a thick-tipped sharpie so it’s easy to see. It also helps keep findings concise.

As you write these findings stick them on a wall – you’ll begin to see that there are similar findings from different participants. It’s a good idea to cluster these findings together. You’ll end up with something like this:


How to run a test

Begin with an introduction. Ask the participant a few questions about themselves in order to make sure they’re the right persona.


Next, remind them about the principles of user testing.


Start the session by setting the scenario. Give the participant a realistic story so they can imagine they’re doing this for real. For example, you could say:


They’re now ready for their first task. This could be:


Each task should cover an area of your product. Keep going until you’ve covered everything you want to test.

Finish up by asking closing questions that aim to discover what the participant thought of the product as a whole, for example:

  • “What was one thing you liked the most?”
  • “What was one thing that you like the least?”
  • “How would you describe this product to a friend?”

Get the most out of your participants

Throughout the session, prompt your participant to give you feedback by doing the following.

Continuously probe

Regularly ask:

  • “What are you thinking now?”
  • “What would you do now?”
  • “What would you do if you were doing this [in the situation you’d normally do this. For example, at home]?”

The echo

If the participant says something like:

“Oh, that’s weird.”

The facilitator should say:

“What’s weird?”

This helps you clearly figure out what the participant is talking about without making assumptions.

The boomerang

Participants will ask the facilitator questions or state something to themselves. It’s good to understand what the participant is thinking or would do if the facilitator wasn’t present. This is where you throw the question or statement back to them like a boomerang.

“Do I need to register?”

“What do you think?”

Ask why

If participants say or do something, ask why. It will help you understand exactly what’s going on which you can use to make design decisions. It’s also good to repeat the participant’s language to remove the possibility of biasing a participant’s response.

“That’s cool”

” Why is that cool?”

After talking to all your participants, analysing the sessions with the group and clustering post-it note findings, you should be ready to iterate your designs.

After testing

Once you’ve tested with at least five participants from each of your persona groups you will be ready to make design changes to your product or service.

After making changes be sure to User Test again. This is to validate that your changes work and to discover more things you can improve.

Keep iterating and testing your designs until your participants begin to ask questions like:

  • “When is this going to be released?”
  • “How much will it cost?”
  • “Can I have it now?”

When you reach this stage, you’ll know your designs are working well.

Show people, don’t ask them

In conclusion, Steve Jobs and Henry Ford were right when they said we shouldn’t ask people what they want.

We should instead show them through User Testing while asking vital questions, which will help us make improvements. We need to continue to do this until people are begging us to release our product or service.

< Previous: How to create value


3 thoughts on “Don’t ask your users what they want – a guide to User Testing

    • Correct! The user isn’t being tested. However, we are testing our product/service with users.

      I prefer not to use ‘Usability Testing’ because we’re not just testing the usability of the product/service. We’re also testing the value of it. As I’m sure you know, overall experience consists of two things; value and usability. Check out my other blog post on this topic: https://painsandgains.co/2016/01/08/experience-over-everything/

      At the end of the day, it’s just a label. ‘Usability Testing’ and ‘User Testing’ are both a bit misleading. Maybe we should call it ‘Experience Testing’ instead?

      Thanks for the question Michael!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s