Usability Testing Boot Camp

In a usability test, we make observations. But in order to fix the design, we need to know why the user behaved in such a way. Sadly, we can't just ask the user for insight. Here's why.

Usability tests don’t require a usability lab, observers, a moderator, screen recording or video recording. All these are ‘nice to have’. Focusing on these alone puts you at risk of running a ‘cargo cult’ usability test.

Dr Philip Hodgson is a psychologist with great insights into the merits of different usability test methodologies. He's also outspoken on why focus groups are such a poor choice for usability testing.

The usability test plan helps all the stakeholders understand and visualise what will happen so that they can comment and make suggestions accordingly.

What would happen if we compressed a usability test plan fit on one page?

Recruiting the right participants is critical to finding genuine usability issues, but how do you go about finding them?

Laura Ivory runs a company that recruits participants for usability tests and market research. Here she describes some of the things she's learnt.

In this lecture, we describe the main setups for running in-person usability tests: the portable test lab; the single room setup; the classic testing lab setup; the classic benchmark test; and the multi-room setup.

Join me on a tour of a usability lab.

Usability testing mobile devices poses some particular challenges. Here's some suggestions on how to overcome them.

In this lecture, I review the advantages and disadvantages of testing in the user’s environment; a rented usability lab; and remotely, via the Internet.

Roads in London with red lines on them are known as red routes: these are the key road arteries in London. Transport for London does everything it can to keep these routes clear. Your product or web site has “red routes” too. They are the critical “user journeys” with your product. You need to identify your products red routes so you can run a task-based usability test.

To develop good test tasks for usability testing, you need to add some contextual information to your red routes.

Here's a useful tip for presenting your test tasks to test participants.

I’m going to talk about three hats you wear when running a usability test. In practice, you tend to play each of these roles yourself, and that’s why this analogy of “hats” works so well. I’ve borrowed this idea from Carolyn Snyder’s excellent book titled 'Paper Prototyping'.

The second hat is the sports commentator hat. Just like in a football game, the facilitator has a better view than the observers. So reinforce any user action that may not have been obvious or visible to observers.

In this video, you'll watch a usability test participant being introduced to the test. You'll also see how the moderator sets up the 'thinking aloud' protocol.

In this lecture, I stop the video at certain points and explain why the test moderator said or did various things.

Now it's your turn to practice welcoming a participant.

Miles Hunter is a veteran user experience consultant and in this lecture he describes several tips for moderating a usability test.

The third hat is the scientist hat where you're responsible for avoiding test bias and recording the data.

If you're running software-based usability tests, there are some tools that can help you record the session and collate the results.

Taking notes in a usability test is an art unto itself. Here Todd Zazelenchuk, the inventor of the Usability Test Datalogger, describes his approach.

Let's get read to watch a live usability test.

In this video, you'll see a participant taking part in a usability test of the Udemy system.

In this lecture I deconstruct the way I moderated the test and highlight some of the things I did well (and some things I did less well).

This video shows how I would datalog this usability test.

It's time for you to run your own test.

Let's review the various types of remote usability test, both moderated and unmoderated.

Further Reading

How to Run a Usability Test with Users Who Are on Your Site Now - This article provides a useful step-by-step tutorial on the practicalities of setting up a remote, moderated session.

One challenge is setting up a remote usability test is helping the participant set up screen sharing. Here's an example of how to do this.

Benchmark tests, preference tests, 5-second tests (‘memory test’), click tests, labelling tests… here's a list of test types you can use.

You need to be careful about the questions you ask with unmoderated tests.

Finally, there’s one other type of remote test that you’ll hear about. These tests have the paraphernalia of a usability test -- such as the PIP video -- but they assume that the moderator’s role is insignificant. The problem is that often users will go off on the wrong track when they’re using your web site. In my experience, you really need a good moderator to keep the session on track.

With formative, thinking aloud tests, the “data” is really usability problems. With summative tests, we have some real numbers we can work with to define our system's usability.

In this lecture, I review two usability test survey tools and show you when to use them: the System Usability Scale (SUS) and the Single Ease Question (SEQ).

Test participants are often reluctant to be critical of a system, even when they've struggled with it. The Microsoft Desirability Toolkit is a way to encourage people to be more critical.

If you want to get more in-depth with using statistics in usability testing, you'll find Jeff's course, Practical Statistics for the User Experience, on Udemy. The course is priced at $199 but Jeff has offered a massive $49 discount to people on this course, which means you get it for $150.

Giving feedback to design teams is difficult because it’s a bit like telling a proud mum that her baby is ugly.

Here are some tips on creating a usability test highlights video that will get watched.

What does a client want from a usability test? In this video, Joanne Aldhous (who has commissioned several usability tests) provides some answers.

Dr Mayhew is a legend in the field of usability and in this interview she relates some tales from the trenches.

I really want you to apply what you've learnt so sign up for refresher training!

Here are some sources you can use to take your learning further.

Please complete this short survey to let me know what you liked about the course and how I can improve it.