My first Usability Test

A few weeks ago, I conducted my first formal Usability Test. It was a 1-person UT conducted on a test image of GNOME 3.14. The most challenging part for the participant (also a first-timer!) was to remember that it was the system that was being tested – not the user. And as a talkative person it was a challenge for me to engage in passive conversation – keeping the lines of communication open (‘okay, what are you looking for?’), but taking care not to interfere with the user’s actions (‘oh, that’s over there!’).

The Usability Test had 20 scenario tasks distributed across four applications in GNOME (gedit, Web, Nautilus and Notes). I used the test structure devised by my mentor Jim Hall. The participant was asked a few questions about their demographic profile and previous experience with computers. The usability test took about 2.5 hours – 1.5 hours to set up and simulate the environment for the tasks, and 1 hour to conduct the actual test.

Since the participant was a friend of mine, she was forthcoming about what she was trying to do at (almost) every step: without worrying about ‘is it silly to not know how to do this?’. This reduced the degree of uncertainty in my notes, and allowed me to focus on why she was doing something. However, I did not categorize my notes by task, but simply by action: thus my report was a little ambiguous about which action was done for which task – this is something I intend to improve upon during my next UT. Also, in future usability tests I will assign numerical ratings (1-5, 5 being the easiest to execute) for each task – this will allow me to generate a heat map for better analysis later.

I learnt three main lessons from my first usability test.
First: what seems obvious to me may not seem obvious to others. Before the test, I assumed that certain tasks would be very quick, simply because I knew certain shortcuts to execute them quickly. However, the participant did not know those shortcuts, and it ended up being a surprisingly long-winded process.
Second: consistency boosts productivity. What took a long time to discover in the first application (gedit) was a breezy matter of a few clicks towards the end, because the participant had discovered a menu/options button that was consistent across all applications within GNOME. It was different from what she was used to earlier (Windows menu bars), but the consistency within GNOME helped increase her productivity as the test progressed.
Third: previous experience defines user ‘reflexes’. Since the participant was a regular user of computers, I expected the entire test to finish very quickly. However I had not considered that the participant had a predominantly Windows background – so the tasks with a Windows-like execution proceeded smoothly, whereas GNOME-specific behaviour had a ‘learning curve’ associated with it.

All in all, I enjoyed the entire experience: and I’m looking forward to conducting bigger and better Usability Tests in the near future! :-)