GNOME Usability Test: Scenario Tasks (Part 1)

A usability test consists of “scenario tasks” that the tester has to complete (read more about scenarios and scenario tasks here). For my usability test of GNOME, I selected four applications – Gedit, Notes, Music and Photos. Of these, Gedit and Notes had already been tested in my mentor Jim Hall’s usability tests, and as per his suggestions the same tasks have been reused for these two applications, with minor changes.

Here’s the scenario task list for Gedit:

1. You want to finish writing a draft of a blog post that you are using in a project. You start the Gedit text editor (this has been done for you).

Please open the file blog post draft.txt from the Documents folder, into the Gedit text  editor.

2. You realize that you got a few details wrong. Please make these edits:

In the first paragraph, change the dash (“—”) to a semicolon (“;”)

from this: Relationships are currency—you

to this: Relationships are currency; you

In the second paragraph, change “me” to “others”.

from relationship to me.

to relationship to  others.

About 2/3 into the document, there’s a list of the “4 I’s” of relationships, but the first two  items are out of order. Put these into the correct order, so the list reads like this:

  1. Initiate
  2. Inquire
  3. Invest
  4. Inspire

When you are done, please save the file. You can use the same filename.

3. Some of the names are incorrect in the file. Please replace every occurrence of  Applejack with Fluttershy, and all instances of Rainbow Dash with Twilight Sparkle.

When you are done, please save the file. You can use the same filename.

4. You’d like to make a copy of the note, using a different name that you can find more easily later. Please save a copy of this note as Leadership lessons.txt in the Documents folder.

For the purposes of this exercise, you do not need to delete the original file.

5. You decide the text in the editor is difficult to read, and you would prefer to use a different style. Please change the text style to be Liberation Serif, 12 point.

And following are the scenario tasks for Notes!

  1. You need to type up a few quick reminders for yourself, so you don’t forget to do them  later. Enter these two reminders into the Notes program. Match the formatting as best as  you can:

First reminder:

Don’t forget: Jeff’s surprise party this Thursday. Check with Mark.

Second reminder:

Things to buy at the grocery store:

  • Milk
  • Eggs
  • Cheese

2. You decide to be more clear in your reminder about Jeff’s surprise party, so you don’t  forget the time.

Update that note to say this:

Don’t forget:

Jeff’s surprise party this Thursday.

Get together at 5:30.

Party at 7:00.

Check with Mark.

3. You decide the text in Notes is difficult to read, and you would prefer to use a different style. Please change the default text style to be Nimbus Sans L Regular Italic, 14 point.

4. This is the last task for Notes. Please delete all notes you may have created during this  part of the usability test.

Part 2 will cover Music and Photos. Read it here.

My first Usability Test

A few weeks ago, I conducted my first formal Usability Test. It was a 1-person UT conducted on a test image of GNOME 3.14. The most challenging part for the participant (also a first-timer!) was to remember that it was the system that was being tested – not the user. And as a talkative person it was a challenge for me to engage in passive conversation – keeping the lines of communication open (‘okay, what are you looking for?’), but taking care not to interfere with the user’s actions (‘oh, that’s over there!’).

The Usability Test had 20 scenario tasks distributed across four applications in GNOME (gedit, Web, Nautilus and Notes). I used the test structure devised by my mentor Jim Hall. The participant was asked a few questions about their demographic profile and previous experience with computers. The usability test took about 2.5 hours – 1.5 hours to set up and simulate the environment for the tasks, and 1 hour to conduct the actual test.

Since the participant was a friend of mine, she was forthcoming about what she was trying to do at (almost) every step: without worrying about ‘is it silly to not know how to do this?’. This reduced the degree of uncertainty in my notes, and allowed me to focus on why she was doing something. However, I did not categorize my notes by task, but simply by action: thus my report was a little ambiguous about which action was done for which task – this is something I intend to improve upon during my next UT. Also, in future usability tests I will assign numerical ratings (1-5, 5 being the easiest to execute) for each task – this will allow me to generate a heat map for better analysis later.

I learnt three main lessons from my first usability test.
First: what seems obvious to me may not seem obvious to others. Before the test, I assumed that certain tasks would be very quick, simply because I knew certain shortcuts to execute them quickly. However, the participant did not know those shortcuts, and it ended up being a surprisingly long-winded process.
Second: consistency boosts productivity. What took a long time to discover in the first application (gedit) was a breezy matter of a few clicks towards the end, because the participant had discovered a menu/options button that was consistent across all applications within GNOME. It was different from what she was used to earlier (Windows menu bars), but the consistency within GNOME helped increase her productivity as the test progressed.
Third: previous experience defines user ‘reflexes’. Since the participant was a regular user of computers, I expected the entire test to finish very quickly. However I had not considered that the participant had a predominantly Windows background – so the tasks with a Windows-like execution proceeded smoothly, whereas GNOME-specific behaviour had a ‘learning curve’ associated with it.

All in all, I enjoyed the entire experience: and I’m looking forward to conducting bigger and better Usability Tests in the near future! :-)