Usability Testing

Executive Summary:

This Usability Testing Report outlines the concept and method of usability testing performed on the KUA Tech Support site, http://kuatech.kua.press.kua.org.

Results are given for the number of clicks and the time needed by site testers to perform two tasks: changing the user’s network password and discovering how to map a network drive.


Download the PDF:

Usability Testing Procedure – KUA Technical Support


Site testers:
Site testers were self-selected from the student body. They responded to an email from me:

Students:
I am looking for 10 volunteers to take part in a usability study. Each volunteer will receive a $5.00 gift certificate from the school store redeemable for the tasty treats of your choice.
The study is simple and consists of you navigating a website while I observe. It should take no more than 10 minutes of your time and can be done in my office or on your own computer in a place convenient to you. Please contact me to arrange a time and/or place.

Volunteers were signed up on a first come, first serve basis.
One student at a time performed the testing.
Methodology:
Test subjects were read an introduction to the study.

Thanks for coming to help me with this usability study. The study is designed to help us improve the department tech support site and is very simple.
You will be given two tasks to perform. I will observe as you perform them. I will be timing you and making some notes but I am not testing your performance. We simply want to see how you navigate in order to improve the site.

They were then directed to a laptop on which was displayed the home page of the site. They were given Task 1 which was worded in the same way for every subject: “I would like you to find out how to change your KUA network / email password.”
I then started the timer, observed as the subjects navigated and counted the clicks of the mouse until the task was accomplished.
They were then given Task 2: “I would like you to find out how to access your network drive, your H drive, from your laptop.”
The timer started, I counted clicks and observed. They were given a chance to add any comments. I used a paper form to record results

Reasoning on methodology:
I gave them the questions one at a time because I didn’t want them knowing ahead of time what the second task was, that might skew the results. I had thought about whether I wanted to ask the questions verbally or hand them the tasks written out on paper. I opted for verbal direction for convenience’s sake, but made sure to use the same wording for each subject.
The wording of the tasks was important too. Many students get their KUA network password confused with the password for their “Portal” the Whipple Hill school website but they all know that the email password can be different from the Portal. Similarly the wording of Task 2 had to include both network drive or” H drive” as many times students do not know what a network drive is, they are more likely to know it as H drive. Some do not know what either the network drive or the H drive mean and one of the subjects did not in fact know that he had a network folder (drive) in which he could store data. He was able to complete the task even so.
Testing Results:
Task 1:
Time to complete (in seconds):

High – 109
Low – 28
Average – 66.13

Number of clicks:

High 5
Low 1
Average 3.75

Task 2:
Time to complete (in seconds):

High – 117
Low – 0
Average – 45.18

Number of clicks:

High 6
Low 0
Average 2.12

The numbers were a little skewed for Task 2 because one tester found the answer to Task 2 on the same archive, or search result page as Task 1 so no further searching was necessary
None of the testers read any of the introductions on the home page that I observed but of course they were intent on completing the task so perhaps this was not really a valid observation.