July 18th, 2009

Usability tests without the users

Everyone is in agreement that usability testing is good.  This normally involves testing users against an interface.  You find appropriate users and through software such as Morae or Silverback you can test and record their actions.  From the user responses you can fix any problems found.  But… how much information can you get without the users?

I was once working on an internal project which involved a map. During the design phase the team members were in disagreement on many usability issues.  For example, should the regions on the map initially be grey and rollover into colour or should the regions be in colour initially and be highlighted on rollover in some other way.  People all had good reasons behind their different ideas. To get past the impasse, I pushed the idea of a rapid prototype.

The prototype was built to test a subset of the different ideas and from this we would move forward to the next version of the interface, test again and so on until we get our final product.  To reduce the technical complexity we only built a small part of the map. We then tested this on users and the information from this decided how we wanted to build the next iteration of the product.

The output of the project was successful and we built a product that was very usable.

But, what could have been done better…

When user testing, you create a script to ask users to carry out actions to test the interface/product.  The steps taken to test the interface were:

  1. Conduct a survey of audience to find out what they want
  2. Brainstorm the product
  3. Create a prototype
  4. Create script for user testing
  5. User test product

The mistake we made was step 4, “Create script for user testing.” This should have done at the start of the project before the prototype was finished.

A user script is the actions you expect a user to carry out when testing, for example: find Egypt.

I found that when we created the script, the solution to some of the usability issues were suddenly obvious, as if a certain design decision was made, it would be impossible for the user to successfully work through the script.  Rather then testing all the different issues with a prototype we could have decided many of them through working through a user script ourselves. Therefore it is always worth doing this step at the start of a project.  The steps should therefore be:

  1. Conduct a survey of audience to find out what they want
  2. Create script for user testing
  3. Brainstorm the product
  4. Create a prototype
  5. User test product

User personas

The idea of user personas is that your split the intended audience of your product into different personas.  You can give them names and a back story.  By thinking about the user actions at the start of the project you are already starting to create a persona. However it may not be worth trying to create an entire back story for them.

For the map project we split our audience into the tasks we were expecting from the survey:

  • User is a browser and is looking for…
  • User is a specific searcher and wants to find out more of this subject…
  • User is looking for inspiration on…

From this we created the user scripts.  If you want, you can expand out and create the entire personas but for this project I felt it was not necessary.

User testing with users is good

Now, user testing with users should always be done and what I am writing here should not be seen as a reason not to do it.  However the point I am making is that you can reduce what you think needs to be tested by using this approach.  Rather than testing diagreements within the team you can solve many of the disagreements with a user script.   Therefore when testing with users you can then test for the most important issues, and these are normally the ones you do not even realise existed.


You should follow me on twitter here

Share the love

Join the conversation

Head of teddy bear appearing over the footer