Pluralistic walkthrough

A pluralistic walkthrough is a usability inspection method whereby representative users (normally played by the evaluators), developers and usability experts, step through a scenario, discussing the usability issues associated with each scenario step. Also known as storyboarding, the method involves the development of a series of tasks, presented to a panel of users in the form of hard-copy panels; the panelists are then asked to write down the actions they would take to complete the task.

How to conduct a pluralistic walkthrough

A pluralistic walkthrough consists of the following steps, first described by Bias (1994):

Generate task scenarios

Task scenarios are generated in the form a series of hard-copy screens, representing a single path through the interface.

End users attempt task scenarios

The scenarios are presented to the user panel and the walkthrough administrator asks the panelists to write out the specific actions they would take in attempting the specified task. This should be completed individually, without conferring with one-another.

Group discussion of users responses

When everyone has written down their responses, the panelists are then invited to discuss their responses and any potential usability problems they may encounter. At this stage the usability experts and product developers remain quiet, until all user comments are heard. The usability experts then give their comments, afterwhich the developers are invited to join in, with an explanation of why the design was the way it was. The panel then moves on to the next round of screens.

It is important that designers and developers are briefed in advance of the walkthrough about remaining neutral throughout the evaluation and to treat all user comments with positive regard.

When should a pluralistic evaluation be considered?

Pluralistic evaluations are usually conducted early in the development cycle or when production time is limited.

How many evaluators are needed?

Between six and ten representative users should be recruited for the evaluations, plus the product developers and usability experts.

Advantages & disadvantages of a pluralistic walkthrough

Advantages

Generates quantitative and qualitative data on users’ actions by way of written responses;
Reduces test-redesign-retest cycle by generating immediate feedback and discussion of design problems and possible solutions while users are present;
Can provide early performance and satisfaction data before costly design strategies have been implemented;
Product developers present during the session gain appreciation for common user frustrations or concerns about designs;
Group atmosphere encourages collaborative, constructive comments from users, developers, and other members of the product team.

Disadvantages

Normally evaluators act as representative users. As such, the evaluation may not gather real user concerns;
Requires all the users and experts to be together at once;
The technique allows for only a limited number of scenarios, due to a fixed sequence of hard-copy (no browsing or exploring);
Product developers might not feel comfortable hearing criticism about their designs;
Because the walkthrough is dependent on all users finishing each task before discussion can begin, the session can feel laborious. Furthermore, only a limited number of scenarios can be explored because of time constraints.

Return to expert evaluation methodologies page

References

The information contained on this page was taken from the following sources:

Bias, R.G. (1994) The pluralistic usability walkthrough - coordinated empathsis. In: Nielsen, J. & Mack, R.L. (eds.) Usability inspection methods. New York: John Wiley & Sons.
Preece, J., Rogers, Y. & Sharp, H. (2002) Interaction design: beyond human-computer interaction. New York: John Wiley & Sons.
Usability Professionals Association (2005) The Usability Body of Knowledge. [accessed 09/06/08]