Cognitive Walkthrough

A cognitive walkthrough is an expert evaluation technique used to identify usability issues, based on how easy it is for first-time users to accomplish specific tasks when interacting with the system.

Cognitive walkthroughs can be applied at any time during the design process once a prototype has been produced, but they are used more often early on in the design process.

How to conduct a cognitive walkthrough

A cognitive walkthrough consist of three main stages:

Identify users and sample tasks

The first step of a cognitive walkthrough is to identify and document the characteristics of the target user population. The use of personas to develop user identities should be considered, as they can help evaluators develop a shared understanding of the real users in terms of their goals, capabilities and contexts. Personas also help prevent "self referential design" whereby people unconsciously project their own mental models on the product design which may be very different from that of the target user population

At this stage, a description or prototype of the system should also be developed, along with a detailed description of the tasks to be completed and a clear sequence of the actions needed for the users to complete the task.

It is important that both the user identities and the sample tasks are developed by suitably qualified people, so as to ensure a proper capture of user characteristics and requirements and to ensure that the tasks are representative of those conducted by the end user.

Bring together of evaluators to conduct the analysis

The next stage in the process is to bring together one or more (preferably more) expert evaluators, along with the designer of the system, to carry out the walkthrough. Evaluators walkthrough (discuss) the tasks with respect to the interface (mock-ups) and tasks, attempting to answer the following questions:

Will the user know what to do to achieve the task?;

Can users identify what is needed to begin/complete the next task? Is it apparent when it is needed?;

Will the user know from the feedback if they have made the right or wrong choice?

Record critical information

As the walkthrough is conducted, a record is then of made of key information arising from the evaluation. In particular this should consist of (but not be limited to):

Details about the types of problems that are encountered;

The causes of the problems encountered;

Details of any side issues affecting the task;

A summary of the design changes that are required to alleviate the problem.

Revise the design

As the final stage of the process, the design of the system or product is then modified to fix the problems encountered during the testing.

It is important that the documentation generated from the walkthrough is sufficiently detailed to allow a review at a later date or subsequent evaluation, including, date of the evaluation, the product version evaluated and the evaluators conducting the evaluation. It is also advisable to use a standardised feedback form, in which answers are given against the three questions raised in step 3. The severity of each problem (i.e. how likely it is to occur and the level of impact on the user if it does) should also be recorded.

When to conduct a cognitive walkthrough?

A cognitive walkthrough should be conducted early in the design and prototyping phase, and also during the test and evaluation cycle.

How many evaluators are needed for a cognitive walkthrough?

At least two expert evaluators (assuming all the relevant personas), plus the product/system designer.

Advantages & disadvantages of a cognitive walkthrough

Advantages

Enables detailed evaluation of user problems, without the user having to be present;
Can be conducted at any stage of product/system development (although usually conducted early in the development cycle);
Possible to conduct the walkthrough prior to a working prototype being available.

Disadvantages

Time consuming and potentially laborious to conduct (particularly of large tasks);
Appear to detect far more potential problems than actually exist;
Technique has a narrow focus and as such can miss potential problems.

Return to expert evaluation methodologies page

References

Preece, J., Rogers, Y. & Sharp H. (2002) Interaction design: beyond human-computer interaction. New York: John Wiley & Sons.
The QA Focus Web site (2005). Introduction to Cognitive Walkthroughs. [accessed 09/06/08].
Wharton, C., Rieman, J., Lewis, C., & Polson, P. (1994) The Cognitive Walkthrough Method: A Practitioner's Guide. In: Nielsen, J. & Mack, R.L. (eds). Usability inspection methods. New York: John Wiley & Sons.