Think-aloud method
The think-aloud method is a form of outsider observational analysis. As its name suggests, the method involves users vocalising their thoughts and actions as they perform a set of specified tasks. Users are asked to say what they are looking at, doing, thinking and feeling, as they go about completing the task.
The advantage of the think-aloud method is that it enables observers to see first-hand the process of task completion (rather than only its final product). A major problem, however, with think-aloud analysis is the occurrence of silence during the evaluation. The observer has no idea what the user is thinking or attempting to do. A way around this problem is for the observer to prompt the user to think aloud, however this may be seen as overbearing. It may also exacerbate another problem associated with think-aloud analysis; unease on the part of the participant in having to speak aloud all of their thoughts and actions. One way around this problem, is for think-aloud analyses to be conducted in pairs or even groups.
When can the think-aloud method be used?
The think-aloud method can be applied throughout the design and development process. The technique is particularly useful for examining prototypes, highlighting potential problems from a user perspective. The technique has also been shown to be very effective when evaluating systems intended to be used simultaneously by different users.
How to conduct a think-aloud analysis
Think-aloud analysis has 4 broad, overlapping stages:
-
State the objectives of the evaluation
As with any evaluation, the first step is to define the aims and objectives of the study. This will help identify the user population and the activities that require evaluating.
-
Define the tasks
Once the objectives have been set and the goals defined, the next stage is to develop the specific tasks for the evaluation. It is important that the tasks are clearly defined - although the processes involved in completing the task may not be implicit, depending on the stage of product development; such is the aim of the think-aloud technique. Clearly defining the task will help the participant focus on the goal and thus the processes involved in achieving that goal. It may also be useful to divide complicated tasks into subtasks, so that more detailed information about each task can be gathered.
-
Conduct the evaluation
Once the objectives have been set and the tasks identified it is time to perform the evaluation. Participants involved in the evaluation should be representative of the general user population so as to provide representative data.
Prior to beginning the evaluation it is important to fully brief the participants. A demonstration of the think-aloud technique should be given and participants should carry out an unrelated practice task. It is also important to remind participants that they should vocalise their thoughts and actions whilst completing the task. It may also be useful to forewarn participants that you may prompt them from time to time during the evaluation, if they fall silent for too long.
Finally, ensure that the participants understand the plan of the evaluation, so that their thoughts and observations are not clouded by comments unrelated to the task. You should also avoid the temptation to prompt the participants, even if their actions appear to be moving off-task (which may be the case with group think-aloud evaluations). Only intervene if: 1) the task has finished; 2) the task has clearly broken down beyond any further progress; or 3) a participant is clearly exhibiting a strong desire to end the evaluation, but is reluctant to say so.
-
Data analysis
Data from the evaluation should be collected ideally in video format. The use of pen and paper notes is not suitable for think-aloud evaluations, as it is difficult to accurately capture all the information. Audio format has the advantage of being less obtrusive than video, but there is no means of relating comments with physical actions.
When analysing the data, try to avoid supplementing the notes with subjective comments based on recollections from the evaluation. Supplementary notes should be used to help clarify the actions and comments from the users, not evaluators' opinions (e.g. the notes should read 'participant selected cash rather than card payment method, but uttered 'I have selected ''pay by card'''; rather than, 'subject appeared to be unable to identify the 'card' payment method).'
Once the analysis of individual users' data has been completed, the next step is to aggregate the data to identify common themes running through the evaluations. The best way to do this is to 'code' responses using a numerical key. This allows responses to be pooled together by type, making the data more manageable. It also allows the findings to be quantified.
Advantages & disadvantages of the think-aloud evaluation technique
Classic observational analysis is a sub-category of direct analysis, and as such has the same advantages and disadvantages as many of the techniques within this group-set. These are as follows:
Advantages
- Provides rich qualitative data;
- Allows first hand insight into the thought processes associated with different tasks.
Disadvantages
- Participants are often resistant to verbalising problems;
- Difficult to identify changes in behaviour due to learning;
- Techniques only work as long as the subject verablises their actions.
Acknowledgements
- Erickson, T.D. & Simon, H. A. (1985) Protocol analysis: Verbal reports as data. Cambridge, USA: The MIT Press.
- Preece, J., Rogers, Y. & Sharp, H. (2002) Interaction design: beyond human-computer interaction. New York: John Wiley & Sons.
- Jaspers, M.W., Steen, T., van den Bos, C. & Geenen, M. (2004) The think aloud method: a guide to user interface design. Int J. Med. Inform., 73(11-12): 781-95.
- Koedinger, K. (n.d) Cognitive Task Analysis: Think alouds and difficulty factors assessments. Learnlab. [accessed 12/06/08].
Last updated: 20.11.2009 © Copyright reserved Website design: Digital Accessibility Team