Observational analysis

Observational analysis involves watching, listening to, monitoring and in some case interacting with users, to determine the way in which they carry out tasks. Observational analysis can take place in a controlled environment, as with usability testing, or in a more natural environment, i.e. in the field.

Observational analysis can be classified as:

Direct observation

Direct observation involves looking at user behaviour during task execution and recording user actions as they occur. An evaluator making notes of user actions during task completion is one example of direct observation; recording the task using a video camera and then analysing the tape for user actions, is another example of direct observation.

Direct observation is useful in that it allows the observer to focus on specific aspects of the task; however, direct observation can cause users to alter their behaviour, according to how they think they should be acting, conscious that they are being observed. This is known as the Hawthorne effect.

Indirect observation

Indirect observation is a method in which task execution is viewed by a means other than first person observation, such as activity logs or diary analysis. The advantage with this method of observation is that it reduces the Hawthorne effect. However, these techniques require increased effort on the part of the participant. Furthermore, individuals often remember events as being better or worse than they really were.

Common types of observational analysis

The adopted approach when conducting observational analysis is dependent primarily on why and where the evaluation is being done.

Direct observational analysis exists along a continuum, according to the level of interaction the observer has with the users. At one end of the continuum are outside observers, where the observer does nothing more than make notes on user activities, whilst at the other end of the continuum are complete inside or participant observers, also referred to as ethnographers, where the evaluator immerses themselves in the environments under observation, to become 'one of the users'.

Indirect techniques, on the other hand, exist as two discrete categories; those that require active user input to obtain the information and those that capture information without direct user input (i.e. passive user input).

The following sections provide information on the common techniques used during observational analysis. The first four techniques listed below are examples of direct analysis. The latter are examples of indirect techniques:

Acknowledgements

Last updated: 20.11.2009   © Copyright reserved    Website design: Digital Accessibility Team