Classic Observational analysis

Classic observational analysis involves the evaluator looking on from the outside, whilst the users undertake the specific activity under evaluation. The goal of this type of analysis is to cause as little disruption to the users as possible. Observers remain just that, although they can interact with the users to gain a better understanding of their actions.

When should classic observational analysis be used?

Classic observational analysis can be used throughout the development cycle of a product or system. One particular technique often used when the product is early in development, is the 'Quick and Dirty' method. This involves watching and talking to users in a casual way to gain immediate feedback about a prototype. Following implementation of a product or system, a full observational analysis may be performed, in which the observer remains completely separate from the users and observes their actions anonymously.

How to conduct observational analysis

Observational analysis has 4 main steps:

  1. State the objectives of the evaluation

As with every evaluation, it is important first to state the aim and objectives of the work and define the observational plan prior to conducting the analysis. This will help set out the extent of the observer-user interaction and the type of observation tools (i.e. pen & paper, video camera) to be used. It is important to remember that a number of goals will often be pre-determined by those funding the study and other key stakeholders.

Stating the objectives of the evaluation will also help identify what permissions must be sought before the observation begins (e.g. permission to access the site of the observation, and permission from participants themselves).

  1. Decide how to record the events

An array of equipment is available to the evaluator with which to observe the user. This is discussed in more detail in the section entitled 'Tools of data collection'. Remember that not all equipment will be suitable for every situation, depending on the location of the evaluation and what is being observed. Plan ahead to ensure that the equipment is fit for purpose and that it works.

  1. Conduct the evaluation

The evaluator should observe the interaction between the user and the product or system, taking care not to disturb the behaviour of users. This may consist of note-taking, audio recordings and/or audio-visual recordings. If notes are taken, ensure the notes are sufficiently detailed (and coherent) to allow subsequent off-line analysis.

Prior to the evaluation it is often useful to develop a framework, such as that below, suggested by Goetz and LeCompt (1984), which can help focus on the objectives of the evaluation and the information required:

  • Who is present? What is their role?
  • What is happening?
  • When does the action occur?
  • Where is it happening?
  • Why is it happening?
  • How is the activity organised?
  1. Analyse the data

Analysis of the data should be performed as soon after the completion of the evaluation as possible, whilst data is still fresh in the mind. Same day analysis should normally serve as best practice.

During the analysis, it is important to distinguish between opinion and factual events. It is also important to remove any ambiguity from data. The use of audio-video notes help can help to alleviate these potential issues. Finally, be prepared to refocus the study as you analyse what you see.

Advantages & disadvantages of classic observational analysis

Classic observational analysis is a sub-category of direct analysis, and as such has the same advantages and disadvantages as many of the techniques within this group-set. These are as follows:

Advantages

Disadvantages

 

Return to Observational Analysis page

 

References

The information contained on this page was taken from the following sources:

Last updated: 12.08.2009   © Copyright reserved    Website design: Digital Accessibility Team