Diary analysis studies

Diary studies, as the name suggests, involves the user keeping a diary record of what they did, how they did it and what they thought about using a particular system or product.

The diary analysis method is particularly useful as a means of gathering data over prolonged periods of time, or where data collection may present a problem due to the personal or sensitive nature of the information. The diary method can also be useful where information is hard to remember - providing a means of instantly capturing data. Diary analysis is reliant, however, on subjects remembering to complete the diary, as well as providing an accurate account of what took place.

Diaries may be open format, allowing respondents to record activities and events in their own words, or closed format, where all activities are pre-categorised. The advantage of the first method is that the diary is much simpler to construct and allows for a greater coding of responses and thus the potential to capture a wider range of opinions. However, it is much more intensive in terms of labour to collect and analyse the data. A closed format allows for more consistent data collection and simpler data analysis, however, the information obtained from closed format diaries is often less rich.

When can diary analysis studies be used?

Diary analysis is not really a suitable technique until a working prototype has been developed. The method is best suited to the evaluation of a high-fidelity prototype where users can interact with the system or product as a whole, rather than having to complete prescriptive tasks.

How to conduct a diary analysis study

Which type of diary analysis technique to use is dependent on a number of factors: 1) the type of evaluation being conducted; 2) the amount of time and resources available for the evaluation; and 3) the level of maturity of the product. Irrespective of this, there are a number of principles that apply to all diary analyses. Corti (1993), suggested the following guidelines when conducting a diary study:

  1. Where a paper booklet is used to collect data, an A4 booklet of about 5 to 20 pages is desirable, depending on the nature of the diary.
  2. The diary should be accompanied by a clear set of instructions on how to complete the diary. The instructions should stress the importance of recording events as soon as possible after they occur and how the respondent should try not to let the diary keeping influence their behaviour.
  3. A model example of a correctly completed diary should feature at the beginning of the diary.
  4. Depending on how long a period the diary will cover, each page should clearly denote the time period it covers, with prominent headings and enough space to enter all the desired information (such as what the respondent was doing, at what time, where, who with and how they felt at the time, and so on).
  5. Checklists of the items, events or behaviour to help jog the diary keeper's memory should be available somewhere prominent. Very long lists should be avoided since they may be off-putting and confusing to respondents.
  6. If the diary is a "fixed time block" format, an exhaustive list of all possible relevant activities should be listed together with the appropriate codes. Where more than one type of activity is to be entered, that is, primary and secondary (or background) activities, guidance should be given on how to deal with "competing" or multiple activities.
  7. There should be an explanation of what is meant by the unit of observation, such as a "session", an "event" or a "fixed time block". Where respondents are given more freedom in naming their activities and the activities are to be coded later, it is important to give strict guidelines on what type of behaviour to include, what definitely to exclude and the level of detail required. Time budget diaries without fixed time blocks should include columns for start and finish times for activities.
  8. Appropriate terminology or lists of activities should be designed to meet the needs of the sample under study, and if necessary, different versions of the diary should be used for different groups.
  9. Following the diary pages it is useful to include a simple set of questions for the respondent to complete, asking, among other things, whether the diary keeping period was atypical in any way compared to usual daily life. It is also good practice to include a section at the end asking for the respondents' own comments and clarifications of any peculiarities relating to their entries.

It is also worth noting that advancements in ICT mean it is now possible to develop an activity diary according to a number of different formats. Electronic diaries, such as an internet diary, provide a means whereby the user can log-on to the diary, complete the entry and then submit it to the evaluator when finished. This reduces the need for the user to carry a hard-copy diary (and also the chance of losing it), however, a key facet of the diary method, the recording of an event as soon as possible after it has occurred, may be missing. Recently, is has become possible for users to record audio diaries (in a similar fashion to the think-aloud method) and upload these direct to a file sharing site to be accessed by the evaluator. This reduces the need for the subject to write everything down, but it does limit the scope of the diary (i.e. it is not practical for a "fixed time block" format). There is also the requirement for additional equipment.

As mentioned above, the particular type of diary method selected (i.e. open or closed, fixed time block or free recall) will vary from study to study, depending on the nature of the information required, the time and resources available and the level of knowledge/expertise of the users.

Advantages and disadvantages of diary analysis studies

Advantages

  1. Traditionally inexpensive to develop and require no special equipment or training;
  2. Useful when users are scattered or difficult to reach;
  3. Provide a means of overcoming the problems associated with collecting sensitive information by personal interview;
  4. Can be used to supplement interview data;
  5. Templates can be created which provide standardised data formats, making analysis simpler. This is particularly true for online diaries.

Disadvantages

  1. Diary studies rely on the subject being able/willing to complete the diary;
  2. Participants often remember the completion of the task as better or worse than it actually was, or taking less or more time than it did;
  3. The large time requirement placed upon the subject means that the diaries must be straightforward and simple to complete. This can have implications on the breadth and depth of the data collected;
  4. The labour intensive work required to prepare and make sense of the data may render it unrealistic for projects lacking time and resources, or where the sample is large. Although this is now less of an issue with the advent of electronic format diaries.

 

Return to Observational Analysis page

References

The information contained on this page was taken from the following sources:

Last updated: 20.11.2009   © Copyright reserved    Website design: Digital Accessibility Team