Expert evaluation methodologies

Expert evaluation is the appraisal of a product or service by someone who has the professional training or experience to make an informed judgment on the design.

The inclusive design toolkit, developed by i~design in association with BT, outlines 3 goals for expert appraisal:

Identify possible causes of design exclusion;
Suggest improvements to reduce this exclusion;
Increase user satisfaction.

Expert evaluation is a useful tool in that it is efficient and provides prescriptive feedback. It is also less expensive than other techniques requiring subject involvement and experts often provide solutions to the problems that they identify.

Common types of expert evaluation

There are several common techniques that can be used when performing an expert evaluation. These are:

Heuristic evaluation
A technique in which one or more experts evaluate the system or product against a list of design principles (commonly referred to as heuristics).

Pluralistic walkthrough
A method whereby evaluators conduct a series of paper-based tasks that represent using the proposed product or system. The tasks are first completed individually and then the findings discussed as a group, highlighting the usability issues associated with completing that task.

Cognitive walkthrough
A technique in which a group of evaluators walkthrough a specific sequence of steps or actions required by a user to accomplish a predetermined task, highlighting and discussing issues along the way.

More information on each technique can be found by clicking on the title of the particular technique.

When to use expert evaluations

Expert evaluations can be used at any stage during the development cycle. The specific technique chosen will depend on the stage of development of the product.

Tips for effective experts appraisals

Conduct expert evaluations at regular intervals throughout development

At an early stage of development, expert evaluation can provide a significant improvement in the design of a product for a relatively small outlay of capital. At the later stages of development expert appraisal can detect critical problems and provide priorities for exploration with users.

Avoid bias when performing expert evaluation

Reduce the likelihood of bias during evaluations by choosing an evaluator who has not been involved in the development of the product or system.

Ensure the evaluator has sufficient technical knowledge

If the evaluator does not have sufficient knowledge, there is a risk that the findings may be inaccurate and even counter-productive.

Return to methodologies for evaluating accessibility page

References

The information contained on this page was taken from the following sources:

Cambridge Engineering Design Centre (n.d). Inclusive design toolkit. [accessed 06/06/08].
European Design for All e-Accessibility Network (n.d.). [accessed 05/06/08]
Nielsen, J. (1994) Heuristic evaluation. In: Nielsen, J. & Mack, R.L. (eds). Usability inspection methods. New York: John Wiley & Sons.
Preece, J., Rogers, Y. & Sharp, H. (2002) Interaction design: beyond human-computer interaction. New York: John Wiley & Sons.
The Department of Trade and Industry. (1990) A Guide to Usability. Exeter, UK: BPCC Wheatons Ltd.
Usability Net (2003) [accessed 06/06/08].