Cost 219ter

1.0 Introduction

1.1 Background to this Project

 

This project was conceived to carry out part of the main objective of the COST 219ter Action:

"The main objective of the Action is to increase the accessibility of next generation telecommunication network services and equipment to elderly people and people with disabilities by design or, alternatively, by adaptation when required."

In order to improve the accessibility of next generation telecommunication equipment, it is necessary to understand how mobile phones are evaluated for accessibility.  It was decided that research should be carried out to determine to what extent the test houses of Europe were working in this field and what methodologies were they following.  It was also decided that an analysis of any evaluation methodologies that were being used by members should be undertaken. 

A Short Term Scientific Mission (STSM) was conducted by Kristian Kristiansson of the Swedish Handicap Institute.  The STSM highlighted the use of a set of guidelines known as heuristics that the RNIB evaluation team used to determine whether a product was accessible or not.  One of the outcomes of this initial visit was that these heuristics were not specific to one device and only evaluated the hardware side of a product and they only considered the needs of blind and partially sighted people.

Therefore a proposal was put forward that these heuristics should be modified to solely focus on mobile phones, and should be extended to include the operating system, mobile web pages and finally incorporate the needs of other disabled user groups. 

In order to understand the evaluation method, it is important to understand what accessibility and usability are, what evaluation is and why it is important and why involve disabled users.  These are all discussed below.

1.2 Accessibility and usability synergy

 

Harrison et al (2006) and Graupp et al (2003) agree that there is a synergy between accessibility and usability. In order to say how these work together, it is necessary to first define what these terms mean.

There are many terms, which define accessibility from the built environment to the World Wide Web.  Many of the definitions are interchangeable however for the purposes of this evaluation the definition of accessibility can be given as:

“Successful access to information by people who have disabilities.”

The international standard ISO 9241 Part 11 defines usability as;

“The extent to which a product or system can be used by users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” 

If a product or system is inaccessible then it is also unusable to the set of users that are unable to access it. This in turn limits the amount of end users that will be able to access and use the product or system. Therefore to open the product up to a much wider audience the product must be designed to be “inclusive” so that it is both accessible and easy to use.

1.3 What is evaluation?

 

Benyon et al (2005) says that evaluation enables evaluators to assess a design, a product or system to determine whether it is learnable, effective and accommodating for its intended user population. Dix et al (2005) and Chandler et al (2005) agree with this statement. Chandler et al (2005) goes on to say:
 
“Evaluation is a tool which can be used to determine whether a product is fit for its purpose that it was designed to do.”

1.4 Why evaluation is important

 

Evaluation of a product or system plays an important role in the project lifecycle. Dix et al (2005) emphasises the importance of evaluation as it can assess the usability and/or the accessibility of a product or system. It may also be used to assess the users’ experience, and to identify any specific problems with the design. 

Dix et al (2005) and Benyon et al (2005) agree that evaluation can be employed when a design, a product or system is being developed or when it is available on the market.  When the evaluation occurs in the design phase the results of the evaluation can be fed back in order to modify and improve the design.

Zwick et al (2005) highlights that a product or system should be evaluated as soon as an initial mock up is ready. This helps to keep the cost of design modifications to a minimum when compared to making a change later in the design process or after the item has been released to market. Benyon et al (2005) agrees with this emphasising that by carrying out iterative evaluations many design issues will be eradicated.

Once the product or system is on the market any flaws which have not been designed out may cause users not to use it as efficiently or successfully as it could be used. These flaws may even make the product or system inaccessible to certain user groups.  It is therefore imperative that a product or system should be evaluated so that issues are resolved.

Evaluation is important to ensure that users can carry out a range of activities with the product or system and that it meets the needs of the users.

1.5 Why evaluate mobile phones?

 

Benyon et al (2005), Chandler et al (2005) and Zwick et al (2005) agree that the miniaturisation trend of technology has resulted in new problems being discovered. Buttons have been put closer together, sit flush to the surface, and have different modes of operation, which allow them to be used for multiple functions. Its small screen size, is increasingly being accessed under different ambient lighting conditions.  This has increased the difficulty that some users already have in accessing the information. This makes it difficult for some users to access the information stored on their mobile phone (Zwick et al 2005 and Benyon et al 2005).

The miniaturisation trend has made it difficult for many people to be able to access and use mobile phones.  As well as this, people with disabilities are being excluded altogether from using a mobile phone as their needs are not being taken into account. Chandler et al (2005) argues that disabled people have a stronger need for mobile communication due to their specific circumstances (e.g. not being able to walk to a payphone or not being able to remember the phone numbers).

Other mobile phone issues highlighted by Benyon et al (2005) include the:

• Limited screen space: due to the limited size of the display fewer functions are displayed at top level, therefore users have to drill down in order to access these, which may lead to difficulties with navigation if designed poorly.
• Poor feedback, which does not guide the user what to do next.
• Inconsistencies with the interface design: This can cause confusion amongst users particularly when changing to a different model of mobile phone (e.g. with some mobile phone models the user is expected to press the right hand button to answer a call, whilst for others the left).
• Poor personalisation of menus
• Obscure symbols on many buttons, which do not easily convey information for newcomers.
• Understand ability of documentation (e.g. the terminology used “scroll”, “icon”, “roaming”).

 

Benyon et al (2005) emphasises that by carrying out iterative evaluations many of these issues will be eradicated.  Benyon et al (2005) goes on to say that to deal with the complexity of the mobile phone it is important to look at the underlying issues, which affect use.  These issues range from the ergonomics of the small keypad to the amount of information the user has to remember in order to operate the device. Changes to improve the accessibility (e.g. by improving colour contrast, font size etc.) will benefit most users as an end result.

Problems with the design, which are not removed may cause confusion amongst users or make the mobile phone to be inaccessible or unusable. It is therefore imperative that mobile phones are evaluated so that issues are resolved as much as possible.

 

1.6 Why involve users with disabilities?

Evidence from current research such as by the Disability Rights Commission has shown that people with disabilities are more likely to find problems with the accessibility of the product or system therefore there is a strong case for users with disabilities to be involved during the evaluation process so a product or system can be designed to be as inclusive as possible to support the wider population.

People with disabilities represent a high proportion of the potential end user population; therefore by designing a product or system to be inclusive manufacturers are opening up the end market to reach more people.  The Disability Rights Commission estimates the spending power of people with disabilities, to be at £80bn per annum. This spending power could make a strong business case for designing inclusive products or systems.  Dong et al (2006) has demonstrated that inclusive products can increase long-term profits, competitiveness, and produce higher quality products that are accessible and easy to use.

In addition to the increased consumer market it has been highlighted by Clarkson et al (2003) that by considering the needs of people with various disabilities during the product or system design, more people will be able to use the product and system as an end result.

1.7 Evaluating for accessibility and ease of use

 

Harrison et al (2006) and Alexander (2004) point out that a product or system is often evaluated for ease of use, and not accessibility, or vice versa. It is assumed that this may be because many of the literatures reviewed were found to encourage users to consider accessibility, with no guidance given on how to adapt current usability methodologies to cover accessibility. Also, many of the design guidelines used have always kept accessibility and usability apart.  This makes them less efficient as the inspection of the product or system has to be performed twice, once for accessibility and once for usability.

Alexander (2004) puts forward a persuasive case for using usability methodologies to check the accessibility of a product or system. This is because there are strong commonalities found between checking for accessibility and ease of use as pointed out by Alexander (2004). For example, both:

• Use an inspection method to evaluate the product or system with experts
• Follow design guidelines
• Use end user evaluation

When using usability methods to evaluate for accessibility, the usability methods chosen should be optimised to focus on any areas where there may be potential concern about the accessibility (e.g. tasks would focus on these areas). The inspection method chosen would include evaluators who have expert knowledge of accessibility.

The guidelines used may differ depending on whether the product or system is being checked for usability or accessibility. When checking the accessibility the "principles of universal design" is usually followed.

For end user testing and beta testing it is important that end users with disabilities are involved to ensure that the product or system meets their needs.

Although a number of methodologies exist for evaluating products or systems, it is not obviously apparent that any are thorough and specific enough to evaluate a mobile phone for accessibility across disabled user groups in any depth.

A new methodology is therefore needed to address this gap found so mobile phones can be designed to be as inclusive as possible. Through experience from using various expert methodologies it was felt that the possibility of the adaptation of an existing methodology should be investigated.

 

1.8 Heuristic Evaluation for Mobile Phones

Following on from the Short Term Scientific Mission conducted by Kristian Kristiansson, the RNIB Innovation Unit reviewed and extended their current heuristics as mentioned earlier so that they focused solely on a mobile phone.  The heuristic evaluation that was formed was broken down into three sections, hardware (e.g. the physical mobile phone), the software (e.g. the phone applications and system and the services (e.g. the web pages that a phone can access). 

To generate a complete set of heuristics too not only include blind and partially sighted people, the RNIB worked with the Swedish Handicap Institute and the Royal National Institute for Deaf people.

Once a new set of heuristics had been created they needed to be validated externally with end users.  This was required to determine whether the checkpoints written matched end users opinions.  To achieve this validation an evaluation was planned which would be conducted in the UK, Cyprus and Portugal.

The results of the user trials in the UK, Cyprus and Portugal would be used to test the heuristics to determine whether or not the heuristics are valid as a method of capturing and assessing the accessibility needs of disabled users.

This report highlights the findings of the comparison between the heuristic evaluation and the end user evaluation, which will show whether or not the heuristics matched end user opinions.


Sources:

Disability Rights Commission: www.drc.org.uk/

Universal Design Principles: www.design.ncsu.edu/cud/newweb/about_ud/udprinciples.htm

Contents

Next chapter