Cost 219ter logo Skip to main content

Home > Projects > Cost 219ter > Towards an inclusive future > 6. The role of evaluation of accessibility

Cost 219ter

Towards an inclusive future: Impact and wider potential of information and communication technologies

6. The Role of Evaluation of Accessibility

6.1 Introduction

By Steve Tyler

A key objective of COST 219ter is to devise a mechanism through which industry can engage with and deliver on the development of devices and systems that are accessible to the widest possible audience. Not only do we want to ensure full accessibility to people with disabilities or for those that find technology challenging to use (to allow full participation in modern society and life), but there is clear evidence that by engaging this market with products that are genuinely usable, take up and income generation possibilities are potentially extremely lucrative.

Marketing in this particular sector is notoriously challenging not least because manufacturers and service providers are not clear about the benefits of their products, not clear how to assess benefits and usability, and not clear which channels to use to guarantee maximum effectiveness. As a result, the sector as a rule has been treated as "niche" or "specialist".

This is odd since in fact users with any form of disability largely want to be a citizen and communicate with people in the same way as everyone else does. The needs of user groups are no different in terms of services in many cases - but the way in which products are used or interacted with may be very different.

Additionally, it is clear in any research in this area that good accessibility and usability design is good accessibility and usability for all users. Many of the products we take for granted today are direct results of well-designed products originally developed for the disability community; they have stood the test of time either because of good design principles or because they address user requirements that turn out to be very mainstream - OCR systems, the typewriter (qwerty keyboard of course being a natural derivative) and the original gramophone are all good examples.

Understanding the challenges of and working with the manufacturing and service provider industry has led the COST 219ter Action not only to focus on what we mean by accessibility and usability for end users, but most importantly, how we could devise systems and processes that industry could follow - systems and processes that could be easily implemented, that fit in to frameworks of testing and evaluation that already exist.

To achieve this most comprehensively, the team:

As a result of this work, this chapter describes the process by which we arrived at the development of the evaluation toolkit - a toolkit we believe is probably the most valuable way of achieving our objective - and we believe the objective of the service provider and manufacturer community - selling more good quality, well designed products that will appeal to the maximum number of users.

We adopted the following process: Having trawled for knowledge in the field, a similar exercise was conducted in the disability community across organisations and research institutions. From this work, a team of Ergonomist’s derived an industry usable methodology that could be imbedded in to standard practice and that would measure the accessibility of network services and terminals.

In new and next generation mobile services, networks are often intrinsically linked to the handsets, often using back-end systems that represent information to users appropriate to the device, browser, etc. Services and devices are inextricably linked and form a unified offering. Evaluating these poses real challenges.The next sections describe how we arrived at our methodology as well as giving a current example of accessibility testing and the benefits it can bring to design principles.

6.2 Evaluation trends

By Edward Chandler, Elizabeth Dixon and Steve Tyler

An initial piece of work was undertaken to establish what types of evaluation or usability assessments were carried out by test houses, industry laboratories and other institutions that currently conduct testing on behalf of manufacturers.

It is clear that testing and evaluation for usability or ergonomic design does take place in some domains more than others - Web Accessibility Testing for example is extremely common as an approach to legislative compliance or good user experience. This and other areas were examined to assess whether or not there were usable tools that were transferable into the telecoms industry.

Finally, an existing methodology currently being used by RNIB that adopts a pan-disability approach was modified and tested with live users to find out if it was possible to achieve a high correlation between real user experience and reported results on the one hand and the expert evaluation toolkit output on the other, modified through experience of this project, the toolkit could be validated.

6.2.1 Review of web accessibility evaluation

By Julio Abascal, Myriam Arrue, Markel Vigo, Elizabeth Dixon and Steve Tyler

From the inception of the W3C (Worldwide Web Consortium), the Web Accessibility Initiative played a vital part, in that it has played and plays a monitoring role with a keen eye on accessibility. As new guidance and recommendations are produced by Consortium members, accessibility testing and accessibility requirements are added in to the emerging recommendation.

As a result of this approach, the methodologies and toolkits developed in this sphere are instructive since accessibility is so closely allied to anything published by the consortium. Furthermore, the web as a carrier and distributor of information or a network from which telecoms and personal devices derive information, plays an increasingly important role in telecoms services - directly or as a value-added provider.

Vital lessons can be learnt from the evolution of the evaluation of web pages for accessibility. The evaluation of the web page involves the evaluator(s) initially checking the web page against a set of accessibility guidelines manually. This was found to be laborious at times and prone to errors with the guidelines being frequently misinterpreted.Therefore in order to help speed up the evaluation process and minimize potential errors some of these guidelines were automated. These tools are usually in the form of a software program or online service. Telecommunication devices may also benefit from having such tools to use as a basis for a wider approach to accessibility evaluation.

Tools for automated web accessibility evaluation (including those for repair) are currently very advanced. These tools are usually able to search the (X)HTML code to check the accomplishment of accessibility guidelines.Other features, such as CSS or XML code conformance with standards can also be automatically checked.

The Web Content Accessibility Guidelines (WCAG) 1.0 [Chisholm et al, 1999] issued by WAI are recognised worldwide as useful recommendations for web accessibility evaluation. Marincu and McMillan, 2004 state that these guidelines are now a reference point in achieving Web accessibility in many of the E.U.’s member states.

Some countries have released their own interpretation of these accessibility guidelines. E.U. member states which have included or adapted the WCAG 1.0 into their web accessibility program include:

In the USA there is the "Section 508 of the Rehabilitation Act in the USA" which provides the equivalent information to the WCAG.

Nevertheless, due to the different formats and structures used for defining accessibility guidelines, interpreting them and determining the possible conflicts between them require a great amount of effort from web developers. Moreover, the ambiguity and the low level of expressiveness in the definition of some accessibility guidelines may result in different interpretations of the same guideline. In this situation, web accessibility evaluation is a complex task which can be simplified by using an automatic accessibility evaluation tool.

There are several free automatic evaluation and repair tools such as WAVE and A-Prompt which are based on specific accessibility guidelines sets. These evaluation tools have a strict guideline specification since guidelines are implemented within the evaluation tool.However, there are some more flexible evaluation tools that are able to verify the accomplishment of any set of guidelines specified in a machine-readable language, usually a version of XML such as GXML [Abascal et al, 2004], GDL [Bereikdar et al, 2005] and GAL [Leporini et al, 2006]. This allows for easy updating to the last version of accessibility guidelines, to define specific guidelines for groups of users such as the elderly [Kurniawan et al, 2005], and also to evaluate any other characteristic that can be expressed using the adequate language.

Research to create or adapt tools that can automatically evaluate the accessibility in other domains can provide a first step for people interested in accessibility evaluation. In this way, experiences with automatic evaluation of the accessibility of web content can be transferred to a limited extent to evaluate web pages for small screens. This can be used as a first step forward to enlarging the field of automatic accessibility evaluation.In this sense, Mobile Web Best Practices are being discussed and a web accessibility evaluation tool has been adapted in order to evaluate mobile web interfaces [Rabin and McCathieNevile, 2006 and Arrue et al, 2006].

It is evident that the possibility of performing a considerable part of the accessibility evaluation automatically would make it cheaper and more popular. For this reason, efficient tools and sound methods are needed to facilitate all phases of a complete evaluation. For instance, one of the problems of manual evaluation is reporting the results in a way that is useful for developers. Therefore, evaluators – experts and users – must be provided with tools for reporting accessibility failures in an efficient and understandable way.

A conclusion that can be drawn from this is that testing and compliance or usability tools can be written for wider software applications and platforms. Accessibility checkers and methodological toolkits are being developed for Microsoft Windows programs focussing on such issues as "Is there a keyboard shortcut for all mouse commands?", "Does the software conform to standard keyboard expectations (e.g. Alt - o for open?)" etc.
 
Clearly this can be extended to mobile phone and PDA handsets - indeed, the Symbian 60 phones and most other products from Nokia conform to a common look-and-feel interface.

In the web environment, although much can be automated, experience shows that real user testing or at least a clear methodology carried out by experts is still the most accurate mechanism for establishing the accessibility level of a website and/or software.

Next generation mobile devices will become more reliant on mobile web-based or wireless services - the interaction between phone software, network based services and web-based (external) systems will become more seamless - accessibility and usability testing in all of these environments will become crucial if the end user experience is to be a pleasant and productive one.A joined up holistic approach to accessibility testing is therefore crucial.

6.2.2 Current trends in accessibility evaluation

Web accessibility evaluation has provided an exemplar of the widespread resources available which developers can use to develop accessible web sites. However, for the evaluation of other products and services, accessibility evaluation is still very much in its infancy with little resource available to assist. To determine the extent of the problem it was crucial to establish what the current trends in industry were by:

To fulfil the objectives a two stage questionnaire process was carried out: Two hundred European organisations that tested products or systems received the stage one questionnaire. Of these, only twenty-six responded. Twenty-four organisations informed COST 219ter that they conducted research into the accessibility of next generation network services and terminals for disabled and elderly people. The results showed that of the twenty-six respondents, nineteen performed evaluations in usability and accessibility, with four evaluating for accessibility only and three evaluating for usability only.Finally twenty-three organisations agreed to take part in stage two of the process. A list of organisations that test/evaluate accessibility can be found on the COST 219ter website (http://www.cost219.org).

The second stage questionnaire was distributed to the twenty-three organisations that agreed to take part. Twelve of these responded. The results showed that popular methods used included questionnaires, interviews, observations, focus groups, expert groups and panels, task analysis, user groups and user panels, scenarios, prototyping and finally heuristic evaluations. This information links well with the literature, which cites all these methods as popular [Stanton et al, 2005].

Popular areas of industry that were found to be evaluated included; mobile phones and services, telecommunication services, software, computers, PDA’s and handsets, and finally, smart homes.

The results of the questionnaires suggested that there was only a limited amount of work being carried out into the evaluation of next generation network services and terminals for accessibility with fewer than 15% responding to the first questionnaire. Interestingly, the results from the first questionnaire showed that most of the organisations evaluated for accessibility and usability with only a minority stating they only tested for accessibility or usability. Further information about the findings of the questionnaires can be obtained from the report written by Anna-Liisa Salminen on the COST 219ter website (http://www.cost219.org).

The results also showed that a variety of methodologies were used covering various domains for evaluating next generation of network services and terminals for disabled and elderly people.

Finally, the results of both questionnaires evidently make it clear that the evaluation of next generation network services and terminals is only being conducted to a limited degree. The research has demonstrated that although a number of evaluation methodologies exist and are being used, it is not obviously apparent that any are thorough and specific enough to evaluate mobile phones for accessibility and ease of use across various disabled user groups and the elderly population in any depth.

6.2.3 Methodologies used by COST 219ter members

As part of the process of looking at current trends in industry for evaluating for accessibility of next generation of network services and terminals, it was important also to investigate what members of COST219ter were currently using.The Royal National Institute of the Blind (RNIB), a member of COST 219ter, had indicated that it carried out extensive evaluations on products and systems, including mobile phones. COST 219ter felt that the methodology used by the RNIB should be reviewed to understand it in greater detail and determine if anything could be learnt from this evaluation methodology.The aim of this work was to determine if a methodology exists which is suitable for recommendation for implementation across industry by COST 219ter.

This visit identified that RNIB carried out extensive user evaluations on products to ensure that they were accessible for blind and partially sighted people. From the knowledge and expertise gained from these evaluations, RNIB created a simple set of guidelines as a means of assessing products initially without involving end users.

However, these guidelines had some shortfalls.Firstly, they were not specifically aimed at next generation mobile phones and as such were not detailed enough to cover the exact requirements needed to fully evaluate a device such as a mobile phone. This included the physical aspects unique to a mobile phone, the operating system and the services that users can access from their mobile phone. Secondly, these guidelines only considered the needs of blind and partially sighted people and finally, the guidelines were only written at a very top level without any detail. Further information about the findings of this work can be obtained from the report written by Kristian Kristiansson on the COST 219ter website (http://www.cost219.org).

After further analysis, it was decided that the evaluation method used by RNIB showed potential for being used by COST 219ter for evaluating next generation mobile phones. In order to fulfil the objectives of the Action it was recommended that these guidelines be adapted to address the above issues.

Analysis showed that the specialist organisations that carry out evaluations of any description in the blindness sector across Europe did not have as robust a mechanism as used by RNIB and it was clear that certainly within the sector in Europe, its techniques were verifiable, consistent and replicable in terms of methodology and result.

6.2.4 Suitability of these methodologies for implementation across industry

The results of the COST219ter research showed that at present there is no evaluation methodology that addresses the needs of disabled and elderly users for ensuring that next generation mobile phones are designed to be inclusive. Therefore a toolkit is needed to address this gap to ensure that mobile phones are developed to be as inclusive as possible.

Whilst this toolkit needs to evaluate accurately and effectively the accessibility of a mobile phone, it also needs to be usable by the person or persons using it. Without the usability of the toolkit being accounted for, there would be a danger of having a solution which would not be accepted by the telecommunications industry due to its own complexity. Therefore any toolkit should not assume prior expertise or knowledge, yet it should be easy to understand and complete.

The research has shown that a toolkit could be proposed using RNIB’s evaluation guidelines as its foundation. Such a toolkit would need to encompass the accessibility needs of various disabled user groups and also need to be developed so that it is aimed specifically at next generation mobile phones.

The example of automatic evaluation has been advantageous in web site accessibility. This example deserves attention to determine if similar tools can be produced to evaluate the accessibility of mobile phones particularly for evaluating the accessibility of mobile web pages. This would satisfy manufacturers who do not wish to extend design and manufacturing times whilst satisfying the needs of users that are currently being excluded from using mobile phones.

6.3 Mobile phone evaluation toolkit

By Edward Chandler, Elizabeth Dixon and Steve Tyler

The toolkit is a simple methodology which is capable of evaluating the accessibility of a mobile phone. The toolkit comprises three sets of heuristics which cover the basic accessibility features of the physical part of the phone, the operating system and the services.

Each set has approximately 14 heuristic principles (rules of thumb).For each principle, checkpoints are given to objectively measure the accessibility/usability of the device against a set pass/fail criteria. Accompanying each set of heuristics is a completion form which includes a fail criterion for each checkpoint.It also shows whether or not the checkpoint is a high or low priority and provides a comments section. Finally, it includes a section for any additional information and recommendations for future work on that phone. The heuristic principles are as follows:

Hardware principles

Software principles Web services principles Further information about these heuristics and their checkpoints can be obtained from: www.cost219.org/toolkit.

It is important to remember that these heuristics may be applied at any time during the design phase or at later stages of the development process or once a product has been released to market. However, it is recommended that these guidelines should be applied as soon as an initial mock up or paper prototype of the device becomes available to minimise design costs.

This toolkit should never be used in isolation as a total evaluation process for evaluating mobile phones. It is intended to highlight basic accessibility issues which currently inhibit disabled users from using a mobile phone.

6.3.1 Development

When the toolkit was in development, it was clear from previous work that the scope of the toolkit needed to be closely specified, it should be a simple methodology capable of evaluating the accessibility of a mobile phone and finally it would have to be validated against end user requirements.

Prior to commencing work on the toolkit, it was important to carry out a scoping exercise defining the parameters of the toolkit. Evaluating a mobile device could potentially include a vast array of possibilities. Therefore as part of this process the following questions were addressed:

To take all these points into consideration, adapting and creating a toolkit to address every functionality/service would be impractical.Therefore rather than adapting the toolkit to cover every possibility, the scope of the mobile phone evaluation toolkit was limited to cover: Additional guidelines would be needed in the future to cover other functionalities such as GPS software. These have not been included at this stage as it is likely that these would require separate guidelines specific to technologies and deployed services. Another area likely to require guidelines is for the creation of accessibility advice for SMS services. However, it is vital to focus on the core functionality of the phone first so that disabled and elderly users can use the phone.

The toolkit includes a list of guidelines for creating accessible mobile web services that are provided over the mobile internet. These guidelines build on the foundations already in place for web accessibility for larger screens.

In order to develop a simple methodology it was decided that detailed checkpoints should be provided under each heuristic principle to guide the evaluator completing the evaluation with the type of issues to check for, as well as providing a completion form alongside the heuristics for ease of completion. During the development of the toolkit the heuristic principles were adapted to accommodate their new checkpoints for each principle.

Several types of mobile phones were analysed during the development of the heuristics and their accompanying checkpoints so that a range of different phone features could be accommodated by the methodology.

As part of this work the RNIB worked alongside other parties close to disabled end users. These included the Swedish Handicap Institute, The Royal National Institute for Deaf people (RNID) and Special education and Rehabilitation department of faculty of Human Kinetics (FMH), Lisbon.The Swedish Handicap Institute drafted their own set of heuristics for people with learning difficulties.RNID and FMH, Lisbon provided recommendations to consider the needs of deaf and hard of hearing users, people with limited dexterity and people with learning difficulties. Each organisation provided recommendations in the form of checkpoints which they felt needed to be added. These were inserted under the relevant heuristic principles.

It was vital that the guidelines accurately reflected the accessibility issues that most users frequently found so these guidelines could be used to accurately evaluate a mobile phone. This would enable evaluators with practically no prior knowledge of accessibility issues of mobile phones to be able to use these guidelines to identify and resolve frequently found accessibility issues.

Once a full set of heuristics was available, it was essential that the heuristics were validated to determine whether they accurately evaluated the accessibility of a mobile phone. This validation involved comparing the results of the heuristic evaluation against end user requirements. The results of the comparison would show whether the heuristics and their checkpoints were successful in identifying accessibility shortfalls in a mobile phone.

6.3.2 End user evaluation and toolkit validation

The aim of the validation was to identify whether or not the toolkit was successful at identifying the accessibility issues which users found during the evaluation.

In order to achieve the validation of the toolkit, a project was organised which would perform an evaluation of a mobile phone using the toolkit as well as an in-depth end user evaluation. This study encompassed four user groups:

These users were recruited by Intercollege in Cyprus, the Special education and Rehabilitation department of faculty of Human Kinetics (FMH) in Portugal, RNID and RNIB in the UK. The results were gathered using focus groups, interviews and questionnaires for each group of end users. The results of the user trials in the UK, Cyprus and Portugal were amalgamated and then compared with the results of an expert evaluation using the heuristic principles. Further information about the findings of this work can be obtained from the report written by Edward Chandler, Elizabeth Dixon, Leonor Moniz Pereira and Cristina Espadinha on the COST 219ter website (http://www.cost219.org).

During the evaluation, there were a number of unexpected results. A summary of these has been highlighted below as well as a summary of the validation study as a whole.

In general, the blind and partially sighted participants found they could dismantle and assemble the phone although most had not done this before. These participants stated that they would get a friend/relation or shop assistant to dismantle and reassemble the phone due to the perceived complexity of the task.

As well as this, most of the UK based blind and partially sighted people had avoided predictive text before the evaluation as they found it impossible to use. However, the majority of the blind and partially sighted group thought that using predictive text was fairly easy to do and would consider using it more. These results were mimicked by the participants with learning difficulties too.

Other unlikely results were linked to menu customisation. The results showed that most users wanted to be able to move menu items as required.This showed that these users liked to make the phone easier to use for them in terms of menu structure. Linked to this, there was almost a 50/50 split between whether to display the menu as a grid format or as a list format.These results highlight the need to make the operating system of the phone highly customisable.

Finally, the results showed that nearly all of the participants liked to change profiles on their mobile phone. There was a strong demand from end users to be able to alter how the mobile phone responds in different environments. This highlighted the importance of user profiles and the benefits of user profiling.

The comparison of the results between the toolkit evaluation and the end user evaluation confirmed whether or not the toolkit provided guidance for the type of problems that users typically found. It showed where the toolkit matched typical problems that users found, as well as highlighting any areas where the results did not match. Checkpoints which did not match were highlighted for review.

The validation study has shown that generally the toolkit match what users with disabilities require to determine if mobile phones are accessible and easy to use. The study also identified that additional work was needed to refine some elements of the checkpoints further. This would make the toolkit more precise.

6.3.3 Outcomes

An evaluation methodology has been designed which can assess the accessibility of a mobile phone. This evaluation methodology uses a series of heuristic principles with detailed checkpoints, which can be used to identify the level of accessibility of a mobile phone. The checkpoints have been specifically designed to evaluate the features of a mobile phone only and would need to be modified further to evaluate another product or system.

This methodology can be used to evaluate a mobile phone when it is being designed, the most ideal use of it being during other testing protocols or design checking, or when it is out on the market. It can also be used by someone who has limited or no prior knowledge of heuristics and evaluation for disabled people as the toolkit guides the person completing it.

As the toolkit has been validated with end users, this is a robust method of evaluating the accessibility of a mobile phone.

We will monitor the effectiveness of the toolkit as new generations of technology emerge and modify it accordingly.

6.4 Case study: an example of how evaluation may change the design: ATM project in Portugal

By Leonor Moniz Pereira, António Cordeiro, João Brisson Lopes, Cristina Espadinha, Manuel Ribeiro

6.4.1 Introduction

In today's society ATM machines play an important role in providing financial services at any time, facilitating daily life routines to a large number of people. However, people with disabilities and elderly people may experience many problems using them.

In fact some services may be unavailable to certain groups, e.g., payments and other services not designed with the needs of people with blindness, motor impairment, learning difficulties, those who are cognitively impaired or even the elderly in mind during design.

According to Gill (1998) people with disabilities and elderly people may have accessibility problems using ATMs at least in one of the following behaviours:

Some disabilities have a stronger direct impact than others on the individual ability to use ATMs. The ability may also change in accordance with the level of impairment.

With the intention of improving the situation of ATM usage for disabled people in Portugal, SIBS (Portuguese Inter-bank Society) sponsored the project. SIBS is responsible for the software of all ATMs available in Portugal, and influences the acquisition and the installation process of all the service providers

The project was developed and coordinated by the Technical University of Lisbon during 2005.Three faculties were involved in this project: the Design department of Faculdade de Arquitectura (Faculty of Architecture), the Computer Science department of Instituto Superior Técnico - IST (School of Engineering) and Special Education and Rehabilitation Department of Faculdade de Motricidade Humana - FMH (Faculty of Human Kinetics).

6.4.2. Objectives and Methods

The project aims were:

In order to identify the needs of people with visual, motor or cognitive Impairment and elderly people, a set of questions were generated: In order to answer these questions, an inclusive design approach was used which focused on the following work areas: In order to cross-reference information and check data consistency, a combination of methods was used (interviews, questionnaires, focus groups and observation of user performance).

To prepare the interviews, the questionnaires and the user trial protocol, a survey of ATM’s was carried out as well as a task analysis of the different ATM services focusing on the interaction process (by vision, touch, sound and keyboard usage).

Considering the critical points observed previously, interviews with two users of each group were carried out.These were used to identify potential missing aspects and to ensure that the user’s point of view was taken into account.Based on this, a questionnaire was developed in order to identify difficulties and constraints of ATM usability.

The results from the interviews and questionnaires were applied to a trial phase where alternative hardware and software solutions were assessed and validated with an ATM test bed.For this phase, users were asked to trial the most commonly used services and express their opinions.

Finally, a set of recommendations was established.

6.4.2.1. Questionnaire

The questionnaire comprised three parts. The first part dealt with user data (gender, age, profession, level of education, etc). The second part dealt with ATM usage focusing on the critical issues enumerated below and the third part related to improvements that the user would like to see in the future.

The critical issues in Part II of the questionnaire cover:

The majority of the questions required a “yes” or “no” answer or a classification using a 1 to 5 Likert scale. In a group of “yes” or “no” questions there was also an area for users to highlight the three most critical issues. It was also possible to add additional comments that the users considered important which had not been elicited.

The questionnaire was distributed to associations of disabled persons, day centres, clubs and other institutions frequented by elderly and disabled persons. It was completed by the user or by the researcher if he/she was not able to do it by him/herself. An electronic version of the questionnaire was developed and distributed to blind people.

6.4.2.2. User Trials

The main objective was to check if the potential solutions for the user’s problems improved significantly or made ATM usage possible.

The trial was composed of three stages. An interview took place to identify the users’ main concerns and priorities in relation to ATM usage; the users performing two scenarios, which were explained to the users beforehand; interactions were tested.

User performance was recorded in two different ways. An automatic log file was produced every time a key was pressed (giving information about the number of mistakes and time spent to perform the task), and information was noted regarding quality of the performance (effort, insecurity, and remarks). The last stage consisted of a final interview to evaluate accessibility and to identify the most important issues from the user’s point of view.

6.4.2.3. Sample

The questionnaire sample was completed by 346 disabled people. Of those, 74 were blind or partially sighted (44 were blind and 30 partially sighted), 74 had motor impairments (25 were wheelchair users, 28 used assistive walking devices, 9 were amputees, 7 had limited dexterity and 5 used an electrical wheelchair and had limited dexterity), 118 were cognitively impaired and 80 were elderly people (aged 65 or over). 206 of the participants were ATM users and 150 were not. Deaf and hard of hearing people were not included in this study as all ATM terminals in Portugal can be operated using visual information.

The user trial group included elderly people, blind and partially sighted people, people with learning difficulties, people with limited dexterity, people with limited arm movements (range of movement in the elbow and shoulders) and people with limited mobility.

A sub-sample of 24 people was selected from the participants who had completed the questionnaire to perform the user trial. The purpose of this was to determine if the solutions enable disabled users to use ATM's or improve usage for all users or whether some of the alternatives would prove problematic for other user groups. The sample consisted of people who experienced at least one of the following barriers/perspectives:

6.4.3. Summary of Results

The participants were asked to provide reasons why they wouldn’t use an ATM. Below is a summary of these reasons.

Fear of being robbed:

Not handling their own money or needing help getting money from the ATM: Not trained to operate the ATM: Could not use the ATM (reach the buttons, see the screen or get to the ATM): The ATM is installed to high from the ground: The Card entry slot is installed too high from the ground: The keys situated next to the screen could not be reached: The participants were asked to comment on the equipment design and installation of the ATM. Below is a summary of the comments by participant group.

Comments by visually impaired participants on the design and installation of ATM’s: Comments by mobility impaired participants on the design and installation of ATM’s: Comments by Mentally Impaired participants on the design and installation of ATM’s: Comments by elderly participants on the design and installation of ATM’s: A summary of the main interaction difficulties experienced by visually impaired participants: A summary of the main interaction difficulties experienced by mobility impaired participants: A summary of the main interaction difficulties experienced by mentally impaired participants: A summary of the main interaction difficulties experienced by elderly participants: The following comments were also obtained about the design and installation of the ATM: Practical application of evaluation and recommendations
The users involved in the trials did not show signs of dislike or discomfort towards the adaptations of the ATM, for instance, the ATM installation in lowered position for tall users.

Moreover, the lowered ATM installation was considered an advantage for small users because it allows them to use their body as a shield against sun reflexes or against other people that can stare at the operations with bad intentions.

The evaluation showed the following solutions as being beneficial to the users: In summary, the main recommendations of the research were:

6.4.5 How evaluation will change the ATM project design in Portugal

The project finished with a presentation of the results to SIBS and all the recommendations and results were welcome.SIBS asked the partners to disseminate the findings to the Portuguese banks in order to facilitate the establishment of the installation procedures. Moreover, they have shown interest to influence the next ATM machines acquisitions of more accessible models. At the present moment SIBS is analysing its software and upgrading it in order to reflect the project recommendations.

6.5 Recommendations

By Steve Tyler and Edward Chandler

This chapter has identified that involving users and evaluating for accessibility and usability is paramount so that the needs of the users are met. This work has identified a number of recommendations which should be carried forward by the telecommunications industry and interested standards groups to achieve inclusion by the largest audience possible in services and terminals for next generation mobile networks. These are outlined below.

1. 
     
Utilisation of the Toolkit: the toolkit has been developed for utilisation by testing laboratories, designers, in-house evaluation, etc. The toolkit has been shown to be robust and accurate and it allows industry and manufacturers alike to get a broad idea of product fit to user requirements. The approach adopted is derived from a well tested methodology focused in determining the accessibility of products and services by people with disabilities.

2.    
  
User Interface Profiles: the concept of user interface profiles is further developed and adopted by service provider and manufacturer communities (this concept can be broadly defined as: "a set of parameters that can be altered or implemented that allow a device to deliver or receive information using alternative interfaces"). See opportunities section for more information.

3.

User Interface Consistencies: as the links between service provider and manufacturer become ever tighter and more seamless, so the need for consistency in operation becomes greater, certainly ideally within manufacturer products.A simple example of this might be the long key press of key 1 access voicemail as a speed dial function, or menu behaviours that are consistent throughout the system. The simplest test of this concept is to take any two of a single manufacturer's products - by learning the first, it is theoretically possible to use the second.The best examples of user interface consistency include Nokia Series 60 and 80 products.
 
It is recommended that:

In addition, it is further recommended that a certification mechanism is utilised, ideally one currently in existence, that embraces accessibility and user requirements of this kind. Mechanisms of this nature do currently exist such as in the Microsoft certification process; the Symbian Signed process is an example of a mechanism that could be used.

4.     
 
User Testing: Although the toolkit is designed to give accurate and robust, replicable information, user testing and user panels are ideal ways of ensuring that a product really does meet user requirements in terms of access. There are various ways of obtaining these and information is available at http://www.tiresias.org to facilitate field tests of this nature. It is recommended that user trials should be built into any manufacturing and development process encompassing the needs of people with disabilities.

5.      

Toolkit Guidance Development: It is recommended that: 6.

Within the toolkit developed by COST 219ter and in all research surrounding the area of accessibility, it is clear that the issue of access by people with varying abilities can only satisfactorily be dealt with through implementing optional alternatives to information output and input. This can be done through building support into devices that enable multimodal input and output. It is recommended that: The recommendations above support access to telecommunications systems by communities of people who would otherwise be classified as a “niche market”; by adopting open and supportive standards, people using specialist or niche assistive devices can access mainstream products through these bridging tools.

7.

As an absolute minimum it is recommended that: In order for mainstream access to become a reality for the vast market of people with disabilities, standard advertising and delivery through retail outlets is a way of breaking the circle which currently exists - There is no market as there are no products. This of course does not mean to negate preferred look and feel of products that manufacturers wish to promote - in fact, entirely the opposite; in general, people with disabilities or older people want to access the same devices, services and systems that friends and colleagues are accessing.

8.

To commit to continuous and structured dialogue and consultation between industry bodies and all users – including people with disabilities and older people. This mechanism would review and report on the progress in activities aimed at building a full inclusive telecommunications sector.

9.

It is recommended that:

6.6 Opportunities and Challenges

By Steve Tyler, Edward Chandler and Tony Shipley

As a result of the research and work carried out in COST 219ter and specifically the work attached to the toolkit development, a number of issues have emerged which would seem to present opportunities for research, development and standards groups as well as some challenges to us as a telecommunications industry in meeting the needs of users. These are outlined below.

1.

User Interface Profiles: as an output to research being carried out by COST 219ter, the concept of "user interface profiles" appears to be gaining momentum and from a user perspective the potential of this development is extremely great.

This concept can be broadly defined as:"a set of parameters that can be altered or implemented that allow a device to deliver or receive information using alternative interfaces".

The simplest example of how this approach operates is as follows:

Case Study 1

A totally blind user on receipt of the phone switches it on. The phone displays and announces itself and offers various settings either in user oriented language or interface oriented language. For example:"How would you prefer to access the screen - select from the following options".The options may include audio output. If audio output is selected the various third party screenreader offerings can be made and downloaded on request to the phone.

Case Study 2

A user who has dexterity problems accesses the phone for the first time.On this occasion, he has a specially designed point and click device.On being asked how he wishes to input information to the phone he might pick the "alternative access device".A selection of these would be displayed and the appropriate drivers downloaded to the phone.

Case Study 3

A user has vision. Figures released from mainstream sources (Microsoft) (Vodafone) indicate that just below 50 per cent of users cannot access the screen, either due to not being able to understand the concepts, or not being able to read the letters on screen. Simple magnification, colour contrast, iconic based menu items, extra written help, etc, are possible solutions for this user.

2.

Out of the Box: although it is vital to ensure good usability and accessibility functionality in terminals and services, it is vital to ensure accessibility has been considered from the point of customer interaction with the sales chain and onwards through the experience.For example, the idea of large print, audio or Braille user manuals being available or an audio help system (phone based), obvious and unconfusing ways that allow connection to power chords or chargers, consistent placement of sockets with obvious visual and raised symbols, are crucial considerations in design.Uncomplicated pictorial or descriptive text in Getting Started manuals mean a quick start as well as easy access to those better suited to accessing text compared to graphics, and vice versa.

The out of the box experience for a person with disabilities or alternative user requirements is at least as important as that of an able-bodied person.The dividends in getting it right first time though are very much greater - user loyalty to the brand is likely to be substantially greater if a product appears to be accessible from the outset without a struggle.

3.
Devices are able to deliver any piece of information in at least two modalities (a u dio, vis u a l, tactual etc).Subdivisions may also be implemented when considering any one of the modalities (for example various fonts, background and foreground colours, size of text and icon versus text information; or in the audio modality, volume, pitch, speed, level of punctuation).

4.

There is a need to research and develop a set of acceptable logos, kite marks or certification which clearly articulate to the consumer the type of accessibility offered by the product.

5.

There is a need to put in place an ongoing mechanism to review requirements for accessibility features, standards and guidelines, providing up to date and clear guidance on product and service development in a dynamic marketplace. This ongoing work could be supported by industry and part of its commitment to accessible design.

6.

Equality and disability access:in a number of countries notably the United Kingdom, USA, Canada and Australia, among others, varying levels of disability discrimination legislation as well as additional duties such as the Disability Equality Duty mean a need to comply with a variety of legislative frameworks.It is clear however that although different requirements exist at present the need to address access for people with disabilities is rising up the agenda on a global level.We need to devse a mechanism that allows global manufacturing and service provider companies to make decisions in developing products safe in the knowledge that legislative duties are met.

General anti-discrimination legislation of the kinds mentioned imposes an obligation to treat a disabled person no worse than any other member of the public. The manner of treatment is not specific and anyone who can prove their case can claim redress in the courts of civil law, but only within the scope set by the legislation.The courts will come to a view as to what does constitute discrimination in particular contexts, usually through a body of established case law, and this will develop over time as cultures and technologies change. Anyone defending such an action may cite published standards or codes of practice to show that their attitude had been reasonable, so it is crucial for all parties that these standards and codes are updated in line with public expectations. Compliance with specific standards or codes might be a regulatory requirement in certain jurisdictions and a demonstration of this compliance - by use of test and appraisal procedures - would satisfy the regulator. It would not necessarily satisfy the civil courts, especially if the regulatory requirements were out of date or inadequate. Awareness of disability issues and the state of the art actions would be the best safeguard.

There is no easy answer - suppliers have to be aware.

6.7 References

A-Prompt Project. http://aprompt.snow.utoronto.ca/

ABASCAL, J., ARRUE, M., FAJARDO, I., GARAY, N. AND TOMAS, J., (2004). Use of Guidelines to automatically verify web accessibility. Universal Access in the Information Society. Vol.3, No.1, pp. 71-79.

ARRUE, M., VIGO, M.& ABASCAL, J., (2006). Automatic Evaluation of Mobile Web Accessibility. Accepted for publication in the 9th ERCIM Workshop "User Interfaces for All". LNCS. Springer, 2006.

BEREIKDAR, A., KEITA, M., NOIRHOMME, M., RANDOLET, F., VANDERDONCKT, J. AND MARIAGE, C., (2005). Flexible Reporting for Automated Usability and Accessibility Evaluation of Web Sites. Proceedings of 10th IFIP TC 13 Int.Conf. on Human-Computer Interaction, INTERACT 2005, M.-F.Costabile, F. Paternò (Eds.), Lecture Notes in Computer Science, Vol. 3585, pp.281-294.Springer-Verlag, Berlin.

CHISHOLM, W., VANDERHEIDEN, G., & JACOBS, I. (Eds.), (1999, May 5). Web Content Accessibility Guidelines 1 .0. http://www.w3.org/TR/WAI-WEBCONTENT/.

ENGELEN, J., FLETCHER, R., GILL, J., KLUSELL, L., LINDSTRÖM, J-I, SILVER, J. & SUTHERLAND, B. (n.d). A Vision of Financial Services: Access to Financial Services for Visually Disabled People. RNIB Scientific Research Unit.

GILL, J. M., (1998). Access Prohibited? Information for Designers of Public Access Terminals. RNIB Scientific Research Unit.

GILL, J. M., (2001). Keeping Step - Scientific and Technological Research for Visually Impaired People. RNIB Scientific Research Unit.

GILL, J., (2004). Access-Ability: Making technology more useable by people with disabilities. RNIB Scientific Research Unit.

INCLUDE – Inclusion of Disabled and Elderly in Telematics (1999). A Step Forward – Design for All. Sweden.

KURNIAWAN, S. & ZAPHIRIS, P., (2005). Research-derived web design guidelines for older people. In proceedings of International ACM Conference on Assistive Technologies 2005, ASSETS, 129-135.
 
LEPORINI, B., PATERNÒ, F. & SCORCIA, A., (2006). Flexible tool support for accessibility evaluation. Interacting with Computers. Vol. 18 No. 5, Pp. 869-890.

MARINCU, C & MCMULLIN, B., (2004). A comparative assessment of Web accessibility and technical standards conformance in four EU states. First Monday. Vol. 9, No. 7. http://www.firstmonday.org/issues/issue9_7/marincu/.

RABIN, J. AND MCCATHIENEVILE.C.(Eds.), (2006, June 27). Mobile Web Best Practices. (W3C Candidate Recommendation) http://www.w3.org/TR/mobile-bp/.

STANTON, N., SALMON, N P., WALKER, G., BABER, C., JENKINS, D., (2005). Human Factors Methods; A practical guide for Engineering and Design, TJ International Ltd.

Section 508 of the Rehabilitation Act. http://www.section508.gov/index.cfm?FuseAction=Content&ID=12#Web.

Web Accessibility Initiative (WAI). http://www.w3.org/WAI/.

Web Accessibility Versatile Evaluator 3.0 (WAVE). http://www.wave.webaim.org/.



Previous

 

 

Last updated: 20.11.2007    © Copyright reserved