Cost 219ter

4.0 Results

The main aim of this evaluation was to determine whether the heuristic guidelines on mobile phone accessibility match what end users say thought about the evaluation mobile.

The results can be split into two sections, hardware results and software results.  Due to a lack of results for the service based heuristics these could not be accurately tested so cannot be discussed further.

Section 4.1 below highlights the number of tested checkpoints for the hardware heuristics which match end user results and the number of tested checkpoints which need to be reviewed. Section 4.2 highlights the same but for the software heuristics. The checkpoints requiring review means that they either did not match the end user results or they matched in one part (.e.g. the focus group) but did not match in another part (e.g. the questionnaire). Therefore it is important to review them to determine if they go far enough or are required at all.

Finally, when comparing the results, it was seen that some answers from the end user trials were not covered at all by the heuristics/checkpoints.  These have been highlighted as "potential new checkpoints."

4.1 Results from the Hardware Heuristic comparison

 

Heuristic 1 (The product needs to be easy to orientate), did not have any checkpoints tested.
Heuristic 2 (The printed visual information needs to be clear) did not have any checkpoints tested.
Heuristic 3 (The non printed visual information needs to be clear) did not have any checkpoints tested.
Heuristic 4 (The buttons, switches, dials, and joysticks need to be easy to locate visually) had two applicable checkpoints tested with 50% of them matching and 50% needing review.
Heuristic 5 (The buttons, switches, dials, and joysticks need to be easy to locate touch) had three applicable checkpoints tested with 67% matching and 33% needing review
Heuristic 6 (The connectors, sockets, battery, battery compartment/back cover, SIM card holder, memory card and memory card holders need to be easy to locate by sight) had seven applicable checkpoints tested with 71% matching and 29% needing review.
Heuristic 7 (The connectors, sockets, battery, battery compartment/back cover, SIM card holder, memory cards, memory card slot need to be easy to locate by touch) had seven applicable checkpoints tested with 57% matching and 43% needing review.
Heuristic 8 (The other tactile information needs to be clear (texture/size/differences) had one applicable checkpoint tested and it matched.
Heuristic 9 (The audible information needs to be clear) did not have any checkpoints tested.
Heuristic 10 (The battery, SIM card and memory card must be easy to use) had eleven applicable checkpoints tested with 27% matching and 73% needing review.
Heuristic 11 (The product needs to be physically easy to use) had five applicable checkpoints tested with 80% matching and 20% needing review.
Heuristic 12 (Compatibility) had six applicable checkpoints tested with all of them matching.
Heuristic 13 (Instructions) had two applicable checkpoints tested with all of them matching.
Heuristic 14 (Packaging) did not have any checkpoints tested.

4.2 Results from the Software Heuristics comparison

Heuristic 1 (Optimise the input method) had two applicable checkpoints tested and both matched.
Heuristic 2 (Optimise the output method) did not have any checkpoints tested.
Heuristic 3 (The non printed visual information needs to be clear) had one applicable checkpoint tested and it matched.
Heuristic 4 (Allow personalisation of the input) had four applicable checkpoints tested with 25% of them matching and 75% needing review.
Heuristic 5 (Allow personalisation of the output) had two applicable checkpoints tested with all of them needing review.
Heuristic 6 (Allow personalisation of the alerts) had four applicable checkpoints tested with all of them matching.
Heuristic 7 (Promote flexibility) had five applicable checkpoints tested with all of them matching.
Heuristic 8 (Promote consistency) had three applicable checkpoint tested with 67% matching and 33% needing review.
Heuristic 9 (Provide easy to understand menus) had seven applicable checkpoints tested with 71% matching and 29% needing review.
Heuristic 10 (Provide intuitive prompts and informative feedback) had one applicable checkpoint tested and it matched.
Heuristic 11 (Promote easy to use Softkeys) had six applicable checkpoints tested with all of them matching.
Heuristic 12 (Error prevention and error recovery) did not have any checkpoints tested.
Heuristic 13 (Promote Easy Connectivity and Compatibility with other devices and/or software) had one applicable checkpoint tested and it matched.
Heuristic 14 (Aid Task Completion) had one applicable checkpoint tested and it matched.

4.3 Results not covered by the Heuristics

 

The end user evaluation also highlighted one area that the hardware heuristics missed out and one area that the software heuristics missed out.  These areas need to be reviewed with the possibility of two new heuristics points being included in the appropriate places. They are as follows:

• Tapered keys are better (Hardware)
• Any opening (such as a memory card lid) needs to be easy to open (Hardware)
• The instructions must be written in plain language (Hardware)
• Ability to have an on screen shortcut bar (Software)
• Ability to write text messages in the users preferred language (BLS or Braille grade 2) and the phone to translate it. (Software)

Contents

Next chapter