Cost 219ter

Technological challenges

From a technological perspective, challenges are present from many perspectives: the real availability of the foreseen technology, its integration in a working system, the interaction with the emerging environment, and the acceptability of the resulting services by the potential users.
In the European ISTAG documents, the following ‘key enabling technologies’ are considered as a basis for the emergence of an intelligent environment:

• embedded intelligence,
• middleware and distributed systems,
• IP mobile and wireless,
• multi-domain network management,
• converging core and access networks,
• micro and opto-electronics,
• trust and confidence enabling tools,
• cross-media content,
• multi-modal and adaptive interfaces,
• multi-lingual dialogue mode.

Some of them may be of paramount interest for the inclusion of all citizens. Moreover, the following requirements are considered crucial for the technological developments. The first is that the hardware must be very unobtrusive. Many technologies are conceived as hand-held or wearable, taking advantage of the intelligence embedded in the environment. This means being lightweight, but also available. People will have with them everything they need to perform any tasks. For example, the only foreseen communication item (sufficient for carrying out navigation, environmental control, and communication) is a personal communicator (P-Com). It has disembodied functionality supported by AmI with different interfaces. It is adaptive, and learns from user’s interactions. It is not necessarily a highly sophisticated piece of equipment; the performance will be limited by size, weight, and power.

The necessary intelligence can be in the environment and the only limiting factor can be the bandwidth. This hardware must be integrated in a seamless mobile/fixed web-based communications infrastructure. Complex heterogeneous mobile and fixed networks need to interoperate and to be dynamically reconfigurable. This will allow the deployment of
dynamic and massively distributed device networks. The AmI landscape is a world in which there are almost uncountable interoperating devices (wire, wireless, mobile, and fixed).

Interactions must be through natural human interfaces and with systems that are intuitive in use. This will need artificial intelligence techniques as the basis for intelligent agents and human to machine interactions, which are supposed to be multimodal (multi-user, multilingual, multi-channel and multipurpose), context sensitive and able to filter information. Finally, a very important design requirement is dependability and security. The AmI-world must be safe, dependable and secure, considering all physical and psychological threats that the technologies might imply.

Basic Technologies

One of the main prerequisites of AmI is that interactions must be multimodal. In principle, the different modalities can be used concurrently so as to increase the quantity of available information or, alternatively, to present the same information in different contexts, or, redundantly, to address different interaction channels. Inclusive design is instrumental in deciding how to use multimodality for different users in different contexts and for setting the design goals. Voice synthesis and recognition can be considered as an example. For recognition, the technological goal is the recognition of connected speech in noisy environments, allowing the production of efficient inputs for people who cannot use keyboards or object manipulation techniques. However, the training system must be robust enough to accept also voices of people with cerebral palsy. Correspondingly, voice synthesis needs not only to achieve better quality, but also to incorporate personal characteristics for the implementation of speech prostheses.

Another design target is the development of automatic translation for real-time conversation between people speaking different languages. Automatic translation has the potential of eliminating the barrier among different countries but the level of integration will depend on the languages that are considered. Different sign languages and symbolic languages, such as Bliss, must be part of the considered set.

Special vibrating materials for alerting people will increase the efficiency of alarm systems for people who cannot use auditory signals. These developments are also related to the study of materials capable of sensing touch or producing three-dimensional tactile presentations of information.

Global positioning systems (GPS) are likely to become standard in many devices and services. This will solve the problem of tracing people and will help in navigation (eg for people who cannot see). However, to be useful in all environments, GPS localisation should be integrated by the deployment of networks of sensors in closed spaces.

Smart tags can signal the presence of objects and provide detailed information about them. For example, a person who cannot see could be directly informed about the items on the shelves of a supermarket. At home, the same person could be able to locate small objects, such as a box of pills or spectacles, and have information about the medication and the dosage. The pill box could also be authorised to make the person aware of its presence.

Gesture recognition can be used to implement virtual keyboards and pointing devices, and to produce interfaces for the manipulation of objects on the screen. Traditional switches could become virtual switches. Interfaces based on the recognition of lip movements or sign-language could also be conceived. Correspondingly, animation technology could be used to produce avatars able to sign or to move lips for lip reading.

Visualisation technologies are considered of paramount importance. Screens should be available everywhere. New materials will produce screens that are lightweight and foldable and follow the user (nomadicity and availability). Alternatively, the presentation screen could be virtual, using projection systems, and the presentation of 3-D information should be possible.

The environment as a general facilitator

Some interesting conclusions can be drawn with reference to services available in the environment (eg airport, hotel, city, cafeteria). First of all, the entire AmI is a pervasive and very sophisticated environmental control, alarm and support/control system.

Then, relay services for people who cannot hear and/or speak may be available by default in the AmI environment, if voice recognition and synthesis, automatic translation, gesture recognition (sign language and lip reading) and animation (synthetic sign language and lip movements) are made available.

However the real winning factor is in intelligence. To plan her business travels, Ann (the generic ISTAG character) relies on agents that can look for relevant information and negotiate on her behalf. Her agents negotiate for a rented car at a reduced price and to obtain a discounted permit to enter the city. Another agent helps her in localising her business presentation according to local preferences (colour schemes, the use of language). These possibilities can be crucial for people who have hearing or speech problems impeding interpersonal communication or sight and manipulation problems reducing efficiency in accessing information. The possibility of delegating transactions can also be crucial if Ann is an older person with cognitive limitations.

When arriving at her destination, if Ann is not able to see, the P-Com guides her through the airport (eg by voice, or using haptic cues). This requires knowledge of her position in the airport and the possibility of controlling the presence of obstacles (people, baggage, etc) through a control system able to monitor tagged objects. If Ann has cognitive limitations, the navigation system may tune the level of support to her known abilities or to the perceived present difficulties. In the hotel, if Ann cannot see, AmI is able to describe the layout and functionalities of the room, as well as the functionalities of its devices (eg the remote control). Its functions can be simplified according to her characteristics and preferences.

When moving in the city AmI registers Ann as a client who cannot see and suggests routes and paths that are not too busy. Alternatively, if AmI registers Ann as an older client, it can connect with the control centre to verify whether she may be allowed to travel alone on the metro or she must be assisted. The level of support can be tuned to Ann’s capabilities (probably changing in time). If Ann is moving around in a wheelchair, AmI can suggest an accessible route to her destination.

At home, if Ann has cognitive limitations, she may need guidance through the required actions. With mild cognitive problems, AmI can remind and provide suggestions, just like a friend. With more severe problems, AmI can completely control the situation: preparing a balanced diet, checking the availability of food, ordering it, caring for its delivery at home, and suggesting all the steps necessary for its preparation. This can be done autonomously or in cooperation with a relative or carer.

Additional opportunities offered by AmI are related to the availability of broadband communication facilities. In the hotel room there is an audio/video system, whose video scenes are described if Ann cannot see and automatically captioned if she cannot hear. The system can be used also for communication with her daughter. The fact that Ann and her daughter are able to converse and cooperatively access information, is very important. It introduces a remote socialisation component, which can be crucial to reduce stress, and through which Ann can be supported not by technology but by another person. If Ann has cognitive problems, her daughter can instruct her when performing tasks. In this case the advantage is reciprocal, because the daughter can monitor that everything is ok without being too intrusive.

The individual interacting with the environment

It is interesting to discuss how all the characters in the scenarios can interact with the multimodal AmI. If an inclusive design approach has been used, for example if Ann is not able to see, she can receive audio messages. If she is not able to speak at all, she can use gesture recognition or text, and if she is not able to speak perfectly the voice recognition system can be trained to match the characteristics of the produced audio signal. If she is not able to hear she can receive text or sign messages.

At the same time, interaction will shift from an explicit paradigm, in which the users’ attention is on tasks, towards an implicit paradigm, in which interfaces themselves proactively drive human attention when required according to their goals. Humans will leave the execution of single tasks to intelligent computing units. Task delegation may be anticipated to significantly alleviate both the physical and the cognitive efforts required for interaction.

The user perspective

Many other aspects of the development of an intelligent environment will have impact on the population at large and on people with disabilities in particular. First of all, it is necessary to investigate how human functions will be engaged in the emerging forms of interaction and how this interaction will affect individual perceptive and cognitive spaces. The emerging environment will be very complex and stimulating, from both a sensorial and cognitive perspective. It is not clear whether people will be able to cope with the hyper-stimulation and the corresponding cognitive load. This is particularly true for people with reduced abilities, and principally for people with cognitive limitations.

Therefore, environment must be developed in such a way that the relevant social factors and the capabilities of people are taken into account, for example, in order to balance the distribution of tasks between the user and the intelligent environment itself.

To be really useful AmI must be reliable, continuously available in space and time, consistent in its functionalities and interactions in private and (crowded and potentially hazardous) public spaces. This may be a very difficult task, due to the inherent complexity of the system.

 

Chapter 1

Contents

Chapter 3