KONTAKT - Intelligent eye-controlled communication aid

People with limited communication skills need support to communicate and participate in life. Technological aids, such as tablet computers with on-screen keyboards and speech synthesizers based on eye control, are often slow and frustrating. How can these technologies be improved to make communication faster and more intuitive? How can taking context into account improve speech speed and fluency?

The aim was to enable users to communicate more quickly and easily. This was achieved by developing a user interface specifically for operation via eye control. In addition, we integrated a user-specific knowledge base and contextual knowledge from the user's environment into the system. By using artificial intelligence, a context-based word and sentence suggestion system was developed that provides the user’s system with additional information from the environment and thus generates faster word suggestions. 

In the KONTAKT project, we worked to understand the experiences and needs of the user group. We conducted interviews to determine individual requirements based on context of use analyses. Based on this, we developed and evaluated various interaction concepts for the individual components of the system, including the presentation of predicted words and selection strategies.

One of the biggest challenges was the integration of context-sensitive word and sentence suggestions. Originally, a speech recognition module and camera-based object and person recognition were considered as input for these suggestions. However, after feedback and evaluations, the priority of the camera in the project was lowered, and the work focused instead on more promising approaches such as the microphone, the word suggestion system based on it, and a grammar filter.

We also looked at the ethical, legal, and social implications (ELSI) of the technology. In particular, the legal requirements for the use of the technology were developed in collaboration with lawyers.

By combining human expertise and technological innovation, a demonstrator was developed that can serve as an efficient communication aid to facilitate the participation of people with limited communication skills in life.