Augmentative and alternative communication software with eye tracking

OData support
Supervisor:
Lajtha András Balázs
Department of Telecommunications and Media Informatics

The Augmentative and Alternative Communication (AAC) is a method of the supporting communication, which is used to supplement or replace speech for those with impairments in the production or comprehension of spoken or written language. The most common forms of the AAC is the sign and the gesture language, but not everyone is capable of using them, due their physical disabilities. These people, especially the children, are the target users of my application, which helps them in the communication with the outside world.

In the project I have created a software which grants an eye gaze communication board for the users. These communication boards are built from messengers. A messenger is a pair of an image and a matching meaning. If a user looks at a messenger, then the system, depending on its type, will read the meaning of the messenger or open it. The users are also able to navigate in the messenger structure with their eye gaze. There are two forms of navigation: a simple scrolling between the messenger pages, and a hierarchical navigation in the messenger structure. In the structure hierarchy a messenger mostly contains semantically connected ones. The hierarchy can run through multiple levels. The users also have the opportunity to create sentences from a custom set of messengers, and also send these sentences to other users in a chat message. I have created a manager page as well, where the users can create, modify or delete their messengers, and they even have the opportunity to share them with others. I have also created a logging system, which logs the user activates, and a recommendation system, which creates global statistics from these log entries. The purpose of the recommendation system is to help the users in the building of the communication board by giving advices about the relevancy of the certain messengers, and by helping to choose a suitable image which match to the meaning best.

Downloads

Please sign in to download the files of this thesis.