Task Assistance for Persons with Cognitive Disabilities

2008-10 till 2013-04
Research Areas: 

Persons with cognitive disabilities like dementia, Alzheimer's disease or autism are often not capable of fulfilling activities of daily living (ADLs) which leads to a limitation or even a loss of independence. In TAPED, we develop an automatic prompting system for persons with cognitive disabilities in the healthcare domain. The system assists the user in the daily routine of brushing their teeth while prompts are delivered if necessary, allowing for a flexible task execution.


Methods and Research Questions: 

A major research question in TAPED concerns making decisions under uncertainty. In our particular case, the system has to decide whether the user needs to be assisted in the execution of the task while the user's progress in the task is not fully observable by the system.

In TAPED, we focus on an application-oriented way of research. We established a close cooperation with v. Bodelschwinghsche Stiftungen Bethel as a large care facility in Bielefeld. In particular, we cooperate with Haus Bersaba which is a residential home for 35 persons with different cognitive disabilities, amongst others epilepsy, dementia and autism. In collaboration with the caregivers of Haus Bersaba, we develop an assistance system that guides the inhabitants through the task of brushing their teeth. The chosen task combines relevance in everyday life on the one hand and offers exciting and challenging research questions in the area of cognitive interaction technology on the other hand.

A major challenge in the development of an automatic prompting system involves making a decision as to whether prompting a user is necessary or not. A decision making component needs to detect problems in the execution of a task based on models including both task-related and environment-specific knowledge. The system has to monitor the user's progress in the execution of the task dealing with uncertainty in the perception since user behaviors underlie large individual variation. Models representing the task and its subunits have to cope with this variation. In addition to inter-personal differences the intra-personal variation, which is characteristic for persons with cognitive disabilities like dementia and Alzheimer's disease, extremely increases the complexity of perception. We use a combination of external sensors (e.g. cameras, microphones) and integrated sensors (e.g. accelerometers) embedded in tools to estimate the user's behaviors. Based on the perception of the user's behaviors the system provides assistance in terms of interaction mechanisms which have to comprise several levels of escalation and intensity, highly dependent on the mental state of the user. From a technical point of view, an interaction mechanism combines multimodal cues to guide the attention of the user, e.g. verbally given commands, auditory alerts, visual signals such as pictograms or visual pointers as indexical gestures.

We aim to assess the utility of the system and to evaluate the system's behavior in a real-world scenario by conducting user studies with the inhabitants of Haus Bersaba.



We constructed a washstand setup and equipped it with sensor technology. We conducted a first study with the inhabitants of Haus Bersaba using a Wizard-of-Oz methodology. The study provided us with a teeth-brushing interaction data corpus and first insights into the reaction behavior of users when faced with an automatic prompting system instead of a caregiver. The results help us to improve (1) the perception of the user's behavior based on the recorded data and (2) the interaction between system and user during task execution.

We developed a first prototype of the TEBRA (TEeth BRushing Assistance) system. The main components of the system are the user behavior recognition framework and the planning and decision making component. The user behavior recognition framework is based on a Bayesian Network classifier combined with a Bayesian filtering approach. The planning and decision making component tracks the user's progress in the overall task applying a Finite State Machine and a dynamic timing model which allows for different velocities of task execution. A detailed description of the system can be found in the paper 'TEBRA - An automatic prompting system for persons with cognitive disabilities in brushing teeth'.

We conducted a study with the first prototype of TEBRA and persons with cognitive disabilities at our cooperation partner Haus Bersaba, a residential home of the v. Bodelschwingh Foundation Bethel in Bielefeld. We


are currently evaluating the data corpus and the performance of the TEBRA system. Preliminary results makes us confident that an automatic prompting system fosters the independence of persons with cognitive disabilities in the toothbrushing task.