A major research question in TAPED concerns making decisions under uncertainty. In our particular case, the system has to decide whether the user needs to be assisted in the execution of the task while the user's progress in the task is not fully observable by the system.

In TAPED, we focus on an application-oriented way of research. We established a close cooperation with v. Bodelschwinghsche Stiftungen Bethel as a large care facility in Bielefeld. In particular, we cooperate with Haus Bersaba which is a residential home for 35 persons with different cognitive disabilities, amongst others epilepsy, dementia and autism. In collaboration with the caregivers of Haus Bersaba, we develop an assistance system that guides the inhabitants through the task of brushing their teeth. The chosen task combines relevance in everyday life on the one hand and offers exciting and challenging research questions in the area of cognitive interaction technology on the other hand.

A major challenge in the development of an automatic prompting system involves making a decision as to whether prompting a user is necessary or not. A decision making component needs to detect problems in the execution of a task based on models including both task-related and environment-specific knowledge. The system has to monitor the user's progress in the execution of the task dealing with uncertainty in the perception since user behaviors underlie large individual variation. Models representing the task and its subunits have to cope with this variation. In addition to inter-personal differences the intra-personal variation, which is characteristic for persons with cognitive disabilities like dementia and Alzheimer's disease, extremely increases the complexity of perception. We use a combination of external sensors (e.g. cameras, microphones) and integrated sensors (e.g. accelerometers) embedded in tools to estimate the user's behaviors. Based on the perception of the user's behaviors the system provides assistance in terms of interaction mechanisms which have to comprise several levels of escalation and intensity, highly dependent on the mental state of the user. From a technical point of view, an interaction mechanism combines multimodal cues to guide the attention of the user, e.g. verbally given commands, auditory alerts, visual signals such as pictograms or visual pointers as indexical gestures.

We aim to assess the utility of the system and to evaluate the system's behavior in a real-world scenario by conducting user studies with the inhabitants of Haus Bersaba.