The personal virtual assistant that guides you through the day

The KOMPASS project of Bielefeld Unversity and the v. Bodelschwingh Foundation Bethel supports people with special needs in their everyday life.

Like a compass, a virtual assistant should navigate its user through the uncertainties of daily life: This is the aim of the new project KOMPASS of Bielefeld University. KOMPASS stands for socially cooperative virtual assistants as everyday helpers for people with special needs. Scientists from the cluster of excellence Cognitive Interaction Technology (CITEC) of Bielefeld University are cooperating with the v. Bodelschwingh Foundation Bethel as well as scientists from the University of Duisburg-Essen and the University of Applied Sciences, Bielefeld. The project is funded by the Federal Ministry of Education and Research (BMBF) with 1.98 million Euros and runs for three years.

“In everyday life we usually use a lot of different technologies which we have to adapt to. With KOMPASS, we are developing a system that bundles many functions together and adapts to the person”, says Professor Dr.-Ing. Stefan Kopp. CITEC researcher Kopp leads the Social Cognitive Systems research group.   At the centre of the KOMPASS project is a “virtual flatmate”, which appears on a Smart-TV in the living/dining area and is able to communicate simply through speech. The technology has been developed in particular for older people who may have difficulties to plan and structure their everyday life. The virtual assistant also should be able to support people with intellectual disability.

The virtual avatar discusses attentively the daily structure, for example, and notes appointments in a calendar. It can also remind the user to do exercise or to call friends. It not only helps to structure daily life but also ensures that senior citizens and people with special needs do not become isolated and exercise enough. In tests, the technology has been so accepted by test persons that they ask the avatar on the screen personal questions or explain the reasons for a visit to the doctor. They think of the avatar as more of a friendly counterpart than technical support.

A great challenge for the technical realisation is that the assistant on the one hand has to express itself clearly and on the other hand has to understand what the person wants. “Although the speech recognition is already very advanced, understanding must often be mutually reached and made plain”, explains Kopp. “KOMPASS takes a step forward here because the virtual assistant should be able to recognise any signs of confusion or ambiguity and adjust its dialogue accordingly.” The assistant should also be able to adapt to the individual emotions as well as the intellectual abilities of the person it is caring for and so respond exactly to the needs of the person. In 2017, three test persons will live with the virtual assistant in their flat in Bethel. In this way, the technology can be researched further and tested out. Also, in the future in cooperation with KogniHome, the scientists plan to integrate the technology into the research apartment in Bethel and supplement it with more functions.

The KOMPASS project further develops the digital avatar “Billie”, that evolved from the VASA project. In the research project VASA (Virtual Assistants and their Social Acceptability) CITEC scientists in cooperation with Bethel investigated how people react to a virtual avatar. A central question was what facial expressions, manners or gestures the avatar should have in order to be accepted by people with special needs or senior citizens.

The scientists are taking ethical and legal issues into consideration: Which data should the assistant be allowed to record? When should it be permitted to film the user with a camera? How far can it intervene in the daily life of the user without being patronising? Because of this, Professor Dr.-Ing. Stefan Kopp, informatics scientist Professor Dr.-Ing. Franz Kummert, social psychologist Professor Dr. Nicole Krämer and linguist Professor Dr. Karola Pitsch are working closely with the legal scholar Professor Dr. Brunhilde Steckler and Dr. Arne Manzeschke, the head of the Munich Center for Ethics and Anthropology at the Ludwig-Maximilians-University Munich.

Professor Dr.-Ing. Stefan Kopp, Bielefeld University
Head of the Social Cognitive Systems research group.
Faculty of Technology and the cluster of excellence Cognitive Interaction Technology (CITEC)
Telephone: +49 (0)521 106 12144