“Without Feedback, Humans Can Only Communicate with Difficulty”

Three questions for Dr.-Ing. Hendrik Buschmeier, research associate at CITEC

There is one big question that concerns computer scientist Dr.-Ing. Hendrik Buschmeier, and this question does not even relate to machines. The question is this: how does human communication work? For ten years, Buschmeier has been conducting research at the Cluster of Excellence Cognitive Interaction Technology (CITEC), where he works in the Social Cognitive Systems research group, which is headed by Dr.-Ing. Stefan Kopp. In his doctoral thesis, which he completed last year, Buschmeier dealt primarily with communicative feedback. For a part of his research findings, he recently received a Best Paper award at the AAMAS conference in Stockholm.

Dr. Hendrik Buschmeier is a research associate at the Cluster of Excellence Cognitive Interaction Technology CITECMr. Buschmeier, as a computer scientist, what fascinates you about human communication?
If I start telling you about my field of research, you might not know a lot at the beginning. The more we talk, the better you can follow along. Isn’t it amazing what happens here?
Smooth communication only works because we, for instance, give feedback to the other person during conversation. A nod, a verbal confirmation, or a confused facial expression: these signals are important for me as the speaker to even be able to tell a story or give an effective explanation without getting bogged down. Everything here happens completely automatically and it’s a great feat as I see it. Without feedback, humans can only communicate with great difficulty. Currently, a machine can only classify what an expression means in a certain communicative situation with great difficulty. But here, too, it could be advantageous if machines could react to small feedback signals from us humans and, for example, adapt explanations to the individual.

At the AAMAS conference – a highly regarded conference in the area of autonomous agents – you won the Best Paper award under the rubric of “Socially Interactive Agents.” What’s the study you describe about?
In the article, I outline how feedback in dialogues with attentive conversational agents can work. Conversational agents are artificial systems that are capable of acting, such as Alexa, Siri, or other social robots, which can interact with their users using natural language. For my study, I developed an agent that is able to process feedback. This isn’t so easy, since a “hmm” can mean different things depending on the context. An artificial agent will never be able to interpret as robustly as us humans do. Our agent (at CITEC, we’re working on the virtual agent Billie) can, to a certain degree, interpret different signals in the context of the situation, such as whether what was said was understood, or whether an appointment was turned down. In our study, we were investigating whether users notice if the agent is attentive, and how they evaluate this. We used three agents to test this: one did not individually adapt at all, one explicitly asked for confirmation every time if what was said was understood, and one reacted to feedback signals such like a nod of the head. In this context, it was the agent that was able to interpret feedback signals who proved to be the one who was perceived by the study participants as being the more attentive speaker. Here, the participants had the feeling that the machines were interested in being understood and helped to solve problems of understanding.

How might attentive agents shape our everyday lives?
I imagine, for instance, a navigation system that adjusts to the individual users. In the user’s own neighborhood, for example, the device wouldn’t have to announce every single street. Attentive household robots could make sure that instructions given by humans have actually been understood. Of course, the question always arises here as to how trustworthy and dependable such a system is. For one thing, the issue of how machines can solve problems of understanding still needs to be resolved. The next thing I will be  researching is how conversational agents can adapt to the mental perspective of their interaction partner in order to adjust their explanations based on the their point of view.

Hendrik Buschmeier (born 1983) studied Informatics in the Natural Sciences at Bielefeld University, with a focus in language processing. Since 2008, he has worked as a research associate at the Cluster of Excellence Cognitive Interaction Technology (CITEC). Buschmeier conducts research on natural communication and how this can be applied to communication with machines.

Buschmeier received the Best Paper award for his article titled “Communicative listener feedback in human–agent interaction: Artificial speakers need to be attentive and adaptive.” The award was given under the rubric of “Social Interactive Agents” at AAMAS, the international conference on autonomous agents and multi-agent systems. The conference, which was held from 10–15 July in Stockholm, Sweden, is the largest and most influential conference in the field.

More information is available online at:
AAMAS conference: http://celweb.vuse.vanderbilt.edu/aamas18/home  

Original publication:
Hendrik Buschmeier, Stefan Kopp. Communicative listener feedback in human–agent interaction: Artificial speakers need to be attentive and adaptive. Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, https://pub.uni-bielefeld.de/publication/2916994, published on 9 July 2018.