Research for a more human-like interaction with the computer is a hot topic both in theory and application development where the identification of technologically relevant aspects of multimodal human communication is both a necessity and a challenge. The interdisciplinary HuComTech research group, founded in Debrecen in 2009 has this scientific and technological goal. In particular, we study the basic structure of communication in simulated job interviews and free dialogue with special attention to the multimodal markers of turn-taking as well as certain basic emotions. Theoretical work is aimed at proposing a generative theory of communication with emphasis of bi-directionality (analysis and synthesis). We pay attention to the study of the role of scenarios and special ontologies in communication.

The project has three distinct modules:

Computational linguistics (prosody)

Digital image processing (eye-gaze, lip and head movement recognition)

Communication theory (generative theory of communication)

With purpose of validation we are building a simple demo on the basis of a framework implementing the fusion of data from various modalities.

The talk will give an outline of the project along these lines and give some details of certain results.