Intelligent modeling of multi-component bio-physiological data for smart human-robot interaction

2010-12 till 2012-11
Research Areas: 

This project aims to develop sophisticated models for the analysis of bio-physiological data, mainly EEG, but also skin conductance, heart rate, oculomotor activity and maybe others. The models for the different input channels will form a unified on-line framework to allow for flexible and adaptive human-machine interfaces that do not rely on muscular activity.

Methods and Research Questions: 

The project will exploit bio-physiological signals as input channels for adaptive human-machine interfaces. Based on on-line EEG measurements in combination with other bio-physiological data, e.g., skin conductance and/or oculomotor activity, we will develop models that allow for the detection of specific mental states in humans.

Mental states can indicate user responses to intrinsic intents, e.g., decisions, or external events, e.g., surprise or interaction errors. The models will enable the design of multi-user human-machine interfaces. Such interfaces will map multiple components within the different data streams onto distinct functionalities of the interfaces. These data streams are analyzed and translated in parallel within one unified framework, a framework that guarantees a smooth, efficient and robust interaction between humans and machines.

The project will build upon state-of-the-art algorithms for segmentation, feature extraction and classification of EEG data. These algorithms are suitable for the detection of either P300 potentials or motor imagery. In contrast to these established methods, we will now integrate different EEG components and, furthermore, other bio-physiological measures. These measures will include, for example, heart rate, respiration, skin temperature or pupil size and fixation duration. This allows for a much greater flexibility of the interfaces and largely enhances the interaction features they provide.



The project will not only provide a set of innovative methods to efficiently exploit bio-physiological measures for human-machine interfaces that context-dependently adapt to users’ needs based on their current mental state(s). Furthermore, these methods will form a set of sophisticated tools that can be conveniently used by other CITEC projects for analyzing (off-line and on-line) data from various (neuro-) physiological experiments.
Methods for analyzing bio-physiological signals, especially EEG data, will provide new insights into human information processing in general as well as into cortical processing in particular. To develop these methods, a thorough understanding and enhancement of different state-of-the-art algorithms is essential. Only this enables us to adapt these algorithms to a multi-component, multi-user human-machine interface. The enhancements to the algorithms will be beneficial for other machine-learning projects, too. The proposed project addresses issues of cognitive control and co-representation and joint action by examining how bio-physiological signals respond to certain situations and events and by evaluating in which situations these signals are stable over time and subjects - and in which situations they differ depending on the individual subject and the subjects’ condition. Furthermore, this project contributes to better understanding the sensorimotor basis of cognition by exploiting the relation between external events, intrinsic goals and bio-physiological signals, especially mental activity.