TACES

Task-dependent control of eye movements and mobile visual sensors

Term: 2008-10 till 2012-02
Research Area: B 
CITEC Logo

TACES

Abstract

To decide ‘‘Where-to-look-next?’’ is a central function of attention. We developed a novel computational model that combines task-dependent priority control, inhomogeneous processing of static and dynamic features and their fusion at the level of visual proto-objects. For each proto-object, attentional priorities are computed. The proto-object with the highest priority is the target of the next saccade. The model has been successfully applied to real images and to the real-time control of fast shifting cameras of the “Karlsruhe humanoid robot head”.

Research Questions and Methods

To decide ‘‘Where to look next ?’’ is a central function of the attention system of humans, animals and robots. We developed a novel computational model for the control of attention that integrates three factors, that is, low-level static and dynamic visual features of the environment (bottom-up), medium-level visual features of proto-objects and the task (top-down).

read more »

Outcomes

GordZeitGFirst, the model is the first computational implementation of a novel cognitive neuroscience model of attention. It combines spatially inhomogeneous processing of static features, spatio-temporal motion features and task-dependent priority control with further theoretical key assumptions of TVA and VAM.

Second, our computational model has been applied to several real world image sequences. Based on a visually specified task, the target has always been found within a few simulated sequences of fixations. Moreover, we have shown that these results are robust to parameter variations.

Third, the computational model has been used for the real-time control of a rapidly shifting camera of the “Karlsruhe Humanoid Robot Head”. We could demonstrate that the model was able to efficiently search for toy objects presented on a uniform background. It directs the head and cameras by a few combined head-eye-saccades to a feature-specified search target within an array of toy objects. Key predictions of the model (e.g, ,proto-object-based visual search) will be tested by comparison with eye tracking experiments with human subjects (lab of Werner Schneider).

Publications

OOP: Object-Oriented-Priority for Motion Saliency Maps

Belardinelli A, Schneider W, Steil JJ (2010)
In: Workshop on Brain Inspired Cognitive Systems. Madrid: 370 - 381.
Download:
Conference Proceeding/Paper | Published | English
Authors:
Belardinelli, Anna ; Schneider, Werner ; Steil, Jochen J.
Department:
Center of Excellence - Cognitive Interaction Technology CITEC
Technische Fakultät
Research Institute for Cognition and Robotics
ISBN:
978-84-614-1869-5

Cite this

Link: http://pub.uni-bielefeld.de/publication/1928041

Attending to Motion: an object-based approach

Belardinelli A (2010)
In: Dagstuhl Seminar Proceedings. Lakemeyer G, Levesque HJ , Pirri F (Eds.); Dagstuhl, Germany: Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany.
Conference Proceeding/Paper | Published | English
Authors:
Belardinelli, Anna
Editors:
Lakemeyer, Gerhard ; Levesque, Hector J. ; Pirri, Fiora
Department:
Center of Excellence - Cognitive Interaction Technology CITEC
Abteilung für Psychologie
Fakultät für Psychologie und Sportwissenschaft
ISSN:
1862-4405

Cite this

Link: http://pub.uni-bielefeld.de/publication/1928047

The control of stimulus-driven saccades is subject not to central, but to visual attention limitations

Carbone E, Schneider W (2010)
Attention Perception & Psychophysics 72(8): 2168 - 2175.
Journal Article | Published | English
Authors:
Carbone, Elena ; Schneider, Werner
Department:
Center of Excellence - Cognitive Interaction Technology CITEC
Fakultät für Psychologie und Sportwissenschaft
Abteilung für Psychologie
ISSN:
1943-3921

Cite this

Link: http://pub.uni-bielefeld.de/publication/1968006

Where to Look Next? Combining Static and Dynamic Proto-objects in a TVA-based Model of Visual Attention

Wischnewski M, Belardinelli A, Schneider W, Steil JJ (2010)
Cognitive Computation 2(4): 326 - 343.
Journal Article | Published | English
Authors:
Wischnewski, Marco ; Belardinelli, Anna ; Schneider, Werner ; Steil, Jochen J.
Department:
Technische Fakultät
Fakultät für Psychologie und Sportwissenschaft
Abteilung für Psychologie
Center of Excellence - Cognitive Interaction Technology CITEC
Research Institute for Cognition and Robotics
Abstract:
To decide "Where to look next ?" is a central function of the attention system of humans, animals and robots. Control of attention depends on three factors, that is, low-level static and dynamic visual features of the environment (bottom-up), medium-level visual features of proto-objects and the task (top-down). We present a novel integrated computational model that includes all these factors in a coherent architecture based on findings and constraints from the primate visual system. The model combines spatially inhomogeneous processing of static features, spatio-temporal motion features and task-dependent priority control in the form of the first computational implementation of saliency computation as specified by the "Theory of Visual Attention" (TVA, [7]). Importantly, static and dynamic processing streams are fused at the level of visual proto-objects, that is, ellipsoidal visual units that have the additional medium-level features of position, size, shape and orientation of the principal axis. Proto-objects serve as input to the TVA process that combines top-down and bottom-up information for computing attentional priorities so that relatively complex search tasks can be implemented. To this end, separately computed static and dynamic proto-objects are filtered and subsequently merged into one combined map of proto-objects. For each proto-object, attentional priorities in the form of attentional weights are computed according to TVA. The target of the next saccade is the center of gravity of the proto-object with the highest weight according to the task. We illustrate the approach by applying it to several real world image sequences and show that it is robust to parameter variations.
Keywords:
Natural scenes ; Top-down control ; Inhomogeneity ; Static and dynamic features ; Proto-objects ; TVA ; Modeling visual attention
ISSN:
1866-9956

Cite this

Link: http://pub.uni-bielefeld.de/publication/1928034