Analysis of a sensory-motor interface: from optic flow parameters to action selection

Research Areas: 

Blowflies compose their flights from quick turns interleaved with translatory flight. Alternating turns with translatory sections is thought to facilitate visual tasks like obstacle avoidance. Information regarding the environment is extracted intelligently from the optical flow, the image velocity field on the eye. The insect extracts a small set of features from the optic flow in a highly non-linear process. ASMOTIF addresses how this representation of optic flow based information about the environmental structure is decoded for action selection.


Methods and Research Questions: 

Despite of the significant progress within the areas of machine vision and robot navigation modern autonomous technical systems are still inferior to their counterparts in nature. We aim at biologically inspired algorithms replacing resource hungry computational algorithms which are barely suitable for real time processing and decision making.

In this project we would like to explore such areas where nature has induced much simpler mechanisms which nevertheless provide solutions to complex problems. Interestingly, the results produced by such simple systems are still far beyond the reach of current powerful computing machinery. In principle, this can only be achieved by using simple but intelligent computational strategies.

Intriguing examples are blowflies; their high temporal and low spatial resolution vision system had made them true artists of flight. Their flight behavior is characterized by the alternating occurrence of high velocity turns (saccades) about their vertical axis and virtually rotation-free translations in-between these saccades. Consequently, the image movements on the eyes – called optic flow – are dominated alternately by either rotational or translational motion. Keeping apart these two optic flow components dramatically reduces the computational effort to extract navigationally relevant information contained in the optic flow.

ASMOTIF aims to investigate (1) how the optical flow information is decoded to select the direction of a future saccade; (2) what is the most likely instant to trigger a saccade; (3) what should be the size (in degrees) of the desired saccade.

To achieve the speculated goals, in ASMOTIF a comprehensive and well-defined data set containing neuronal responses from a population of motion sensitive output neurons of the fly visual system has been analyzed using clustering techniques. Before drawing any conclusions from the resulting cluster configurations, they are benchmarked using objective criteria with regard to their quality as well as to their stability. The application of these criteria ensures that the defined clusters represent significant structures within the data set.

In the next phase the cluster configurations which exhibit highest values for both quality and stability are then selected for further analysis. Within these selected cluster configurations, the members of each configuration are characterized according to their representation of the saccade direction, i.e. left or right saccade. This representation provides in turn the basis for the development of the basic engine of the saccade controller employed inside the saccade direction predictor. In principle, this controller works based on a majority vote control strategy. At each given instant in time an individual vote for the subsequent saccade direction is registered and pooled until the triggering of a saccade becomes inevitable. The predicted saccade direction is the dominant direction among the pooled votes.



At present our first milestone – the development of a saccade direction predictor based on the results of the cluster analysis – has been successfully achieved. The mean error of this novel predictor is as low as 19.6%. In principle it takes into account the optic flow and intelligently extracts the intrinsic information about the present environmental structure embedded inside. To ensure its generalization to new independent data, a cross-validation approach is used. However, the questions regarding the most likely instant for triggering a saccade as well as its size still remain unanswered and will be addressed in a future analysis. Moreover, performance enhancement of the currently developed saccade direction predictor is also in the list of future endeavors.

The ultimate objective of ASMOTIF is to combine the gained knowledge for realizing a bio-inspired computationally light-weight locomotion controller suitable for vision based autonomous technical systems in real-time. This in turn will allow agents to control their locomotion in complex natural environments on the basis of self-generated information about ego-motion and environmental structure.