Neuromorphic pre-touch sensing system

Responsible person: Tim Walther
This work is carried out in collaboration with: Jakob Engelmann, AG Active Sensing, CITEC, Uni Bielefeld

In this interdisciplinary project we use knowledge gained by the study of the sensory system of the weakly electric fish, in order to build a neuromorphic pre-touch sensing system suitable for robotic applications. Reliable three-dimensional perception is essential for robotic agents and mandatory for accomplishing - from a human perspective - simple tasks like walking or grasping objects. Current standard approaches in robotics rely on combinations of far range sensors (e.g. vision) and tactile sensors, while lacking sensory feedback on intermediate pre-touch range. This, however, becomes problematic if parts of the field of view are blocked (e.g. by a manipulator), or if the sensory information itself is unreliable. This project tries to fill the gap with a novel biologically-inspired pre-touch sensing system composed of an electric pre-touch sensing device, coupled to a neuromorphic spiking neural network.

Collage of a picture of the weakly electric fish, a schematic view of the network for pretouch sensing, a schematic view of the fabricated chip in which the network can be tested, and a picture of the fabricated pretouch sensor PCB  

Weakly electric fish have the remarkable ability to see with their body by "electrically illuminating" their surroundings. This enables them to detect, track, and localize both objects and prey, even under conditions which are not favourable for vision. Additionally, they are able to actively control the imaging process of their electric sense. This is achieved by causing weak discharges with an electric organ, so called electric organ discharges (EODs). These self-generated signals are prone to subtle modulations of the environment, including alterations caused by the electric fields of other living beings. Specialized electroreceptors, distributed all over their body, are tuned to detect these modulations.

Picture of the fabricated ActivePreTouch sensor PCB

The pre-touch sensors we utilize in this project rely on the same fundamental operating principle: they generate an AC voltage on a transmitting electrode, which induces a displacement current in a receiving electrode. Objects within the actively emitted electric field modify the amplitude of the signal received if their dielectric properties differ from that of air. Additionally, the received sensory data will be locally processed by using signal en- and de-coding approaches known from weakly electric fish, providing an output signal based on neural spike-latency. A long term goal of this project is the creation of custom neuromorphic hardware, which will guarantee real-time and low power computation, while sparse temporal coding will ensure a low bandwith for transmitting pre-touch information to a processing unit.