Researchers at the Technical Research Centre of Finland (VTT) are developing an energy-efficient machine vision system that improves robot and drone perception.
Machine vision allows devices to see the visible environment and interpret the observations about the environment without interruption. In self-driving cars, lidar and computers achieve this perception. However, in robotics and self-flying drones, batteries are not sufficient to power these types of technologies.
The goal of VTT’s multispectral intelligent vision system with embedded low-power neural computing (MISEL) project is to develop a fast, reliable and energy-efficient machine vision system that could be attached to drones, industrial and service robots, and surveillance systems.
“In addition to algorithms, we are developing devices better suited for local perception and analysis than computers that use microprocessors and graphics cards,” said Jacek Flak, senior scientist at VTT, who coordinates the MISEL project. “We are modelling it after the human vision system that works locally, rapidly and energy efficiently. It is essential that the system filters observations to be processed right from the start rather than taking a series of regular photos and going through all data they contain. Filtering is possible when the system focuses on events, i.e., on the changes in the scenes.”
Using three components, the neuromorphic machine vision system mimics the functioning of a human brain. The first component in the system is a photodetector built with quantum dots sensitive to both visible and infrared light. The second component chain mimics the cerebellum, which is the part of the brain located at the back of the head, to give instructions for rapid reflexes. The third component is a processor that imitates the cortex for deeper analysis of data and directs the sensor to focus on objects of interest.
The machine learning methods teach the system to identify different events, such as distinguishing between a flying drone or a bird, and anticipate a chain of events.
The project will last four years and VTT will focus on developing a processor mimicking the functioning of the cerebral cortex and novel non-volatile memory. VTT will also be responsible for the compatibility of the different technologies and the integration of the different components with the processors.