In order to fly safely, drones need to know their precise position and orientation in space at all times. Commercial drones solve this problem using a GPS, but this only works outdoors and is not reliable, especially in urban environments. Conventional cameras that are mounted on drones work only when there is a high amount of light available and the drone’s speed is limited before the image becomes blurred and can’t be used by computer vision algorithms. In order to fix this, professional drones use sensors that are elaborate, expensive and bulky.
A group of researchers from the University of Zurich and the Swiss research consortium NCCR Robotics has now developed an innovative alternative approach. This enables drones to fly in a wide range of conditions with an eye-inspired camera that can easily cope with high-speed motion. It can see in the dark more effectively than the conventional cameras that are currently used by all drones.
"This research is the first of its kind in the fields of artificial intelligence and robotics, and will soon enable drones to fly autonomously and faster than ever, including in low-light environments," said Prof. Davide Scaramuzza, Director of the Robotics and Perception Group at UZH. The research team has already taught drones to use onboard cameras to infer their position and orientation in space.
Event cameras were invented at UZH together with ETH Zurich. The cameras don’t need to capture full light on the entire bio-inspired retina in order to have a clear picture. They only report changes in brightness for each pixel which ensured perfectly sharp vision, even during fast motion or in low-light environments. The UZH researchers have designed new software that efficiently processes the output from the cameras, harnessing this to enable autonomous flight at higher speeds and in lower light than currently possible with commercial drones.
Drones equipped with the event camera and the software designed by the Swiss researchers could assist search and rescue teams in scenarios where conventional drones would have no use. For example, on missions at dusk or dawn when there isn’t enough light for normal cameras to work. They would be able to fly faster in disaster areas where time is critical.
"There is still a lot of work to be done before these drones can be deployed in the real world since the event camera used for our research is an early prototype. We have yet to prove that our software also works reliably outdoors," said Ph.D. Student Henri Rebecq.
Professor Scaramuzza adds, "We think this is achievable, however, and our recent work has already demonstrated that combining a standard camera with an event-based camera improves the accuracy and reliability of the system."
A paper on this research was published in IEEE Robotics and Automation Letters. Read the paper on this topic here.