Event-based vision sensors (EVS) — also known as event cameras, neuromorphic cameras or dynamic vision sensors — represent a shift in machine vision away from traditional frame-based imaging. The aim is to improve high temporal resolution while simultaneously reducing data redundancy particularly in challenging lighting conditions.
EVS cameras are systems that only react when something changes like a shift in brightness or a sudden movement. The sensors accomplish this by:
- Each pixel working on its own.
- Sending a signal only when it sees a change.
- Not wasting time or power if things remain the same.
Event-based vision sensors show the trajectory of an object as it changes rather than taking a photo of the ball in each frame. In a frame-based sensor, the ball and background are captured but, in an event,-based vision sensor takes the differential data showing the trajectory of the moving call. Source: Sony Semiconductor
Features
Event-based vision sensors are fast and can identify fast movements instantly. The temporal resolution — or the minimum measurable time difference between two consecutive pixel events — is less than 100 µs. Meaning the fastest movements can be captured with a comparable image-based frame rate of more than 10,000 FPS without motion blue. The result is no blind spots between images.
The sensors can work in bright or dark places, which makes these sensors suitable for factories with different types of lighting. EVS can produce results with nearly no light as they recognize contrast changes even from 0.08 lux.
And because the sensors ignore the environment if nothing changes, it uses less energy than other sensors and it saves space by sending only relevant data, not full images. That means not just raw data is being sent but also pixel motion is transferred with coordinates and a time stamp, allowing for results being directly available.
Use cases
As a result, EVS technology is gaining traction in a range of sectors.
Event-based vision sensors only register movement as it changes. Anything stationary does not appear allowing users to find faults in machinery or unwanted vibrations and more. Source: IDSIn industrial use cases, EVS can catch fast-moving products on conveyor belts. EVS cameras can be used for 3D measurement of an object as it passes through a conveyor belt or on a factory floor. It then sends this data to a computer where it outputs it as a point cloud. This allows workers to obtain detailed height information about objects on the factory floor with enhanced accuracy.
In robotics, EVS systems allow for drones and robots to dodge objects in their environment and map their surroundings. The sensors transmit to the drone or robot the objects it detects whether that be a change in light or the environment so the machines can adjust or avoid obstacles.
For maintenance, EVS cameras can visualize more efficiently small vibrations that may lead to a machine breaking down or in need of repair before it happens. Event-based vision sensors show just the vibration in the machine and where it might be taking a toll on the device rather than a frame-based camera that is harder to grasp where the vibration is impacting the machine. This could also allow EVS cameras to spot tiny defects in materials or machines that otherwise might only be identified when the machine fails.
In human tracking, people wearing dark clothing are harder to detect. EVS enhanced with machine learning can recognize dark clothed figures continuously even if they are running.
Other use cases include:
- Liquid monitoring
- Human tracking
- Visual inspections
- Scientific measurement and investigation
- Slow motion movement analysis
With EVS systems, vibration monitoring shows just the vibrating parts of the image. In this case, it shows where on the truck is being most affected by the vibrations and where the waves of vibrations are happening to better visualize changes happening. Source: Sony Semiconductor
Future use cases
While those use cases are now beginning to mature, vendors are looking ahead to what EVS technology could potentially be used in the future.
This includes combining an EVS with a standard sensor to create fast blur-less video or photos.
Sports science is another potential sector where an EVS camera is used to analyze body movements. This could show where an athlete is changing their movement in a way to impact performance or how they might improve their performance with adjustments.
In smart cities, EVS cameras could be used to monitor a high-crime area to alert authorities to changes in the environment or buildings. It could also track high volumes of people to allow city managers to plan routes more effectively and more.
A variety of EVS sensors and cameras are inching closer to commercialization being led by IDS Imaging Development Systems and Sony Semiconductors. Source: IDS Imaging Development Systems GmbH
Toward commercialization
Earlier this year, industrial machine vision vendor IDS Imaging Development Systems GmbH and neuromorphic vision systems house Prophesee SA launched a new line of EVS cameras for industrial machine vision applications.
Called the uEye, the EVS cameras are targeted at real-time machine vision processing applications like optical monitoring of vibrations or high-speed motion analyses. The event-based sensor transmits events depending on when and where the brightness in its field of view changes.
