Electronics and Semiconductors

Why microbolometer cameras are key to autonomous vehicle night vision

19 May 2023
Source: Owl AI

Around the world, highway safety organizations, insurance companies and regulatory agencies are urgently pressing for action to counter the steadily rising rates of pedestrian deaths and injuries at night. While connecting cameras in cars to automatic braking systems has reduced incident rates in the daytime, these cameras, even when augmented by radar, fail to provide protection at night. The latest government and industry safety regulations address this shortcoming by requiring improvements in nighttime performance as soon as suitable technology is available. In these regulations, five years is the imperative timeline, driving rapid incorporation of night vision technologies into the automotive manufacturing process.

The solution seems obvious – adapt some of the night vision equipment used in defense to work in cars. In fact, attempts to do this have been tried, and so far, they have all failed. The problem is that in defense, the primary goal is to provide images for the troops while in cars, the images must be acted on by computers. For the same reason, night vision equipment used in commercial security is not suitable in cars.

In cars, cameras have the same operating requirements as other automotive electronics – high tolerance of a wide range of environments, high reliability, low cost, and compact size. Cameras are part of the safety equipment in cars and so must operate continuously and accurately provide good data to the computers responsible for making life and death decisions about driver warnings and braking and steering. Equipment designed for cars must be carefully designed to incorporate all the features and safeguards needed to improve safety for both occupants and those outside.

While automotive requirements impose specific problems on night vision, the fundamental technologies remain applicable.

Two of these night vision technologies dominate existing markets – low-light-level (LLL) cameras and thermal imaging. LLL cameras are essentially daylight cameras with certain modifications made to support operation with less light. In one of the oldest modifications, an image intensifier, which takes in the low light levels and electronically increases their brightness is added to the front of a camera so a regular daylight sensor can detect the result. Some newer sensors directly accept the low light scene and increase the signal internally using an avalanche multiplication process. Other sensors are designed with noise low enough to allow detection of single photons without amplification. However, while these cameras can see on moonless nights, they suffer from the same limitations as daylight cameras – they can’t see through fog or rain and they can be dazzled by bright lighting.

The second night vision technology, thermal imaging, utilizes a different physical phenomenon – detection of electromagnetic radiation that is emitted by all objects above absolute zero. At room temperature, the bulk of this radiation occurs in a band between 8 and 12 µm called the long-wave infrared (LWIR). Fortunately, the atmosphere is transparent in this band so imaging of objects emitting this “thermal radiation” gives good results.

Thermal imaging requires the use of image sensors that can convert the received radiation into electrical signals. Two primary types are in use. Sensors that convert photons in the LWIR band directly to electrons work very well but require cooling to 100K or below, requiring the use of bulky, unreliable coolers. Uncooled detectors using structures called microbolometers that change temperature on exposure to the incoming radiation can drive small changes in electrical current that can be read as image data. Although this detector type is less efficient than the cooled type, the data it produces is eminently suitable for both direct viewing and supplying to computers for analysis and its minimal size, weight and complexity readily supports use in automotive cameras.

Microbolometer cameras, compact, light and rugged, seem to be the solution – but they have one small problem – they are still too expensive. Detecting pedestrians requires something on the order of 1000 pixels across the scene for curb-to-curb coverage, but cameras with sensors this size currently cost many thousands of dollars. Current markets have provided minimal price pressure so new sensor and camera designs have tended to simply duplicate the strategies employed for decades.

Figure 1. A compact thermal camera core with curb-to-curb coverage. Source: Owl AIFigure 1. A compact thermal camera core with curb-to-curb coverage. Source: Owl AI

With the arrival of automotive technical requirements and volume potential, the setting has now changed.

Instantly, a market has emerged that can support significant innovation in the design and manufacture of microbolometer sensors and the optics they require.

Traditionally, the microbolometer array and the circuitry that read out the microbolometer signals were manufactured separately and then bonded, an expensive process. In these devices, the signals from the sensor were analog, requiring external digitization and correction to be suitable for computer processing. Adding further to the challenge, the lenses were made from germanium and other exotic materials, requiring a slow and costly manufacturing process.

Figure 2. A complex thermal infrared lens on a conventional camera. Source: Jinjin Chen/CC BY 4.0Figure 2. A complex thermal infrared lens on a conventional camera. Source: Jinjin Chen/CC BY 4.0

For automotive night vision cameras, all of these expensive legacy processes are now to be cast aside.

In this new paradigm, the sensor array and readout are a single monolithic device incorporating real-time digitizing and correction, providing signals suitable for direct connection to the automotive computer. New optical materials and techniques, such as chalcogenide glasses, metamaterials, diffractive elements and surface molding are being combined to substantially reduce lens cost and bulk.

The result will not be little metal box cameras containing circuit boards and a lot of connectors but a single bonded module ready for mounting in positions in automobiles ideal for observing pedestrians in traffic and on the open road. Using a standard high-speed serial digital interface like GMSL2, these compact, low-cost modules will connect to an automotive electronic control unit hosting artificial intelligence (AI) in the form of convolutional neural networks (CNNs) capable of finding and locating pedestrians in the dark.

This might all seem unlikely if there were not already successful demonstrations of the technologies.

Figure 3. Stopping for a poorly-lighted child at night. Source: Owl AIFigure 3. Stopping for a poorly-lighted child at night. Source: Owl AI

During live testing late in 2022, a prototype thermal night vision system in a production automobile successfully stopped automatically with room to spare when encountering a test dummy the size of a child running on to a dark road twelve out of twelve times.

The automotive manufacturing industry and various Tier One original equipment suppliers are already looking at this technology. Within the next year, the system will be available and ready to protect pedestrians on the open road.

About the Author

Chuck Gershman brings over 30 years of technology and semiconductor industry experience in executive management, marketing, engineering, business development, sales, consulting, and executive advising, including Owl Autonomous Imaging. He has also served as an accomplished executive and board director for two other organizations. Gershman is a member of Drexel University, College of Engineering Alumni Circle of Distinction; in addition, he holds three patents in microprocessor architecture and has been nominated for multiple awards.

Owl Autonomous Imaging delivers monocular 3D thermal ranging computer vision solutions that dramatically enhance safety day or night and in adverse weather conditions, to automotive and industrial mobility markets. Owl AI’s system approach identifies living objects in all conditions from dense urban environments to completely dark country roads, where it is paramount to quickly identify, classify, and determine the distance to an object including all VRUs.

Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the GlobalSpec
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter