While light detection and range (lidar) technology has been around for decades — used in surveying and mapping by governments or for atmospheric research and meteorology — it has become popular as one of the key elements enabling self-driving functions in vehicles.
While the push to create fleets of self-driving cars moves on and progress is made, the move toward fully automated vehicles is likely to be slow, and automated functions are more likely to be rolled out gradually as the technology ramps up.
While lidar companies wait to begin mass producing the sensing systems for vehicles, they are looking to other markets that can also take advantage of the technology today.
What is lidar?
Lidar is a remote sensing method used to measure the distances of objects — rocks, a dog in the street, walls inside a building, curves in the road, the size of packages and more. Much like radar systems that employ radio waves to measure objects, lidar uses lasers to calculate the distance of objects and the light pulses from these lasers generate 3D information about those objects.
A lidar system generally consists of a scanner, a laser and a receiver, however, it is common to find sensors like accelerometers or speedometers, inertial guidance systems and gyroscopes in lidar systems as well as GPS.
On self-driving test vehicles, a lidar system is typically the whirling object mounted on top of the car or truck that spins around firing lasers in all directions to measure objects in the car’s environment. This helps pave the way in real-time for vehicles to drive without the assistance of a human. Outside of vehicles, lidar can be mounted on tripods or fixed to a handheld system with a video camera to collect data for surveying or mapping.
Mapping interiors
Information gathered from lidar scanners for indoor 3D maps could be used to point emergency responders to critical locations such as emergency exits, fire extinguishers or utility shutoffs. Source: NIST
While lidar has become a staple for generating 3D maps for use to study the Earth’s environment and obtain information about the patterns of climate change. Only recently has lidar started to be used to map the interior of buildings and other areas. While mapping interiors is important for surveying or exploration, an added benefit is to emergency personnel that may have to enter a burning building or collapsed structure following a natural disaster to save lives.
The National Institute of Standards and Technology (NIST) has been working with emergency services and academia since 2018 to develop systems for using lidar to generate 3D maps of buildings under its Point Cloud City program. Under the program, scanning indoor structures in 3D was accomplished using a handheld lidar scanning device and could map an entire building in a few hours.
A portable lidar scanner for use to map indoor locations to create 3D maps in a matter of a few hours. Source: NIST The goal is to increase public safety for emergency response so firefighters can have an accurate map before entering a structure that may be compromised or on fire. NIST envisions a way to generate the 3D maps in the field of view of a firefighter through augmented reality (AR) to give sight and orientation to how a structure may have changed during an event. Information in the display would also be provided to first responders such as emergency exits, fire extinguisher locations and utility shutoffs.
Another use could be for schools or universities to provide indoor 3D maps to new students or for workplaces for new employees that could point to key locations or services via a smartphone to help people to avoid getting lost or disoriented.
AR
AR and lidar got a huge boost last month when Apple said it would be including a lidar scanner in its recently released iPhone 12 5G Pro smartphone. This is not the first time that Apple has included lidar in its internet-connected devices. Earlier this year it included a lidar camera sensor in the latest iPad Pro.
The goal for Apple is to include lidar as a way to improve AR experiences and improve autofocus by as much as six times in low-light scenes for more accuracy and reduce the time it takes to capture photos and videos. The technology also allows for night mode portraits, Apple said. Lidar could also be used for face mapping that would allow users to port themselves, or others, into AR for use in gaming or work situations.
The next steps for Apple could include a new version of smart glasses as the device giant has recently applied for patents for “head-mounted display with low light operation" that would include a controller and head-mounted display. While a variety of sensors would be included in such a system — depth, infrared, ultrasonic — the inclusion of lidar in such a system may make it easier for industrial workers to use AR to map equipment or construction work to create 3D maps without having to carry heavy equipment.
Meanwhile, Swiss technology firm ActLight recently released a tunable dynamic photodiode to help lidar scanners in smartphones create the depth map of surrounding space and enable AR apps that are sensitive to ambient light and blinded by bright light.
“The fast adoption of lidar in smartphones has created a big demand in terms of light sensors performance improvement and we are in talks with the major players of the lidar supply chain and their customers to bring this technology to market” said Serguei Okhonin, co-founder and CEO at ActLight.
Drones and logistics
Outside mapping using lidar has been done for decades, but now how this has typically been accomplished is getting a modern-day facelift. DJI, one of the largest drone makers in the world, last month introduced its first unmanned aerial system (UAS) equipped with lidar technology.
The company introduced two surveying drones designed to increase the efficiency and accuracy of data collected from aerial inspections and data collection missions. DJI said that lidar inside a drone allows industrial users to see, cover and understand geospatial context faster and better than from other methods such as on a tripod or on a handheld system.
Lidar can build accurate reality models in either low light or heavy foliage areas as well as reach locations that humans cannot.
Logistics is another use case where lidar is being equipped to increase efficiency and accuracy. Intel last month introduced volumetric measurement software to be used with its RealSense lidar camera designed specifically to measure packages and pallets.
Intel said this solution improves logistics by scanning and calculating the measurement and volume of packages and pallets in seconds, which helps logistics manufacturers to make real-time decisions across inventory management and shipping warehouses.
Critical infrastructure and speed enforcement
Lidar is being used to create 3D maps with detailed information regarding critical infrastructure such as the electronic power grid. Source: Velodyne
Additional use cases for lidar have been announced by Velodyne, a maker of lidar cameras, including the use of the technology to monitor electric power grids and operate the grid more safely and efficiently. The system can detect problems that include lines that are sagging or damaged by a storm as well as keep tabs on these power systems to identify unused capacity on transmission lines to make it easier to integrate renewable energy resources into the power grid.
Velodyne’s lidar systems are also being used in Parifex’s intelligent transportation systems (ITS) aimed to improve connected traffic control services on French streets. The traffic monitoring solutions are unmanned and enable accurate speed enforcement measurement to prevent tailgating, sudden change in direction of vehicles and identify areas of heavy traffic.
Parifex said lidar can be used to gather real-time information on moving objects in its field of view and could eventually be used for smart intersections, access control, smart parking, accident detection and other applications.
Parifex is conducting a pilot project to monitor a fleet of vehicles in a smart parking project that consists of hundreds of cars linked to a key box system. Finding the correct vehicle with the right keys was a challenge, but lidar was able to track and locate each vehicle more efficiently.
