Often the sudden appearance of a game changing technology is the result of several enabling technologies that have been slowly and independently evolving. And then until suddenly, these technologies come together in an entirely new way. This was the case for the iPhone in 2007. All the underlying technologies had been evolving for many years, yet it took Steve Jobs to take those technologies and create the iPhone.
This is the case with the new generation of structured light 3D camera systems. The underlying technology has evolved to the point where an entirely new type of 3D imaging is ready for the factory floor.
Enabling technologies
Like many technologies in recent years, 3D machine vision has been enabled by the exponential increase in processing power. Creating 3D point clouds in under a second requires an order of magnitude more computing power than processing a 2D image. Once the processing power was available, machine vision companies began to experiment with different approaches.
One approach was to replicate human stereo vision with two cameras spaced a short distance apart. Unfortunately, the stereo camera approach to machine vision turned out to be limited by two factors. First the systems had trouble accurately judging depth for certain types of shapes unless the cameras were separated by too far a distance to be practical. Spacing the cameras farther apart then led to occlusion, where sections of the target object blocked the view of one of the cameras.
Second, on smooth parts with few features, the machine vision software had difficulty finding a distinctive feature located in both images to allow stitching the two images into a 3D image. This is called the correspondence problem.
Another early 3D imaging technique that is still widely used is laser triangulation. Laser triangulation uses a laser line generator, angled at about 30° from the vertical, to generate a line across the target object perpendicular to its direction of motion. As the part moves beneath the laser, the laser generates a distinct line on the part that highlights the contours to the overhead camera. Because the laser is monochromatic, a filter on the camera lens isolates ambient light outside the laser wavelength. Laser triangulation gives good depth resolution and precision.
However, it is slow because it relies on a single laser scan line, and it requires the target object to be moving at a constant speed. But the way that the laser line clearly showed the Z-axis contour of the target object clearly pointed the way to a better solution.
Breakthroughs
Two breakthroughs in adjacent fields revolutionized 3D machine vision. First, LEDs became bright enough to be used for machine vision FOV illumination. Single color LEDs are also monochromatic, like a laser, so the camera can filter out ambient light. In most structured light systems, blue LEDs are used for a variety of technical reasons, making these vision systems easy to spot on the factory floor.
Second, Texas Instruments developed a technology called digital light processing (DLP) that uses an array of thousands of micromirrors to project images. This technology, as well as a few other similar technologies, have become the dominant technology in all digital projectors.
The combination of these two technologies revolutionized machine vision. A DLP projector with a monochromatic LED light source allows the projection of not just one contour line on the target object like a laser scanner, but hundreds of lines now covering the entire field of view.
A camera or stereo cameras can snap an image of the entire target object covered in a precise pattern of lines, instead of waiting for the laser to scan across the object.
The pattern can be changed multiple times a second, allowing the machine vision software to integrate multiple images of the object with different line patterns. Structured light system optimized for metrology can accurately measure the dimensions of objects with complex surface contours like castings into the micrmeter range.
In stereo camera systems, the light pattern is varied across the surface in a mathematical pattern, making each line readily identifiable in the image. This allows precise stitching of the two images, eliminating the correspondence problem.
3D machine vision for new applications
One of the challenges to using pick and place robots more widely is the bin picking problem. It is difficult for a robot to identify and pick a single part out of a bin full of randomly oriented parts. Structured light 3D machine vision has finally made robot bin picking viable. The structured light can illuminate the entire parts bin at once, and it can use multiple patterns to ensure that the edges and contours of the tangled or complex parts are clearly differentiated. The imaging and vision software part differentiation can occur in under a second, so the imaging and picking can occur at speeds that are practical on the shop floor.
Another application where structured light 3D imaging excels is metrology. The technology has the capability to rapidly measure an entire compound curved surface like a turbine blade, something that is very difficult using a standard CMM, which can only measure a single point at a time. Because 3D imaging is non-contact, it can also measure the surface contours of flexible polymer parts such as automotive body and interior components.
Some limitations may apply
While structured light 3D machine vision is revolutionary, it is not appropriate for every 3D imaging application. First, both the camera and the object must be stationary while the object is being imaged. So it is not ideal for conveyor belt applications.
Second, it is limited in range, since both the intensity of the area illumination and the absolute resolution of the pattern decreases with the square of the distance from the system. Thus, the ability of the projector to visible pattern of sufficient resolution decreases rapidly with distance. For this reason, the optimal range of the sensor is under 2 m, and the closer to the sensor, the higher the precision of the point cloud.
Laser sensors do not suffer from the intensity drop off with distance because a laser beam is highly collimated. This is why various laser based sensors dominate in the vehicle navigation space where longer range sensors are a necessity.
Summary
Like many technologies that were just waiting for the underlying technology to evolve, Structured Light 3D imaging systems were the subject of a few academic papers, and then suddenly, the technology is everywhere. All major vendors of machine vision systems now have at least a first generation in their product line, and there are a fast moving start-ups that are just introducing their second generation products. So, this is a good time to take a look the technology and see if it can work for you.
