The following is a Q&A with Marc Landman, Vice President, CCS America in which computational imaging technologies, benefits, products, and applications within machine vision are discussed.
You recently released the LSS-2404 lighting controller for computational imaging. What led to this product being developed, and why?
Precision illumination is always critical to high-performance machine vision systems. CCS recognized the fact that by combining illumination with computation, one can take imaging technology to the next level.
The LSS-2404 programmable lighting controller was developed primarily to serve as the “heart” of any computational imaging system. Image acquisition has not changed appreciably since the beginning of the machine vision industry roughly 40 years ago. Cameras take a single exposure, using fixed optics and lighting conditions.
When system builders are faced with tough imaging challenges, this model doesn’t work very well. Typical solutions are to try different lighting and/or lensing or to “fix” the problem of poor image quality in software. Both approaches add extra time and cost to imaging solutions, and still often fall short in producing optimal images.
But with today’s high-speed image sensors and data interfaces, there is now an alternative approach that can often improve image quality or produce images that were previously impossible. This approach is called multi-shot or computational imaging, and it relies on capturing a sequence of images taken under different lighting or optical conditions. These images are processed with image processing software, to create an output image that enhances image quality or extracts the features needed for a specific machine vision task.
These computational imaging sequences can be performed at high speeds with most digital cameras and an ordinary PC or processor, so even rapidly-moving parts are supported. By focusing on the image properties most important to a particular machine vision task, computational imaging offers powerful advantages over traditional one-shot imaging.
In your words, what is computational imaging, and why would someone decide to utilize the technology?
Computational imaging (CI) refers to digital image capture and processing techniques that combine computation and optical encoding. Fundamentally, CI relies on data extracted and computed from a series of input images captured under different lighting or optical conditions. There are a few basic principles involved:
· Computation is inherent in the image formation process
· CI combines special lighting and/or optics along with image processing during image capture
· CI typically involves a sequence of images with different illumination for each frame
· CI covers a wide variety of techniques, all designed to output better images or images with unique characteristics
· CI ends with the image acquisition process – it does not involve image analysis
The benefits of CI are very clear. First and foremost, CI permits better or previously impossible images for machine vision systems to be created at lower cost. Forget about iterative attempts to get the lighting and optics right, forget about post-processing to enhance images in software. CI, with its targeted feature extraction, directly outputs the image your vision algorithm needs. This is turn allows you to shorten development time and create superior machine vision solutions.
Whether you need depth information, better color, more image contrast, more depth of field, or even if you’d like to combine lighting techniques or leverage multispectral information, CI allows you to get the image you need, in a very efficient way. Finally, CI allows you to innovate. With CI, system designers can start to think in new ways about creating solutions to difficult imaging problems.
What does this product do differently than other similar products?
Well, at this point in time products for CI are just beginning to appear in the machine vision industry. One of the first tools to be introduced was photometric stereo, which uses a technique called shape from shading to separate the shape of an object from its 2D texture. In this way, photometric stereo accentuates 3D surface structure, making it an ideal tool to inspect surface details like scratches, dents, pin holes, raised printing, or engraved characters, especially on low contrast surfaces.
But while photometric stereo works by capturing images illuminated from multiple directions, much more is possible when you vary the lighting intensity, wavelength, angle, focus or other optical parameters.
Tools such as high dynamic range (HDR) imaging, ultra-resolution color with a monochrome imager, extended depth of field, and multispectral analysis, are all achievable when you use multi-shot imaging. CCS has demonstrated each of the above-mentioned techniques in real-time, along with others, using a variety of industry-standard hardware and software.
Although this technology is firmly established in the digital photography domain, CI has not yet reached widespread deployment in the machine vision industry. As an emerging technology, little CI software exists and what does exist is largely proprietary and platform-specific. In fact, the main barrier is the closed-architecture nature of most existing solutions.
The LSS-2404 series from CCS changes that. It is an open-architecture lighting system for computational imaging and is designed to be integrated into any machine vison camera and many smart cameras. CCS is working with many of the leading machine vision companies to integrate the LSS series products and provide the necessary software support. You can combine the LSS-2404 controller and lighting products with cameras and software from nearly every major machine vision supplier, and easily create computational imaging-based solutions.
The LSS controller is programmable and extremely flexible. With it, you can create arbitrarily complex multi-shot image sequences with the required timing and outputs, to enable virtually any conceivable computational imaging technique.
What further plans do you have, that you are willing/able to talk about, for products within computational imaging?
CCS as a company is very committed to the adoption of computational imaging in the machine vision industry. I mentioned earlier how commonplace this technology is in digital photography, with features like HDR, optical image stabilization, red eye reduction, face/smile detection, and post-capture refocusing available in many consumer cameras and smartphones.
Computational imaging in machine vision is the next progression towards achieving higher levels of image quality and ultimately more reliable solutions. CCS is fully committed to making CI a mainstream technology through innovative products, education and industry-wide partnerships. >>> Learn more here.
There are many more advances to be made in computational imaging. It is currently a very active research area in the scientific and academic community. Although we can’t disclose specific plans here, CCS intends to develop and release a continuous stream of CI techniques to offer the best possible imaging tools to our partners and customers.
Are there any types of applications that would be improved upon by using computational imaging technologies?
Of course, there are many! When you struggle to achieve good images using conventional lighting and optics, CI can often provide them. Whether you need better contrast, better color, more depth of field, 3D surface information, or many other targeted image characteristics, CI offers techniques to get you images that are optimized for your vision algorithm. The output of many CI tools are images that are already segmented or have the features relevant to your task highlighted or extracted.
What new technologies and requirements do you see as key machine vision application drivers?
We see a number of changes occurring in the machine vision industry. Overall, applications for machine vision technology are increasing in their demands, and the market is widening. There are many sectors now outside traditional industrial inspection. For manufacturing, increased detail and data about objects being inspected is becoming increasingly important. So is faster throughput in manufacturing processes. In many plants, greater flexibility is required in production machinery and manufacturing processes, which results in greater use of automation and robotics including machine vision integrated with automation control systems.
Increased intelligence and performance from machine vision systems as well as new market sectors are driving developments in imaging and lighting technologies. For example, so-called “intelligent lighting” can provide lighting lifetime data for preventive maintenance via Industry 4.0 as well as guaranteed consistent lighting intensities and resolution through in-built lighting feedback. Where increased throughput is required for high-speed processes, very high-intensity LED lighting has been developed for both continuous line scan and pulsed applications.
As 3D inspection systems continue to expand, new capabilities in structured lighting are being developed. Deep learning software, when combined with advanced lighting and imaging technologies, has the possibility to provide even higher levels of performance than today’s conventionally programmed vision systems.
In short, as lighting becomes further integrated into machine vision systems, it is critical that these technologies be delivered in an open-architecture way, and advanced functionality such as CI should be provided within easy to use systems and available with industry-standard imaging software.
