The breadth of applications for which multispectral/hyperspectral imaging is an appropriate technology is constantly expanding. This technology is a main driver for non-factory vision/imaging applications. Because this imaging technique is still being tested for usefulness across myriad industries, the list of applications continues to increase, but it is impossible to list every one in one article. Following are emerging applications commonly cited in our query of various multispectral/hyperspectral imaging technology providers and a round-up of current product offerings.
Optical Sorting
The word “sorting” is used a lot when talking about where multispectral/hyperspectral imaging is being employed, and “optical sorting” is a term used to describe many of the applications for which multispectral/hyperspectral imaging is a solution. Optical sorting includes applications in many areas. “With the worldwide drive toward increased recycling, multispectral and hyperspectral applications in this industry are growing,” comments Simon Stanley, Director of Technology, ProPhotonix (Salem, NH, USA; www.prophotonix.com). “Multispectral and hyperspectral imaging can be used in the sorting of recyclable material, particularly in the recycling of textiles, where the different wavelengths are used to identify the difference between organic and synthetic materials, for example.”
Among other examples, Xing-Fei He, Senior Product Manager, Teledyne DALSA (Waterloo, ON, Canada; www.teledynedalsa.com) cites food sorting and recycling sorting as examples, where the NIR channel provides further spectral information about rice, tea, nuts, etc. that enables more accurate sorting based on properties that are invisible to the eye, such as the moisture levels in food items. This lets foreign materials, such as glass and silicone to be detected easily with NIR imaging and removed.
Specific to recycling, Minna Törmälä, Global Marketing Manager, SPECIM Spectrum Imaging, Ltd. (Oulu, Finland; www.specim.fi), states that this type of imaging is suitable even for black plastics sorting, which is impossible with conventional imaging techniques, and textile waste sorting, which is a big challenge worldwide.
Steve Kinney, Director of Training, Compliance + Technical Solutions for Smart Vision Lights (Norton Shores, MI, USA; www.smartvisionlights.com), says, “A large percentage of the US food supply is inspected with machine vision somewhere between its origin and reaching the consumer shelf. Examples include using NIR or SWIR to remove rocks or other inorganic material from various types of beans, rice, and grains; using a combination of color and NIR/SWIR to detect if fruit is ripe (color) and find bruises (NIR/SWIR) in the same system; and separating harvestable product from chaff (corn kernels from husk fragments).”
Regarding food, Stanley adds, “Optical sorting has long been used in the food industry to sort and grade food products. Multispectral and hyperspectral technology has enabled superior optical sorting in this industry, with these systems now being utilized to detect moisture content, foreign objects, and contaminants. In sorting and grading fish, hyperspectral imaging is now being used to inspect for parasites and to grade the quality of the fish based on the levels of certain organic material within the fish itself.”
Rei Yamaguchi, Vice President, Industrial Optics & New Business Development, Tamron USA (Commack, NY, USA; www.tamron-usa.com), comments, “Applications like food inspection and sorting require the SWIR wavelength to see, for example, bruising of fruit that cannot be seen in visible light since the shorter wavelengths provide better contrast and resolution compared with visible light.”
Inspection
There are a variety of inspections for which multispectral/hyperspectral imaging is suitable.
For example, He notes that it can be used for print inspection, e.g. banknotes and security documents. “The NIR channel is used to inspect security inks printed on the banknotes or documents that only reflect NIR light; this is a measure to prevent counterfeiting,” he says. He adds that it can also be used to inspect PCB substrates. “The NIR channel has deeper penetration depth so it can further detect defects underneath the surface.”
Regarding print inspection, Stanley says, “Multispectral and hyperspectral imaging techniques are also being used in print inspection including currency inspection where UV or IR wavelengths can be used to inspect security features while other wavelengths are used to inspect the printing.”
Kinney says, “Increasingly, we see UV, NIR, and/or SWIR wavelengths used in conjunction with either monochrome or color visible imaging. A recent example of this that we worked on was helping a manufacturer of heating pads. The heating pad internals consisted of a serpentine heating element mounted to a web mesh with ~1-cm square openings. The visible inspection was necessary to locate and place the element onto the web along with properly placing the thermostat and electrical controls. The manufacturer used a neutral color, silicone-based sealant over the serpentine heating element to affix it to the web backing. It was difficult to distinguish the silicone adhesive from the similarly colored web backing to check if the adhesive was in the right place. This is important because heating pads are flexible and the silicone had to be present along 100% of the wire length to ensure there wasn’t any separation or movement of the coil as the product was flexed during use. This required both visible and UV images to complete all inspection tasks reliably.”
Mark Kolvites, Senior Technical Sales Manager, Metaphase Technologies (Bristol, PA, USA; www.metaphase-tech.com), adds, “Automated inspection of nuts (e.g., walnuts, hazlenuts, pecans, etc.) flowing on a conveyor for contaminants can be performed with typical machine vision cameras and monochrome illumination to a certain degree, but a much more robust solution is possible when using hyperspectral. Machine vision users find typical machine vision systems can be configured to detect known defects (metal bolt, different color piece of plastic, etc.) but if the contaminants can be anything, with no prior knowledge of the defect types, including defects that look just like the nut, but are in fact are not a nut, hyperspectral technology is required. To increase reliability in this type of inspection, the system needs to detect anything that is not a nut. As we know, hyperspectral is ideal when the system needs to differentiate between objects that look the same as the item, but are in fact not the same, i.e. nut vs. stone, sugar vs. salt.”
Pharmaceutical inspection is another area where hyperspectral/multispectral imaging is finding favor. Although traditional vision systems have been used in blister pack inspection for some time, Stanley says, “Hyperspectral imaging can now be used not only to inspect packaging but to analyze the pills within the blister pack for verification and quality control purposes.” Kinney adds, “Multispectral imaging has uses in pharmaceutical production and the related track, trace, and verify operations now being required by legislation to ensure product safety and anti-tampering. Most security and tax seals, as well as some hidden product and manufacturing codes, use multi-band protections. A multispectral imaging system and specific knowledge is required to correctly identify or decode security and tax seals or reveal the manufacturing lot codes and other info hidden from the consumer.”
Agriculture
“An interesting emerging application for hyperspectral and multispectral imaging is agriculture,” says Nick Sischka, Director of Imaging Product Development, Edmund Optics (Barrington, NJ, USA; www.edmundoptics.com). “Imaging with additional wavelengths outside of the visible spectra allows for the observation of important things like the presence of disease in crops, relative water levels, and the ripeness of fruit. This additional information can be analyzed because of the crops’ unique absorption, transmission, and reflection of infrared and ultraviolet wavelengths.”
Similar to optical sorting, Kinney says, “Multispectral imaging is being used in the field, in greenhouses, and (increasingly now) for vertical farming. Multispectral imaging can help decide if a product is ripe for harvesting; identify harvestable product from the plant; tag fungus and diseases on crops for eradication before they become a problem; and identify plants from weeds to precisely target and apply fertilizers and pesticides only to the plant, reducing ground contamination.”
“More than 50% of our applications are in agriculture, forestry, smart farming or food safety and security,” says Andras Jung, Co-founder of Cubert, Inc. (Ulm, Germany; www.cubert-hyperspectral.com) He adds that there is a growing demand for water- and underwater spectroscopy.
Additional Applications
Other applications that are emerging include: color measurement, for manufacturing quality control or art heritage analysis; packaging; IR microscopy; semiconductor inspection; medical; aerial mapping; remote sensing in agriculture/horticulture industries; waste sorting; moisture measurement; chemical imaging; threat detection; methane gas detection; and oil, gas, and mineral exploration. “In medical applications,” says Yamaguchi, “the longer wavelength allows for deep tissue imaging, e.g., vessels not seen with visible light are now able to be seen.” Jung also adds that applications for neurologic brain research are emerging.
Samiul Haque, Senior Product Manager, Emberion, Ltd. (Espoo, Lapland, Finland; www.emberion.com), explains that the industry is keen to detect spectral signatures in wider wavelength ranges to optimize the type of materials detection and ensure selectivity during their sorting (food and waste recycling) or detection process. One example is the water peak at 1920 nm that provides a high contrast compared with the traditional 1450-nm peak. Cameras with wide wavelengths can detect that high contrast peak for moisture detection and moisture ingress.
Mark Donaghy, VP, Sales and Marketing, Raptor Photonics (Larne, Co. Antrim, UK; www.raptorphotonics.com), adds, “The high spectral resolution of a hyperspectral imaging system allows for detection, identification, and quantification of surface materials, as well as inferring biological and chemical processes. We see an uptick from factory line-belts, viewing platforms, robots, and drones all the way up to Mini/Cubesats in space. It is a very time-efficient technique offering instantaneous data.”
Lighting
Lighting can be challenging in any machine vision application, and there are various hyperspectral light sources from which systems integrators can choose.
“The number of high-spec hyperspectral cameras at reasonable prices is growing,” says Daiki Suzuki, New Application Development Section, Section Manager, CCS, Inc. (Kyoto, Japan; www.ccs-grp.com). “The range of applications—and hyperspectral light sources—is expanding accordingly. There are several types of lighting available for hyperspectral cameras.”
Suzuki explains that to maximize a hyperspectral camera’s performance, it is best to obtain spectroscopic information over a wide wavelength range. Different sensors are sensitive to different wavelengths, so lighting for hyperspectral cameras thus requires a wide wavelength range from visible to SWIR.
“Applications for hyperspectral cameras are expected to expand further in the future,” he adds. “The specifications required for lighting will become more strict and more diversified in turn.”
Solutions
The LUCID Vision Labs (Richmond, BC, Canada; www.thinklucid.com) Atlas™ SWIR (Figure 1) is a GigE PoE+ camera featuring wide-band and high-sensitivity Sony SenSWIR 1.3-MPixel IMX990 and 0.3-MPixel IMX991 InGaAs sensors, capable of capturing images across both visible and invisible light spectrums, and boasting a miniaturized pixel size of 5 μm.
AVAL Global’s (Yongin-si, Gyeonggi-do, The Republic of Korea; www.aval-global.com) AHS-003VIR hyperspectral camera (Figure 2) can simultaneously acquire two-dimensional spatial information of XY coordinates and data of wavelength axis (spectrum information). Wavelengths of 450 to 1700 nm are spectroscopically 640 pixel lines and acquired as spectral information of 512 bands.
Using ultra-low dispersion glass and low partial dispersion glass, Computar Optics Group’s (Cary, NC, US; www.computar.com) ViSWIR Hyper/Multi-Spectral Lens Series (Figure 3) minimizes focus shift within a few microns in the 400- to 1700-nm visible and shortwave infrared range. A broadband anti-reflection coating applied to all lens elements minimizes adverse lighting effects.
Teledyne DALSA’s Linea ML 8k multispectral camera (Figure 4) provides spectrally independent RGB and NIR outputs with minimal spectral crosstalk. For inspection applications such as banknotes and other high security prints, the Linea ML 8k multispectral camera accurately detects security inks printed on the paper that only reflect near infrared (NIR) light. In other applications like semiconductor wafer and printed circuit board (PCB) inspection, the camera easily reveals defects below the surface penetrated by the NIR light. Previously, customers who required both color and NIR imaging needed to use one color camera and another NIR camera separately. By combining both capabilities into one camera, the ML 8k simplifies system designs.
The DoAll light (Figure 5), by Smart Vision Lights, is an integrated product combining six form factors into one. Because of its RGBW and NIR ring lights included in the standard configuration, the DoAll has native multispectral capabilities. Special orders allow UV and other wavelengths to be added. The DoAll also features two auxiliary ports, allowing lights of any spectrum to be added and controlled, further extending the multispectral possibilities.
The Specim FX series cameras (Figure 6) provide reliable classification results based on the target’s chemical composition. Specim FX series cameras are designed specifically for industrial use. The cameras’ high frame rates meet industry speed requirements, and robust structures and small sizes allow flexible installation locations. They are widely used in food and recycling industries for sorting and quality control.
Metaphase has introduced the RGB + IR LED Line Light (Figure 7). Traditionally multispectral line scan inspection applications, such as agriculture inspection, required multiple inspection stations, each with its own camera and unique single wavelength (color) LED light. Multispectral cameras along with these line lights configured with individually controllable RGB & IR LEDs can perform these inspections at a single station. The lights are individually controllable RGB and IR LEDs to maximize contrast.
TECHSPEC® Zinc Sulfide Multispectral Windows (Figure 8), from Edmund Optics, are manufactured from a water-free, multispectral grade of zinc sulfide that has low absorption and scatter properties over its transmission range of 0.4 to 12 μm. Zinc Sulfide Multispectral is a chemically vapor deposited material that is commonly used in thermal imaging. Zinc Sulfide Multispectral is suitable for systems that have both visible and mid-wave or long-wave infrared sensors. It is slightly harder and more chemically resistant than zinc selenide.
ProPhotonix’s COBRA HyperSpec (Figure 9) offers hyperspectral tunable illumination for optimal image acquisition. Discrete control of up to 12 different wavelengths allows for the creation of a broad spectrum with excellent HO spatial and spectral uniformity and high-quality imaging. Visible and IR options (400 to 1000 nm) are available as well as a new SWIR option (950 to 1750 nm).
Tamron’s SMA series (Figure 10) covers the whole spectrum from visible light to SWIR band (400 to 1700 nm). The SMA series (12 mm, 16 mm, and 25 mm) avoids defocused objects and provides consistent high-resolution images and high spectral transmittance across the entire spectrum from visible light to SWIR band. The main character of the SMA series is that the need to refocus for different spectral bands becomes obsolete.
Cubert’s Ultris 5 (Figure 11) is a miniature (30 × 30 × 50 mm, 120 g) snapshot hyperspectral imaging camera based on light field technology, where both the intensity and direction of incident light rays are used to produce 80000 spectra/image. The camera combines a continuously variable bandpass filter, a multi-lens array and a high-speed CMOS sensor. This spectral camera comes with a native resolution of 290 × 275 pixels.
Emberion’s upcoming VS20 GigE Vision high speed 400-fps camera (Figure 12) with VGA resolution features sensors that have high pixel operability of greater than 99.98% and HDR up to 120 dB. The images the cameras capture enable detection of both bright and dark objects and allow for multispectral imaging and enabling hyperspectral solutions.
Raptor Photonics offers the Owl 320 HS (Figure 13), a high-speed Vis-SWIR camera, offering resolution from 600-1700 nm. It runs at 349 Hz in full resolution (faster if ROI’d or binned).
The EFFI-FLEX-HSI hyperspectral LED line light (Figure 14) uses broadband LEDs. The irradiation range can be adjusted by changing the lens position. In addition, it is possible to select from different diffusion plates that emphasize brightness or uniformity or a cylindrical lens that focuses light and attach it to the front of the light-emitting surface.
