Chris Mc Loone
The past two years have made it difficult for systems integrators, OEMs, and end users to travel to see the numerous options available to them to solve their own vision/imaging challenges or those of their customers. That has not stopped manufacturers from innovating to improve products already on the market and developing new ones. The 3D imaging segment of machine vision is no different. Developments have led to continued expansion of 3D imaging’s scope as well as ways to simplify its implementation.
3D imaging is a very suitable solution for a variety of applications. André Kasper, CTO, AT - Automation Technology (Bad Oldesloe, Germany; www.automationtechnology.de), observes, “3D is now much more commonly used in the industry than 2D because 3D imaging, especially by means of laser triangulation, not only provides much more information, but also more reliable and stable results.”
AT - Automation has launched the first sensor module of its C6 series: the cx4090HS module (Figure 1), which complements the company’s modular 3D sensor concept (MCS). “The cx4090HS product is a high-speed module that is nearly twice as fast as all of our other 4K 3D sensors,” says Kasper. “In addition, we have introduced our new C6 firmware that includes new innovative features like MultiPart, MultiPeak, and also GenICam3D.” With the C6 firmware, laser triangulation can now be used in applications such as glass inspection or inspections with secondary and disturbing reflections. Kasper adds, “Our 3D sensors are particularly well suited for surface inspection, geometry control (metrology), and also robot guidance.” The company plans to add another product to its C6 family in 2022.
Renata Sprencz, Head of Marketing, LUCID Vision Labs (Richmond, BC, Canada; www.thinklucid.com), says, “We’ve seen a wide adoption of our 3D ToF cameras in recent years due to improved 3D depth data precision and accuracy thanks to Sony’s DepthSense™ IMX556PLR back-illuminated ToF image sensor technology. In addition, our Factory Tough™ industrial platform, compact camera size, new feature sets such as HDR and High-Speed Mode, and competitive price point present a new set of opportunities for users developing 3D imaging systems and enable deployment in challenging environments.”
In 2021, LUCID Vision Labs introduced the Helios2+ 3D ToF camera (Figure 2), an advanced version of the company’s Helios2 ToF camera, that adds two new on-camera depth processing modes: High Dynamic Range Mode (HDR) and High-Speed Mode. The HDR Mode of the Helios2+ ToF camera combines multiple exposures in the phase domain to provide accurate depth information in high-contrast, complex scenes containing both high- and low-reflectivity objects. The High-Speed Mode enables depth perception using a single-phase measurement, allowing for faster acquisition speed and higher frame rates compatible with moving object perception. In High-Speed Mode the Helios2+ camera can run up to 103 fps at 2.5-m distance. Sprencz comments, “Our Helios2+ ToF camera is ideal for many demanding 3D applications such as 3D inspection, automated material handling, and robotics to run more efficiently while reducing overall system cost.”
Sahara Yang, CEO, Vzense (Newark, CA, USA; www.vzense.com), says “3D imaging will change the world we have already realized. There will be more applications we can expect and work on.” The Vzense DCAM550 is capable of interpreting reliable and accurate 3D depth data and is easily customizable for various logistics and mobile robot integrations like obstacle avoidance, palletizing, and parcel singulation. The DCAM550 has a wide field of view (69°, 51°, and up to 120° horizontally customizable) and VGA resolution (640 × 480 pixels) along with optional IP67 water resistance. The Vzense DCAM710 (Figure 3) allows customers to evaluate or study small quantity pilot production. With the wide ToF field of view (69°, 51°), RGB field of view (73°, 42°), VGA resolution (640 × 480 pixels), and RGB resolution (1920 × 1080 pixels), the DCAM710 is suitable for research on logistics, mobile robot, industrial automation, people behavior detection, etc. The Vzense DCAM560C RGB-Depth camera for industrial use will be launched in Q1, 2022.
“Recently, we are seeing an increased number of customers looking for 3D solutions to solve problems that conventional 2D inspection couldn’t address, says Mathieu Larouche, Product Manager for 3D Sensors, Matrox® Imaging (Dorval, Quebec, Canada; www.matrox.com/imaging). “These would be applications involving, for example, distinguishing the different sizes, shapes, and positioning of an object.
“A challenge, however, is that many of these customers are new to or not entirely comfortable with 3D technologies and how these integrate into existing vision systems, or even how to begin development of a new 3D-based system. They are looking for a simple, streamlined solution for using 3D imaging, and that’s exactly where Matrox Imaging is currently focusing our efforts, through the creation of tools like Matrox Capture Works, whose graphical interface is easy to navigate and control, as well as through regular releases and updates to software and firmware that make it easier to develop 3D-based applications.”
In late 2020, Matrox Imaging released its Matrox AltiZ high-fidelity 3D profile sensors (Figure 4) for machine vision. They are used to generate consistent profiles, depth maps, or point clouds. To streamline integration, the Matrox AltiZ makes use of a GigE Vision interface, with GenICam GenDC support, which makes it fully compatible with Matrox Imaging software and third-party software as well. During 2021, the company worked on expanding its offering and to further flesh out its capabilities in this space. “First, we worked on enhancing the performance to achieve even higher scan rates; the improved scan rates can reach upward of 11,000 profiles per second in alternating acquisition mode,” says Larouche. “We improved the firmware of the Matrox AltiZ to make 3D acquisition even easier to do because we are always looking to improve the user experience and make integrating 3D technology into vision applications an easier process.” To facilitate uptake and integration of 3D technology, Matrox Imaging developed Matrox Capture Works, a configuration and setup utility provided with all its vision software—Matrox Imaging Library (MIL) X software development kit, its subset MIL-Lite X, and the Matrox Design Assistant® X integrated development environment. “We also released new 3D tools in the latest service pack for the MIL X SDK and in the latest version of Matrox Design Assistant X, our flowchart-based IDE for machine vision,” adds Larouche.
Matrox Imaging’s 3D sensors are optimized for use in any industry that leverages 3D technology, including quality control applications, especially in the food and beverage sectors; packaging; as well as in the automotive manufacturing industry.
Sanna Leinius, Product Market Manager 3D Business, Basler AG (Ahrensburg, Germany; www.baslerweb.com), says 3D image processing is used, in particular, whenever volumes, shapes, or the 3D position of objects have to be analyzed or to improve systems’ reliability. The additional depth information helps for demanding vision tasks in constantly changing environments often found in logistics and factories. “Robots are thus enabled to perceive their environment in real time and act flexibly in dynamic processes,” she says. “Every 3D imaging technology has its pros and cons. It is important to thoroughly examine which solution is best for the specific vision task.”
In the first quarter of 2022, Basler will introduce stereo vision solutions (Figure 5) to enable any robotic system to reliably perceive its environment in real time. Stereo vision allows position, orientation, and object size detection with the combination of classical and AI-based methods. The Basler stereo vision solution consists of industrial-grade stereo cameras in two baselines: a 65-mm variant for on-arm robot-mounted 3D vision and a 160-mm variant suitable for static mounting e.g. in a robot cell. Aligned high-end accessories as well as application-oriented software modules complete the offering.
Besides the new stereo vision offering, Basler also further enhanced its blaze ToF camera, introducing new features like HDR and integrating the camera’s SDK into the Basler pylon software environment. The Basler stereo vision system is a plug-and-play machine vision solution that requires neither experience nor expertise in machine vision. It is designed to support robotic applications ranging from bin picking to navigation. With its onboard processing capabilities, it can be integrated directly, with typically no need for external computers. Its intuitive Web interface makes it easy to implement.
Both 3D imaging technologies, stereo vision and ToF, are suitable for various machine vision tasks in logistics, industrial automation, agriculture, and healthcare.
Heiko Seitz, Technical Author, IDS Imaging Development Systems GmbH (Obersulm, Germany; www.ids-imaging.com), says, “3D camera technology enables robots to recognize their environment in order to react independently to different conditions. The perception of the environment with 3D camera data thus enables many innovative applications that could previously only be done by humans. It is thus an essential building block for automation processes. However, as with 2D imaging, the 3D camera hardware and its imaging processes are not the only key to good 3D imaging.”
All Ensenso 3D camera families are easy to set up and operate via the Ensenso SDK. In addition to wizards for easy setup and to support 3D camera calibration, the SDK also includes the option for GPU-based image processing for even faster 3D data processing. With sample programs and source code, it can be used to realize 3D point clouds from multiple perspectives of Ensensos in multicamera operation or to integrate 2D features, barcodes, and color information via IDS industrial cameras. Even frequently required tasks, such as object recognition, are now an integrated part of the Ensenso software package with the “Partfinder.” The more user-friendly and complete the software, the better the overall 3D system.
“With the Ensenso S (Figure 6), we have expanded our portfolio with an ultra-compact and consistently cost-optimized 3D camera,” says Seitz. “It is an industrial-grade alternative for applications where previous systems were too expensive and complex. For this purpose, we have optimized the 3D technology with structured laser light to maximize the quality and robustness of the data and at the same time reduce costs.” AI technology supports the matching and triangulation of the 3D raw data. The S series, according to Seitz, is therefore an affordable 3D universal camera that can be used in a wide range of applications, e.g. surveillance, autonomously driving vehicles, or in robotics. At the same time, it is 100% compatible with the Ensenso SDK.
Mark Davidson, CRO, DreamVu (Philadelphia, PA, USA; www.dreamvu.com), citessoftware and hardware innovations for 3D imaging in 2021 from his company. “From a software point of view, 2021 was really the big year in software. One is obstacle detection for mobile robots, and the innovation is 3D obstacle detection across the whole horizontal field of view and vertical field of view but flattened into a 2D laser scan,” he explains. “Because what we found is so much of the customers’ infrastructure today is using 2D Lidar for obstacle detection, and they’re used to everything being a 2D data set. For them to take advantage of 3D obstacle detection requires so much extra memory, processing, etc. So, flattening our 3D into a 2D scan has been a big impact on our ability to give customers this 3D obstacle detection in an architecture that they can take advantage of.”
Another software innovation he cites is accurately implementing people detection with depth. “The big challenge is tracking,” he says. “And when I say tracking, I mean if I identify you in the field of view and I label you as person 001, making sure that I don’t lose you as 001 as you move around and other people come in and crisscross the scene. Because the field of view is so large, you never leave the frame. This provides the ability to find somebody and not lose them. We had one of our people go around the corner and come back, and it was able to hold on to that person. So, person detection with tracking and, of course, tracking plus the coordinate information telling you exactly where that person is. And then when I know the where—we update that about 10 times per second—I can do flow management. I can predict where these people are going. One use case is having an AMR follow a person and not get confused. Let’s say in a commercial cleaning environment, somebody is cleaning an airport, and they want the robot to follow them from point A to point B. Well, airports are busy with lots of people. That’s one use case that we see or in a warehouse as well.”
On the hardware side, DreamVu has been working to combine 3D vision with active projection or IR laser. “We have prototype cameras in the lab right now,” says Davidson. “We’ve shared some data with customers, but we will, in the first quarter, be sampling that product, combining vision and IR projection with our 360° field of view. This solution can work in complete darkness, and IR projection gives better depth accuracy—better than what stereo vision does—so I’m very, very excited about that one.”
At the turn of 2021, Photoneo (Bratislava, Slovakia; www.photoneo.com) officially released its 3D camera, MotionCam-3D (Figure 7), which remained the major focus of the company’s activities throughout 2021. MotionCam-3D is powered by the patented Parallel Structured Light, which is enabled by Photoneo’s proprietary CMOS sensor. The MotionCam-3D area scan 3D camera enables the capture of fast-moving objects and their 3D reconstruction in high resolution and accuracy without motion artifacts. The challenge of high-quality 3D scanning of dynamic scenes could not be overcome with conventional 3D sensing approaches powering area scan cameras, which can either provide a high quality of 3D image data or fast scanning speed. MotionCam-3D can capture objects moving up to 144 kilometers per hour and provide a point cloud resolution of 0.9 MPixels and an accuracy of 300 to 1.250 μm across its five models. The camera can also be used for scanning static scenes, in which case it can provide a resolution and accuracy of up to 2 MPixels and 150 to 900 μm respectively. Andrea Pufflerova, PR Specialist with Photoneo, says, “MotionCam-3D decreases cycle times and latency, which also greatly benefits hand-eye applications—a robot does not need to stop its movement to make a scan, as is the case with standard 3D sensing technologies. Thanks to its effective resistance to vibrations and movement, the camera also enables the automation of completely new tasks in agriculture for the scanning of animals and plants, or in the medical sector for the capture of human body parts.”
Besides MotionCam-3D, Photoneo also provides industrial-grade 3D scanners of the PhoXi family, which provide submillimeter resolution, accuracy, and a high level of detail for scanning static scenes.
Vincenzo Forte, Industry 4.0 Project and Product Manager, Terabee Inc. (Saint-Genis-Pouilly, France; www.terabee.com), says 3D imaging is attracting interest in many industrial and non-industrial fields. “The importance of having industrial-compliant devices at affordable prices is key for developing innovative applications,” he says.
The Terabee 3Dcam VGA’s (Figure 8) open system allows computer and machine vision integrators to develop custom applications with high level coding flexibility on a camera with high resolution and field of view. It features open on-board, Linux-based computing power. Its resolution, more than 300 k pixels (640 × 480), and several outputs, i.e. depth, near-infrared signal level, point cloud, and image quality representation, allow the device to spot 3D details up to the millimeter scale. The camera is laser class 1 and has a range up to 5 m, as well as a field of view of 90° × 67.5°.
The device is provided in a bundle with user manual and tutorial videos. In addition, a Terabee 3Dcam VGA SDK includes: APIs for applications development on client and onboard, with examples in C++ and Python; a GUI both in Windows and Linux; and Ethernet, RS485, and digital/PWM outputs to be easily integrated in control systems.
Yoann Lochardet, Marketing Manager, 3D, Teledyne e2v (Seville, Spain; imaging.teledyne-e2v.com), says, “Conceiving and developing a ToF system is normally significantly more complex than a 2D camera since the performance of the distance measurement is dependent on a variety of system-level aspects—timing accuracy, illumination source including eye-safety, optics, etc.”
Inder Kohli, Senior Product Manager, Vision Solutions, Teledyne DALSA, says, “With advancements in processing technology and fueled by AI, customers are looking for ways to extract more information from each processing step. Z-Trak2 sensors provide 3D data that is not easily available in conventional 2D imaging techniques. 3D systems are becoming more reliable, affordable, and are easier to deploy and maintain for factory applications that require 100% inline inspection.”
Teledyne’s main 3D imaging solutions are the Hydra3D ToF sensor and Z-Trak series of laser profilers. The Z-Trak2 (Figure 9) is a factory-calibrated, 3D laser profile sensor that uses 5GigE to deliver 45.5-kHz profile rates. Designed for high-speed 3D applications Z-Trak2 has integrated features to enhance profiles by allowing users to select finite impulse response (FIR) filter stages, single-pass High Dynamic Range (HDR) and real-time inline image filters. Higher quality profiles enable improved measurements and more reliable results. Compliant with the EMVA’s GenICam 3.0 and SFNC 2.3, the Z-Trak family can output data in various 3D formats and works with off-the-shelf 3D software packages.
Hydra3D is the latest high-resolution ToF CMOS image sensor from Teledyne e2v. This sensor introduces several innovations: a three-tap high-speed pixel, a powerful functionality to increase linear dynamic range in a very flexible way, and a multisystem management feature embedded on-chip. This sensor is suitable for multiple markets and applications including vision-guided robotics, logistics, and automated guided vehicles.
Dr. Maximilian Klammer, Head of R&D at Chromasens (Konstanz, Baden Wurttemberg, Germany; www.chromasens.de) states, “3D cameras enable complex measurement tasks to be solved and allow quality inspection to be raised to a new level.”
In 2021, Chromasens introduced the 3DPIXA wave dual 7μm camera (Figure 10). Florian Thienger, Field Application Engineer, 3D at Chromasens, says, “The 3DPIXA wave dual 7μm is capable of scanning at ultra-high resolution of 6.8 µm per pixel with an extremely large scanning width of 100 mm and 14700 measuring points at a high speed of 124 mm/s. The 3DPIXA simultaneously delivers 3D data and color images. All types of illumination can be used for image capturing with 3DPIXA.” Thienger adds that the camera is suitable for applications such as pin inspection/metal sealing surface inspection, BGA inspection/wirebond inspection, cookie inspection and meat inspection, and metal surface inspection.
The primary 3D solution Cognex (Natick, MA, USA; www.cognex.com) introduced in 2021 is the In-Sight® 3D-L4000 (Figure 11), an all-in-one 3D vision system combining 3D laser displacement technology with a smart camera and easy-to-use software. The system enables users to quickly, accurately, and cost effectively solve a wide variety of inspections using true three-dimensional part imaging via point clouds. Unlike typical 3D systems that transform 3D images into a representational 2D height map for basic tool processing, the In-Sight 3D-L4000 allows developers to place vision tools directly on a 3D point cloud of the part. True 3D inspections increase the accuracy of results and expand the types of inspections that can be performed. Because inspections are in 3D, users can immediately experience how the vision tools operate on the actual part or component.
The In-Sight 3D-L4000 includes traditional 3D measurement tools, like plane and height finding, while introducing new, more robust tools designed from the ground up to leverage inspections in true 3D space. The new tools are designed with the simplicity of 2D vision, making them accessible to a range of different users and skill levels. The In-Sight spreadsheet interface, within the In-Sight 3D-L4000 system, allows users to quickly and easily set up and run 3D applications without the need for programming. This intuitive environment simplifies application development and streamlines factory integration with a full I/O and communications function set.
LMI Technologies (Burnaby, BC, Canada; www.lmi3d.com) offers its Gocator® 3D smart sensors (Figure 12) for inline inspection, factory automation, inline quality inspection, material optimization, and smart 3D robot vision in industrial inline applications. This line of laser and structured light sensors delivers 3D scanning, measurement, and control in a single package. Gocator features a continuously expanding suite of FactorySmart® features and functionality designed to solve real-world factory challenges.
Tordivel (Oslo, Norway; www.scorpionvision.com) has introduced several 3D cameras. The Scorpion Foodsafe 3D cameras are based on stereo vision or 3D laser triangulation and offer a working distance of 400 to 3500 mm and integrated polarized light. The Scorpion Foodsafe 3D Venom camera offers single-camera stereo vision, a working range of 200 to 600 mm, integrated polarized light, and resolution of 0.01 mm. Support of AI and neural networks is offered through its Scorpion 3D Stinger Neural PC (Figure 13), which can be panel-mounted. It can be mounted in a Foodsafe cabinet. Everything is powered by Scorpion Neural Factory Version 2.0 for object location, segmentation, and defect classification.
“With today’s demands for higher throughputs; more accurate quality control; and flexible, batch-oriented production, the pressure is mounting to deliver more speed and accuracy from 3D machine vision systems,” says Ryan Morris, Machine Vision Product Manager for SICK USA (Minneapolis, MN, USA; www.sickusa.com). He continues, “The SICK Ruler3000 is an exciting prospect for OEMs, systems integrators and accomplished 3D vision end users. Because so much of the configuration and commissioning work is already done, you can rapidly shortcut to integrating a continuous stream of industry-standard data into higher level systems or robot controls. And, with accurate 3D, reflectance and scattered light measurements in one device, you can evaluate several aspects of the inspection simultaneously to achieve optimum control and measurement confidence.”
The Ruler3000 (Figure 14) combines SICK’s Ranger3 streaming camera with a Class 2 eye-safe laser, preselected optics, and fixed geometries to enable simpler configuration and commissioning. With industry-standard compliance giving comprehensive access to machine vision software tools, the Ruler3000 cuts time and complexity when integrating more demanding inspection, measurement, and robot guidance tasks across a wide range of industries. Powered by SICK’s highly sensitive CMOS sensor and Rapid On-Chip Calculation technology, the Ruler3000’s integrated streaming camera scans reliably at rapid production speeds. It processes up to 15.4 Gbps to enable up to 7000 full-frame 3D profiles per second. The Ruler3000 extracts the true 3D shape of an object, regardless of its contrast or color, as well as simultaneously capturing greyscale and scattered light measurements, so image processing and measurement precision can be optimized.
Thomas Embla Bonnerud, CEO, Zivid (Oslo, Norway; www.zivid.com), says, “3D cameras need to get tougher. They are most often deployed in tough, unforgiving environments. We have built Zivid Two to exacting industrial standards so that it remains calibrated and performant in quite hostile circumstances such as vibration, shock, and temperature fluctuations. Most robot downtime is associated with its vision system; we built a camera that stops this being the case going forward.”
The Zivid Two industrial color 3D camera (Figure 15) is designed for on-arm robot operation, fits in the palm, and weighs less than 1 kg. Zivid has also introduced a full range of accessories for on-arm operation for Zivid Two, including various mounts, enclosures, and industrial, high-tolerance power and Ethernet communication cables for long-life operation and flexibility. “Additionally, we have been releasing SDKs packed with new features for customers to take advantage of with Zivid 3D color cameras,” adds Bonnerud.
Zivid color 3D cameras offer point precision of <50 μm, dimensional trueness error of <0.2%, and rich vivid color. Because the Zivid Two 3D color camera is designed to work mounted on a robot arm, it offers extended overall FoV and multiple FoV zones, allowing a robot to work from multiple bins/operating stations and across a large working area, like the side of a car, all with one robot and one 3D camera.
