Responding to a corporate goal to be environmentally sensitive, Nestlé HealthScience decided to change the color of measuring scoops (from red or green to transparent) included in cardboard canisters, or boxes, of adult supplemental nutrition powder.
“The colored scoops are very hard to recycle because of the colored pigments,” explains Klaus Keitel, national account manager for strategic customers at SICK (Waldkirch, Germany), which designed an AI-enabled machine vision system for Nestlé (Vevey, Switzerland).
But the color change had an unintended effect on the automated inspection process at the company’s manufacturing plant in Osthofen, Germany, where the nutritional supplement is produced: The automated inspection system could no longer verify the presence of the scoops in the cardboard canisters.
When the problem first surfaced, plant managers assigned an employee on each shift to the task of watching the process and pulling canisters without scoops from the line. “After an hour, it was a hard job,” explains Armando Simoni, a technician focused on measurement and project management at the company’s plant in Osthofen.
Not surprisingly, Nestlé opted to replace the machine vision system with the new one from SICK, which uses deep learning algorithms to detect the transparent scoops.
Automated Production Process
On the manufacturing line, a measuring scoop drops from 20 cm above the production line into an empty canister, which is positioned upside down, so that the scoop ends up near the top of the can, just under an aluminum seal, when the production process is complete. After the presence of the scoop is verified, the powdered mix is added. This happens at a rate of between 40 and 80 canisters per minute.
About 13 different adult nutrition products—different formulas and can sizes—are produced on a rotating basis on a single line at the plant. The brands include Optifibre, Resource Dextrine Maltose, Resource Maltodextrin, Thicken up Junior, Resource Thicken Up, and Resource Espesante.
Positioned on the line, the old machine vision camera system snapped an image right after the scoop was added. It would then count the colored pixels in the image to determine if the scoop was present. This process worked fine with a colored scoop. But this was not the case with the transparent scoop because the gray aluminum seal, which is what the consumer removes to access the powder, reflected onto the transparent plastic of the scoop, making it appear gray. With two gray surfaces, the camera system couldn’t distinguish between the scoop and aluminum seal.
“The classical vision system could only count whether it's a dark or it's a white pixel,” Keitel explains.
The Deep Learning-enabled Inspection System
Once Nestlé decided to replace the vision system, plant managers reached out to numerous machine vision companies for proposals. Most of the companies quoted a 90% accuracy rate, while SICK’s sales representatives said their system would achieve a 99% rate by using algorithms enabled with AI, Simoni recalls.
The food and nutrition company’s RFP was challenging for a variety of reasons, according to Keitel. First, Nestlé went through a transition period in which it used both colored and transparent scoops because it wanted to deplete its inventory of the old scoops. Second, Nestle’s plant managers wanted to transition from filling one type of container to the next on the production line without adjusting settings on the camera, according to Keitel.
Work began on design and testing of the SICK system in April 2021; it has been operating in a production capacity since June 2022.
Related: Deep Learning/AI System Provides 100% Inspection for Food Producer
Machine Vision Components
To design a system that can detect the presence of the transparent scoop, SICK incorporated deep learning algorithms with a monochrome 2D snapshot camera, picoCam2, which is a global shutter CMOS camera with a C-mount lens mount. The camera is compatible with the GigE interface for video and data transmission over Ethernet networks.
The system is positioned above the line, so it can snap images of the canisters after the scoop is added. The line is illuminated with a rectangular LED backlight from planistar Lichttechnik GmbH (Himmelstadt, Germany).
To develop the algorithms, SICK used its deep learning software, dStudio, which works with the company’s cameras.
Now that the algorithms have been developed for this application, Simoni can modify them as necessary to accommodate changes in canisters or scoops.
Once the camera takes an image, the algorithm determines whether a scoop is present, and this processing occurs on the camera, which transmits the result to a PLC from Siemens (Munich, Germany). It usually takes about 50 milliseconds for the camera to produce a result and send it to the PLC, Simoni explains.
The PLC is in a control cabinet near the station where the canisters are filled with the powdered supplement. A 5 m 12-wire cable attaches the camera from SICK to the PLC.
Related: Optimizing Machine Vision Lighting for Food and Beverage Inspection
Overcoming Challenges
As is typically the case with deep learning algorithms, their performance in a production environment depends on how well they are trained.
Simoni says Nestlé and SICK learned this lesson while implementing the new machine vision system.
The employees from SICK and Nestlé involved in the project did not get the results they wanted after a first round of training. For round two, they added images of scoops in a variety of orientations, as the exact position of a scoop depends on how well the canister is lined up under the automated mechanism when the scoop drops from above. The conveyor is in constant motion, so the position of the canisters can vary.
They also realized that they needed two neural networks to accommodate the different canister sizes: one for those with diameters of 73 or 99 mm, and a second neural network for canisters with diameters of 140 mm.
System Updates Over Time
In 2023, the plant managers at Nestlé added an automated mechanism to push canisters without scoops off the production line.
With the original setup, the SICK system sent a signal to the PLC to stop the line if it detected a canister without a scoop. Once the missing scoop was added, the camera software detected it and then sent a signal to the PLC to restart the line.
Simoni says the new system is more efficient because it solves the missing scoop problem without halting production.
Overall, the system has been quite successful, Simoni adds, because it identifies canisters without scoops with nearly 100% accuracy.
