FireWire cameras, LED lighting, and conventional blob analysis team up to inspect chocolates at a rapid rate
Cloetta Fazer is the largest candy manufaccturer in Scandinavia, producing vast quantities of its Polly treat and many other confections on a continual basis. It is very important to Cloetta that a Polly always looks like a Polly. To free its workers from the manual inspection process previously used to achieve this, the company turned to Mabema Consulting, a small machine-vision system integrator providing turnkey automatic detection and positioning systems for the past six years, to build an automated Polly inspection and sorting system (see Fig. 1).
A Polly is a small ball with a flat bottom and a characteristic surface topography with irregular furrows and facets. It’s a small marshmallow mound, covered by light or dark chocolate. A Polly is roughly 1.5 cm across at the base and a little shorter than it is wide.
null
Pollies In, Pollies Out
The candy enters the system by means of an external pocket feeder, which delivers it to a vibrating sieve mechanism with three layers of screen plates. These plates contain different size holes that sort out Polly clumps and deviant Pollies, and the whole mechanism vibrates to encourage right-sized pieces onto a conveyer belt. Here, the candies are mechanically sorted into five single-Polly lanes on a conveyor, which moves them back and forth to prevent sticking and sweeps them forward to a staging area (see Fig. 2). The work of the Mabema system ends at a hopper that feeds acceptable Pollies into a large bin, ready to package and ship.
The system moves the Pollies into a series of five holes, through which they drop into five vision subsystems. Each of the five vision subsystems in the sorting system contains a set of sensors, illumination, and three cameras in a ring-shaped housing (see Fig. 3). In operation, as the candy falls into a ring, it triggers a sensor, which triggers an array of LEDs and then an image-capture operation.
Because of the desire for high speed in this application, Mabema gave each subsystem its own dedicated analysis PC responsible for capturing Polly images, making good/bad decisions, and activating (or not activating) a set of air nozzles that blow faulty Pollies into a reject bin. Good Pollies simply fall into the hopper.
The five PCs reside in a conventional 19-in. computer rack near the equipment, along with a sixth PC that consolidates the inspection application, providing centralized control, assigning settings, gathering statistics, and monitoring status. All six PCs, which communicate over 100-Mbit/s Ethernet using an ACE/TAO real-time CORBA interface, are based on the same Asus P5W64-WS PRO I975X SKT-77, which is a PCI motherboard with a 2.4-GHz Intel CORE 2 E6600 microprocessor and 2 Gbytes of 667-MHz DDR2 memory, running the Windows XP Pro operating system.
The control PC also has a plug-in PCI board for control functions such as the conveyor belts: an Advantech PCI-1750 board, which provides 32 channels of isolated digital I/O and timer/counter. The analysis PCs, in turn, each have a Matrox Meteor-II PCI frame-grabber board, which captures the output of up to six cameras at a maximum 30-MHz sampling rate, provides buffer memory to prevent lost images, and streams images to host memory at a maximum sustained rate of 130 Mbytes/s.
The company uses three IEEE 1394 interfaces on each Matrox board to transfer monochrome images from the three cameras in each ring of detection to the associated analysis PC. A trio of Firefly MV cameras from Point Grey Research is placed at equidistant sites around the inside of the ring. The Meteor-II board provides trigger and timer functions.
An IEEE 1394 interface is used because of earlier successes, and because it is present on the Firefly MV camera selected. The original 1394a version of the interface provides sufficient speed for the application.
The integrator chose this particular board-mounted camera for its combination of small size and high speed. Based on a 1/3-in. CMOS imager from Micron Technology, the camera has a native 752 × 480-pixel, wide-VGA format, a typical 60-frame/s capability, and good power efficiency, as well as a small 25 × 40-mm form factor. The company is using the cameras in binning mode, which combines high speed with a reduced 376 × 240-pixel image size.
There is a twofold human interface to the control computer of the inspection and sorting machine. A keyboard-video-mouse controller on the top of the rack is provided for access by the authorized personnel who set up the application and decide on acceptance criteria, and a simple operator terminal on the rack door provides for in-process control and status reporting.
The ring of detection
After mechanical gyrations cause a piece of candy to fall into a vision subsystem, it takes very little time for its image to be captured from three perspectives, judged, and acted upon. The inspection rate at each of the five subsystems is 20 pieces/s, measured from “sensor trigger to blow,” and the system is capable of a Polly-to-Polly cycle time as low as 30 ms when the operations in a sequence are overlapped to take advantage of built-in delays.
When a candy enters a detection ring, it activates one or more photosensors arranged in a 360º array around the ring and triggers the cameras to act, but not until it has waited about 25 ms to account for the chocolate’s continuing descent. Once a trio of images is captured, the three camera outputs travel simultaneously to the analysis PC over three individual IEEE 1384a links. Analysis and decision making takes less than 10 ms for all three images using conventional blob analysis and then, if a Polly is deemed imperfect, a delay of about 10 ms is introduced before triggering the air nozzles for about 20 ms to reject a chocolate.
Mabema noted that lighting is a key component in the success of its Polly inspection system, playing a major role in ensuring that quality images of the Polly can be captured. The company made use of an illumination ring sourced from lighting specialist LAT electronik to provide backlighting and dim front lighting for the three cameras. It contains several hundred small red LEDs mounted in retainers in a 360º array located within each detection ring in the system.
Polly blob analysis
Blob analysis is used for inspecting the candies. Some versions of edge and geometric shape analysis might have been more efficient, but they would also require more time to complete than blob analysis.
Mabema found simple thresholding very effective at detecting areas of missing chocolate on a Polly, which show up as white spots. On the other hand, establishing the parameters that define a Polly shape proved more difficult, since a correct shape is not possible to define well. It is more a matter of what a human perceives as correct. The blob analysis takes size, height, and contours into account, checking for concavities of certain depths and widths and other features.
The application software running on the system was developed with Visual Studio 2005 C++ and Trolltech Qt 4.3, using the Matrox Imaging Library (MIL 8.0; see Fig. 4). In setup mode, the programmer sets thresholds with the on-screen sliders, defining such details as size, shape, degree of chocolate coverage, and various delays.
Once the application is set up, the operator interacts with the machine through a simple operator terminal on the door of the computer rack, which interfaces to the control computer over a MODbus connection. MODbus is an established industrial control interface with a very mature programming library for this realm of applications. The terminal has an alphanumeric keypad, a set of control and cursor keys, a small LCD for basic identification and status information, and 16 indicator-equipped “soft” keys—that is, keys whose function is labeled on the terminal’s LCD.
Features, advantages, benefits
Says Magnus Lundberg, managing director of Mabema, “The challenge for the Polly-sorting system was to avoid a trade-off between thoroughness and speed, providing 100% inspection of 10 to 12 tons of candy per day. To do this, we emphasized the use of ‘methods that are not very time-consuming.’ It combined a multistage mechanical sorting system and a tailored imaging subsystem with a design goal of inspecting 100 Pollies/s. The system is running at Cloetta today at a typical rate of 50 to 60 pieces/s, but it has also been run as fast as 115 to 120 pieces/s with good results.”
