Multi-camera vision system verifies sterile medical device and supply package integrity.
John Fouts, Bhaskar Ramakrishnan, and Luo Cheng
It goes without saying that Class II and III medical device and supply manufacturers can ill afford the risk of sterile product contamination. However, as hygienic as the packaging process is, the potential still exists for foreign materials to become trapped between layers of heat-sealed material. In some cases, such debris may form micro channels large enough to breach the seal to ambient, allowing bacteria to enter later.
Manual inspection of package seals tends to be time consuming and error prone. After all, contaminants such as hair and hidden particles in the seal can be extremely challenging for people to see, leading to mistakes or oversights. Moreover, damaged package seals may also have microscopic holes or tears that human eyes cannot see.
Needless to say, if such packages make their way into hospitals and other medical settings, it can compromise not just the sterility of the medical devices and supplies, but also the health of patients.
Understanding this, engineers at DWFritz Automation Inc. (Wilsonville, OR, USA;www.DWFritz.com) have developed an automated seal inspection system called ASI. The system relies on a multi-camera machine vision system designed to perform 100% inline inspection of packages on a manufacturing line to ensure that sterile products remain sterile throughout the supply chain.
The ASI system was demonstrated in April at the Automate 2017 trade show and conference in Chicago. Designed to inspect for seal integrity and other defects on packages formed from flexible poly films, this system was designed to inspect packages formed into eight individual pouches in two parallel rows of four each, filled with surgical devices and sealed with perforated and breathable Tyvek lidding.
To inspect all of the seals surrounding each of the eight pouches, the ASI material-handling system picks up each package and precisely moves it at a rate of 250mm/sec over two arrays of upward-facing cameras with integrated strobe lighting (Figure 1). During the 12-second inspection cycle, no less than nine cameras acquire 378 images to detect and classify defects down to less than 50 microns.
|Figure 1: The DWFritz automated seal inspection system inspects all sides of the seal by drawing in packages on an automated conveyor. A specially designed vacuum chuck, capable of picking up even porous packages then moves the packages over several rows of upward-facing cameras.|
Production line integration
While the trade show demonstration was a standalone system inspecting surgical sponges, at the actual customer site the ASI system is an integral part of the production line that's linked via automated conveyor to the packaging equipment, which fills similar packages with various types of medical devices and supplies.
During operation, as the package enters the system via infeed conveyor, it's sensed and then input to a Control Logix PLC from Allen-Bradley (Milwaukee, WI, USA;www.ab.rockwellautomation.com), which controls machine operation, triggers an output to raise lift fingers under the package. When the package is in position on the elevated infeed stage, the PLC triggers an In-Sight 5605 smart camera vision system from Cognex (Natick, MA, USA; www.cognex.com) to acquire an image.
The image is first analyzed by the camera's on-board processor for pre-inspection alignment to determine package orientation and provide X, Y and T coordinates so that a vacuum chuck on the gantry can pick up the package in the correct orientation for inspection.
Since between the eight pouches on each package are four seal lanes in one direction and five lanes in the other direction (Figure 2), engineers specified no less than nine cameras in the design of the two upward-facing optical array assemblies.
|Figure 2: Tyvek lidding sealed onto a 12 x 12 inch, eight-pouch array of flexible polyethylene hermetically seals medical devices up to 2-inches high. Custom illumination was required to properly illuminate the seal areas, shown in tan, to detect defects down to less than 50 microns.|
During operation, the material handling system first moves the package over the y-axis optical array which includes four camera/lens stack and light assemblies. Then the material handling system moves the package over the x-axis optical array which includes five camera/lens stack and light assemblies.
For each camera/lens stack and light assembly, engineers specified an Ace 2048 x 2048 monochrome camera from Basler (Ahrensburg, Germany;www.baslerweb.com), which features a CMV4000 CMOS image sensor from CMOSIS (Now ams Sensors Belgium; Antwerp, Belgium; www.cmosis.com).
To meet the resolution requirements of the application, each camera's 4 MPixel global shutter sensor, which has a 5.5 μm pixel size, is paired with a 40mm lens from Schneider Optics (Hauppauge, NY, USA;www.schneideroptics.com). This arrangement achieves a 25-mm field of view.
The four cameras of the y-axis optical array connect to an industrial PC (iPC) from Advantech (Milpitas, CA, USA;www.advantech.com). Likewise, an identical iPC controls the five-camera x-axis optical array. The iPCs are arranged in a master-slave configuration with the master running an OPC server to communicate with the PLC machine controller over Ethernet.
During operation, image acquisition and part handling are tightly synchronized. Encoder feedback from the PLC to the iPCs triggers each camera array to acquire images every 7.5 mm of travel, as the package passes overhead and high- and low-angle lights are alternately strobed.
This creates sequential, but overlapping images, which are transferred via USB3 interface to the industrial PCs, where they are stitched together, reconstructing two images of the 12 x 12-inch package with different lighting.
The image analysis software uses various algorithms to perform certain vision tasks. One identifies channels across the seal and another looks for wrinkles in the Tyvek. There is an algorithm that measures the width of the seal along the entire perimeter of each item and ensures that it doesn't fall below a specific minimum to identify areas of the seal that may be too narrow.
Other algorithms are applied to the images to detect particulates captured inside the seal, wrinkles in the Tyvek lidding, spoiled seals, glazed seals, and to locate any holes or punctures that may exist. Several algorithms are required to detect hairs. One for dark hair, one for blond hair, and another for gray hair.
After image analysis is complete, the master iPC communicates the result to the PLC, which in turn actuates one of two air cylinders required to pass the package, or divert it to the reject bin.
Another challenge was illumination. Due to the shape of the package itself, and because the seal areas are recessed in lanes between the eight cuboid-shaped pouches, engineers had to develop a compact, custom illumination system that fit up inside the lanes between the pouches, very close to the seal areas.
The highly-specialized strobe lighting solution is an integral part of each camera/lens stack and light assembly, and provides the contrast required for the machine vision software to detect the required seal defects.
Detecting 50-micron defects requires a resolution of 10 or 12 microns per pixel, the material handling system had to achieve tight axis control in terms of positioning tolerances to maintain a +/-1-mm depth of field when presenting the seal areas to the cameras.
|Figure3: The low-profile and highly-specialized strobe lighting solution is an integral part of each camera/lens stack and light assembly, and provides the contrast required for the machine vision software to detect seal defects.|
Package handling challenges
During ASI development, the engineering teams tasked with specifying the part-handling and machine vision system had to address several design challenges. First, the fact that each eight-pouch package may contain numerous items of varying sizes and weights, which, when combined with the non-rigid nature of the package, make the packaging tend to flop around during handling.
Another design challenge is the breathable nature of the Tyvek lidding, which makes it difficult to pick up with vacuum. Plus, the flexible material's wavy and uneven surfaces challenge part presentation by the gripper to the machine vision cameras for inspection.
Despite the flexible nature of the packages and the fact that they may contain various medical devices and supplies of different weights and sizes, the material handling system must maintain package flatness due to depth of field requirements of +/- 1 mm over a 1-ft, y-axis stroke and a 2-foot, x-axis stroke.
Because this required the use of a relatively heavy vacuum chuck that was powerful enough to hold flat the porous packaging despite the weight of its contents, engineers chose a three-axis gantry system over a wristed robot because it provided the smooth, precise and repeatable movement required.
Besides minimizing the risk of sterile Class II and III medical device and supply product contamination, the ASI could potentially impact the packaging of food, military supplies, consumer products or virtually any other application that relies on flexible packaging with intact hermetic seals.
John Fouts, Senior Controls Software Engineer, Bhaskar Ramakrishnan, Technical Sales Engineer and Luo Cheng, Senior Vision Engineer, DWFritz Automation Inc., Wilsonville, OR, USA (www.DWFritz.com).
Milpitas, CA, USA
Milwaukee, WI, USA
Natick, MA, USA
Hauppauge, NY, USA