Smart cameras target automotive inspection

On production lines where products may vary in size, shape, texture and placement, adaptable vision systems are required.

James Reed

During an automated wheel, tire, and wheel/tire verification process, a global manufacturer of automobiles and trucks was experiencing unacceptably high levels of false fails. In addition to using a vision system that could not account for variations in depth of field or textures on the metal surfaces of the wheel, the system's wheel training routine rarely worked and the proprietary implementation of the training routine code meant that operators could not amend the code.

Rather than employ this proprietary system, LEONI Vision Solutions Group determined that by employing a combination dome light and brick light configuration and multiple smart cameras for some of the inspection nodes, the correct wheels, tires, and resulting assemblies could be verified. This was accomplished by groups of Cognex In-Sight 5000-series cameras that could be trained by system operators, not just the systems integrator.

Moving targets

Although automotive production lines may only produce one type of car at a time, fitted wheels and tires will change depending on the options package required. Each car has four standard tires and a spare wheel and if both types pass down the same conveyor, automated machine vision systems must inspect both, regardless of where each tire resides on the conveyor. To accomplish all the required inspections for the volume of cars produced across the large expanse of the production plant, the automotive OEM required nine inspection stations in total. Four of these stations were used to identify the inner diameter and tread of the tire to confirm the tire model number. Since the metal wheels were of different color, depending on the car type for which they were designed, a further three inspection stations were used to identify the color of each wheel.

Due to the distance between the initial wheel and tire verification stations and the final assembly verification station –and the chance that a wheel or tire might be removed or added between initial and final verification – it was necessary to perform a final check of the wheel-tire assembly before the tires were installed on the car. Thus, given the volume of production at the plant, two final inspection stations were used to confirm that the final wheel and tire assembly were correct and that four standard tires and one spare tire were in a consecutive line for final assembly. One of these final assembly stations was fed by two wheel conveyors and two tire conveyors; while the other was fed by two wheel conveyors and one tire conveyor.

Diameter and tread

At the first inspection station, measuring the inner diameter of the tire and the tread pattern allowed each tire to be identified and verified against data from a supervisory PLC connected to the plant's Ethernet/IP network (Figure 1). To measure the tire's inner diameter, it was necessary to eliminate any shadows that would make machine vision analysis imprecise. By placing a downward looking camera in the middle of a dome light from Advanced illumination (Rochester, VT, USA; www.advancedillumination.com) and a matching color band-pass filter from Midwest Optical Systems (Palatine, IL, USA; www.midopt.com), shadows inside the tire were minimized. A 1600 x 1200-pixel In-Sight 5300 camera from Cognex (Natick, MA; USA; www.cognex.com) fitted with a Fujinon lens from Fujifilm (Wayne, NJ, USA; www.fujifilmusa.com) then provided sufficient resolution at maximum stand off to accommodate the dome light and tire height while providing sufficient spatial resolution to measure the inner diameter of the tire.

Figure 1: To image the tread of the tire, a lower-resolution camera was mounted to the side of the conveyor. Bracketed by a pair of brick lights to illuminate the thread from both sides, the camera had sufficient resolution and depth of field to acquire sharp images of the tire tread.

To image the tread of the tire, a lower-resolution 640x480-pixel In-Sight 5100 camera, also from Cognex, with a similar optical configuration was mounted to the side of the conveyor. Bracketed by a pair of brick lights from Smart Vision Lights (Muskegon, MI, USA; www.smartvisionlights.com) to illuminate the thread from both sides, the camera had sufficient resolution and depth of field to acquire sharp images of the tire tread regardless of distance between camera and tire. As the tire breaks the beam of a photo-eye spanning the conveyor, both In-Sight cameras acquire images.

Once the images are acquired, Cognex In-Sight software performs a number of preprocessing steps to enhance the tread pattern in the image. Cognex's PatMax algorithm is then used to extract tread shape and compare it with stored tread patterns indexed based on the expected tire tread configuration issued by the plant's network.

Overcoming shading

Wheel inspection presented a different set of challenges. It is fairly straightforward to differentiate between different styles of wheels when the only variance is a texture difference between polished metals. To accomplish this, many systems use color cameras and color image processing to accomplish this task. However, by carefully choosing the correct color of light for the overhead dome light and a matching band pass filter, gray-scale images can be used to differentiate different metal finishes and shades of silver, grey and black.

Finding the best colored illumination to differentiate colored products depends on many factors including light, optics and camera response, material properties, the target colors and how closely they resemble one another. By testing a number of different LED illumination systems, a dome light with an appropriately colored LED illumination system was fitted with an In-Sight 5300 camera and mounted above the three wheel inspection stations. Here, the higher resolution cameras are used to identify the grey-scale values of textured areas on the polished metal wheels (Figure 2).

Figure 2: By carefully choosing the correct color of light for the overhead dome light and a matching band pass filter, gray-scale images are used to differentiate different metal finishes and shades of silver, grey and black.

As the wheel breaks a photo-eye on the conveyor, the 5300 camera acquires an image and feeds the data to the camera's embedded In-Sight software. To identify the spatial and grayscale features and patterns on the wheels that are indicative of the wheel model, the cameras use edge detection, blob analysis and intensity tools functions and compares the resulting wheel model data with stored mathematical descriptions based on information from the plant network. In the case of a failed verification, out of sequence tires/wheels, or groups of wheels of the wrong number, the In-Sight camera issues an alarm so that operators can manually verify the wheel type.

The vision software for the tire and wheel inspection systems was developed using the spread sheet version of the Cognex In-Sight Explorer vision software v 4.5.0. This software was used to program and control the cameras of three tire inspection stations each with two cameras and a nearby wheel inspection station (Figure 3).

Figure 3: The vision software for the tire and wheel inspection systems was developed using the spread sheet version of the Cognex In-Sight Explorer vision software v 4.5.0. This software was used to program and control the cameras of three tire inspection stations each with two cameras and a nearby wheel inspection station.

While it was not necessary, each verification station has a PC host installed from the previous inspection system that runs Cognex VisionPro software. Ironically, one of the problems with the previous system was that it would not show the operator images of the tires as they were verified, in the event of an alarm or during training. However, using the monitor of these PCs, images from each group of In-Sight cameras could be viewed as well as the system's HMI for the plant network and conveyor controls. These PCs were also used to facilitate training of new products and provide operators the ability to view and change tire, wheel and final assembly spatial and threshold parameters as needed.

Checks and balances

After the tires passed by four verification stations and wheels pass three verification stations, tires and wheels are assembled and passed to the final pair of inspection stations. These final verification stations employ a similar configuration to the tire inspection station with a dome light placed above the wheel-tire assembly and an In-Sight 5300 camera pointing downwards to confirm tire and wheel diameter. As each tire passes through these inspection stations, a side-mounted In-Sight 5100 camera with brick lights on either side reads the tire tread and again compares the patterns with stored values indexed by data from a nearby plant PLC. If an alarm condition existed, the In-Sight cameras notify the nearby PLC, which halts the conveyor and alerts nearby operators.

Since fielding this smart-camera wheel and tire inspection network, LEONI has installed several similar systems. In some cases, a lift mechanism or other accessory is required to spin the tire to check for good seals between wheel and tire, surface defects, matches between tire pressure monitoring sensor (TPMS) location and a sticker that indicates the tire is in balance. In other cases, systems have been deployed to check the presence and Lposition of center caps on the wheel.

James Reed, LEONI Vision Solutions (Lake Orion, MI, USA; www.leonivisionsolutions.com )