Vision system inspects motorcycles
Smart vision system directs robot to move motorcycle engines within an inspection cell to automatically check parts orientation.
Smart vision system directs robot to move motorcycle engines within an inspection cell to automatically check parts orientation.
By C. G. Masi,Contributing Editor
When motorcycle-manufacturer Harley Davidson (Milwaukee, WI) approached Scott Woida, president of Midwest Engineering Systems (West Allis, WI), to design a test system that would verify the correct timing-chain installation in its new Twin-Cam 88 engine, Harley envisioned making only the one pass-fail inspection. "As soon as Harley engineers learned that we were considering a vision system to do this test," Woida recalls, "they started to say: 'Well, maybe we could also ensure that the washers are put on correctly and the sprockets are tightened down all the way. Then, we could see that the guides are pressed down completely. As long as we are looking at these things that span the entire engine, let's look at all this stuff.' So, we ended up performing about 30 inspections on the engine."
To make the new engine run more smoothly than its previous versions, Harley designed the motor with two balance shafts installed parallel to the crankshaft: one ahead and one behind the crankshaft (see Fig. 1). Each balance shaft carries an eccentric weight. The forces set up by the rotation of these weights counterbalance the vibrational forces set up by the motion of the pistons and connecting rods. A timing chain running over sprockets on the three shafts couples these forces together.
The original inspection was needed to ensure that the timing chain was installed correctly; the timing chain synchronizes the balance shafts with the crankshaft. When the timing chain is installed correctly, marks cast into the sprockets line up with discolored links in the timing chain (see Fig. 2). This is a straightforward task for a machine-vision-based inspection system.
Additional inspection tasks that involve verifying the presence of all the components that should have been installed by this phase in the assembly process are also straightforward for an automated inspection system. When Harley asked that the vision system be able to verify that certain components were properly tightened, however, test requirements became complicated.
For example, consider the two spring-loaded chain tensioners. To ensure proper chain tension, the tensioners have to be forced into their sockets. The vision system can verify that they were forced home by measuring the distance from the casting surface to the top of the chain-tensioner body. Making that measurement, however, requires precise positioning of the camera with respect to the target.
Precisely positioning the camera is complicated by the fact that the engines are carried through the assembly area on J-hooks, which are approximately 12 ft (4 m) long, suspended from an overhead conveyor. These J-hooks can only position the engine in the inspection cell to within ±2 in. (approximately 5 cm) in the x, y, and z axes and several degrees of rotation around the vertical (z) axis.
Well into the vision-system design, Midwest engineers discovered that there was also a large planarity error, as well. That is, engines do not always sit flat on the J-hook. Therefore, the plane in which the timing chain runs can not be relied on to line up with the x, y (horizontal) plane.
The Midwest engineers' first idea was to use a mechanical fixture to position the engine under inspection with respect to several cameras. They soon realized, however, that they could make a more robust, flexible system that was faster and easier to use by mounting a single camera on a six-axis robot arm (see Fig. 3 and Fig. 4 on p. 43).
The camera is a Series 600 SmartImage Sensor from DVT Corp. (Norcross, GA), which incorporates an image-processing subsystem within the camera housing. The robot is an IRB 1400 industrial robot manufactured by ABB Flexible Automation (New Berlin, WI). A desktop PC is used as the human-machine interface (HMI) terminal. Custom software directs communications with the other system components as nodes on an Ethernet network. They also can communicate with the Harley factorywide intranet. The robot's internal computer controls the inspection process. It contains a list of required inspections and the test sequence.
Within its housing, the DVT Series 600 SmartSensor incorporates a 640 x 480-pixel, variable-frame-rate CCD camera, frame-grabber electronics, and an image-processing processor. It initially acquires and analyzes the inspection images. Then it delivers system information such as quality-control inspection results, coordinate data for motion controllers, statistical process-control information, and two-dimensional code verification. The built-in image processor runs the company's FrameWork 1.4 software interface. This software provides analysis tools such as blob analysis, translation and rotation, intensity, edge count, feature count, precision measurement, mathematical operations, trainable OCR reader, template matching, and object finding for converting raw image data to useful data.
Combining the camera, image-acquisition, and image-processing functions into one housing permits dynamic access of the CCD without image float or flutter and eliminates external signal-conversion electronics, fixed frame rates, and limited gray-scale quantization. An external PC is needed only for system setup. "It's compact for Harley. The company doesn't want a big, complicated vision system on the factory floor," says Woida.
The robot's internal computer controls the inspection process. It directs which inspections should be performed and in what order. For its part, the SmartSensor camera is configured with all the necessary image-processing routines and the image-analysis procedures needed for each inspection.
The sensor communicates its outputs and receives control inputs through various networking options. The TCP/IP Ethernet network allows engineers to program or monitor the system from any network node, such as for directing the robot.
Woida divides the procedures into two classes: orientations and inspections. Orientations are visual-feedback-directed robot motions used to move the camera to the precise location and orientation to capture the image needed. An inspection is the capture and analysis of that image.
Vision inspections are generally performed in up to six groups. Thus, the system might perform an orientation to capture an image while the camera is looking at the timing chain edge-on (see Fig. 5 on p. 44). The camera captures the image and analyses it with a group of inspections. These inspections might include determining whether the sprockets have been tightened down or whether the chain is aligned with the horizontal plane. The camera's internal computer then sends an image analysis as a packet through the Ethernet hub to the HMI to inform the operator whether the unit under test passes or fails. The Ethernet packet is formatted to carry results of up to six tests.
FIGURE 3. Final version of SmartSensor vision inspection system consists of a single camera mounted on a six-axis robot arm. The system incorporates a camera and housing that contains built-in image-acquisition and analysis-processing electronics. It also provides feedback signals for guiding the robot arm to the exact location for each inspection task. (Photo courtesy of Midwest Engineering Systems)
If any of the tests indicates a failure, the camera sends a failed-image file for storage in the PC database. Then, the vision system goes on to complete the rest of the inspection sequence to find any other failures.
After finishing the entire inspection sequence, the PC signals the operator whether the unit passes or, if one or more failures exist what corrective action to take.
Using the Ethernet to tie the system components together produces several advantages. Installation and cabling become easier. Each major system component (smart camera, PC, and robot) has its own IP address so that intrasystem communication can use standard network protocols and software. Moreover, Ethernet allows a seamless integration into the Harley factorywide network so that the vision inspection system does not become an island of automation apart from the factory management system.
"I can sit anywhere in the plant and monitor the camera," says Woida. "I can see what it's seeing. If I have the correct access and security, I can program the camera from anywhere on the network."
MAKING IT WORK
The inspection cell is curtained to control lighting variations as well as to prevent human intrusion into the operational space while the robot is operating. Three lighting sources are used. First, area floodlights provide bright and even illumination throughout the cell. The SmartSensor camera comes with a high-intensity light-emitting-diode source mounted on its housing. The inspection system uses this light source at the start of an inspection sequence when it tries to locate the engine in the cell. Then, when the camera is so close that its housing may block the area lighting, a ringlight mounted around the camera lens floods the visual field with even illumination.
An inspection sequence starts with the robot positioned far from the J-hook carrying the engine to be inspected. Supposedly, the J-hook is locked into a certain orientation, carries the engine lying flat on its side (with the crankshaft pointed vertically), and stops in a certain spot.
However, any of those assumptions could be wrong. The inspection system, therefore, has to start by making sure it isn't going to damage itself by bumping into the engine or the hook. From that rough evaluation of the test scene, it can determine where it should start maneuvering.
All the orientation procedures are designed as feedback loops. To locate itself with respect to the engine in the x,y plane, for example, the camera takes an overhead image of the engine. The camera's image processor locates the crankshaft end and calculates how far it appears from the image center. The processor then sends a two-dimensional displacement vector over the Ethernet network to the robot, which moves the camera. The camera then takes another image and sends a correction back to the robot, which moves the camera again. The feedback loop keeps going until the displacement error falls within a preset tolerance. Once the camera is centered over the crankshaft end, the system uses a second feedback loop to rotate itself into q-alignment using a second feature to determine the required rotation angle at each step.
To locate itself in the z direction, the vision system locates a certain machined hole in the engine case that has a known diameter. The image processor measures the hole's diameter (in pixels) at the image plane, compares it to a target's apparent diameter, and estimates how far to move the camera toward or away from the engine to achieve that target. That estimate is passed to the robot, which moves the camera, and so forth, until the apparent diameter comes within its tolerance.
Finding the planarity error is the most complicated orientation procedure. To estimate the planarity error, the image processor measures the apparent diameters of three holes on the engine case and uses solid trigonometry to calculate a correction. These measurements comprise a three-dimensional displacement and a j-angle correction. Again, the feedback loop continues until the correction magnitude falls below a tolerance.
It took about a year from the time Harley Davidson first approached Midwest Engineering Systems with the inspection problem to the time the first system was fully operational. The engineers spent about six months analyzing the problem, developing a solution, and building the system. Then, it took about four months to install the system on the production floor—mostly because the installation had to be coordinated with Harley's annual shutdown. "Then," Woida recalls, "we took another two months to make all of those changes. You watch 100 parts go by and make sure the system can handle all the different variations between the parts."
Parts variations emerged because Harley uses multiple suppliers for many engine components. Harley's designers define the significant engineering specifications, but many properties are left to the suppliers' discretion. For example, the dimensions of a washer inside the engine were defined, but the finish (which makes no difference to the engine's operation) was not. So, washers arrived with anything from a polished surface to a matte anodized finish. Although that variation didn't make any difference to the engine, it slowed down the inspection process.
For some variations, it was possible to open up the tolerances on what the vision system would accept. In other instances, such as the washer situation, Harley had to add a finish specification to their part requirements.
"This was our initial project," Woida points out. "It was a learning experience. Since then, we have developed software that almost automates the creation of the inspection routine. A lot less custom code had to be written for subsequent versions."
ABB Flexible Automation
New Berlin, WI 53151
Norcross, GA 30093
Milwaukee, WI 53208
Midwest Engineering Systems Inc.
West Allis, WI 53227