Plumbing Parts Assembly

Integrating machine vision with a six-axis robot has allowed complex plumbing fixtures to be automatically assembled

Andrew Wilson, Editor

As a business partner to some of the world’s best-known plumbing and heating suppliers, the Pegler Yorkshire Group offers numerous types of pipe fittings in different sizes and configurations. When manufacturing copper and brass elbow joints, tees, and valves, various pipe components must be assembled before the finished item can be shipped to the customer.

“Because of the variety of parts that the company produces,” says Robert Fisher, partner with FS Systems, “mass production requires many different types of assembly machines, each of which were fed manually by skilled machine operators.” As a UK partner of Kawasaki Robotics and a distributor of machine-vision products from both Vision & Control and Vision Components, FS Systems was called upon to provide machine vision to help automate this manufacturing process (see video below).

Pick and place

“In the past, an operator would pick a number of semi-finished parts from a bin and place into a solder or assembly machine to join it to other parts,” says Fisher. In the design of FS Systems’ RoboVis, however, the company has not chosen to automate thisbin-picking operation. Parts are first placed into a bowl feeder and fed onto a 200-mm-wide conveyor belt running at 5 m/min.

“In this way,” says Fisher, “the problem of bin picking is avoided, making it easier and more accurate to pick parts.” Unlike many other conveyor-based systems, the conveyor does not run continuously. Instead, after a part reaches the end of the conveyor, it is detected by a wide beam sensor that is in turn connected to a programmable logic controller that is used to stop the conveyor. This is especially useful for cylindrical parts that tend to roll on the conveyor; a specific type of conveyor belt is chosen to help with this and provide minimum reflection from the illumination source back to the camera.

As many as 10 different parts can be randomly located across a 260 × 200-mm area of the conveyor. For the robotic system to properly locate each part correctly,illumination plays a vital role. Because brass and copper parts are specular, they are difficult to illuminate. The system must also compensate for metal fragments from the production process that may be present on the conveyor belt.

Lights, camera…

“Ringlights are not suitable for such an application,” says Fisher, “because light will reflect from the part in all directions, making it difficult to obtain an image with good, consistent contrast across the viewed area.” Instead, FS Systems contracted Vision & Control to build a custom 300 × 240-mm, 630-nm (red) LED panel (FALKD300x240-R) that was placed approximately 1 m above the conveyor (see Fig. 1). By placing a camera in the center of this panel, reflected direct-on-axis illumination could be captured by a Marlin F-131B FireWire camera from Allied Vision Technologies.

FIGURE 1. FS Systems’ RoboVis is a vision-based robotics system currently deployed by Pegler Yorkshire Group for the automatic assembly of pipe fittings.

Images from the camera were then transferred to a host PC, which was interfaced to the robot controller of a six-axis robot from Kawasaki Robotics over an Ethernet interface (see Fig. 2). “By interfacing the conveyor sensors to the robot controller, images from the 1280 × 1024-pixel CCD camera were captured after the conveyor was in a stationary position,” explains Fisher.

FIGURE 2. Images captured from a camera located at the center of an LED panel are transferred to a host PC, which is interfaced to a robot controller. Analyzed image data are then used to instruct the robot as to which part on the conveyor can be optimally picked.

“Since the products on the conveyor were stationary, capturing one image every 2–3 s was sufficient,” he continues. However, before picking any specific part from the conveyor, the system must determine which part can be selected most efficiently. After this is determined, the system checks whether there will be any potential collisions with any other parts on the conveyor. If there is no such conflict, then the part is picked; otherwise, the robot must pick another part from the conveyor. In the event that no parts can be picked, the conveyor is advanced and all the parts are recycled back to the hopper for re-presentation.

“In the RoboVis systems installed at Pegler Yorkshire Group,” says Fisher, “approximately 95% of the parts presented can be picked, leaving only 5% to be recycled through the system.” Determining which parts can be picked in this manner is achieved by analyzing the images captured by the CCD camera. As a Certified Integration Partner for MVTec Software, FS Systems usedHalcon running on the host PC to perform the analysis.

Shape recognition

Before images can be processed, the system must be calibrated. A known good part is placed on the conveyor belt and the image is digitized. Any metal fragments within the image are removed by smoothing the image. To identify the position of part, the image is then thresholded to segment the pipe fitting.

After segmentation, Halcon’s geometric shape-recognition function maps these regions as an affine-invariant function. Once such a function is developed, it is stored in the host PC as a reference.

In operation, the system captures images of multiple pipe fittings within the camera’s field of view. Once again, smoothing, thresholding, and geometric shape-recognition algorithms are applied.

This process results in a number of affine-invariant transforms that represent the selected part at different orientations within the image. The transforms are compared with the known good reference to determine the best matched object for the robot to pick.

Part location

The system must still determine the location of other parts within the image before picking to avoid collision with multiple parts. Using the shape recognition tool in Halcon, the orientation inx, y, and θ of each part is determined and the optimum path of the robot computed.

The robot is moved to the location of the part where a robot gripper enters the orifice of the pipe, expands, and picks the part from the conveyor. Once picked, the part is placed in the automated assembly machine and is processed; the robot returns to pick the next part.

“Because of the variety of parts that multiple RoboVis systems are required to recognize,” says Fisher, “it was necessary to develop a user interface that allowed the operator to visualize the system in action, amend high-level image-processing functions, and test the system before it could be placed in production.” FS Systems chose Visual Basic to front-end the Halcon library (see Fig. 3).

FIGURE 3. Using the system interface, operators can visualize objects, calibrate the system using a known object area, and select the number and order of image-processing functions.

Objects can be seen as they appear on the conveyor and functions such as thresholding can be highlighted to show the operator which objects the system will pick as well as its coordinates (top left). Using a Model and Tool Control, the operator can also calibrate the system using a known object area and select the number and order of image-processing functions that the system will run. According to Fisher, more than 15 RoboVis systems are now in use at Pegler Yorkshire alone in the UK, USA, and Hungary.