Machine-vision software narrows complexity gap

By Jeff Child

Easier-to-use environments and new pattern-matching schemes smooth software-development chores for systems integrators and users.

Machine-vision application developers rank ease of use high on their list of design goals. Recent advances in machine-vision software are supporting that on two levels. On one hand, software vendors are enhancing users' ability to get software up and running easily and quickly through the use of comprehensive graphical environments. These environments are geared to let developers avoid complex programming and algorithm development if they so choose. On the other hand, more vendors are introducing new technologies such as geometric pattern-matching into their offerings. Such technologies help make the overall vision application simpler by doing more advanced processing within the software.

Exemplifying these trends, Cognex Corp. (Natick, MA) has developed its VisionPro software environment (see Fig. 1). VisionPro rolls the complete Cognex Vision Library (CVL) into an ActiveX development environment that lets users develop software in a COM-based open environment. It moves all the capabilities that were in the Cognex C++ software environment, including PatMax and optical-character-reader functions, into an ActiveX package. Users also can make use of functions from the Cognex Object Manager Interface in VisionPro.

VisionPro runs on WindowsNT and Windows2000 operating systems. The approach is similar to the ActiveMIL product from Matrox Electronic Systems (Dorval, Quebec, Canada), which integrates the Matrox Imaging Library (MIL) into an ActiveX environment. Along similar lines, Halcon 6.0, a machine-vision software library by MVTec Software, (Munich, Germany) and distributed by Leutrek Vision Inc. (Burlington, MA) in the United States, can be accessed from C, C++, and ActiveX (COM).

VisionPro also includes a feature called Quickstart that lets users drop in vision tools from a palette. Once dropped in, the software automatically creates links among the vision tools and lets users test out parameters using ActiveX controls. Users can then customize their application using Visual Basic or C++. Sample code is also provided to enable developers to craft applications fast.

Steve Cruickshank, Cognex product marketing manager, sees support for ActiveX and COM environments as a key way to open up development efforts. "We envision that developers would use our environment to let them drop in other COM objects such as machine controllers, input/output controllers, and specialty application functions," he says.

An open strategy has also been in the works at Coreco Imaging (Bedford, MA). Once a closed system, Sherlock, the company's deployable vision development environment, became an open architecture in version 5.8 two years ago. The latest version, Sherlock 6.1, expands on that and lets users create and add their own algorithms into the Sherlock framework (see Fig. 2).

Because Sherlock is a deployable software package, it can be used as is with no modification. If used as is, the user interface is the Sherlock interface. Simon Melikian, director of vision software at Coreco Imaging, says that a high degree of openness is vital to meet the diverse range of vision-systems applications. "Many OEMs have their own user interface, their own button icons, and their own database interface. They can create their interface with Visual Basic or Visual C and just make calls to Sherlock," he says. "The underlying Sherlock software would then do the image processing, interface with the cameras, and perform control functions. On the other hand, there are simple applications where users don't really care about the interface—where a machine sits all day and displays or records pass or fail data. In such applications it makes no sense to spend extra time working on the user interface," adds Melikian.

Sherlock 6.1 includes new features and incremental improvements, including support for the Camera Link interface. This version also lets users include "in-line" Visual Basic Script code as part of their investigation. This allows user-defined computations and access to automation servers like Excel.

Another feature relates to linescan cameras and accommodates applications of variable length frames. This significantly shrinks acquisition time in applications that look at different sized moving objects. New algorithms also have been added, such as support for "Interleaved 2 of 5" barcode and "Perspective Calibration." The latter provides compensation for distortion that occurs as a result of the camera and sensor planes not being positioned in parallel.

Among the most significant of these technologies is geometric pattern-matching. By avoiding the pixel grid of brightness values traditionally used to represent and match patterns, geometric pattern-matching can handle arbitrary orientation, size, and shading variations and deliver higher accuracy, while not sacrificing robustness, ease of training, and speed. Cognex, a leader in such technology, introduced its first geometric pattern-matching tool three years ago and has continued to enhance it. Its PatMax software is ported into its VisionPro software environment.

Other vendors also are bringing various forms of geometric pattern-matching into their machine-vision products. Geometric pattern-matching looks at the boundaries of the object, not the texture of the object. In contrast, older pattern-matching schemes, such as correlation, look at the darkness or brightness of the object and try to find a match. As a result, geometric pattern-matching can find objects at any angle. Correlation can do that but requires intensive timing and computation. That means objects don't have to be as strictly ordered as they come down a conveyor belt, for example, because the vision system can recognize the objects at any angle.

Coreco's Melikian points out that geometric pattern-matching is particularly helpful for objects that really have no texture but are rather just an outline. "Think of etching on a glass, for instance," he says, "That's where geometric pattern-matching flies." Coreco incorporates geometric pattern-matching in Sherlock 6.1.

For its part, Matrox has added a model-based type of geometric pattern-matching to release 7.0 of its MIL. Called Geometric Model Finder, this tool uses geometric features to quickly find patterns (see Fig. 3). It can search an image for patterns located at any position including angle, any scale from 50% to 200% with occlusion, and handle nonuniform changes in contrast, including contrast reversal. Geometric Model Finder also incorporates calibration, which transparently compensates for optical and perspective distortions and provides results in real-world coordinates. Pattern models can be easily shared among calibrated systems.

According to Stephane Maurice, Matrox imaging-software manager, model-based pattern match is popular in fields such as robotics. "Many find it much easier to play with a model rather than a regular pattern," he says. Matrox crafted its geometric pattern-matching technology from scratch, according to Maurice, and the company has two pending patents. The company intends to evolve the technology, by adding three-dimensional capability, for example. Maurice distinguishes the Matrox software from other vendors' approaches that offer only an edge-detect version of geometric pattern-matching. Such schemes, he says, are much less robust and don't provide the features and speed needed with true geometric pattern-matching.

Taking a different approach, National Instruments (NI; Austin, TX) offers a pattern-matching tool that's not based on geometric techniques (see Fig. 4). Its technique is based on a patented sampling technique. According to Stefanie Breyer, NI group manager for IMAQ research and development, its pattern-matching approach is just as valid as geometric pattern-matching and offers some different benefits.

Along similar lines, the company has added color pattern-matching to its IMAQ vision software. This software combines NI pattern-matching technology with its color-processing technology. "As a result, the software can not only find patterns based off gray-scale objects and edges, but also can base it off the color information in the pattern as well," says Breyer.

Many sophisticated vision-system developers prefer to do some of their own algorithm development or algorithm integration. The latest release of MatLab Image Processing Toolbox from The MathWorks (Natick, MA) supports a number of capabilities that serve that level of development. Among these algorithms are spatial transformation functions for applying a variety of spatial transformations to images and points.

Supported transform types include affine, projective, polynomial, piecewise linear, local weighted mean, and user-defined.

Also in the latest release are image-registration functions for registering (aligning) two images, including an interactive control-point selection tool, subpixel control-point refinement, and fast normalized cross-correlation. Rounding out the new release are some new fundamental algorithms for image de-blurring. Algorithms include Lucy-Richardson, regularized, and Wiener deblurring, each of which supports multidimensional problems.

Last year, Data Translation (Marlboro, MA) released its Information Exchange (DtiX), a COM-based, peer-to-peer communication system that simplifies the integration of both vision and sensor-based measurements. DTiX allows users to share variable and image data between DT Vision Foundry, DT's machine-vision software, and Measure Foundry, the company's test-and-measurement software. This is targeted to developers of machine control systems who want to perform both imaging and machine control (see "Automated systems use off-the-shelf machine-vision software". Using the software, developers can integrate data acquisition measurements with machine vision data and correlate the results under a single user interface.

At last year's Vision Show (Boston, MA) the company demonstrated a version of DTiX simulating an assembly line of fuel injectors moving at high speed. A DT3152 frame grabber captures the images and brings them into DT Vision Foundry; the software exports the images and variables through DTiX to DT Measure Foundry. Here, variables include good parts, bad parts, total parts, and the diameter of the fuel injectors.

The integration of test and measurement and vision software and hardware in one system allows a user to correlate the data to monitor the complete system or make adjustments as needed. These adjustments can range from controlling the speed of the motor to synchronize with the speed of the camera, to determining if the vibration of the motor is impacting the reliability of the camera. Since vision and data-acquisition measurements are automated, testing can take place remotely without additional resources monitoring the process.

_____________________________________

Designed to image and process coins at a rate of five Eisenhower-dollar-sized coins per second, the system performs image processing using Vision Foundry software from Data Translation (Marlboro, MA). This was chosen because it can perform fast and accurate gray-scale image comparisons. A CVM-10 progressive scan 640 x 480-pixel monochrome camera from JAI (Laguna Hills, CA) was used. to image the tokens

An important challenge was to optimize the image such that the classifier speed and accuracy were maximized. To maximize these variables it was necessary to optimize the field of view (FOV) to the coin. If the coin occupies too much of the FOV, image-processing time increases. If the coin is too small, identification accuracy is adversely affected. And to constrain perpendicular motion, to the axis of travel, of the coin as it passed under the camera, the application needed to be invariant to coin rotation.

Each module is designed to accept a TTL input trigger, which comes from a part sensor that indicates that a token is in position to start the image capture. After the image capture step, analysis of the image using Vision Foundry is performed, and a TTL output signal generated if a mismatch is detected. If the coin does not match the template, it is rejected. A rejection occurs if the coin is too worn, or if it is from another casino. Interface software allows the operator to train the system to recognize the coin of interest. After the system is trained, it passes coins matching the pass criteria and rejects all others.

The final system was at least five times faster than a human being and system recognition accuracy is superior to human performance. Unlike manual operators, the system can operate 24 hours a day, seven days a week without any degradation in accuracy or inspection throughput.

~ John A. Little,Vision 1, Bozeman, MT 59715; e-mail: [email protected]

_____________________________________

Cognex Corp.
Natick, MA 01760
Web: www.cognex.com

Coreco Imaging
Bedford, MA 01730
Web: www.imaging.com

Data Translation
Marlboro, MA
Web: www.datx.com

Leutrek Vision Inc.
Burlington, MA 01803
Web: www.leutrek.com

The MathWorks Inc.
Natick, MA 01760
Web: www.mathworks.com

Matrox Imaging
Dorval, QC, H9P 2T4 Canada
Web: www.matrox.com/imaging

MVTec Software
Munich, Germany
Web: www.mvtec.com

National Instruments
Austin, TX 78759
Web: www.ni.com