Software developers are incorporating Intel's multimedia instructions into their packages to speed up image-processing functions.
By Andrew Wilson,EditorSystems integrators are becoming more reliant on general-purpose CPUs to power image-processing functions. Accordingly, they are turning to off-the-shelf software packages that take advantage of Intel Corp. (Santa Clara, CA) multimedia instructions (MMX) to speed up image-processing functions. Because these packages directly support Intel's Pentium and other MMX-enabled processors, they can take advantage of any processor performance increase while offering systems developers an easy upgrade path.
Although some commercially available image-processing packages are using Intel's Image Performance Library (IPL) or Computer Vision Library (CVL) software, developers also are augmenting these packages with novel image-processing features. For example, the IPL software provides a set of image-manipulating dynamic-link-library functions optimized for MMX processors, and the CVL software includes recognition, measurement, and geometric image-processing functions (see Vision Systems Design, Aug. 2000, p. 10).
Says Gary Bradski of Intel's Microprocessor Research Laboratory, "CVL and the image-performance libraries have identical structures. But CVL is being developed by the Vision and Pattern Recognition Group in Intel's Microprocessor Research Labs, and IPL is offered by the company's Microprocessor Software Labs."
Although Intel's IPL is not supplied as source code, the company does provide developers with rights to redistribute binary code in their products. The CVL software, however, is provided as open source code, and Intel provides developers with rights to use and redistribute source or binary code for research or commercial use (see Fig. 1). "To keep this open source code pure," says Bradski, "the recent release of CVL no longer relies on IPL. However, IPL will remain compatible with CVL."
Open or closedBecause Intel does not charge for either IPL or CVL, a number of researchers and developers of third-party image-processing and machine-vision software are embracing the technologies. One researcher, Daniel Lyddy, a graduate student at California PATH/UC Berkeley Vision Group (Richmond, CA), is using CVL in a remote vehicle-trajectory monitoring system.The car-traffic application problem under investigation involves the intersection of the Bay Bridge and the interchange of Interstates 80, 580, 880, and 980. Says Lyddy, "Westbound drivers often make weaving maneuvers or sudden lane changes so that their car speed drops to less than 20 mph. These maneuvers occur daily for more than eight of the 14 hours between 5 am and 7 pm." To obtain traffic-surveillance data, Lyddy and his colleagues are combining data from several different sensors and developing traffic-simulation and prediction models using machine-vision and magnetic-loop detection.
During system development, Lyddy dismissed off-the-shelf vision and imaging packages in favor of Intel's free CVL. "Commercial software vendors protect their trade secrets by not opening up their source code, but they still can produce mediocre software. Open-source systems such as OpenCV will win in the long run because bugs and faulty assumptions will eventually be revealed under the scrutiny of thousands of programmers," says Lyddy.
Yang Ming at the Institute of Systems and Robotics (Coimbra, Portugal) is also considering OpenCV software in a vision-based intelligent-vehicle-guidance application. At present, Ming is using an off-the-shelf frame grabber with that company's image-processing package. "This software," says Ming, "only includes basic capture functions. For the intelligent-vehicle-guidance system, we will require more image-processing functions, and, because our application is real-time based, we need MMX-based instructions." For Ming, however, the commercial package is expensive, and he plans to use OpenCV.
Third-party vendorsThird-party image-processing vendors that have either tested or used Intel's performance suite, such as Matrox Imaging (Dorval, Quebec, Canada), Adept Technology Canada (Sainte Foy, Canada), Coreco Imaging (St. Laurent, Quebec, Canada), Datacube (Danvers, MA), and Data Translation (Marlboro, MA), offer varying opinions. In 1997, Matrox Imaging became the first image-processing vendor to exploit Intel's MMX technology in version 5.0 of its scientific/industrial imaging software library, the Matrox Imaging Library (MIL). "Adopting MMX for MIL provided performance gains for host-based image-processing applications," says Pierantonio Boriero, product line manager for Matrox Imaging."When Intel introduced its own performance library, we tested and compared it to the MIL and found that our library performed as well as Intel's IPL, if not better in certain areas," he adds. "As Intel continues to release versions of its IPL, we will test it against MIL's capabilities. In the future, we may adopt Intel's libraries if they are more optimized than our own," says Boriero.
Companies such as Adept Technology Canada have chosen to incorporate Intel's performance suite in their software packages for robot guidance, visual programming, and complete machine-vision toolkits. The company's HexSight software provides robot guidance and parts-inspection functions. As an open system product that runs on a PC, HexSight uses the geometry of parts to locate them in a two-dimensional field of view. Its algorithms can find the contours of parts in the image and then identify the position, scale, and orientation of each part. This information can be sent to a robot or machine controller to guide parts handling (see Fig. 2).
Says Jean Noel-Berube, Adept's international sales manager, "Adept is using the Intel library to implement a number of auxiliary functions in the image-processing tool used in HexSight. These functions include various image-processing algorithms, such as arithmetic, assignment, logical, filtering, morphology, and histogram. However, Adept is not using the Intel library as part of the HexSight code where the strength of our technology lies."
Coreco also uses the Intel's IPL in its WiT visual programming package that combines multiple Pentium execution with MMX-based image-processing operators. Using block diagrams to express algorithms, WiT provides a visual programming environment for image-processing development with a collection of C-callable vision libraries for object-character recognition, alignment, barcode reading, and web inspection.
Terry Arden, director of operations at Coreco Imaging, says, "Since Intel's IPL does not always offer what our customers need, Coreco's IPL-based WiT 7.0 is augmented with other MMX-based functions. Even though IPL can perform a 3 x 3 dilation, the structuring element or kernel must be set to use all 1s. Our own version allows arbitrary kernel values. Also, IPL does not offer blob analysis or run-length encoding (RLE), whereas WiT offers an MMX-based RLE generator. Over time, IPL will offer more traditional image-processing components, and we will build on this with image-analysis, alignment, OCR, and barcode-reading tools."
Both Datacube and Data Translation are also considering the addition of more IPL functions to their software offerings. "On Datacube's MaxVideo 2000 PCI-based image processing card, IPL is used when its speed and performance are superior to any other solution," says Larry Taitlebaum, Datacube marketing manager. "In most cases, however, our own internally developed MMX code is faster, more flexible, or more tailored to a specific image-processing tool," he adds. To date, the IPL has been used by Datacube to implement gray-scale morphology, affine and perspective transformations, and a Prewitt edge filter.
Data Translation is exploring the IPL for an upcoming release of DT Vision Foundry, a PC-based machine-vision platform for visual-inspection and automated industrial-imaging applications (see Fig. 3). "Data Translation has looked at Intel's performance suite for use in DT Vision Foundry," says Rob Gregory, Data Translation marketing manager. "But since we are already in the middle of the next release of the software, scheduled for February or March, this will be implemented in a future release."
Just say noAlthough many PC-based image-software vendors have embraced Intel's MMX software, others are not supporting the product. Instead, vendors such as The MathWorks (Natick, MA) and MVTec Software (Munich, Germany) are building software packages that can run on a number of processors and operating systems."We last looked at the Intel Performance Library in the summer of 1998," says Steve Eddins, development team leader at The MathWorks. "At that time it was unsuitable for use with MatLab. Our image-processing tools for use with MatLab, which are available in the MathWorks' Image Processing Toolbox, run on PCs using Windows or Linux and on UNIX workstations from Sun, Hewlett-Packard, IBM, and SGI. Intel's IPL is limited to PCs and seems especially optimized for 8-bit data, whereas our scientific and engineering customers are more interested in 16-bit and floating-point data, which the IPL did not support. In some 16-bit benchmarks that included median filtering and image histogram functions, our algorithms implemented in portable C code ran faster than IPL," he adds.
In recently released MatLab 6, fast Fourier transforms (FFTs) are computed using the FFTW library from the Massachusetts Institute of Technology (Cambridge, MA). Although written in portable C code, FFTW algorithms have been shown to outperform even vendor-optimized FFT codes (see Vision Systems Design, Nov. 1999, p. 12). The FFTW algorithms compute prime-length FFTs so that developers do not need to pad their FFT sizes to powers-of-two, resulting in memory savings.
Another company targeting general-purpose image processing is MVTec Software. The company's Halcon machine-vision tool provides operators for factory automation, quality control, remote sensing and aerial image interpretation, medical image analysis, and surveillance tasks (see Fig. 4). These operators can be used in ActiveX/COM, C, or C++ applications to speed the design of machine-vision applications.
Olaf Munkelt of MVTec says, "With one exception, Halcon doesn't use any MMX operations. In fact, Intel's performance library suite is not used at all. Because Halcon is a general-purpose package, it is not optimized for a particular processor, but runs under 64-bit Solaris, UNIX, and IRIX operating systems. Interestingly, the performance gain (of IPL) cannot be verified, as can be seen on the Web at:
developer.intel.com/software/products/perflib/ipl/iplperfspec.htm."
Berkeley Vision Group's Lyddy sees the limitations of Windows as compounding the problem. "Windows was designed from the ground up as an interactive operating system, so its human interface is optimized at the expense of overall stability, reliability, and remote usability. Windows applications sometimes crash for no apparent reason and generally refuse to run unattended. My software will eventually be deployed in traffic cabinets by the sides of highways in California. So, the need to run for months or years without human interaction is a high priority. Because of this, any library or system that runs exclusively on Windows was immediately rejected."
Company InformationDue to space limitations, this Product Focus article does not include all of the manufacturers of the described product category. For information on other suppliers of machine-vision software, see the 2000 Vision Systems Design Buyers Guide (Vision Systems Design, Feb. 2000, p. 88). Adept Technology Canada Sainte Foy, QC, G1V 3V9 CanadaWeb: www.hexavision.comCalifornia PATH/UC Berkeley Vision Group
Richmond, CA 94804
Web: www.cs.berkeley.edu/
Coreco Imaging
St. Laurent, QC, Canada H4T 1V8
Web: www.coreco.com
Datacube
Danvers, MA 01923
Web: www.datacube.com
Institute of Systems and Robotics
Coimbra, Portugal
Web: www.isr.uc.pt
Intel Corporation
Santa Clara, CA 95052
Web: www.intel.com
Matrox Electronic Systems
Dorval, QC, Canada H9P 2T4
Web: www.matrox/imaging
MVTec Software
Munich, Germany
Web: www.MVTec.com/
The MathWorks
Natick, MA 01760
Web: www.mathworks.com
