High speed calls for optical image processing in pattern-matching systems
By Andrew Wilson, Editor at Large
Pattern-matching systems using optical correlators were originally pursued as a solution for fast target-recognition in military systems. Compact, fast, and accurate, these systems are generally used to analyze data from synthetic aperture radar (SAR) and hyperspectral sensors and provide target cuing and identification. In these applications, optical-correlator-based systems overcome the bottlenecks associated with processing large SAR or hyperspectral imaging data.
Several researchers are now using the technology for nonmilitary applications such as cervical-smear analysis, security systems, and
remote-sensing applications, in which correlations between large data sets must be rapidly accomplished. Indeed, the recent improvements in commercially available spatial light modulators from Boulder Nonlinear Systems (BNS; Boulder, CO), Displaytech (Longmont, CO), and others have made it possible to develop compact analog optical processors that are orders of magnitude faster than their digital counterparts.
Correlator design
In most popular optical correlators, the image to be processed is first displayed on an input spatial light modulator (SLM) and then illuminated by a coherent beam of light from a laser diode (see Fig. 1). This approach is taken by Bertrand Noharet, a research engineer at the Institute of Optical Research (Stockholm, Sweden). In operation, the SLM, a 256 ¥ 256 ferroelectric-liquid-crystal (FLC) device from Displaytech, reflects the coherent beam and is then Fourier transformed and multiplied by a two-dimensional function displayed on a second SLM.
"This function most often carries information about the object to be detected, possibly about other objects, or noise and clutter to be rejected," says Noharet. After a second optical Fourier transform, the correlation peaks of high intensities are detected by a CA-D1-56 CCD camera from Dalsa (Waterloo, Ontario, Canada) and then digitized into a personal computer (PC) using a frame grabber from Imaging Technology (Bedford, MA).
Next, the input images are read from PC memory or grabbed as a live sequence with another CCD camera that is capable of acquiring as many as 220 images per second. "To overcome the poor modulation capability of the FLC SLMs, different types of preprocessing are implemented on the frame grabber," says Noharet. "Preprocessing, which involves edge-enhancement operators, enhances the relevant features for correlation and can improve the discrimination capability of the correlator," he adds.
In the original system design, however, system throughput was limited by the slow interface between the input camera and the optical correlator. To overcome this, Noharet and his colleagues designed and built a dedicated hardware interface placed on the frame grabber that converts the signal from the preprocessor into a signal that can directly feed the SLM. "This interface has been tested at 40 frames/s and is currently being improved to reach 200 frames/s, the limit of the CCD camera," says Noharet.
To show how effective optical correlation is at pattern recognition, Simon Boothroyd of the National Research Council of Canada (NRC; Ottawa, Ontario, Canada) has used optical-correlation techniques in a prototype face-recognition system. When two similar images are presented to the correlator, a high degree of similarity results (see Fig. 2). Conversely, when two different images are presented to the device, the output of the correlator produces no peak, corresponding to a lack of correlation in the two images.
SLM choices
In its SLM products, BNS allows developers a choice of liquid-crystal-based SLM types, including nematic, smectic C, and smectic A. According to BNS, all the SLMs modulate light due to their inherent birefringence, but each has characteristics that work well only with certain applications.
Whereas nematic devices are useful for displays or for applications in which a large phase-only modulation depth is needed, they offer fast optical risetimes (50 to 500 ms) but slow falltimes (about 2 to 3 ms).
The smectic-C devices are the most popular due to their high speeds and modulation characteristics. When very high switching speeds are required, smectic-A devices are usually used. These devices must be operated at elevated temperatures and typically exhibit short lifetimes relative to the nematic or smectic-C devices.
In the development of a 128 ¥ 128 optical correlator based on the company`s SLM technology, BNS used a Dalsa Model DA-C1-0128A camera as the input device to the SLM (see Fig. 3). According to Kipp Bauchert, a BNS electro-optics engineer, the camera is capable of loading new images into the SLM at 732 frames/s with every other frame being inverted to maintain a balanced electrical field. This inversion results in a useful 366 frames/s.
The Dalsa output camera is also driven at 732 frames/s, with each frame captured by a proprietary frame grabber. Captured images are transferred to host memory for processing by a peak-detection algorithm. The results show an input image of X and O and a correlation with a small X-filter using the binary optical correlator. The peaks indicate the correlation between the two images.
Other approaches
In correlator designs, the data describing the object to be recognized are presented as a complex spatial filter at the Fourier-transform plane. And, although the SLMs provide high-spatial resolution and light modulation in either intensity or phase, they are relatively expensive, specialized devices. In an effort to lower costs, Mark Gardener and his colleagues at the University of London (London, England) are experimenting with the use of liquid-crystal-television (LCTV) display panels to modulate the light in an optical-correlator design.
"In a twisted-nematic LCTV, the voltage across the pixel tilts the liquid-crystal molecular detectors," says Gardener. "Besides the polarization-changing property, this action presents a different refractive index to the incident light, and thus two different degrees of tilt will slow the incident light by different amounts and modulate the light in phase."
Unfortunately, whereas a phase-only filter offers the best performance in optical correlators, the phase modulation from a LCTV is always accompanied by intensity or polarization modulation due to its inherently twisted structure. By careful choosing operation configurations and polarizer orientations, Gardener and his colleagues hope to optimize the LCTV for maximum phase modulation with minimum accompanied intensity modulation so that such devices can be used for effective correlation.
Applications
Although much industry research has focused on target recognition as the major application for optical correlators, researchers are now looking to perform other image-processing functions with the devices. For example, Mark Begbie at the University of Edinburgh (Edinburgh, Scotland) is studying the use of the devices as image-processing building blocks. "In essence," says Begbie, "an optical correlator performs the mathematical correlation of an input image with a target image. Although this may appear to be very restrictive, any processing function that can be written in terms of correlation can be implemented using such architectures," he adds.
Functions that Begbie is studying include morphological operators. With the processing power of an optical correlator operating at 1000 images per second, real-time morphological video processing can be accomplished. One such application is removing noise in an image (see Fig. 4). "By performing four successive morphological operations, it is possible to remove both positive noise in the background and dropout noise in the image while keeping the image data intact," says Begbie.
Pap-smear analysis
An important medical application of optical-correlator technology is in the area of automated screening of cervical-smear slides. Typically, each slide is screened at 20X, and 3600 images measuring 400 ¥ 400 mm are processed. To automate this procedure, researchers Ramkumar Narayanswamy, John Metz, Rosemary Stewart, and Kristina Johnson of the Optoelectronic Computing Systems Center of the University of Colorado (Boulder, CO) are developing an optoelectronic automated two-step screening system.
In the first processing step, the slide is examined at high speed (up to 1000 images per second) to locate and mark suspicious regions. "This region-of-interest (ROI) detection reduces the amount of data requiring computer classification by a factor of 1000," says Metz. The classification system looks at various features of the nuclei, such as texture of the nucleoplasm and fractal dimension of the nuclear membrane, using traditional image-processing techniques on a PC. Because the PC classification step takes a long time for large data sets, data reduction without loss of important information becomes essential.
"Our optical/electronic ROI detector provides this high-speed data reduction without eliminating the crucial pieces of information necessary for diagnosis. Thus, most of the "hay" in the "needle-in-a-haystack" problem has been removed, making finding the needle much easier," says Metz. The ROI detection is performed using mathematical morphology feature detection in the form of a hit/miss transform (HMT) to detect the shape and size of the cell nucleus. If the nucleus is circular with a diameter larger than 10 µm and smaller than 20 mm, the cell is identified as suspicious. "This eliminates unwanted areas of the slide such as white blood cells and larger items such as air bubbles and dust particles," says Metz.
Computer simulation of the algorithm tested on 184 Pap-smear images detected 95% of the suspect regions as suspect, while tagging just 5% of the normal regions as suspect (see Fig. 5). The HMT transform is currently being performed optoelectronically using an optical correlator. This morphological processor uses an optically addressed SLM for image acquisition with a laboratory microscope; it also performs incoherent-to-coherent optical conversion. A 256 ¥ 256 electrically addressed SLM operating in the phase mode at the filter plane and a custom-designed smart-pixel VLSI integrated circuit at the output plane perform fast detection and calculation of the HMT output.
FIGURE 1. In optical correlators, the image to be processed is first displayed on an input spatial light modulator (SLM) and an illuminated laser diode. In this design, developed at the Institute of Optical Research (Stockholm, Sweden), the SLM reflects the beam and is Fourier transformed and multiplied by a two-dimensional function displayed on a second SLM. After a second optical Fourier transform, the correlation peaks of high intensities are detected by a CCD camera and digitized using a PC frame grabber (top).
FIGURE 2. To show how effective optical correlation is at pattern recognition, Simon Boothroyd of the National Research Council of Canada uses optical-correlation techniques to recognize faces. When two similar facial images are presented to the correlator, a high degree of similarity results when the correlator output contains a peak (top). Conversely, when two different facial images are presented, the output of the correlator does not produce a peak (bottom), corresponding to a lack of correlation
FIGURE 3. In its 128 ¥ 128 optical correlator, Boulder Nonlinear Systems uses a Dalsa CCD camera as the input device to a spatial light modulator. Captured images are then transferred to host memory for processing by a peak-detection algorithm (top). Correlating an input image of Xs and Os with a small X filter (bottom left) using the binary optical correlator shows intensity peaks that indicate the correlation between the two images (bottom right).
FIGURE 4. By performing four successive morphological operations on an image (left), optical correlator can remove both the positive noise in the background and the dropout noise in the image while keeping the image data intact (right).
FIGURE 5. A typical cervical-smear image shows a region of abnormal cells in the center of the image with normal cells on both sides and across the entire bottom of the image (top). Abnormal cells are distinguished by their larger nuclei and lower nuclear cytoplasm ratio (bottom). A computer simulation of the hit/miss transform tested on 184 Pap-smear images detected 95% of the suspect regions as suspect while tagging just 5% of the normal regions as suspect.
