Sophisticated software helps machine vision check bank notes
By R. Winn Hardin, Contributing Editor
New $100, $50, and $20 bills include microprinting, watermarks, and fluorescing security threads. To inspect these security features, the US Bureau of Engraving and Printing (Washington, DC) has implemented novel imaging-inspection techniques to ensure the quality of
the bank-note-printing process. To do so, the bureau awarded a contract to the engineering division of Giesecke & Devrient America (Bedford, MA) for an off-line currency-inspection system.
For inspection of watermarks and security threads, the company subcontracted a machine-vision system from Vision Machines (North Reading, MA) to inspect the notes. Vision Machines developed a transmissive inspection subystem (TIS) capable of detecting watermarks and security threads. Using a Cognex (Natick, MA) based vision platform, hierarchical search processes, and normalized-correlation techniques to validate the presence of the new security features, the system reduces area-specific bank-note reject rates to approximately 0.03%.
Operating as a subsystem to Giesecke & Devrient`s Image Guard print inspection, the TIS can also be used as a stand-alone unit with a programmable logic controller (PLC), depending on the application. While the Image Guard system uses CCD cameras and dedicated processors to inspect a sheet of 32 bank notes with fine-line printing and color-shifting ink, the role of the TIS is to image and analyze two bank notes per sheet (see "The color of money," p. 28, and Fig. 1).
"Asynchronous operation is important because the system does not know when a currency sheet is in position," says Marc Landman, president of Vision Machines. The TIS analyzes two bank notes in two columns on an individual sheet. Operating at speeds to 10,000 sheets/hour, system line speed dictates that only 100 to 200 ms must transpire between captured frames.
Imaging techniques
Before inspection of the watermark and security thread, the TIS locates the upper left-hand corner of the bank note. This corner serves as an origin from which feature positions can be determined using normalized-correlation pattern-location techniques. Although this process is mathematically intensive compared to binary correlation and blob analysis, it is made easier by hierarchical search procedures that first identify which areas in an image are most likely to yield a positive match with a given template.
According to Landman, the Cognex 5400 board uses two types of imaging models. These models or templates are examples of what a specific feature in an image should look like. If the image and the template match, then the part or object is considered "good." The Cognex 5400 board can use either a mathematical expression of an object or be trained to recognize specific features by storing real-world images as bitmaps for future use. ©
"The corner of a bank note is an example of when to use a synthetic model because corners are scale-invariant," says Landman. The process begins with a hierarchical search that uses dedicated hardware on the Cognex board to filter and sample the image to generate several versions with varying levels of resolution. In operation, low-pass filtering followed by sampling every other pixel in both dimensions produces a half-resolution image; the process is then repeated for subsequent levels.
Starting with the lowest-resolution image, the board begins a search (scanning the template across the image) using
where N is the number of pixels, image pixel = Ii = I(u+xi, v+yi), M is the corresponding model pixel at the relative offset (xi, yi), and r is the correlation coefficient, expressing the level of similarity between the template and a specific image area.
In this application, r ranges between -1 and 1, where -1 is a perfect mismatch and 1 is a perfect match. According to Bill Silver, vice president of research and development at Cognex, normalized-correlation techniques take the correlation equation one step further by inserting the correlation equation
n(u,v) = max (r (u,v), 0)2
This is a normalized equation that yields n, a normalized correlation value between 0 and 1, with an expanded range in the upper values. Cognex converts this near-percentage to a range between 0 and 1000 for analysis purposes.
After searching across the entire lowest-resolution version of the bank note and identifying positions that match the model of the upper left-hand corner, the search engine begins again with the next-highest-resolution image, then the next, focusing on areas that show the highest degree of similarity. The process is often characterized as hill-climbing, where hills represent the most likely prospects for a match, the highest hill being the best match. The process decides which are likely areas and later prioritizes these areas for a "best first search" by using accept and confusion thresholds.
The accept threshold is the lowest correlation value a feature can have and still be considered a possible instance of the feature being searched for. The confusion threshold represents the correlation value above which a candidate feature is guaranteed to be an instance of the feature being searched for. The confusion threshold, along with the number of expected results ("hills"), allows the search to be terminated before all hills have been explored.
An accept threshold set at 500 says that any area yielding a normalized correlation value higher than 500 should be kept as a likely match. If one of these matches exceeds the confusion threshold, which is typically set higher than the accept threshold, then the image-analysis functions should pursue this likely match to its completion because it is most likely the feature under search. By using this hill-climbing technique, the Cognex board reduces processing time without missing a matching feature for one that is similar.
Normalized-correlation techniques possess several properties that make them useful in locating patterns. First, the correlation value is independent of linear brightness changes in both the image or the model. In currency-sheet inspection, correlation values are largely unaffected as light output from the strobe decreases with age. Second, the correlation coefficient is dimensionless and provides an absolute indication of pattern similarity under different imaging conditions. Third, normalized correlation can provide positional information to subpixel accuracy that is less affected by image degradation than blob-analysis methods based upon a feature`s center of mass.
After the TIS has identified the upper left-hand corner of the bank note, it locates and analyzes the security thread and the watermark (see Fig. 2). Rather than using a 2-D edge filter to detect local changes in intensity, the caliper tool uses 1-D convolution (see Fig. 3). In this process, 2-D image data are projected onto a 1-D image, and a 1-D edge filter is used to locate each edge in the image.
After this edge has been found, a scoring algorithm discriminates correct from incorrect edges by using expected position, expected size, contrast, and edge polarity constraints. These constraints are important because the security thread is intentionally nearly invisible. Thread location is made more difficult because of variations in paper density, presence of colored fibers in the paper, and dust from the printing process. By performing a pixel-to-inch conversion based upon system calibration, system software determines the location of the thread. Because the thread`s horizontal position in the bank note is indicative of the denomination, the software can verify that $100 notes have been printed on $100 paper and have security threads for proof of engraving.
Degraded images
In the second part of the TIS analysis, software draws a window where the watermark should be and searches for that region using normalized correlation. Because the system is transmissive, both sides of the printed paper are seen by the imaging system. The watermark, located to the right of the portrait on the note, resembles this portrait and can change from bank note to bank note because it is not printed but created by varying paper density in a small area during the papermaking process. Consequently, watermarks are difficult to counterfeit and also difficult for machine-vision systems to quantify.
To establish an adequate template or model, the TIS is statistically trained on the watermark by combining multiple watermark images on different bank notes and averaging the results (see Fig. 4). This results in a bitmap template with a range of acceptable values for each pixel.
Choosing the proper size model is also an important task. The model selected should be sufficiently singular to positively identify the watermark and sufficiently different from watermarks of other bank-note denominations to prevent confusion. Larger models yield more reliable results, because they are less likely to suffer from image degradation and not give high correlation values; smaller models increase system throughput and lessen problems caused by interference from front or back ink areas as they vary relative to watermark positions.
After the TIS has verified the presence and location of the security thread and the watermark, results are sent to the Printrak subsystem, where they notify a PLC in the event of a reject. Current sheet results and batch totals are displayed on an operator monitor and can be logged to a PC file via a 32-kbyte shared memory interface between the Cognex board and the host PC. "Both the vision and nonvision analyses are performed on the Cognex board.
The operator can view the image analysis in a text format on the PC monitor or see the image captured by the Hitachi camera on an RGB monitor placed next to the PC monitor. The operator can also switch the setup for different currency denominations at a user interface on the Printrak, where the signal is relayed via serial cable to the TIS, or from the TIS interface itself, in the case of stand-alone operation.
Vision Machines` transmissive inspection system uses normalized- correlation pattern-location techniques to perform image analysis while avoiding the pitfalls of inadequate lighting found in many manufacturing environments.
FIGURE 1. As a sheet approaches the TIS, ImageGuard sends a tracking number across an RS-232 link to the TIS system. After issuing the tracking number, the ImageGuard system delivers a trigger signal to the TIS over a parallel I/O line. The Cognex 5400 processor then triggers a MVS-5000 strobe from EG&G Electro-Optics underneath the currency sheet and a KP-M1 2/3-in monochrome interline transfer CCD camera from Hitachi Denshi installed above the bank note sheet. After capturing the analog RS-170 768 ¥ 493 image, the Cognex board digitizes the image for further processing.
FIGURE 2. After the TIS has identified the upper left-hand corner of the bank note, it locates and analyzes the security thread and the watermark. Using the bank-note corner as a reference, regions of interest are determined along the bottom and top of the bank note in the white border where the thread is most visible. Thread verification is based upon Cognex`s caliper tool, which uses edge-detection to locate edges or edge pairs despite noise in the image background.
FIGURE 3. Rather than using a 2-D edge filter to detect local changes in intensity, the caliper tool uses 1-D convolution. In this process, 2-D image data are projected onto a 1-D image, and a 1-D edge filter is used to locate each edge in the image. Analogous to taking the first derivative of the intensity profile, the software searches for maximum rates of intensity change. Once it determines a significant rate of change, the potential edge of interest has been found.
FIGURE 4. To establish an adequate template or model, the TIS is statistically trained on the watermark by combining multiple watermark images on different bank notes and averaging the results. This results in a bitmap template with a range of acceptable values for each pixel. When a pixel varies more than a defined standard deviation value, the software declares it a "don`t care" pixel, and it is not used in the analysis. This process results in a detailed template that proves important to correlating matches for watermarks containing substantial amounts of distortion and contrast variations.
The color of money
From the 1920s to the early 1990s, US currency remained the same in appearance and content. However, recent technical advances in ink-jet-printer, digital-scanner, and color-copier technologies and their use by counterfeiters have attracted the attention of officials of the US Department of Treasury (DoT).
In response, the DoT turned to the National Academy of Sciences for answers on how to counter these printing, scanning, and copying technologies. In 1993, the US Bureau of Engraving and Printing evaluated 120 security improvements to US currency and selected several for implementation including using enlarged portraits placed slightly off-center to reduce the amount of wear. As a result of moving the portrait to the side, the bureau also incorporated a watermark by varying the density of the paper until it resembles a replica of the portrait.
A security thread is also embedded in the currency with the message USA HUNDRED, FIFTY, or TWENTY printed on the thread. When subjected to ultraviolet (UV) light, the $20 bill thread fluoresces green, the $50 bill glows yellow, and the $100 bill glows red.
Special ink is also used to create the denomination numeral on the lower right-hand corner of the bank note. When viewed at an angle, the ink changes from green to black. This helps eliminate the danger of standard or even advanced reprographic counterfeiting.
New money production also involves micro- or fine-line printing. Microprinting is used in the background of the portrait, in the building image on the opposite side of the bank note, in the lower left-hand corner of the $20 and $100 bills, in the border of the $50 bills, in Ulysses Grant`s collar, and in Benjamin Franklin`s coat. When new bills are counterfeited, this fine-line printing results in a blurred copy being produced.
R. W. H.
