Imaging Systems Identify License Plates of Moving Vehicles
Similar to microprocessor developments, basic machine-vision technologies are finding their way into embedded applications that provide higher levels of functionality, such as intelligent traffic systems. These automatic operating systems use machine-vision, image-processing, recording, and radar technologies to monitor traffic- and vehicle-associated functions with high speed, accuracy, and low cost.
By Mike Muehlemann
Similar to microprocessor developments, basic machine-vision technologies are finding their way into embedded applications that provide higher levels of functionality, such as intelligent traffic systems. These automatic operating systems use machine-vision, image-processing, recording, and radar technologies to monitor traffic- and vehicle-associated functions with high speed, accuracy, and low cost. Application functions range from speed enforcement, license plate reading, toll verification, parking security, and traffic-light violation.
FIGURE 1. The E-ZPass system has been installed on the New York State Thruway Authority System to automate the collection of tolls for moving vehicles. This traffic imaging system identifies moving vehicles via their license-plate number and transmits this information to the preregistered driver's billing account. These tollbooths reduce traffic congestion and save labor and operational costs.
These traffic imaging systems integrate machine-vision and radar technologies to provide a turnkey law-enforcement solution for detecting vehicle offenses. For example, the radar system monitors an approaching vehicle's speed, and, if the speed is faster than a preset level, it acquires an image of the vehicle. The image (or in many cases multiple images) forms a record of the type and make of vehicle, along with license-plate information. All of the information is then wired to database and ticket-processing centers, where hard-copy tickets are printed and mailed to offenders. Typically, these traffic imaging systems are built into mobile platforms that can be moved at random and positioned in key problem areas and at highway tollbooths.
Traffic-light enforcement is another application that has benefited from improved vehicle-vision technologies. Violation of traffic signals, for example, running past red lights, is a major safety issue and causes numerous fatalities. Imaging systems are now capable of monitoring both the color of the traffic light and the movement of vehicles through intersections. When a vehicle moves into the intersection against a red light, the imaging system detects the offender and records the traffic-light color, the make and model of the vehicle, and the license plate. All this information is automatically sent from permanent intersection installations to ticket-processing centers.
For parking-lot security, rental-car agencies are using automated imaging systems to monitor vehicles that enter and exit their facilities. High-security areas, including government, military, corporate, or public facilities, are using license-plate imaging systems for monitoring and archiving vehicle flow. These imaging systems are providing high levels of security and documentation files for applications ranging from larceny to terrorism.
Other areas that have rapidly implemented automated imaging systems are tollbooth collections and customs processing at border crossings. These systems image the vehicle and acquire information regarding the make and model, the license-plate number, and the border-crossing time. This information enables the correlation with future information, such as the length of stay.
Automated toll collection
Analogous to border-crossing applications, many state highway agencies are installing traffic imaging systems for use on high-traffic toll roads, bridges, and tunnels (see Fig. 1). The push for tollbooth-collection automation is increasing dramatically, especially in densely populated metropolitan areas, as traffic volume increases and the associated time delays of manual tolls cause staggering traffic jams. In the United States, several successful preliminary installations of automated vehicle imaging include the SunPass system in Florida, the E-ZPass system on the New York State Thruway Authority (NYSTA) system, and the Fast Lane system in Massachusetts.
FIGURE 3. Typical traffic imaging system comprises a camera, lighting, and OCR software modules. Generally mounted on a tollbooth, underpass, or bridge, it is triggered by a laser-based vehicle-location system (not shown).
These systems use a variety of imaging technologies to automate the collection of tolls and to minimize the driver time spent at the tollbooth. Currently, these systems use radio tags within the vehicle that communicate with the tollbooth imaging system. Within milliseconds, the approaching vehicle is identified and the toll is automatically withdrawn from the driver's prepaid account. Most installed systems direct the user to exit in special 5- to 15-mph lanes.
These imaging systems are used predominantly as enforcement tools for violators of the special automated lanes. If a vehicle exits the automated lane without the special radio tag or with an overdrawn account, the imaged license-plate information is forwarded to a ticketing center.
The commonality of all these imaging applications is the acquisition of the license-plate image and the subsequent reading of the information using optical-character-recognition (OCR) techniques. Over the past decade, these systems have provided solutions for various traffic applications, with minor variations to standard vision/imaging hardware, software, and lighting components.
Automated tollbooth stations have demonstrated improved traffic flow, by improving service to drivers, and increased revenues for state toll collections. In North America, several progressive organizations have begun developing the next generation of imaging systems for higher-speed toll collection. For example, the Canadian providence of Ontario has installed a vision-based system that uses no radio tags or user registration other than license-plate information.
Similarly, in the United States, the NYSTA has performed extensive roadside tests over the past 24 months to implement automated toll-collection systems that allow vehicles to pass through tollbooths at highway speeds. The primary reason for installing this type of system is to eliminate tollbooths in areas where major highways converge. This setup permits vehicles to transfer on and off tollways without slowing down.
However, updated high-speed toll-collection systems provide additional challenges over those of current systems. The new systems require that all vehicles using the throughways be properly identified at both entrances and exits. Currently, on the NYSTA, vehicles require proper identification as defined by eight toll classes and ten subclasses.
Intense NYSTA testing has demonstrated that laser-based imaging systems help provide the best vehicle information regarding length, number of trailers, and number of wheels for proper toll classification. Similarly, these imaging systems also identify the vehicle road and lane positions necessary for a camera-based acquisition system to acquire both front- and rear-license-plate images for proper identification (see Fig. 2). Based on these images, the OCR software reads the license-plate numbers and transfers the information to a centralized processing station.
The basic components for traffic imaging systems involve tailored digital cameras and frame grabbers, specialized lighting devices, and application-specific software. Whereas these components are similar to those developed for generic machine-vision applications, they have been tailored to meet specific parameters for license-plate-recognition applications.
For the demanding NYSTA installations, turnkey license-plate reader systems developed by Tecnicon International Inc. (Haymarket, VA) have exhibited reliable operation for more than 18 months (see Fig. 3). The camera used within these systems is the Model 1015T traffic camera from Eastman Kodak Co. (Rochester, NY). This digital camera was chosen because it uses a 1K x 1K-pixel CCD sensor that delivers the required resolution over the specified field of view.
Because a vehicle's license plate can potentially pass anywhere within a highway lane, the camera's field of view must be more than 12 feet in width, and yet the license-plate characters must still be identifiable by the OCR algorithm. Another major design consideration centers on the fiberoptic system interface. A gigabit fiberoptic ring network allows as many as four cameras per fiber pair to operate at 15 frame/s per camera.
The fiberoptic ring is used in these applications because of its wide bandwidth and high noise immunity. A typical traffic-imaging-system installation requires long lengths of signal and control cables for the successful operation of a multilaned tollbooth collection station. Copper-based signal lines are limited to about 30 m in length for carrying digital camera transmissions, whereas the fiberoptic platform allows a transmission range of up to 2 km.
In addition to carrying the cameras' digital outputs simultaneously, the fiberoptic platform also provides additional bandwidth for accommodating such input/output controls as strobe triggers, camera operation, and pan/tilt/zoom commands. While the fiberoptic-link installation provides several advantages for this niche application, it did encounter a serious problem. At system design time, no frame grabbers were commercially available with gigabit fiberoptic interfaces. To solve this problem, Tecnicon developed a special frame grabber with the proper fiberoptic interface.
The software used in the NYSTA systems is called TollEnforcer; it uses ImageView32 to generate the traffic citations and reports (both from Tecnicon). The software kernel includes a proprietary back-end OCR processing system for plate recognition and a Department of Motor Vehicle database interface that is modified to match a particular state or country protocol.
The main technical issues surrounding the development of the lighting systems for these applications included overall output intensity, spectral properties, working distance, field of view, lifetime, operational reliability, robust construction for outdoor operation, and overall power requirements.
To meet these needs, Illumination Technologies (East Syracuse, NY) developed custom light-emitting-diode (LED) lighting systems.
The intensity requirement is defined by the sensitivity of the camera, the reflectance of the license plate under investigation, and the integration time required to obtain a blur-free image. The intensity requirement increases in direct proportion to the speed of the vehicle under inspection because the duration of the image acquisition must be minimized to eliminate blurring. In addition, high-speed license-plate reading has increased the intensity requirements over earlier low-speed systems.
A key issue in the successful implementation of these high-speed imaging systems is how to produce enough light without creating a nuisance, distraction, or potential safety hazard for the vehicle's driver. Earlier slow-speed or no-speed systems used large floodlight systems mounted close to the lane to illuminate license plates. When high-speed application demands emerged, more sophisticated high-output lighting devices were needed.
The most logical choice for these applications is near-infrared (NIR) illumination. A subset of the NIR illumination range from about 700 to 900 nm is invisible to the human eye but is visible to most cameras using either CCD or CMOS sensor technology. Technically, this type of illumination can be produced in a highly desirable pulsed fashion using LEDs. These solid-state devices provide the reliability, lifetime, and covert nature needed for such applications. The developments of this technology resulted in the issuance of a US patent to Illumination Technologies.
Whereas NIR illumination for vehicle recognition is the de facto standard in many parts of the world, the variety of license plates manufactured in the United States provides ongoing challenges. The variability of license-plate type, color, and background reflectivity continues to increase as states add entire new lines of designer and vanity plates, each with different color combinations and graphical layouts. Some states maintain sufficient license-plate control (specifically, using absorbing numerals on retroreflective backgrounds) and find that they still have latitude to develop artistic variations that are invisible under NIR light (see Fig. 4).
Many East Coast installed traffic imaging systems, however, must be capable of reading plates from several different states. Testing has shown that the variety of plates currently available, especially in these regions, make NIR illumination an impossible solution. In these areas, system specifications dictate which plates are to be successfully read, and detailed spectral testing is conducted to determine the proper wavelengths that must be included in the strobe lighting. The ever-increasing output of many visible wavelength LEDs has made it possible to supply solid-state strobe units with two or three distinct spectral outputs that solve the pending applications.
MIKE MUEHLEMANN is president of Illumination Technologies Inc., East Syracuse, NY 13057; e-mail: firstname.lastname@example.org.
Eastman Kodak Co.
Rochester, NY 14650
Illumination Technologies Inc.
East Syracuse, NY 13057
Tecnicon International Inc.
Haymarket, VA 21069