Ned Lecky
Due to the diverse processing scenarios involved in thetransportation imaging business, system integrators must be able to implement their software solutions on many different platforms.
These may range from embedded single-board computers, to server farms that are collecting thousands of images per second, to supercomputers, which might be analyzing thousands of images for vehicle recognition or monitoring the flow of traffic.
The computer hardware used invision systems for transportation applications covers the entire spectrum of price and performance (see Fig. 1). But whether such systems employ a mobile GigE-embedded PC, a lower-power embedded PC, or just a bare Pico-ITX dual-core single-board computer, even basic applications will demand real-time processing of the image stream to handle image comparisons or pattern-matching, real-time optical character recognition (OCR), lighting, exposure, and gain adjustment, as well as some form of compression.
Currently, there is a great deal of interest in developing applications for smart-phone platforms, too. Such software would allow users to interrogate or diagnose vision systems remotely, or provide a way to upgrade the software running on them in the field. Many new applications are expected to be developed to fulfill these requirements in the near future, and many transportation imaging vendors are currently investigating the possibilities.
In terms of operating systems (OSs), Microsoft's Windows XP—and now Windows 7—are supported by all of the networking interconnection schemes and cameras. At the same time, Linux OSs, such as Ubuntu, are much less expensive than Windows while offering users higher performance and greater reliability.
In applications involving very high camera counts—such as those that employ a mix of numerous types ofline- and area-scan cameras that are all capturing data simultaneously—Windows can prove inadequate for the task, while Linux does not encounter similar difficulties.
For that reason, Linux is becoming increasingly popular in more complex image-processing applications. Testifying to that fact, Lecky Integration has developed Linux systems that have been continuously acquiring images on ten cameras for the better part of a year or longer without having been rebooted once.
The OS becomes a significant part of the overall component expenses when the final transportation inspection system must be relatively inexpensive. Therefore, eliminating that cost by using open system software is an attractive option that provides a reasonably easy solution to implement.
Fortunately, there are nearly as many development tools running under Linux as there are on Windows. Furthermore, the development tools that can be used on Linux—such as NetBeans—are very solid, well-known, free, and open source.
There are also a number ofimage-processing algorithm libraries available. Some are free; other providers offer a free fixed library of algorithms while charging a premium for tools that are more complex or highly application specific. Since those are available for Linux, it is possible to develop the same kind of applications under Linux as one might by using Windows.
Image-processing tasks
Image-processing tasks can be handled by five classes of algorithms. These may detect the presence or absence of objects, calibrate or gauge their size or shape, recognize and verify optical characters, detect and read barcodes, or perform pattern recognition.
The number of software tools available to accomplish these tasks is astounding, and there is a lot of parity between all the different types of products in the marketplace. Developers have been writing such algorithms for a long time; today they are not only robust but offer high performance as well.
The problem is that, because of the unique demands of the transportation market, algorithms that perform well in industrial machine-vision systems may be unsuitable for tasks in transportation applications.
Detecting the absence or the presence of a part or feature is usually very straightforward in a machine-vision environment. But such detection in a transportation application can be substantially more challenging.
Consider a simple case of developing a system to identify the arrival and departure of a train. Although the problem sounds trivial, the system must take into account other moving objects that may be behind the camera's field of view such as people or other trains, as well as reflections from these objects and changing weather conditions.
Furthermore, since no two trains are alike, attempting to detect the presence and absence of the train will require looking for an object in a scene that is common to them all. Often, much development effort is required to determine what that object and its size and motion profile characteristics might be, and, just as important, how they might be measured.
Similar issues arise inmetrology applications. Even if a system is required to take a low-accuracy measurement of a truck's height, it must be able to identify the specific edges of interest in the image, compensate for the variability in the distance and angle where the truck is located in relation to the camera, and be capable of handling varying degrees of illumination, as well as shadows that are cast on the truck.
There are many different high-speed algorithms that have been developed over the years to handle OCR. Because many companies have been working on developing OCR software, most software products that are on the market have little problem identifying characters effectively.
License plate reading (LPR), however, is a more difficult problem to solve. In the transportation industry, there is often enormous variation in the number of fonts that are used on number plates. The location, scale, and angle at which the plates are presented to the system can be highly variable. Adding to the problem is that many plates may have deteriorated over the years, resulting in fonts that are difficult to identify. Off-the-shelf algorithms are not capable of handling such tasks, and so it is necessary to develop an OCR algorithm that is suitable.
Identifying the location of numbered plates on moving vehicles under a variety of illumination conditions is quite complex; therefore, LPR is a computationally intensive process that involves performing a number of iterative procedures to find the image of the plate.
In LPR systems, images are segmented into specific areas and processed to determine whether the objects in those areas represent lines or blocks that might indicate the presence of a plate. During the identification process, many different image-enhancement operations may be performed on the same image before an image of the plate is recognized. Once it is, the results can be processed by an OCR engine that can read the characters.
Reading barcodes in a transportation environment presents similar difficulties. Like OCR algorithms, barcode algorithms are also widely available and robust, but because of the need to capture images in focus at a high resolution while also taking into account any image blur, the processing time for reading the barcodes tends to be higher and the read rate considerably lower than in traditional industrial applications (see Fig. 2).
False readings
Although today's machine-vision systems can identify characters as effectively as a human being, the accuracy of any OCR system is dependent on the quality of the markings on the object to be inspected (see Fig. 3). One of the issues involved in identifying vehicle ID numbers is that, in some situations, vehicle owners or vandals will make efforts to obscure them. In situations like these, of course, the failure rate of a system will inevitably be 100%.
However, in the transportation market, there are means by which false readings can be reduced. In the case of subway cars, the ID number is repeated once on the front and once on the back of the car. By comparing those two numbers, the false read rate can be lowered dramatically. Frequently, the ID numbers on a vehicle will also appear on both sides and on the top. Similarly, by reading and comparing them all, the number of false reads can be reduced.
Algorithms for pattern recognition have been developed to an incredible extent in the area of industrial machine vision. Pattern recognition in the transportation industry remains problematic unless the system is constrained to recognize features on a vehicle-by-vehicle basis. There are no off-the-shelf machine vision pattern-recognition algorithms, for example, that have the ability to identify a car in three different perspectives.
Onerous demands
There is no doubt that the transportation market places some unique demands on developers of vision systems—far more onerous than those faced by developers of traditional machine-vision applications.
As a result, components such as lighting, camera optics, computer systems, and software that is developed to run on them are almost inevitably very application-specific and tailored to meet the needs of the individual image-processing environment.
Although it is possible to rely on some image-processing libraries and some off-the-shelf software algorithms, frequently custom algorithms and interfaces must be developed to make an application rugged, reliable, and problem-free in the field.
