Updates On The State Of The Licence Plate Identification / Recognition System

In others, like commercial vehicle operations or secure-access control, a vehicle’s license plate is checked against a database of acceptable ones to determine whether a truck can bypass a weigh station or a car can enter a gated community or parking lot.
LPI/R can be used to issue violations to speeders or simply to offer a reminder by displaying a plate number with the vehicle’s speed on a variable messaging sign. It can facilitate emissions testing by recording a plate’s alphanumeric sequence while automatically analyzing tailpipe effluents, or help identify and fine violators. LPI/R also can monitor the time it takes vehicles to travel from one point toanother, keeping traffic management centers apprised of transit times along busy streets and highways.
At international border crossings, license plates - the only universal vehicle identifier - can be checked against a database of "hot" cars to locate stolen vehicles and plates or those registered to fugitives, criminals, or smuggling suspects.

SYSTEM COMPONENTS AND HOW THEY WORK

A typical LPI/R system is comprised of a video imageacquisition subsystem, a CPU for image processing and control, a hardware- or software-based character recognition engine, and a storage or transmission facility for electronically recording plate contents and data such as date, time, and location.
At any LPI/R system’s heart is its recognition engine and the embedded algorithm. It is important to understand that the system supplier decides how to code the recognition engine. The user, OEM, or integrator takes that algorithm along with the system. A basic understanding of how the recognition engine interprets image content is central to confirming that the overall "solution" can handle a given application.
The correlation or template matching approach to character recognition is straight-forward and can be reliable, provided the target is "cooperative" and the application remains invariant. As the name implies, once each character is isolated, the recognition engine attempts to match it against a set of predefined standards. Any condition3lighting, viewing angle, obscuration, plate size, font3that causes a character to vary from the standard is likely to confuse the engine and return a questionable result.
Structural analysis uses a decision-tree to assess the geometric features of each character’s contour. The technique can be somewhat tolerant of variations in size, tilt, and perspective. As a simple example, take the characters B, D, 6, and 9. Features that might be used to distinguish them are the number of loops3one or two3and the vertical position of the loop3top, central, or bottom. Two loops point to a B, and one loop leads to the next branch of the tree. A loop at the top indicates a 9; if the loop is central, it’s a D, and a loop at the bottom means a 6. Characters without loops (E, M, N, hyphen, etc.) require additionally complex, time-consuming analysis.
Neural networks are trained by example instead of being programmed in a conventional sense. While learning to recognize a recurring pattern, the network constructs statistical models that adapt to individual characters’ distinctive features. Therefore, neural networks tend to be resilient to noise, and performance usually is not compromised under changing operational conditions. However, each modification (e.g. a new font) that is presented to the neural network may require a significant investment in retraining.

WHAT TO LOOK FOR

Evaluating a system’s capabilities can require that one look beyond the recognition engine’s proficiency with a standard character set under nearly ideal test conditions. It is prudent to determine all factors that influence operations and to learn the effects of those variables that cannot be held constant. They include:
· vehicle speed
· volume of vehicle flow
· ambient illumination (day, night, sun, shadow)
· weather
· vehicle type (passenger car, truck, tractor-trailer, etc.)
· plate mounting (rear only or front and rear)
· plate variety
· plate jurisdiction (and attendant fonts)
· camera-to-plate distance
· plate tilt, rotation, skew
· intervehicular spacing
· presence of a trailer hitch, decorative frame, or other obscuring factors

Applications differ in their requirements. Some mandate that information from the plate be reported to the issuing jurisdiction, others need additional I/O, such as interfacing to a gate controller, an off-site database, or remote archival server. In those cases, it becomes essential to make sure the recognition engine and its host computer can handle the added computational load. Even if that degree of complexity is not contemplated immediately, it is wise to consider whether future enhancements will be possible without a major system reconfiguration.
Buyers should distinguish between absolute identification of each and every character, as for a traffic violation application - an example that requires doing so accurately and reproducibly - versus recognizing a plate the system has seen before; as in inventory control where the universe of plates is finite and known. The degree of computer processing, the recognition engine’s precision, and internal accuracy monitoring are of greater import in the former scenario.
Presently, LPI/R systems excel at inventory control. Many already are being used for vehicle surveillance, monitoring, and origin-destination surveying. In those situations, a single character error is less significant than in enforcement applications. It’s easier for a system to recognize an entire plate it has seen previously3whether or not it was interpreted accurately3than to properly identify each and every character, time and again, under all operating conditions. Clearly, a system designed to identify and fine violators must be sufficiently accurate to prevent law-abiding drivers from receiving undeserved tickets.

A WORD ON ACCURACY

Determining the accuracy of an LPI/R system is complex and depends on the application, operating conditions, and assumptions made during testing. When evaluating a system, it is important to use those criteria to examine manufacturers’ claims carefully.
System performance is difficult to quantify. It’s tempting to expect a machine to be perfect and to assume one hundred percent accuracy in interpreting license plate content.
Of course, a machine can only identify the alphanumerics after it properly recognizes that a plate is present in the fieldof- view. The success rate of each step thus must be figured into the overall calculation:

A = (T ´ I) ´ 100 (1), where
A = total system accuracy, expressed as a percent
T = rate of successful plate recognition, expressed as a decimal number
I = rate of successful interpretation of entire plate content.

If the interpretation of every character is assessed individually, the equation becomes:

A = (T ´ I1 ´ I2 ´ . . . In) ´ 100 (2), where
A = total system accuracy
T = rate of successful plate recognition
I1 = rate of successful interpretation of first character
I2 = rate of successful interpretation of second character
In = rate of successful interpretation of nth character.

But a system’s overall accuracy cannot be extrapolated directly from its individual character accuracy. For example, let us assume a system recognizes and identifies ten thousand license plates with seven characters on each plate for a total of seventy thousand characters. If the system reads the first six characters correctly on each plate but misses the last character on every plate, one might be inclined to state the overall accuracy as (60,000 ÷ 70,000) ´ 100, or 85.7 percent. However, using Equation 2, the true system accuracy in this case is zero.

A = (T ´ I1 ´ I2 ´ I3 ´ I4 ´ I5 ´ I6 ´ I7) ´ 100 (3)
A = (1 ´ 1 ´ 1 ´ 1 ´ 1 ´ 1 ´ 1 ´ 0) ´ 100 (4)
A = 0 percent (5)

Performance attributes must be specified carefully to be meaningful. Simple statements like "eighty-five to ninety percent accurate" can hide important assumptions and often engender unrealistic expectations. Avoiding the pitfalls of operator bias while truthfully demonstrating statistically significant results about correct interpretations requires a large sample size, studied under varying conditions of illumination, speed, camera offset angle, precipitation, etc. Usually time and money are too limited for developers to perform such rigorous testing.
However, some plates cannot be read at all - neither by machine nor by eye - due to dirt, poor lighting, damage, or obscuration. An automated system, therefore, should not be expected to achieve perfection, even under ideal conditions.
One way to measure success is as the percentage of license plates correctly identified by the machine that can be verified by a person looking at the raw video signal on a monitor. If a person must guess from a poor video representation, it is probable the machine also will produce a lower-confidence answer. Looking at clear transmissions, the human is less likely to err, and an automated system similarly will return a higher degree of accuracy.
The supplier should clearly and precisely define the conditions under which the system achieved the stated accuracy rate. A system may correctly identify plates ninety-five percent of the time under controlled conditions, but only fifty percent of the time under less ideal conditions. The supplier also must explicitly state the definition of failure: a failure could signify a missed plate (no recognition) or an error in the interpretation (identification) of one or more characters.
The importance of a failed identification depends on the application. For example, in secure-access control, any system failure would be unacceptable because that could admit someone not authorized for entry (false positive) or deny admission to an authorized person (false negative). Different threshold settings may predispose a system to one or the other type of error. Operational constraints determine which type of error to favor, allowing designers to set thresholds appropriately.
Using the access control example, post processing may compensate for imperfect interpretation. Following the recognition step, each plate’s sequence of characters is submitted for matching against a database of known sets. A single character error would be insignificant if the system were instructed to grant admission under the following conditions:

Given a finite database populated with plate ABV123, designers must consider the likelihood of encountering the true plate ABW123 and the implications of admitting an unauthorized vehicle.

WHAT’S IN STORE

Computer-based plate recognition emerged in the 1980’s. In 1993, LPI/R technology made a successful transition from the research bench to the commercial marketplace. Recently, as supply and demand dynamics took hold, off-the-shelf components started to become available from a greater number of vendors. Now, with eighty-seven suppliers offering commercial LPI/R products, the technology is finding its way into progressively more solutions-oriented intelligent transportation systems (ITS).
As LPI/R enjoys heightened visibility among law enforcement, commercial, and private sector communities, potential users will be exposed to the wealth of AVI (automatic vehicle identification), AVL (automatic vehicle location), ETTM (electronic tolling and traffic management), ITS, and video violation enforcement (VES) applications that can be addressed with the technology. Likewise, new challenges are sure to be found in ever-increasing plate varieties, smaller-sized alphanumeric characters, decorative fonts, and a more prevalent call for reporting not only plates’ letters and numbers but the issuing jurisdiction as well. (Note: not all U.S. States, for example, have the complete State name on the plate. Even when the name is present, it’s usually too small to be captured by the image acquisition subsystem.)
Those observations not withstanding, several companies are rising to the occasion with innovative and creative outcomes.
For plates where foreground/background contrast makes even visual identification difficult, one vendor is employing a dual-camera configuration using visible and infrared regions of the electro-magnetic spectrum. Filtering appropriate wavelengths significantly improves character readability for plates such as the one from the State of Illinois, shown in the below figure.

Leading half-height characters in the U.S. federal government-issued plate (figure below) are unreadable at typical CCTV resolution. Another company integrates ultrahigh definition image capture. At 1300 ´ 1030-pixels, those important alphanumerics can be imaged with sufficient resolution to allow automatic recognition.

An innovative firm’s pattern matching methodology not only reads plate alphanumerics, the application software analyzes the entire image of the plate and queries a database for potential matches. Under that scenario, it may become less vital to report the issuing jurisdiction which3as on the State of Tennessee sample (figure below)3is not always indicated on the plate.

As transportation and law enforcement organizations tap into near realtime access to who (or which vehicle) is where, we all may benefit from heightened safety and violation enforcement, faster transit times, and guidance to less congested routes. Prospects seem positive for continued growth of an international market to help realize that potential.

Contact information:

Lee J. Nelson, Principal Systems Consultant
Electro-Optical Technologies, Inc.
Box 3125, Falls Church, Virginia 22043-0125 USA
Telephone: +1-703-893-0744
E-mail: lnelson@rcn.com