AUTOMATIC NUMBER PLATE RECOGNITION SYSTEM

USING CONVOLUTIONAL NEURAL NETWORKS

K.P. KRISHNA SAGAR, ASSISTANT PROFESSOR, [email protected]
E. GNANESWARI, ASSISTANT PROFESSOR, [email protected]
C. MURALI MOHAN, ASSISTANT PROFESSOR [email protected]
Department of ECE, Sri Venkateswara Institute of Technology, N.H 44, Hampapuram,
Rapthadu, Anantapuramu, Andhra Pradesh 515722

of information is not as relevant as manually

Abstract: Because of its various real-world
identifying the vehicle and its owner. There are a
uses, Automatic Licence Plate Recognition
number of approaches to building an ANPR
(ALPR) has been the subject of much study.
architecture, and automatic number plate
Unfortunately, a lot of the existing solutions
recognition is a crucial component of an intelligent
still rely on a lot of limits and aren't strong
traffic network [1]. Although there is a strong
enough to handle real-world scenarios. This
correlation between the shooting distance and the
research introduces a cutting-edge ALPR
fraction of the licence plate in the picture, the
system that makes use of the YOLO object
ANPR architecture [7] is not simple to balance in.
detector, which is both efficient and reliable.
On the other hand, the vast open area and clearly
Each ALPR stage trains and fine-tunes the
discernible licence plate photos make it impossible
Convolutional Neural Networks (CNNs) to
to catch moving vehicles. Determination is
make them resilient under varied situations,
required to discern a licence plate in images that
such as changes in camera, lighting, and
are both small and distorted. Using a CCD camera
backdrop. We provide a two-pronged
that can pan, tilt, and zoom (PTZ) is one solution
strategy that uses basic data augmentation
[2]. Each phase has its own set of advantages and
techniques, such inverted Licence Plates
disadvantages, and different scholars have
(LPs) and flipped characters, for character
suggested different approaches. There are
segmentation and recognition in particular.
primarily three stages to the process of licence

Keywords: CNN, character recognition, plate recognition. That area is where the extraction
is going to take place.
1. Introduction:

Licence plate recognition provides a identifying licence plates and personalities.

wealth of information useful for finding
and identifying records. However, the Device programme ANPR allows for licence plate
standard procedure for licence plate recognition. Vehicle licence plates may be seen in
identification is tedious. Detecting camera-captured images. Reading and unfolding
licence plates to recover buried riches licence plates is the main function of ANPR. The
 acronym ALPR stands for "Automatic detection and identification. From the literature
Licence Plate Recognition" and study, we may see many contemporary strategies in
describes ANPR as well. This system's action.
software keeps track of licence plates
and how they correlate with other data, By adhering to a learning methodology, K.K. Kim
including time, date, and GPS et al. (2000) constructed a system that could
coordinates. It reads licence plates recognise licence plates. Pictured above is the
using optical character recognition automobile detecting module's interior as seen via
technology. An important part of the camera. As a result, you will get an image of the
automated parking is LPR. Industry potential area. In order to locate the licence plate,
demand for higher-profile commercial the two TDNNs were used as horizontal and vertical
parking management projects in smart filters, respectively. With a recognition rate of and a
city zones is likely to drive the segmentation rate of 97.5%,
inclusion of this functionality. Among
its numerous potential applications, it 97.2% for the suggested setup [1].
has used as a security measure in toll
collecting systems, traffic management,
petrol stations and more [8]. Because of
its many uses, including highway In 2007, Chin-Chuan Han et al. proposed a system
surveillance, urban logistics, traffic law that can detect several targets and capture high-
enforcement, and many more, ITS is a quality photographs of licence plate numbers. In
key component in making smart cities a order to keep an eye on moving conveyance in an
reality [6]. open field, the author has constructed a computer
with a tuned dual-camera system. This system
consists of a fixed camera and a pan-tilt-zoom
camera. In order to recognise the licence plate, a
2. Literature Survey:
CNN classifier has progressively recognised each
In this part, we will go over certain one. Data was painstakingly constructed from the
protocols that have already been defined. scientific photographs, and this method properly
A lot of effort has been put in over the discovered 59 IDs, as 64 cars entered this zone
unlawfully [2].
on deep learning and image processing
techniques for object recognition in the
last several years. This area has seen the
development of a number of distinct An ANPR architecture that emphasises
algorithms for vehicle reconnaissance convolutional neural networks' learning ability was
created by Madhusree Mondal et al. Having an average speed of 0.185s identifying 454
(2017). Because it can differentiate plates with 90.8% accuracy, detection was
between the vehicle's states based on the somewhat low but had an excellent success rate.
licence plate, the self-synthesized feature Using optical character recognition, the process
of CNN was used in this case. In this took an average of 0.031 seconds and almost 60%
study, the system was structured as an of the detected plates were correctly recognised [4].
echelon network of feature detectors that
sequentially processed visual input In their 2018 proposal for an ALPR system,
related to the visual cortex's dominant Rayson-Laroca et al. examined the efficacy and
processing experience; this data reliability of a framework built on the cutting-edge
impacted the CNN's computational YOLO artefact detector. The CNN are trained and
model. Although there were fewer adjusted for each ALPR stage so they can withstand
training samples used in this study, the various situations. Using simple data augmentation
results showed a 90% improvement in techniques like inverted licence plates and character
accuracy [3]. returns, the author of this study devised a two-stage
approach to explicitly segmenting and identifying
Drawing on the features and variances of characters. Both commercial systems fell short of
the plates therein, Andrew S. Agbemenu the 70% recognition threshold, making it impossible
et. al. (2018) put up an ANPR approach. to draw any firm conclusions from the UFPR-ALPR
In this paper, the author suggests an dataset. A greater recognition rate of 78.33% was
improved method for use with Ghanaian achieved using the suggested technique, but [5].
licence plates in transportation. Two
candidate identification algorithms were In order to assist ITS in recognising licence plates
used in the model's design: one for edge from Bangladesh, Prashengit Dhar et al. (2018)
detection and another for algorithm created an automated LPR programme. The white
matching the template. In order to backdrop and black lettering on this work plate are
eliminate noise, skew, and character quite visible. The detection was carried out by
arrangement issues, the device used the Prewitt operators.
character segmentation approach, which
is especially effective when working
with square plates. The tesseract OCR
engine was used to produce character in order to divide the margins of the licence plate.
recognition at the final point. Feature To make the points stand out, morphological
dilatation was used. The reconnaissance mission
was eventually completed using deep CNN. A
remarkable 99.6% accuracy rate was shown by the
approach in character categorization [6]. of the cases. The YOLOv3 model, which has 53
convolutional layers, was used for the detection
Using Mask-RCNN for multiple oblique investigation. Following identification, the next step
pictures and different shooting angles, is to segment the picture and identify the characters
Cheng-Hung Lin et al. (2019) developed it contains. At this stage, the gadget extracts the
a three-stage method for licence plate licence plate area and passes it on to the second
identification. In the previous step for YOLO model for platform picture identification and
vehicle detection, the author used segmentation. Thus, the model was tested with 200
YOLOv2 for the related conveyance. photos and managed to accurately identify the
Once we found the licence plate, the number plate number in 199 of them, yielding an
next step was to run YOLOv2 once accuracy rating of 98.5% [8].
again to identify the plate. At this stage,
YOLOv2 divides the 19 × 19 grids that
3. Methodology:
included the pictures of the cars that
were recorded in phase I. Finally, It is our honest opinion that many ITS uses rely on
character identification was actual ANPR systems in the real world.
accomplished using Mask R-CNN. In Variations in weather and illumination, high vehicle
addition to achieving a mAP grade of speeds, and poor plate clarity all reduce the
about 91%, the findings show that the effectiveness of automatic number plate recognition
suggested model could categorise car systems.
number plates with bevel angles greater As an example, when it comes to filthy plates, the
than 0-60 degrees [7]. majority of popular algorithms for plate
identification and recognition don't work.
Using convolutional neural networks,
NazmusSaif et al. (2019) presented a With the right hardware platforms and real-time,
method to identify Bangla licence plates resilient, and novel algorithms, this article lays out a
in car images. The key choice for the system that solves these problems.
proposed system in this study is a
convolution neural network due of its When used in real-world web applications, the
system can handle a wide range of scenarios,
including but not limited to: poor plate clarity, high
setting up the whole process. In their vehicle speeds, variable ambient conditions,
instance, CNN did far better than languages, and formats.
traditional image processing algorithms, We are expanding the work utilising hog and CNN
and when comparing generalised CNN to achieve high accuracy and detection for number
models, CNN models fared better in all plate and feature extraxtion. High detection and
identification accuracies on dirty plates identification, and licence plate detection. These
are the major advantages of our components are shown in Figure 1. We use SSD, a
technology. well-known deep learning-based object
Two novel data sets were developed and identification method, for object detection in this
used in this study to accomplish credible work [8]. Results for object identification tests
assessments: the "Highway Data set" for demonstrate that the SSD model outperforms
vehicle counts on highways and the alternatives with respect to both speed and accuracy
"Crossroad Data set" for violation [11]. The SSD model is able to recognise items of
detection. Our system achieves 98.7% varying sizes since it examines objects in feature
accuracy for plate detection, 99.2% maps from different layers. We carry out the region
accuracy for character segmentation, and and character identification of licence plates using
97.6% accuracy for plate identification the SSD object detector, as will be detailed later on.
on the Crossroad Data set. The detection I. Locating Licence Plates Locating the arriving
rate for vehicle counting applications vehicle's licence plate inside the recorded picture is
across the Highway data set is 99.1%, the first step in the licence plate recognition job. In
while the false alarm rate is 0.5%. Using a licence recognition job, the area around the
a publicly accessible English plate data licence plate is the most important part. For the sake
collection, we also evaluated our method of our work, we will disregard the rest of the
and managed to reach an overall picture. We use SSD model [8] for licence plate
accuracy of 97%. Many previously detection operation. The
reported ANPR systems are compared to
the proposed system from various A training dataset that is annotated with licence
angles.Practical considerations led to the places is used to develop the model.
installation of several ANPR systems at
various routes and crossings throughout

Ahvaz, the Iranian capital. After a year

of nonstop testing in a variety of weather Character Detection

situations (rain, snow, dust, etc.), these

On the detected license plate region, license plate
systems have shown to be both sturdy
characters are localized with an object detector. In
and dependable.
this stage, we compare SSD object detection
Here we outline the specifics of the
method with DPM [12], which is an effective model
licence plate recognition system that has
for character detection on license plates as shown in
been suggested. There are three
[3]. The trained models can detect 33 different
components to the suggested solution:
characters; 23 English letters (excluding „Q‟,
character detection, licence plate
„W‟ and „X‟) and 10 numbers from „0‟ overlapping ratio is greater than 70 %, the
to „9‟. The details of these methods are one with the highest detection score is
as follows: SSD Model: In this used as the detected character. The final
approach, SSD object detector [8] is rule is, the first two and the last two
utilized to detect the characters within character of the plate should be number.
the input image. For this operation a Thus, any letter detected on this range is
character detection SSD model is trained ignored.
using a character regions annotated
training dataset. DPM Model: We Proposed Block Diagram:
utilized a deformable part based
character detection model in which each
part is a node on the tree (we used 3
nodes in the tree) and mixture model
captures the structure of the 33 different
characters.

License Plate Recognition

After the character detection task, license

plate is recognized. For an accurate
recognition, some rules are obtained. The
first rule is, the detected character is
ordered with respect to their center pixel
points. The second rule is, if the detected
character region is overlapped with
another detected character region and this
Fig 1 shows the block diagram for the
proposed model

Step 1-Prepare Dataset For our project a

prepared dataset of numbers and letters to
recognize characters on vehicle number
plate, which in spite of any environmental
conditions and at any capturing angle is
stored. Two folders are created,
 A. Training and models in license plate detection and
B. Validation folder. character detection stage and for DPM
A.Training Folder- In training set, model in the character detection stage.
total 36 classes and each class contains SSD Model: During the training process,
800 images for training. Training folder we utilized transfer learning approach to
data is on which the neural network works. make the training process more efficient.
B.Validation Folder- Similarly, In We utilize a base SSD model presented.
validation set total 36 classes and each Using this base model, we fine-tuned it
class containing 200 images for validation. with our specific datasets. Fine tuning
The neural network while learning through operation is performed by freezing the
the training set also checks the loss from weights of the first three convolutional
the validation set. blocks of the model. The rationale behind
Step 2- Build Sequential Model Build the this strategy is based on two facts. First
model using deep neural network library in three convolutional blocks trained with a
matlab as backend. We are using large dataset behave as a feature extractor.
sequential object to model our neural Thus, there is no need to update these
network. weights with our relatively small dataset.
Step 3-Compile and train the model We Secondly, since the first feature map to be
use categorical cross entropy as loss analyzed to detect objects fall into fourth
function, because we utilize multi- convolutional block, it is logical to update
classification in our project. Adam weights starting from there. In our fine
optimizer is used to reduce the loss tuning operations, we set the batch size as
function. After training for 10 epochs, our 16. As learning hyper parameters, Adam
model achieved an accuracy of 98.57 %. optimizer with a relatively small learning
In this study, either a single channel NIR rate 0.0003 is utilized. Also we applied
or a three channel RGB image are utilized learning rate decay strategy shown in Eq. 1
in the decision making process. Instead of where ⋋ is the learning rate, i is the epoch
creating different models for two types of number.
image source, we convert single channel ⋋𝑖+1= ⋋𝑖∗ 0.9 ------------------------ (1)
NIR images to 3 channel NIR images by DPM Model: Proposed tree model T = (V,
cloning them channel-wise and generate a E) is a pictorial structure where V is the set
single model using an NIR or an RGB of parts, and E is the set of edges between
image. Below, we outline procedures and parts. [14] defined a score for a particular
hyper parameter selections for SSD
configuration of parts L = {𝑙𝑖}, for a given
image I as shown in Eq. (2), where 𝜑 is
4. Results:

the histogram of gradients features (for the

landmark points) extracted at pixel

location 𝑙𝑖 = (xi, yi). First term sums the

appearance evidence for placing the ith
template, 𝑤𝑖 at location 𝑙 . Second term
score the spatial arrangement of the set of Fig 2 Shows the input image
parts 𝐿, where dx (dy) term represents the
spatial deformation in x (y) axis between
parts i and j. This model can be viewed as
a linear classifier [15] with unknown
parameters w and deformation parameters
{a, b, c, d} learned during training using
latent classification.illustrates the DPM
models obtained for several characters. For
Fig 3 Feature Selection Model For the
a given test image I, we maximize Eq. (3)
image
using dynamic programing to find the best
configuration of parts.

𝑆∗ (𝐼) = 𝑚𝑎𝑥𝑚 [𝑚𝑎𝑥𝐿(𝐼, 𝐿)]

--------------- (3)
 Fig 4 Recognized plate There is need for improvement in the numbering
system, number plate type (background), moving
distance photographs, tilted or side view photos, and
image processing techniques when it comes to number
plate recognition using neural networks. Neural
networks and object detection are great at spotting

Fig 5: Extracted Output distant, skewed, or side views, as well as moving

photos. In order to achieve excellent accuracies and
enhanced accuracy for future recognition, high-
resolution cameras with an increased number of frames
are being considered for use in possible recognition
systems.

6. References:

1. Kim, K.K., Kim, K.I., Kim, J.B. and

Kim, H.J., 2000, December. Learning-
based approach for license plate
recognition. In Neural Networks for Signal
Processing X. Proceedings of the 2000
Fig 6: performance analysis IEEE Signal Processing Society Workshop
(Cat. No. 00TH8501) (Vol. 2, pp. 614-
5. Conclusion:
623). IEEE.

Research on the use of licence plate recognition 2. Han, C.C., Hsieh, C.T., Chen, Y.N., Ho,
for traffic monitoring is detailed in this article. G.F., Fan, K.C. and Tsai, C.L., 2007,
For effective traffic surveillance, ANPR is a October. License plate detection and
great and trustworthy tool. Easy detection of recognition using a dual-camera module in
interested automobiles from all angles and a large space. In 2007 41st Annual IEEE
access to owner information are outputs of a International Carnahan Conference on
device with a sophisticated image processing Security Technology (pp. 307-312). IEEE.
technology. When it comes to expanding the
smart transport network, ANPR systems are
crucial.
3. Mondal, M., Mondal, P., Saha, N. and Advanced Mechatronic Systems
Chattopadhyay, P., 2017, December. (ICAMechS) (pp. 229-234). IEEE.
Automatic number plate recognition using
8. Saif, N., Ahmmed, N., Pasha, S.,
CNN based self-synthesized feature
Shahrin, M.S.K., Hasan, M.M., Islam, S.
learning. In 2017 IEEE Calcutta
and Jameel, A.S.M.M., 2019, October.
Conference (CALCON) (pp. 378-381).
Automatic License Plate Recognition
IEEE.
System for Bangla License Plates using
4. Agbemenu, A.S., Yankey, J. and Addo, Convolutional Neural Network. In
E.O., 2018. An automatic number plate TENCON 2019-2019 IEEE Region 10
recognition system using opencv and Conference (TENCON) (pp. 925-930).
tesseract ocr engine. International Journal IEEE.
of Computer Applications, 180, pp.1-5.

5. Laroca, R., Severo, E., Zanlorensi, L.A.,

Oliveira, L.S., Gonçalves, G.R., Schwartz,
W.R. and Menotti, D., 2018, July. A
robust real-time automatic license plate
recognition based on the YOLO detector.
In 2018 International Joint Conference on
Neural Networks (IJCNN) (pp. 1-10).
IEEE.

6. Dhar, P., Guha, S., Biswas, T. and

Abedin, M.Z., 2018, February. A system
design for license plate recognition by
using edge detection and convolution
neural network. In 2018 International
Conference on Computer, Communication,
Chemical, Material and Electronic
Engineering (IC4ME2) (pp. 1-4). IEEE.

7. Lin, C.H. and Li, Y., 2019, August. A

License Plate Recognition System for
Severe Tilt Angles Using Mask RCNN. In
2019 International Conference on