Multi Object Detection:acomputer vision approach for efficient object localization and

recognition
 Table of Contents

Page

List of Figures 6

Chapter

I. Introduction..................................................................................................................... 8

Overview................................................................................................................. 8

Motivation............................................................................................................... 8

Research aim........................................................................................................... 9

Problem description and Research Questions......................................................... 9

What is an image in Computer Science? ................................................................ 9

What is Object Detection? .................................................................................... 13

What is Image Recognition?................................................................................. 14

II. Background ................................................................................................................... 17

Overview............................................................................................................... 17

Image Processing .................................................................................................. 17

Artificial intelligence and learning ....................................................................... 18

Machine Learning ................................................................................................. 19

Machine Learning Evaluation............................................................................... 21

Deep Learning....................................................................................................... 22

Neural Network..................................................................................................... 23

Unified Detection Model YOLO ....................................................................... 28

Chapter Page

Pytesseract............................................................................................................. 39

Regular Expressions.............................................................................................. 41

III. Experiment .................................................................................................................... 43

Overview............................................................................................................... 43

Data Preparation.................................................................................................... 43

Training................................................................................................................. 45

Evaluation Metric.................................................................................................. 47

IV. Results ........................................................................................................................... 49

Overview.............................................................................................................. .
49
Detection Accuracy..............................................................................................
.
Font Style and Size ..............................................................................................49

Input Size ..............................................................................................................

52
V. Conclusion....................................................................................................................
.
Overview.............................................................................................................. 54

Application........................................................................................................... .
55
Limitations of Work.............................................................................................
.
Future work..........................................................................................................55

References .................................................................................................................................
55
1. Image Recognition and Object Detection difference [36]. ...............................................

2. Enhancing grayscale images with histogram equalization [37]........................................

3. Structure of Perceptron [35]..............................................................................................

4. Output of a Perceptron [35]. .............................................................................................

5. Activation functions. (1) the left curve is a sigmoid function curve. (2) the right curve is a

tanh function curve [35]....................................................................................................

6. A sample fully connected neural network with only one hidden layer [35].....................

7. Visualization of a Training Process [35]. .........................................................................

8. Max-pooling. Pooling from 24 × 24 to 12 × 12 [35]. .......................................................

9. Structure of a CNN Based Object Detection Model [35]. ................................................

10. YOLO Structure [35]. .......................................................................................................

11. YOLO Network Architecture [42]....................................................................................

12. Bounding Box prediction formula [32].............................................................................

13. Image before pre-processing [43]. ....................................................................................

14. Image after pre-processing [43]. .......................................................................................

15. Word bank used to generate our training image data. ......................................................

16. Sample generated image using our custom-built tagging tool..........................................

17. Sample grid generated by YOLO for our detection algorithm. ........................................

18. Our detection model summary..........................................................................................

19. Graphical View of the IoU equation [41]. ........................................................................

20. The higher the IoU, the better the performance [24]. .......................................................

21. (a) Object detection algorithm after 10 epochs (b) Object detection algorithm for an

image without any link (c) Improved Object detection algorithm after 200 epochs. .......

22. Accuracy between two Optical Character Recognition techniques and our model on ten

im-ages using regular font style........................................................................................

23. Regular Expression used to recognize a URL in a string of text. .....................................

24. Our model experiencing overfitting..................................................................................

25. OCR image conversion to text versus our object detection model...................................

26. Example italicized text......................................................................................................

27. Detection Accuracy between Optical Character Recognition techniques and our model on

ten images using italicized font style. ...............................................................................

28. Detection on different input image sizes. .........................................................................

29. Our models performance on slightly varying font size.....................................................

1.

2.
3.

4.

5.
14
15
Figure 1. Image Recognition and Object Detection difference [36].
Figure 2. Enhancing grayscale images with histogram equalization [37].
Figure 3. Structure of Perceptron [35].
 Figure 4. Output of a Perceptron [35].

Figure 5. Activation functions. (1) the left curve is a sigmoid function curve. (2) the right curve
is a tanh function curve [35].
Figure 6. A sample fully connected neural network with only one hidden layer [35].
Figure 7. Visualization of a Training Process [35].
Figure 8. Max-pooling. Pooling from 24 × 24 to 12 × 12 [35].
Figure 9. Structure of a CNN Based Object Detection Model [35].
1.

2.

3.

4.

5.

6.
 Figure 10. YOLO Structure [35].

Figure 11. YOLO Network Architecture [42].

Figure 12. Bounding Box prediction formula [32].
Figure 13. Image before pre-processing [43].
Figure 14. Image after pre-processing [43].
 Figure 15. Word bank used to generate our training image data.

Figure 16. Sample generated image using our custom-built tagging tool.
Figure 17. Sample grid generated by YOLO for our detection algorithm.
Figure 20. The higher the IoU, the better the performance [24].
Figure 21. (a) Object detection algorithm after 10 epochs (b) Object detection algorithm for an
image without any link (c) Improved Object detection algorithm after 200 epochs.
Figure 22. Accuracy between two Optical Character Recognition techniques and our model on
ten im-ages using regular font style.

Figure 23. Regular Expression used to recognize a URL in a string of text.

Figure 24. Our model experiencing overfitting.
 Figure 26. Example italicized text.

Detection Accuracy on italicized texts

80
70
60
50
40
30
20
10
0

Thesis OCR Pattern 1 OCR Pattern 2 - regEx

ErrorsOverfittingExact

Figure 27. Detection Accuracy between Optical Character Recognition techniques and our
model on ten images using italicized font style.
 59

[1] P. Chakravorty, "What Is a Signal? [Lecture Notes]," in IEEE Signal Processing Magazine,
vol. 35, no. 5, pp. 175-177, Sept. 2018, doi: 10.1109/MSP.2018.2832195.
[2] M. Rouse, [Online]. Available at: https://whatis.techtarget.com/definition/image.
[Accessed: 01/10/20]
[3] Merriam-Webster [Online]. Available at: https://www.merriam-webster.com/dic-
tionary/image, [Assessed: 01/10/20]
[4] Wikipedia, Image [Online]. Available at: https://en.wikipedia.org/wiki/Image, [Assessed:
01/10/20]
[5] The Editors of Encyclopedia Britannica, Image-processing [Online], Available at:
https://www.britannica.com/technology/image-processing. [Assessed: 01/10/20]
[6] Wikipedia, [Online], Available at:
https://www.bbc.co.uk/bitesize/guides/zqyrq6f/revision/3. [Assessed: 02/10/20]
[7] Object (image processing) [Online], Available at: https://en.wikipedia.org/wiki/Object_(im-
age_processing) , [Assessed: 02/10/20]
[8] P. Ganesh, Object Detection: Simplified [Online], Available at: https://towardsdatasci-
ence.com/object-detection-simplified-e07aa3830954, [Assessed: 02/10/20]
[9] Tensorflow, Available at: https://www.tensorflow.org/lite/models/object_detection/overview,
[Assessed: 03/10/20]
[10] Wikipedia, Available at: https://en.wikipedia.org/wiki/Object_detection,
[Assessed: 02/10/20]
[11] Fritz, Detection Available at: https://www.fritz.ai/object-detection/, [As-
sessed: 03/10/20]
[12] Wikipedia, of object Available at: https://en.wikipedia.org/wiki/Out-
line_of_object_recognition , [Assessed: 02/10/20]
[13] Object Recognition, Available at: cse.usf.edu/~r1k/MachineVisionBook/Machine-
Vision.files/MachineVision_Chapter15.pdf, [Assessed: 03/10/20]
[14] N. Pinto, D. D. Cox, and J.J. DiCarlo, Why is Real-World Visual Object Recognition
(2008) PLoS Comput Biol 4(1): e27.
[15] A. Gulli and P. Sujit, Learning with Keras (2017), Available at:
https://1lib.us/book/3411804/7ea47a?id=3411804. [Assessed: 03/10/20]
[16] Definitive Glossary of Higher Mathematical Jargon - Algorithm". Available at:
https://mathvault.ca/math-glossary/ [Assessed: 03/10/20]
[17] of ALGORITHM". Merriam-Webster Online Dictionary. Available at:
https://www.merriam-webster.com/dictionary/algorithm [Assessed: 04/10/20]
[18] Y. Gavrilova, Artificial Intelligence vs. Machine Learning vs. Deep Learning:
Available at https://serokell.io/blog/ai-ml-dl-difference [Assessed: 04/10/20]
[19] J. Brownlee, Gentle Introduction to Object Recognition with Deep (2018)
Available at: https://machinelearningmastery.com/object-recognition-with-deep-learn-
ing/ [Assessed: 01/10/20]
 60

[20] A. Kamal, YOLOv2 and YOLOv3: All You want to Available at:
https://medium.com/@amrokamal_47691/yolo-yolov2-and-yolov3-all-you-want-to-
know-7e3e92dc4899 [Assessed: 04/10/20]
[21] You Only Look Once: Unified, Real-Time Object Detection, 2015. Available at:
https://arxiv.org/abs/1506.02640, [Assessed: 04/10/20]
[22] A. Rosebrock., over Union (IoU) for object (2016) Available at:
https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-de-
tection/, [Assessed: 04/10/20]
[23] I. Tan, Labelling Quality with IOU and F1 Available at: https://me-
dium.com/supahands-techblog/measuring-labelling-quality-with-iou-and-f1-score-
1717e29e492f , [Assessed: 01/10/20]
[24] StackOverflow, Over Union (IoU) ground truth in Available at:
https://stackoverflow.com/questions/61758075/intersection-over-union-iou-ground-truth-
in-yolo, [Assessed: 02/10/20]
[25] J. Redmon & A. Farhadi, (University of Washington), YOLO9000: Better, Faster,
Available at: https://pjreddie.com/media/files/papers/YOLO9000.pdf, [As-
sessed: 02/10/20]
[26] v2 Object Available at: https://www.geeksforgeeks.org/yolo-v2-ob-
ject-detection/ [Assessed: 04/10/20]
[27] A. Aggarwal, Explained Available at: https://medium.com/analytics-vidhya/yolo-
explained-5b6f4564f31 [Assessed: 04/10/20]
[28] K. Mahesh Babu, M.V. Raghunadh, Vehicle number plate detection and recognition using
bounding box method, May 2016, pp 106 110
[29] L. Cai, F. Jiang, W. Zhou, and K. Li, Design and Application of An Attractiveness Index for
Urban Hotspots Based on GPS Trajectory Data, (Fellow, IEEE), pg 4
[30] Wikipedia, bounding Available at: https://en.wikipedia.org/wiki/Mini-
mum_bounding_box, [Assessed: 04/10/20]
[31] Dive into Deep Learning, Object Detection and Bounding Available at:
https://d2l.ai/chapter_computer-vision/bounding-box.html , [Assessed: 04/10/20]
[32] J. Redmon & A. Farhadi, YOLOv3: An Incremental Improvement, University of Washing-
ton Available at: https://arxiv.org/abs/1804.02767 [Assessed: 05/10/20]
[33] YOLO: You Only Look Once, Available at: jeremyjordan.me/object-detection-one-
stage/#yolo, [Assessed: 05/10/20]
[34] Longman Dictionary, Definition of training, Available at: https://www.ldoceonline.com/dic-
tionary/training [Assessed: 05/10/20]
[35] Guangrui Liu -Time Object Detection for Autonomous Driving Based on Deep Learn-
(2017), Available at: https://tamucc-ir.tdl.org/handle/1969.6/5637 [Assessed:
03/18/21]
[36] A. Abdulkader & C. Vlahija -time vehicle and pedestrian detection, a data-driven rec-
ommendation focusing on safety as a perception to autonomous Available at:
http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1479957&dswid=-3676 [As-
sessed: 03/18/21]
 61

[37] Enhancement methods in image processing Available at: https://www.mathworks.com/dis-

covery/image-enhancement.html. [Assessed: 03/18/21]
[38] Evaluating a machine learning model. Available at: https://www.jeremyjordan.me/evaluat-
ing-a-machine-learning-model/. [Assessed: 03/18/21]
[39] A Comprehensive Guide to Convolutional Neural Networks the ELI5 way. Available at
https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-net-
works-the-eli5-way-3bd2b1164a53. [Assessed: 03/18/21]
[40] J. Jokela Counter Using Real-Time Object Detection and a Small Neural
Turku University of Applied Sciences. Available at: https://www.theseus.fi/bit-
stream/handle/10024/153489/Jokela_Jussi.pdf?sequence=1&isAllowed=y. [Assessed:
03/18/21]
[41] Manishgupta You Only Look Available at: https://towardsdatasci-
ence.com/yolo-you-only-look-once-3dbdbb608ec4. [Assessed: 03/18/21]
[42] E. Y. Li Really Deep into YOLO v3: A Available at: https://to-
wardsdatascience.com/dive-really-deep-into-yolo-v3-a-beginners-guide-9e3d2666280e.
[Assessed: 03/18/21]
[43] F. Zelic & A. Sable A comprehensive guide to OCR with Tesseract, OpenCV and Py-
thon. Available at: https://nanonets.com/blog/ocr-with-tesseract/. [Assessed: 03/18/21]
[44] Tessdoc. Available at: https://github.com/tesseract-ocr/tessdoc/blob/master/ImproveQual-
ity.md. [Assessed: 06/04/21]
[45] J. Goyvaerts Expressions: The Complete Available at: https://www.reg-
ular-expressions.info/print.html. [Assessed: 06/04/21]
[46] M. Erwig & R. Gopinath, for Regular Available at:
https://web.engr.oregonstate.edu/~erwig/papers/ExplRegExp_FASE12.pdf. [Assessed:
06/04/21]