EYE AND VOICE CONTROLLED WHEEL CHAIR

A PROJECT REPORT

Submitted by

CHINNAKOTLA SREEHARSHA 721420243010

KONDETI SAI NIKHITHA 721420243028

SANGAVI S 721420243046

in partial fulfillment for the award of the degree

of

BACHELOR OF TECHNOLOGY

in

ARTIFICIAL INTELLIGENCE AND DATA SCIENCE

NEHRU INSTITUTE OF ENGINEERING AND TECHNOLOGY

(AUTONOMOUS)

COIMBATORE – 641105

ANNA UNIVERSITY : CHENNAI 600 025

MAY - 2024
 ANNA UNIVERSITY: CHENNAI 600 025

BONAFIDE CERTIFICATE

Certified that this project report “ EYE AND VOICE CONTROLLED

WHEEL CHAIR ” is the bonafide work of “CHINNAKOTLA
SREEHARSHA (721420243010) , KONDETI SAI NIKHITHA
(721420243028), SANGAVI S (721420243046) ” who carried out the project
under my supervision.

SIGNATURE SIGNATURE

Dr.V.T.KRISHNAPRASATH Ms. SUJITHRA G

HEAD OF THE DEPARTMENT INTERNAL GUIDE

Associate Professor Assistant Professor

Department of Artificial Intelligence Department of Artificial Intelligence

And Data Science And Data Science

Nehru Institute of Engineering And Nehru Institute of Engineering And

Technology (Autonomous) Technology (Autonomous)

Coimbatore – 641 105 Coimbatore – 641 105

Submitted for the Anna University Project viva-voce examination held on

……………………... at Nehru Institute of Engineering And Technology
(Autonomous).

INTERNAL EXAMINER EXTERNAL EXAMINER

ii
 ACKNOWLEDGEMENT

At this pleasing moment of successful completion of our project, We thank God

Almighty for his blessings.

We convey our sincere thanks to our respected Founder Chairman Late

Shri. P.K. DAS, Chairman and Managing Trustee Adv. Dr. P KRISHNADAS
and CEO & Secretary Dr. P. KRISHNAKUMAR who provided the facilities
for us to complete the project successfully.

We express our deep gratitude to our beloved and the honourable

Principal, Dr. P. MANIIARASAN for his valuable support.

We owe grateful and significant thanks to Dr.V.T KRISHNAPRASATH, our

Head of the Department, Artificial Intelligence and Data Science, and our
sincere thanks to our internal guide Ms. SUJITHRA G, Assistant Professor /
AI&DS, who has given constant support and encouragement throughout the
completion of our course and our project work.

We thank our Project Coordinator Ms. SIKA K, Assistant Professor / AI&DS,

who rendered us all possible help along with her valuable suggestions and
constructive guidance for successful completion of our project.

We are thankful for the support and guidance from all the Faculty of
ARTIFICIAL INTELLIGENCE AND DATA SCIENCE DEPARTMENT
and sincere thanks to our supporting staff for their timely assistance. Finally, We
would like to express our gratitude towards our parents for their motivation and
support which helped us in the completion of this project.

iii
 ABSTRACT

The Eye and Voice controlled wheel chair offers a revolutionary approach
to enhancing electric vehicle maintenance by deploying a network of sensors
within vehicle wheels to collect real-time data on tire pressure, temperature,
tread wear, and alignment, which is transmitted wirelessly to a central control
unit. Employing advanced algorithms incorporating machine learning and
predictive analytics, interprets this data to detect anomalies and predict
maintenance needs, providing proactive alerts and recommendations to prevent
breakdowns and optimize operational efficiency. Accessible via mobile or web
applications, the system offers users comprehensive insights and diagnostic
reports, enabling informed decision-making and efficient maintenance planning.
By promoting proactive maintenance and optimizing performance, the EWHMS
(E-Wheel Health Monitoring System) contributes to safety, sustainability, and
cost reduction in electric vehicle operations, positioning itself as an
indispensable tool for manufacturers, service providers, and end-users.

iv
 TABLE OF CONTENTS

CHAPTER NO. TITLE PAGE NO.

ABSTRACT iv

LIST OF FIGURES viii

LIST OF ABBREVATIONS
ix

I INTRODUCTION 1

1.1 PROBLEM STATEMENT 2

1.2 THE WHEEL MONITORING SYSTEM 2
1.3 OBJECTIVES 3
1.4 BENEFITS AND IMPLICATIONS 4

II LITERATURE SURVEY 6

2.1 INTRODUCTION 6

2.2 EXSISTING METHODS

7

2.2.1 VOICE-CONTROLLED WHEELCHAIR

SYSTEM FOR DISABLED PEOPLE (2015) 7

2.2.2 DEVELOPMENT OF A LOW-COST EYE-

GAZE CONTROLLED WHEELCHAIR (2017)

7

2.2.3 REAL-TIME VOICE CONTROLLED WHEEL

v
 CHAIR SYSTEM USING RASPBERRYPI (2018)
8

2.2.4 INTEGRATION OF DEEP LEARNING WITH

EYE TRACKING FOR IMPROVED CONTROL

OF ASSISTIVE DEVICES (2022) 8

2.2.5 EYE-CONTROLLED WHEELCHAIR (2011)

8

2.2.6 VOICE-CONTROLLED WHEELCHAIR

NAVIGATION WITH AUTOMATIC USER

ADAPTATION (2015) 8

2.2.7 AN INTELLIGENT WHEELCHAIR WITH

INTEGRATED VOICE AND GAZE

CONTROL (2019) 9

III REQUIREMENTS SPECIFICATION

10

3.1 HARDWARE REQUIREMENTS

10

3.1.1 ESP 32 DEV KIT 10

3.1.2 EMBEDDED WEBCAM 11

3.1.3 SERVO MOTORS 13

3.1.4 MOTOR DRIVERS 14

vi
 3.1.5 LED DISPLAY 15

3.2 SOFTWARE REQUIREMENTS 17

3.2.1 CV2 17

3.2.2 MEDIAPIPE 17

3.2.3 TIME 18

3.2.4 UTILS AND MATH

18

3.2.5 NUMPY 19

3.2.6 SERIAL 20

IV PROPOSED MODEL 21

4.1 INTRODCUTION 21

4.2 WORKING 22

4.3 WORK FLOW 24

V EXPERIMENTAL ANALYSIS AND RESULTS 27

5.1 SOURCE CODE

27

5.2 OUTPUT
42

VI CONCLUSION 45

6.1 FUTURE SCOPE 45

APPENDIX I 48

SOURCE CODE 48

vii
 APPENDIX II 63

OUTPUT SNIPPETS
63

REFERENCES
74

LIST OF FIGURES

FIGURE FIGURE NAME PAGE

NO. NO.

3.1 ESP32 Dev Kit 11

3.2 Embedded Webcam 12

3.3 Diagram of Servo Motor 14

3.4 Motor Driver 15

3.5 LED Display 16

4.1 Flow Chart 24

4.2 Circuit Diagram 26

viii
5.1 Front View 42

5.2 Side View 42

5.3 Command in LED Display 42

5.4 Navigating through Eye Ball Motion 44

5.5 Voice Command using User Interface 44

LIST OF ABBREVATIONS

EV – Electric Vehicle

EWHMS – E-Wheel Health Monitoring System

NUMPY - Numerical Python

UI - User Interface

ML - Machine Learning

LED - Light Emitting Diode

ix