A Mini Project Report
on
VOICE CONTROLLED ROBOTIC VEHICLE
Submitted in partial fulfillment of the
Requirements for the award of degree of
Bachelor of Technology
in
Computer Science and Engineering
by
P. Sai Ragasri,
(18H61A0595)
P.V.N.S. Sasank Teja,
(18H61A0599)
G. Jayasree,
(19H65A0507)
Under the Guidance of
Dr. G. Balakrishna,
Assistant Professor
Department of Computer Science and Engineering
ANURAG GROUP OF INSTITUTIONS
(Formerly CVSR College of Engineering)
(An Autonomous Institution, Approved by AICTE and NBA Accredited)
Venkatapur (V), Ghatkesar (M), Medchal(D)., T.S-500088
(2018-2022)
CERTIFICATE
This is to certify that the project entitled “Voice Controlled Robotic Vehicle”
being submitted by P. Sai Ragasri bearing the Hall Ticket number 18H61A0595,
P.V.N.S. Sasank Teja bearing the Hall Ticket number 18H61A0599 and G. Jayasree
bearing the Hall Ticket number 19H65A0507 in partial fulfillment of the requirements
for the award of the degree of the Bachelor of Technology in Computer Science and
Engineering to Anurag Group of Institutions (Formerly CVSR College of
Engineering) is a record of bonafide work carried out by them under my guidance and
supervision from September 2021 to December 2021.
The results presented in this project have been verified and found to be
satisfactory. The results embodied in this project report have not been submitted to any
other University for the award of any other degree or diploma.
Internal Guide External Examiner
Dr. G. Balakrishna,
(Assistant Professor)
Dr.G.Vishnu Murthy,
Professor & Head, Dept. of CSE
ACKNOWLEDGEMENT
It is our privilege and pleasure to express profound sense of respect, gratitude and
indebtedness to our guide Dr. G. Balakrishna, Assistant Professor, Dept. of Computer
Science and Engineering, Anurag Group of Institutions (Formerly CVSR College of
Engineering), for his indefatigable inspiration, guidance, cogent discussion, constructive
criticisms and encouragement throughout this dissertation work.
We express our sincere gratitude to Dr. G.Vishnu Murthy, Professor & Head,
Department of Computer Science and Engineering, Anurag Group of Institutions
(Formerly CVSR College of Engineering), for his suggestions, motivations and co-
operation for the successful completion of the work.
We extend our sincere thanks to Dr.V.Vijaya Kumar, Dean, Research and
Development, Anurag Group of Institutions, for his encouragement and constant help.
P. Sai Ragasri (18H61A0595)
P.V.N.S. Sasank Teja (18H61A0599)
G. Jayasree (19H65A0507)
DECLARATION
We hereby declare that the project work entitled “ Voice Controlled Robotic
Vehicle” submitted to the Anurag Group of Institutions(Formerly CVSR College of
Engineering) in partial fulfillment of the requirements for the award of the degree of
Bachelor of Technology (B.Tech) in Computer Science and Engineering is a record of
an original work done by us under the guidance of Dr. G. Balakrishna, Assistant
Professor and this project work have not been submitted to any other university for the
award of any other degree or diploma.
P. Sai Ragasri(18H61A0595)
P.V.N.S. Sasank Teja (18H61A0599)
G. Jayasree(19H65A0507)
Date: / /2021
ABSTARCT
Our proposed project aims at a robotic vehicle operated by human speech
commands. The system operates with the use of an android device which transmits voice
commands to a microcontroller to achieve this functionality. The transmitter consists of
the android phone Wi-Fi device. The voice commands recognized by the module are
transmitted by through the Wi-Fi transmitter. These commands are detected by the robotic
vehicle in order to move it in left, right, backwards and front directions. The Wi-Receiver
mounted on top of the vehicle is used to recognize the transmitted commands and decode
them. After decoding these commands are passed on to the microcontroller. The
microcontroller then drives the vehicle motors to move it accordingly. This is done with
the use of a driver IC used to control the motor movements. The Wi-Fi technology used
to transmit and receive data allows for remotely operating the system within a good range.
CONTENTS
S.NO PAGE NO.
1. Introduction 1
1.1. Motivation 1
1.2. Problem Definition 1
1.3. Objective of the Project 2
2. Literature Survey 3
3. Analysis 6
3.1. Existing System 6
3.2. Proposed System 6
3.3. Software Requirement Specification 6
3.3.1. Purpose 7
3.3.2. Scope 7
3.3.3. Overall Description 7
4. Design 8
4.1. UML diagrams 9
5. Implementation 14
5.1. Modules 14
5.2. Module description 13
5.3. Introduction of technologies used 17
5.4. Sample code 18
6. Test cases 21
7. Screenshots 22
8. Conclusion 26
9. Future Enhancement 27
Bibliography 28
1. INTRODUCTION
Speech signals are the most important means of communication in human beings.
Almost every conversation to interact is done by means of voice signals. Sounds and
various speech signals can be converted into electrical form using a microphone. Voice
recognition is a technology which is used to convert the speech signals into a computer
text format. This voice recognition technology can be used to control and generate speech
acknowledgement using some external server. Robot voice has the ability to understand
thousands of voice commands and perform required action. The voice recognition is a bit
difficult task because each person has his own accent. For that, Robot voice uses Bit
Voicer Server which supports 17 languages from 26 countries and regions. These robotic
assistants can be used for shaping, manufacturing and tooling purposes in various sectors
such as manufacturing, defense etc. In hospitals, these robotic assistants can be used for
the purpose of performing surgeries and operations with high precision. In this paper, we
develop an assistant robot that can be operated using speech commands
1.1. Motivation
The motivation behind this undertaking is to plan a robot for the exchange of
information from one end to other in firms to make the data development less demanding
by only a transmitting voice charge from smart phone, with the assistance of an android
application on through Wi-Fi arrangement.
1.2. Problem Definition
The surprising raise in the utilizing of robots and automation offers various
advantages as well as it has drawn the attention of both academic investigation and
commercial programs. The analysis on numerous techniques of controlling robot has
accomplished quite a few successes by introducing a number of innovative & unique
methods of robot movement control. Verbal interaction intended for robot controlling is
actually sort of an innovative process among many methods which are introduced
regarding robotics control.
1
1.3. Objective of the project
The main objective of this project is to develop a simplified robot that works
based on human voice commands. There are various previous robots based on this
principle but their implementations are harder compared to our prototype. Our main
objective is to develop our robot in such a way that it also comes to use in military
purposes, transportation purposes, object detection, bomb detection and in numerous
fields.
2. LITERATURE SURVEY
2
Many researchers have worked in the development of robots and have come up
with new approaches. These researches serve as the basis for the new innovations. An
attempt has been made to list out some papers mentioning the technology used and
advantages of each.
“Robot Control Design Using Android Smartphone” by Mrumal.K.Pathak, Javed Khan,
Aarushi Koul, Reshma Kalane, Raunak Varshney This paper explains the controlling of
robot through phone adopting Bluetooth technology and some of its features, mobile
component and robot. The solutions derived are very much comfortable and they act as
platform for robot building. The cost is very less, computation speed and sensing of the
control device, smartphone is good. The paper aims to provide simple robot hardware
architecture with sophisticated android domain which is powerful enough[1].
“Smart Phone Controlled Robot Using ATMEGA328 Microcontroller “by Aniket R.
Yeole, Sapana M. Bramhankar, Monali D. Wani, Mukesh P. Mahajan. The paper is about
robot designing making use of the application of android phone. In this paper, the
direction of the robot and what is the distance of the robot from the obstacle will be
communicated. This information shared through phone through Bluetooth. The control
commands are sent incorporating features like motor speed control and sensing the
data[2].
“Android Mobile Phone Controlled Bluetooth Robot Using 8051 Microcontroller” by
Ritika Pahuja, Naren A robot can be defined as electro-mechanical machine governed by
programming computer and electronic hardware. Robots are been used in factories all
over the world in the Manufacturing units. In this paper, motion of the robot is controlled
by the buttons of the android application. Bluetooth module is used for interfacing of
smartphone and the controller. This is done through the help of UART protocol[3].
“Robot Controlled Car Using Wi-Fi Module” by S R Madkar, Vipul Mehta, Nitin
Bhuwania, Maitri Parida. In this paper, robot-controlled car designed in android
application making use of Wi-Fi in the smartphone. The devices can be controlled even
though android phone is not physically present through the SMS. In this project, inclusion
of spy camera using which live videos can be streamed to the user making use of Wi-Fi.
Instead of using normal lithium-ion battery, this project makes use of solar cells leading
to energy efficiency[4].
3
“Lab VIEW Model of Voice Commands for Mobile Robot Motion Control using Internet
of Thinks Module” by Snezhana Georgieva Pleshkova, Aleksander Bogdanov Bekyarski
and Zahari Todorov Zahariev. The interaction of the people with the robots are been
satisfied through many mobile robot methodologies done by many of the researchers. In
all the methodologies, commonly the voice commands are the interaction domain. This
is required for operating the motion of the mobile robot. This is accomplished in this
article by proposing a creation of LabVIEW model for the reception and recognition of
voice commands which is transmitted from the person to the mobile robot. The
experimentation and testing of LabVIEW model are carried out in the Internet of Thinks
which is sophistically adopted on the mobile robot domain[5].
“Implementation and Testing of Voice Control in a Mobile Robot for Navigation” by
Sudeep Sharan, Trung Quoc Nguyen, Peter Nauth1 and RuiAra´ujo This paper is about
integration of a voice control software system done for the robot mobile system named
as ROSWITHA.The integration is with the SLAM algorithm of Robotic Operating
Systems The robot is also been controlled in numerous ways through the development of
Graphical User Interface (GUI). There are many tabs for controlling like to start the
navigation on the map which may be created or available, for the detection of voice, in
order to enter the location to which we are intended to go and for the immediate stopping
of the system in emergency. In real time and in numerous environments, and with
differentiation in the speaker, the experimentation and testing are carried out. This is done
to improve the accuracy of the designed system[6].
“A Voice Command Detection system for controlling Movement of SCOUT Robot” by
S. Azargoshasb, A. H. Korayem, Sh. Tabibian. The robot termed as SCOUT robot is
controlled through the detection system using voice commands. Hidden Markov model
is used for the implementation of the detection system for the transmission of voice
commands. A database for speech is created incorporating needed commands. The
database is non-native and it has different commands like ready, left, right, stop, forward,
backward, fast, slow and go. Twenty Persian language speakers have uttered each of the
command. Among them ten are male and ten are female. On the basis of HMM, the
detection of voice commands done. The Bluetooth is used for the transmission of detected
command to the robot. The evaluation results of the designed system show the
performance improvement in the interaction of human and robot[7].
4
“Design and Implementation of a Voice Controlled Robot with Human Interaction
Ability” by Humayun Rashid, Iftekhar Uddin Ahmed, Sayed Bin Osman, Qader Newaz,
Md. Rasheduzzaman and S M Taslim Reza. The system of design involves
microcontroller connected to the smart phone through Bluetooth module for the
transmission and reception of the commands in voice mode. The movement of the robot
can be controlled through the transmission of the voice command which is converted to
text using the app of the smartphone and for transmission of data to the microcontroller.
The data reception is followed by the responses of the robot which are given
corresponding to the voice commands. Voice commands are used to make specified
movement in the specified direction. The talking system of the robot is developed with
the SD card consisting of pre-recoded human voice. This audio file is then made use in
talking system of the robot. The robot will speak the sentences, will operate based on the
instruction after receiving each of the commands from the device[8].
“Voice Control Robot Using Android Application”, by Soniya Zope, Preeti Muluk,
Rupali Mohite, Aishwarya Lanke, Megha Bamankar. Controlling the robotic device or
vehicle through the voice commands and remote operation control in the manual method
is designed in the project. The microcontroller ATMEGA32 and Bluetooth device is used
incorporating interfacing of control unit and sense the signals which will be sent by the
android app. The Android app sends the serial data received from the Bluetooth module
which is interfaced to the ATMEGA32. The paper deliberates the controlling of robot car
making use of Wi- Fi module and android application of the smartphone[9].
3.ANALYSIS
3.1 Existing System
5
Previous works on voice-controlled robots shows that the design of those robot
was complicated and none of them were able to interact with users. Robots are anticipated
to socialize along with its user however it has not yet arrived at this kind of level. There
are numbers of techniques to control robot using voice identification yet it is reasonably
limited. The development of a voice-controlled robot is demonstrated in this paper which
has the ability to follow voice command from user and does communicate with user by
using pre-recorded human voice sound. Previously developed robot used ZigBee which
is a costly device. Another Voice Controlled Robotic Vehicle utilized computer with a
sound card and a microphone which was not user friendly. A technique to give voice
command using android based smart phone using Bluetooth is presented to construct the
robot based on microcontroller.
3.2 Proposed System
In this proposed system a human voice is used to control robot. In this system an
android application is used to recognize and process human voice which is further
converted into text (making use of google speech to text converter). This text is
transferred to the robot using Wi-Fi. This text is further processed by the microcontroller
to control the robot accordingly. The purpose of this research is to provide simpler robot
hardware architecture but with powerful computational platforms so that robot’s designer
can focus on their research and tests instead of Wi-Fi connection infrastructure.
3.3. Software Requirement Specification
• Arduino IDE
• Embedded C/C++
3.3.1. Purpose
The main purpose of this project is to develop a simplified robot that works
based on human voice commands. There are various previous robots based on this
principle but their implementations are harder compared to our prototype. Our project
6
can also be used in military purposes, transportation purposes, object detection, bomb
detection and in numerous fields.
3.3.2. Scope
The Future Implications are very vast of this type of technology. This technology
can also be used with the object detection and other technologies. It can also be made
more advanced such that it can be used even in transportation. Voice controlled robotic
system is very beneficial in areas where there is high risk for humans to enter. It can
easily recognize the voice commands and runs smoothly. Further enhancement in project
can be used for Home security and military purposes where the commands can be given
to robot without risk by increasing the range and by installing cameras.
3.3.3. Overall Description
The robot will be based on microcontroller Arduino Uno because of its versatile features
along with numerous advantages which is based on Atmega328P and an open-source
platform with the benefit of physical computing. The system will utilize Wi-Fi
technology and Standard communication interface safe, less power consuming device to
connect and exchange data between devices without using of any kind of physical contact
like wires and cable.
4. DESIGN
Block Diagram: A block diagram is a diagram of a system in which the principal parts
or functions are represented by blocks connected by lines that show the relationships of
7
the blocks. They are heavily used in engineering in hardware design, electronic design,
software design, and process flow diagrams.
Fig.4 Block diagram of the robotic Vehicle
4.1. UML Diagrams
4.1.1. User case Diagram: In the Unified Modeling Language (UML), a use case diagram
can summarize the details of your system's users (also known as actors) and their
8
interactions with the system. To build one, you'll use a set of specialized symbols and
connectors.
System
arduino uno
App
<<include>> <<include>>
Wi-Fi module
<<include>>
U/P()
U/P()
Start
Robo
<<include>>
User
front/back/right/left
Stop
arduino_uno Wi-Fi-module
User
operations
Robo
android app
Fig.4.1.1. User Case Diagrams for proposed system
4.1.2. Component Diagram: Component diagram is a special kind of diagram in UML.
Thus, from that point of view, component diagrams are used to visualize the physical
components in a system. These components are libraries, packages, files, etc. Component
diagrams can also be described as a static implementation view of a system.
9
android studio arduino ide
APP
Fig.4.1.2. Component Diagram for proposed system
4.1.3. Deployment Diagram: A deployment diagram is a UML diagram type that shows
the execution architecture of a system, including nodes such as hardware or software
execution environments, and the middleware connecting them. Deployment diagrams are
typically used to visualize the physical hardware and software of a system.
10
On/off Wi-Fi module Start front/back/right/left Stop
: User : Robo
1 : App()
2 : pairing()
3 : On()
4 : Start()
5 : operations()
6 : Action()
7 : Off()
8 : stop()
Fig 4.1.3. Deployment Diagram for proposed system
4.1.4. Sequence Diagram: A sequence diagram or system sequence diagram (SSD)
shows object interactions arranged in time sequence in the field of software engineering.
It depicts the objects involved in the scenario and the sequence of messages exchanged
between the objects needed to carry out the functionality of scenario.
11
front/back/right/left
: Robo
Wi-Fi module
Start
Stop
: User
On/off
Fig.4.1.4. Sequence Diagram for proposed system
4.1.5. Flow Chart: A flowchart is a picture of the separate steps of a process in sequential
order. It is a generic tool that can be adapted for a wide variety of purposes, and can be
used to describe various processes, such as a manufacturing process, an administrative or
service process, or a project plan.
12
App
App
Start
Off
arduino uno
ON
Wi-Fi module
bluetooth module
front/back/right/left Robo
Operations
front/back/right/left
Start/Stop
Robo Stop
Fig.4.1.5. Flow chart diagrams for proposed system
5. IMPLEMENTATION
5.1. Modules
• Wi-Fi module
13
• Motor Driver IC
• Battery
• DC motors
• Arduino
5.2. Module Description
5.2.1. Wi-Fi module
ESP8266 is a complete and self-contained Wi-Fi network solutions that can carry
software applications, or through Another application processor uninstall all Wi-Fi
networking capabilities.ESP8266 when the device is mounted and as the only application
of the application processor, the flash memory can be started directly from an external
Move. Built-in cache memory will help improve system performance and reduce memory
requirements.
5.2.2. Motor driver IC
A motor driver IC is an integrated circuit chip which is usually used to control
motors in autonomous robots. Motor driver ICs act as an interface between
14
microprocessors in robots and the motors in the robot. The most commonly used motor
driver IC’s are from the L293 series such as L293D, L293NE, etc. These ICs are designed
to control 2 DC motors simultaneously. L293D consist of two H-bridge. H-bridge is the
simplest circuit for controlling a low current rated motor.
5.2.3. Battery
Batteries operate by converting chemical energy into electrical energy through
electrochemical discharge reactions. Batteries are composed of one or more cells, each
containing a positive electrode, negative electrode, separator, and electrolyte. Cells can
be divided into two major classes: primary and secondary. Primary cells are not
rechargeable and must be replaced once the reactants are depleted. Secondary cells are
rechargeable and require a DC charging source to restore reactants to their fully charged
state.
5.2.4. DC Motor
A DC motor is any of a class of rotary electrical machines that converts
direct current electrical energy into mechanical energy. The most common types rely on
the forces produced by magnetic fields. Nearly all types of DC motors have some
15
internal mechanism, either electromechanical or electronic, to periodically change the
direction of current flow in part of the motor.
5.2.5. Arduino
The Arduino Uno is a microcontroller board based on the ATmega328
(datasheet). It has 14 digital input/output pins (of which 6 can be used as PWM
outputs), 6 analog inputs, a 16 MHz ceramic resonator, a USB connection, a power jack,
an ICSP header, and a reset button. It contains everything needed to support the
microcontroller; simply connect it to a computer with a USB cable or power it with a
AC-to-DC adapter or battery to get started.
5.3. Introduction of technologies used
Arduino IDE:
16
The Arduino Integrated Development Environment (IDE) is a bridge system
for Windows, macOS, and Linux published in C and C++ functions. This is used to write
and upload computer code to Embedded system boards and several other
entrepreneurship development boards with 3rd-party core support. The source code of the
IDE is protected by the GNU General Public License, edition 2. To endorse the languages
C and C++, the Arduino IDE employs code generator structuring laws. The Connectivity
project contains a library function in the Arduino IDE that includes various of standard
input and output procedures. To launch the simulation and run the main programmed
loop, app code only requires two basic calculations, which are assembled and linked with
a configured stub main () into an executable file executive programmed using the GNU
toolchain, which is updated to reflect IDE distribution. The avrdude software is used by
the Arduino IDE to convert source codes to a text file in hexadecimal encoding, which is
then loaded into the Arduino board's chipset by a loader. Avrdude is the default sharing
tool for uploading code to approved Arduino boards. The below figure shows the Arduino
IDE where the code is been written in Embedded C Language to run to voice control
vehicle.
(Fig. 5.3.1. shows Arduino IDE)
5.4. Sample Code
#define MPR 4
#define MPL 5
17
#define MNR 6
#define MNL 7
String str;
char val;
void setup () {
Serial.begin(9600);
pinMode(MPR,OUTPUT);
pinMode(MPL,OUTPUT);
pinMode(MNR,OUTPUT);
pinMode(MNL,OUTPUT);
void loop() {
while(Serial.available())
{
if(Serial.find("'"))
{
str=Serial.readString();
}
val=str[0];
Serial.println(val);
}
if(val=='1')
{
front();
Serial.println("front");
}
else if(val=='2')
{
Serial.println("back");
18
back();
}
else if(val=='3')
{
Serial.println("Left");
left();
}
else if(val=='4')
{
Serial.println("Right");
right();
}
else if(val=='5')
{
Serial.println("Stop");
Stop();
}
}
void front()
{
digitalWrite(MPR,0);
digitalWrite(MPL,1);
digitalWrite(MNR,1);
digitalWrite(MNL,0);
}
void back()
{
digitalWrite(MPR,1);
digitalWrite(MPL,0);
digitalWrite(MNR,0);
digitalWrite(MNL,1);
19
}
void left()
{
digitalWrite(MPR,0);
digitalWrite(MPL,1);
digitalWrite(MNR,0);
digitalWrite(MNL,0);
}
void right()
{
digitalWrite(MPR,0);
digitalWrite(MPL,0);
digitalWrite(MNR,1);
digitalWrite(MNL,0);
}
void Stop()
{
digitalWrite(MPR,0);
digitalWrite(MPL,0);
digitalWrite(MNR,0);
digitalWrite(MNL,0);
}
6. TEST CASES
20
Test case Output Success/Failure
When the voice is not It warns the user to try Failure
recognized by application again
When the voice is The message will be sent Success
recognized to perform respective
operation
When a correct The robotic vehicle Success
command(direction) is moves in that direction
given
When a wrong command is The application alerts the Failure
given user with a wrong
message
7. SCREENSHOTS
21
Fig.7.1. This is the android-based application that recognizes the voice of the user.
Fig.7.2. A warning is given by the application if the user’s voice is not detected.
22
Fig.7.3. After successful recognition of the left command, the data is sent to the device
and the respective operation takes place.
Fig.7.4. After successful recognition of the front command, the data is sent to the
device and the respective operation takes place.
23
Fig.7.5. The application alerts the user with a message if invalid command is given
Fig.7.6. Component’s picture of the robotic vehicle
24
Fig.7.7. Active View of the robotic vehicle at the time of its running
25
8. CONCLUSION
We present a voice recognition system for the mobile robot control. The
accuracy reaches a level of 95% which is much higher than the ones provided by
commercial systems in the market. The core acoustic modelling was achieved by using
the HMM method and more words than the five commands are built in the linguist
database to improve the accuracy by reading the past recognized words in the buffer and
increase or decrease of the possible occurrence of the output words. The system has been
successfully implemented and tested on a voice based controlling robot.
26
9. FUTURE ENHANCEMENT
As we know nothing in this world is perfect everything is trying to make itself
better and more effective compared to others. So, on the same note, this technology also
requires lots more development. Thus, expanding its applications farther where at present
we can’t think of. Further enhancement in project can be used for Home security and
military purposes where the commands can be given to robot without risk by increasing
the range and by installing cameras.
27
BIBLIOGRAPHY
[1] Bojan Kulji, Simon János and Szakáll Tibor, Mobile robot controlled by voice
,International Symposium on Intelligent Systems and Info f rmatics,189-192, 2007.
[2] Peter X. Liu, A. D. C. Chan, R. Chen, K. Wang, Y. Zhu, Voice Based Robot Control,
International Conference on Information Acquisition,543- 547, 2005.
[3] Rahmadi Kurnia, Md. Altab Hossain, Akio Nakamura, and Yoshinori Kuno, Object
Recognition through Human-Robot Interaction by Speech, International Workshop on
Robot and Human Interactive Communication, 619-624, 2004.
[4] Luo Zhizeng, Zhao Jingbing, Speech Recognition and Its Application in Voice-based
Robot Control System, International Conference on Intelligent Mechatronics arid
Automation, 960-963, 2004.
[5] Katsushi Sakai, Tsutomu Asada, Developing a Service Robot with Communication
Abilities, International Workshop on Robots and Human Interactive Communication, 91-
96, 2005.
[6] Sidney D’Mello Lee McCauley James Markham,A Mechanism for Human – Robot
Interaction through Informal Voice Commands, International Workshop on Robots and
Human Interactive Communication,184-189, 2005.
28
