PROJECT REPORT
ON
Virtual Assistance
AT
GD Goenka University
PROGRAMME
Bachelor’s of Computer Application
Submitted By:
Keerti Gulati (210160212005)
Livpreet Singh (210160212013)
Shubham Yadav (210160212092)
Nikhil Yadav (210160212059)
1.
INTRODUCTION
In this Project Jarvis is Digital Life Assistant which uses mainly human communication means
such instant message and voice to create two way connections between human and his
apartment, controlling lights and appliances, notify him of breaking news, Facebook’s
Notifications and many more. In our project we mainly use voice as communication means
so the Jarvis is basically the Speech recognition application.
The concept of speech technology really encompasses two technologies: Synthesizer and
recognizer. A speech synthesizer takes as input and produces an audio stream as output. A
speech recognizer on the other hand does opposite. It takes an audio stream as input and
thus turns it into text transcription. The voice is a signal of infinite information. A direct
analysis and synthesizing the complex voice signal is due to too much information contained
in the signal. Therefore the digital signal processes such as Feature Extraction and Feature
Matching are introduced to represent the voice signal. In this project we directly use speech
engine which use Feature extraction technique as Mel scaled frequency cepstral. The
melscaled frequency cepstral coefficients (MFCCs) derived from Fourier transform and filter
bank analysis are perhaps the most widely used front- ends in state-of-the-art speech
recognition systems. Our aim is to create more and more functionalities which can help
human to assist in their daily life and also reduces their efforts. In our test we check all this
functionality is working properly. We test this on 2 speakers (1 Female and 1 Male) for
accuracy purpose.
What can this A.I. virtual assistance accomplish for you?
It can answer basic questions fed to it.
It can play music and videos on Youtube.
It can do Wikipedia looks for you.
It is equipped for opening sites like Google (listens to queries and searches them on
Google) and Chrome.
Making queries is an essential part of one’s life, and nothing changes even for a developer
working on Windows. We have addressed the essential part of a netizen’s life by enabling our
voice assistant to search the web. Here we have used web browser module for extracting the
result from the web as well as displaying it to the user. Jarvis supports a plethora of search
engines like Google and displays the result by scraping the searched queries.
Evolution of Virtual Assistance:
The concept of virtual assistance dates back to the early days of computing when rudimentary
programs attempted to simulate human-like interactions. However, significant advancements
in artificial intelligence (AI), natural language processing (NLP), and machine learning (ML)
have propelled virtual assistance into a new era. The launch of Siri by Apple in 2011 marked
2.
a significant milestone, followed by other major players such as Amazon's Alexa, Google
Assistant, and Microsoft's Cortana.
SYSTEM DEVELOPMENT
Tools and technologies used
Language used: Java script, HTML, CSS
Modules used:
Annyang: A small, standalone library for adding voice commands to your web pages.
Artyom.js: A voice-controlled assistant library for the web.
Web Speech API: A built-in API in modern browsers that allows web developers to
incorporate speech recognition and synthesis into their applications.
Assistant.js: A JavaScript framework for building conversational interfaces.
Botpress: While primarily a bot-building platform, it also provides JavaScript SDK
for building virtual assistants.
Dialogflow: While it's primarily a natural language understanding platform, it
provides a JavaScript SDK for integrating conversational experiences
into applications.
Functions created:
User Input Processing Functions: These functions handle the user's input, whether it's
text input or voice input. They preprocess and interpret the input to determine the
user's intent.
Response Generation Functions: These functions generate the virtual assistant's
response based on the interpreted user input. They can generate text, voice responses,
or trigger actions.
API Interaction Functions: If your virtual assistant needs to fetch data from external
APIs (e.g., weather, news, or third-party services), you'll have functions to handle
these interactions.
Action Execution Functions: For virtual assistants that perform actions (e.g., setting
reminders, sending emails), you'll have functions to execute these actions.
3.
Natural Language Processing (NLP) Functions: If you're implementing NLP features,
you'll have functions to process and analyze text for intent extraction, entity
recognition, etc.
Conversation Flow Control Functions: These functions manage the flow of the
conversation, deciding when to prompt the user for more information or provide
additional context.
We implemented the code on Visual studio code.
Problem statement
We are all well aware about Cortana, Siri, Google Assistant and many other virtual assistants
which are designed to aid the tasks of users in Windows, Android and iOS platforms. But to
our surprise, there is no such virtual assistant available for the Developers i.e. Windows
platform.
PURPOSE: This Software aims at developing a personal assistant for Windows-based
systems. The main purpose of the software is to perform the tasks of the user at certain
commands, provided in either of the ways, speech or text. It will ease most of the work of the
user as a complete task can be done on a single command. Jarvis draws its inspiration from
Virtual assistants like Cortana for Windows and Siri for iOS. Users can interact with the
assistant either through voice commands or keyboard input.
PRODUCT GOALS AND OBJECTIVES: Currently, the project aims to provide the
Windows Users with a Virtual Assistant that would not only aid in their daily routine tasks
like searching the web, extracting weather data, vocabulary help and many others but also
help in automation of various activities. In the long run, we aim to develop a server assistant,
by automating the entire server management process - deployment, logging, monitoring and
make it smart enough to act as a replacement for a 6 general server administrator.
PRODUCT DESCRIPTION: As a personal assistant, Jarvis assists the end-user with day-
to-day activities like general human conversation, searching queries in various search engines
like Google, Bing or Yahoo, searching for videos, retrieving images, live weather conditions,
word meanings, searching for medicine details, health recommendations based on symptoms
and reminding the user about the scheduled events and tasks. The user statements/commands
are analysed with the help of machine learning to give an optimal solution.
OBJECTIVES:
Allow the A.I. to speak a given piece of text.
4.
Make a function to open websites which asked to be opened
Make a function which opens the latest uploaded video on Youtube with the title said
by the user
Make a function to search for the query on Google for something that the A.I. doesn’t
understand.
Feed some questions and answers to make A.I. talk like a human being.
Applicability
The mass adoption of artificial intelligence in
users’ everyday lives is also fueling the
shift towards voice. The number of IoT
devices such as smart thermostats and speakers
are
giving voice assistants more utility in a
connected user’s life. Smart speakers are the
number
one way we are seeing voice being used. Many
industry experts even predict that nearly every
application will integrate voice technology in
some way in the next 5 years.
The use of virtual assistants can also enhance the
system of IoT (Internet of Things).
Twenty years from now, Microsoft and its
competitors will be offering personal digital
assistants that will offer the services of a full-
time employee usually reserved for the rich and
famous
5.
Applicability
The mass adoption of artificial intelligence in
users’ everyday lives is also fueling the
shift towards voice. The number of IoT
devices such as smart thermostats and speakers
are
giving voice assistants more utility in a
connected user’s life. Smart speakers are the
number
one way we are seeing voice being used. Many
industry experts even predict that nearly every
application will integrate voice technology in
some way in the next 5 years.
The use of virtual assistants can also enhance the
system of IoT (Internet of Things).
Twenty years from now, Microsoft and its
competitors will be offering personal digital
assistants that will offer the services of a full-
time employee usually reserved for the rich and
famous
Evolution of Virtual Assistance:
The concept of virtual assistance dates back to the early days of computing when rudimentary
programs attempted to simulate human-like interactions. However, significant advancements
in artificial intelligence (AI), natural language processing (NLP), and machine learning (ML)
have propelled virtual assistance into a new era. The launch of Siri by Apple in 2011 marked
a significant milestone, followed by other major players such as Amazon's Alexa, Google
Assistant, and Microsoft's Cortana.
6.
Functionalities of Virtual Assistance:
Virtual assistants offer a wide array of functionalities, ranging from basic tasks like setting
reminders and sending messages to more complex operations such as managing schedules,
making reservations, and even providing personalized recommendations. They leverage AI
algorithms to understand user queries, interpret intent, and provide relevant responses in real-
time across multiple platforms and devices.
Challenges and Considerations:
1) Privacy Concerns: Collecting and processing user data raise privacy concerns
regarding data security and potential misuse.
2) Reliability: Virtual assistants may sometimes misinterpret queries or provide
inaccurate responses, leading to user frustration and distrust.
3) Ethical Considerations: As virtual assistants become more sophisticated, ethical
questions arise regarding their influence on user behavior and decision-making
processes.
4) Integration Complexity: Integrating virtual assistants with existing systems and
workflows can be challenging, requiring significant customization and development
efforts.
5) Bias and Fairness: There's a risk of bias in virtual assistants' algorithms, potentially
leading to discriminatory outcomes, particularly in sensitive domains like hiring or
financial services.
CONCLUSIONS
Through this voice assistant, we have automated various services using a single line
command. It eases most of the tasks of the user like searching the web, retrieving weather
forecast details, vocabulary help and medical related queries. We aim to make this project a
server assistant and make it smart enough to act as a replacement for a general server
administration. The future plans include integrating Jarvis with mobile to provide a
synchronised experience between the two connected devices. Further, in the long run, Jarvis
is planned to feature auto deployment supporting elastic beanstalk, backup files, and all
operations which a general Server Administrator does. The functionality would be seamless
enough to replace the Server Administrator with Jarvis.
7.
