AMITY UNIVERSITY

---------Uttar Pradesh-------

TERM PAPER on

Augmented Reality in Healthcare and Education

Submitted to
Amity Institute of Information Technology
In partial fulfillment of the requirements for the award of
the degree of
Bachelor of Computer Applications

Guided By Submitted by
S.K.Jha Manavi Nair
A1004818079
Augmented Reality Page 1
 Declaration by student

I, Manavi Nair student of BCA hereby declare that

the term paper titled “Augmented Reality in
Healthcare and Education” which is submitted by
me to Prof. S.K.Jha, Amity Institute of Information
Technology, Amity University, Uttar Pradesh,
Noida, in partial fulfillment of requirement for the
award of the degree of BCA, has not been previously
formed on the basis for the award of any degree,
diploma or other similar title or recognition.
The author attests that permission has been obtained
for the use of any copy righted material appearing in
the Project report other than brief excerpts requiring
only proper acknowledgement in scholarly writing
and all such is acknowledged.

Place: Noida _________________

Manavi Nair
A1004818079
2018-2021
Augmented Reality Page 2
 Guide Certificate

I hereby certificate that the Project Report by Manavi

Nair, student of BCA, SEM: 03, Enrollment no:
A1004818079 with the title “Augmented Reality in
healthcare and education” which is submitted to

Augmented Reality Page 3

 Acknowledgement

It is high privilege for me to express my deep sense

of gratitude to those entire faculty

Augmented Reality Page 4

 Abstract

Augmented Reality Page 5

Index
S.no. Topic Page
No.
1. Declaration 2
2. Certificate 3
3. Acknowledgement 4
4. Abstract 6
5. Introduction 7
6. History 9
7. Types of AR 17
8. How does an AR device work? 22
9. How AR devices are controlled? 27
10. AR Frameworks 28
11. AR Models 34
12. Disadvantages of AR 40
13. Conclusion 41
14. References

Augmented Reality Page 6

 Introduction
This study aims to explore the use of augmented reality in
healthcare and education. There are several benefits of
utilizing augmented reality as a medium of learning.
Augmented reality (AR) is a technology that superimposes a
computer generated image on a user’s view of the real
world, thus providing a composite view.
AR appears in direct view of an existing environment and
adds sounds, videos, graphics to it. A view of the physical
real-world environment with superimposed computer-
generated images, thus changing the perception of reality,
is the AR.
The primary value of augmented reality is derived from
the manner in which components of the digital world
blend into a person’s perception of the real world, not as a
simple display of data, but through the integration of
immersive sensations, which are perceived as natural
parts of an environment.
Augmented reality is actually a mixture of real life and
virtual reality, somewhere in between the two, so it’s often
referred to as mixed reality. With the help of advanced AR
technologies the information about the surrounding real

Augmented Reality Page 7

world of the user becomes interactive and digitally
manipulable. For instance, seeing
Immersive perceptual information is sometimes combined
with supplemental information like scores over a live
video feed of a sporting event. This combines the benefits
of both augmented reality technology and heads up
display technology (HUD).

Augmented Reality Page 8

 HISTORY

AR in the 1960s

Figure 1.First head-mounted display.

In 1968 Ivan Sutherland and Bob Sproull created a

first head-mounted display; they called it “The
Sword of Damocles”. Obviously, it was a rough
device that displayed primitive computer graphics.

Augmented Reality Page 9

  AR in the 1970s

Figure 2. Video Place-an artificial reality technology.

In 1975 Myron Krueger created Video place- an

artificial reality laboratory. The scientist envisioned
the interaction with digital stuff by human
movements. This concept later was used for certain
projectors, video cameras and onscreen silhouettes.

Augmented Reality Page 10

  AR in the 1980s

Figure 3. First portable computer called EyeTap.

In 1980 Steve Mann developed a first portable

computer called EyeTap, designed to be worn in
front of the eye. It recorded the scene to
superimposed effects on it later, and shows it all
to a user who could also play with it via head
movements. In 1987 Douglas George and Robert
Morris developed the prototype of heads-up display
(HUD). It displayed astronomical data over the real
sky.

Augmented Reality Page 11

  AR in the 1990s

Figure 4.Virtual Fixtures created by Louis Rosenberg.

The year 1990 marked the birth of the “augmented reality”

term. It first appeared in the work of Thomas Caudell and
David Mizell- Boeing company researchers. In 1992 Louis
Rosenberg of the US Air Force created the AR system
called “Virtual Fixtures”. In 1999, a group of scientists led
by Frank Delgado and Mike Abernathy tested new
navigation software, which generated runaways and
streets data from a helicopter video.

Augmented Reality Page 12

  AR in the 2000s

Figure 5. AR Travel Guide by Wikitude.

In 2000 a Japanese scientist Hirokazu Kato developed

and published ARToolKit – an open-source SDK.
Later it adjusted to work with Adobe. In 2004 Trimble
Navigation presented an outdoor helmet-mounted
AR system. In 2008 Wikitude made the AR Travel
Guide for android users.

Augmented Reality Page 13

  AR Today

Figure 6. Microsoft HoloLens.

In 2013 Google beta tested the Google Glass – with the

internet connection via Bluetooth. In 2015 Microsoft
presented two brand new technologies: Windows
Holographic and HoloLens (an AR goggles with lots
of sensors to display HD holograms). In 2016 Niantic
launched Pokemon Go game for mobile devices. The
app blew the gaming industry up and earned $2
million in a just first week.

Augmented Reality Page 14

  AR in the Future

Figure 7.AR in the future.

Phones and tablets and glasses will not be the only

venue for AR. For example, the development of
augmented reality earbuds, which allow you to adjust
the sounds that come in from your surroundings.
Research continues a pace on including AR
functionality in contact lenses and other wearable and
self-operated devices.

Augmented Reality Page 15

 The ultimate goal of augmented reality is to create a
convenient and natural immersion, so there’s a sense
that phones and tablets will get replaced, though it
isn’t clear what those replacements will be. Even
glasses might take on a new form, as “smart glasses”
are developed for blind people.

Augmented Reality Page 16

 Types of Augmented Reality

Several categories of augmented reality technology

exist, each with varying differences in their objectives
and applicational use cases. Below, we explore the
various types of technologies that make up AR:

 Recognition Based AR

Figure 8.Recognition Based AR.

Recognition based AR (also called marker based

AR) uses a camera and some type of visual
marker, such as QR/2D code, to produce a result
only when the marker is sensed by a reader.
Marker based applications use a camera on the
device to distinguish a marker from any other
Augmented Reality Page 17
 real world object. Distinct, but simple patterns
(such as a QR code) are used as the markers,
because they can be easily recognized and do not
require a lot of processing power to read. The
position and orientation is also calculated, in
which some type of content and/or information
is then overlaid the marker.

 Location Based AR

Figure 9.Location Based AR

As one of the most widely implemented

applications of AR, location based (also called
marker less) augmented reality, uses a GPS,
digital compass, velocity meter, or accelerometer

Augmented Reality Page 18

 which is embedded in the devices to provide data
based on your location. A strong force behind
marker less augmented reality technology is the
wide availability of smart phones and location
detection features they provide. It is most
commonly used for mapping directions, finding
nearby businesses, and other location-centric
mobile applications.

 Projection Based AR

Figure 10. Projection Based AR

Projection based AR works by projecting artificial

light onto real world surfaces. Projection based
AR allows for human interaction by sending light

Augmented Reality Page 19

 onto real world surface and then sensing the
touch of that projected light. Detecting the user’s
interaction is done by differentiating between an
expected (or known) projection and the altered
projection (caused by the user’s interaction).
Another interesting application of projection
based AR utilizes laser plasma technology to
project a 3D interactive hologram into mid-air.

 Superimposition Based AR

Figure 11. Superimposition Based AR.

Augmented Reality Page 20

 Superimposition based AR either partially or
fully replaces the original view of an object with
a newly augmented view of that same object. In
superimposition based AR, object recognition
plays a vital role because the application can’t
replace the original view with an augmented one
if it can’t determine what the object is. A strong
consumer-facing example of superimposition
based AR could be found in the Wayfair AR app.
By downloading an app and scanning selected
pages in their catalogue, users can place virtual
Wayfair furniture in their own home with the
help of AR.

Augmented Reality Page 21

 How does an Augmented
Reality device work?

The objective of AR is to bring computer generated objects

into the real world, which only the user can see.

Augmented realities can be displayed on a wide variety of

displays, from screens and monitors, to handheld devices
or glasses. Google Glass and other head-up displays
(HUD) put augmented reality directly onto your face,
usually in the form of glasses. Handheld devices employ
small displays that fit in users’ hands, including smart
phones and tablets.

Figure 12.Working of AR
Augmented Reality Page 22
Key components of AR devices:

1. Sensors and cameras

Figure 13. Sensors and cameras.

Sensors are usually on the outside of the AR device, and

gather a user’s real world interactions and communicate
them to be processed and interpreted. Cameras are also
located on the outside if the device, and visually scan to
collect data about the surrounding area. The device takes
this information, which often determines where
surrounding physical objects are located, and then
formulates a digital model to determine appropriate
output.
Augmented Reality Page 23
 2. Projection

While “Projection Based AR” is a category in itself, I’m

specifically referring to a miniature projector which can
essentially turn any surface into an interactive
environment. The information taken by the camera is
projected onto a surface in front of the user; which could
be a wrist, a wall or even a person. The use of projection in
AR devices makes the screen real estate a lesser important
component.

3. Processing

Figure 14. Processing.

Augmented Reality Page 24

AR devices are mini-supercomputers packed into tiny
wearable devices. These components include a CPU, a
GPU, flash memory, RAM, Bluetooth/Wi-Fi microchip,
GPS microchip, and more. Advanced AR devices, such as
the Microsoft Hololens utilize an accelerometer (to
measure the speed in which your head is moving), a
gyroscope (to measure the tilt and orientation of your
head), and a magnetometer (to function as a compass and
figure out which direction your head is pointing) to
provide for truly immersive experience.

4. Reflection

Figure 15.Reflection.

Augmented Reality Page 25

Mirrors are used in augmented reality devices to assist
with the way your eye views the virtual image. Some
augmented reality devices may have “an array of many
small curved mirrors” (as with the Magic Leap augmented
reality device) and others may have a simple double-sided
mirror with one surface reflecting incoming light to a side-
mounted camera and the other surface reflecting light
from a side-mounted display to the user’s eye.

Augmented Reality Page 26

 How AR devices are
controlled?
Augmented reality devices are often controlled either by
touch a pad or voice commands. The touch pads are often
somewhere on the device that is easily reachable. They
work by sensing the pressure changes that occur when a
user taps or swipes a specific spot. Voice commands work
very similar to the way they do on our smart phones. A
tiny microphone on the device will pick up your voice and
then a microprocessor will interpret the commands. Voice
commands, such as those on the Google Glass augmented
reality device, are preprogrammed from a list of
commands that you can use.

Figure 16.Controlling AR device by touch.

Augmented Reality Page 27

 Augmented Reality
Frameworks
AR Frameworks, also known as Software Development
Kits(SDK), the tools used for the development of AR
applications. The role of the AR SDK is to perform the
non-trivial task of fusing digital content and information
with the real world. The AR SDK is responsible for many
components of the applications, which are currently
available, including content rendering, AR tracking, and
scene recognition.

 The recognition component works as the brain of the

AR app.
 The tracking component can be stated as the eyes of
the AR experience.
 Content rendering is simply imaginative virtual
objects and scenes on the real-time information.

Figure 17. AR Framework.

Augmented Reality Page 28

Some examples of AR Frameworks are given below:
1. Vuforia

Figure 18. Vuforia

Supported platforms: Android, iOS, UWP and Unity

Editor and Smart Glasses.
Features:
 Mixed Reality and Eyewear Support including
Microsoft HoloLens.
 Recognize and track a broader set of objects, Image.
 Vuforia Ground Plane will be included in Unity as
part of a new free deployment option.
 Natural Interactions with Virtual Buttons
 Vuforia Object Scanner
 Model Targets: CAD based detection to detect real-
world objects like a car, bike, machine etc.
Augmented Reality Page 29
 2. ARKit

Figure 19. ARKit

Supported platforms: iOS, Metal, SceneKit, and third-

party tools like Unity and Unreal Engine.
Features:
 TrueDepth Camera
 Visual Inertial Odometry
 Scene Understanding and Lighting Estimation
 High-Performance Hardware and Rendering
Optimizations
 Great community support

Augmented Reality Page 30

 3. Google ARCore

Figure 20. Google ARCore

Supported platforms: Android Studio, Unity, Unreal

Engine, Web.
Features:
 Environmental understanding: Allows virtual objects
to be placed in a way that physically connects with
the real world.
 Motion Tracking: Allows users to walk around and
interact with virtual content that is rendered in the 3D
world.
 Light Estimation: Create realistic looking objects by
having its own light change dynamically according to
the environment lighting.

Augmented Reality Page 31

 4. ARToolKit+

Figure 21. ARToolKit+

Supported platforms: Android, iOS, Unity Editor.

Features:

 Robust Tracking, including Natural Feature Tracking

 Strong Camera Calibration Support
 Simultaneous Tracking and Stereo Camera Support
 Simple to use unity plugin: without prior registering
targets
 Full Unity3D and Open Scene Graph Support

Augmented Reality Page 32

 5. Wikitude

Figure 22. Wikitude

Supported platforms: Android, iOS, Unity Editor, Smart

Glasses, JavaScript, Xamarin.
Features:
 image recognition & tracking
 Object Recognition
 Instant Tracking
 3D tracking technology (SLAM-based)
 Cloud recognition

Augmented Reality Page 33

 AR Models

1. AccuVein

Figure 23. Searching the vein using AccuVein

AccuVein is using AR technology to make both nurses

and patients lives easier. 40% of IVs (intravenous
injections) miss the vein on the first stick, with the
numbers getting worse for children and the elderly.
AccuVein uses augmented reality by using a handheld
scanner that projects over skin and shows nurses and
doctors where veins are in the patients bodies. AccuVein
has been used on about 10 million patients, making
finding a vein on the first stick 3.5x more likely.

Augmented Reality Page 34

 2. EchoPixel

Figure 24. 3D model of a patient's abdomen

EchoPixel develops medical imaging devices that enable

doctors to use CT images of a patient’s abdomen and
display a 3D model. Their products allow doctors to
render patient-specific anatomy leading to increased
clinical knowledge, faster operations, and better care.
Their EchoPixel True 3D AR product uses a wide variety
of current medical image datasets to enable radiologists,
cardiologists, pediatric cardiologists, and more to see
patient specific anatomy in an open 3D space.

Augmented Reality Page 35

 3. Immersive Touch

Figure 25. Training using Immersive Touch

ImmersiveTouch provides cutting-edge VR and AR

training and education solutions to medical professionals,
students, and the healthcare industry. ImmersiveTouch
combines their innovative technologies with strong
academic partnerships to promote world-class medical
education and patient safety. Their company’s
comprehensive education solutions include next
generation surgical simulators and learning management
systems.

Augmented Reality Page 36

 4. Touch Surgery

Figure 26. Practicing surgery using Touch Surgery

Touch Surgery offers an app that allows users to practice

surgery at any place and at any time. They use an
interactive mobile surgical simulator that guides you step-
by-step through every part of an operation, and every
decision that’s made along the way. Using sophisticated
technology, Touch Surgery is creating accurate and
valuable surgical content, disseminating the best
techniques and procedures to improve the quality of
surgery worldwide.

Augmented Reality Page 37

 5. Elements 4D

Figure 27. Combining different elements using elements 4D

An AR app for studying chemistry. It allows combining

different elements as the simulation, to see how they
would react in reality. To start it special triggers on
printed cards are used.
6. AugThat

Figure 28.3D experiences of different subjects in classroom

Augmented Reality Page 38

Designed by a former teacher, is the application that
brings AR in a classroom. AugThat mainly targets
students who lack motivation with help of 360-
degree virtual photos and multiple 3D experiences.

7. ArLoon Plants

Figure 29. ArLoon Plants app

This app can be used to learn about plants and animals.

Find out all about plants: their parts, their life cycles and
their different species. It helps youto delve into the way
they adapt to ecosystems and the seasons to survive and
discover a host of curious facts.

Augmented Reality Page 39

 Disadvantages of AR

 Headsets can’t be worn in the field for long

period of time.
 Headset battery life is limited. Therefore, it can’t
be used in a long surgery.
 Difficult to use outdoors due to the glare created
on the screen from the sun.
 Not accessible to differently-able people. For
instance: Blind people.
 Lack of privacy is a concern in AR based
applications.
 Dangers of reality modification. Example:
Pokémon Go game has created controversy due
to associated accidents and even deaths.
 It is very costly to develop AR enabled devices.
 With the continue usage of augmented reality
applications can effect on human health and may
cause low eyesight.

Augmented Reality Page 40

 Conclusion
Despite the rising use of Augmented Reality in many
areas of the modern era, augmented reality in education
and healthcare is still new and unsettled. Though
possibilities of AR in teaching/studying are great,
providing new ways of learning. Teachers get to catch the
attention of students and motivate them better,
while students get new tools to visualize their subjects and
complex concepts, as well as obtain practical skills.
Moreover, even parents can benefit – by engaging their
children to study with AR.
Also AR plays a significant role in the case of healthcare,
as it Augmented Reality medicine apps allow medical
students to study the human body and better visualize
health issues. AR apps display better how the human
body operates by overlaying anatomical details on a 3D-
printed human skeleton. While it’s crucial that medical
students understand a theory, they must also understand
its real life applications. New AR technology allows them
to access real-time data and patient information faster
than ever before, which greatly improves the learning
process, and then diagnose patients correctly.

Augmented Reality Page 41