AR software development

& VR systems
Module 2

1
AR Software Development:
Developing AR software involves creating applications that blend digital content with
the real world. This can be achieved by using various AR frameworks like ARKit (for
iOS), ARCore (for Android), or open-source frameworks like Vuforia or OpenCV. The
development process often includes integrating computer vision, 3D rendering, and
real-time interaction with the user.
•Popular AR Development Frameworks:
•Unity with AR Foundation: Supports ARKit (iOS) and ARCore (Android).
•Vuforia: A popular platform for marker-based AR.
•Microsoft Mixed Reality Toolkit (MRTK): For HoloLens and cross-platform AR.
•AR.js: A lightweight web-based AR library.
•Languages and Tools:
•C#: Common for Unity-based AR applications.
•Java/Kotlin (Android) and Swift/Objective-C (iOS): For platform-specific development.
•OpenCV: For advanced image processing.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 2

Camera parameters and camera calibration
•Importance of Camera Calibration:
•Ensures accurate placement of virtual objects in the physical environment.
•Compensates for lens distortion, perspective issues, and intrinsic camera properties.
•Key Camera Parameters:
•Intrinsic Parameters: Focal length, principal point, skew coefficient, and lens distortion
coefficients.
•Extrinsic Parameters: Position and orientation of the camera in the world space.
•Camera Calibration Techniques:
•Checkerboard pattern: Capture multiple images of a known pattern from various angles.
•Tools:
•OpenCV: Provides camera calibration and distortion correction methods.
•MATLAB: Popular for academic and research purposes.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 3

Marker-based augmented reality, AR Toolkit
Definition: Uses visual markers (e.g., QR codes or predefined patterns) to trigger AR content.
How It Works:
Detects markers using the camera feed.
Calculates the marker's position and orientation relative to the camera.
Renders AR content on top of the marker.
Popular Libraries:
ARToolKit: An open-source library for marker-based AR.
Vuforia: Supports custom markers (called image targets).
OpenCV: Can be used for custom marker detection and tracking.
Marker Types:
2D Markers: QR codes, fiducial markers, or custom patterns.
3D Markers: Physical objects or models.
4. ARToolKit:
What It Is: A pioneering open-source library for developing AR applications.
Features:
Marker-based tracking.
High-speed tracking of simple patterns.
Integration with OpenGL for rendering.
Usage:
Configure markers and their corresponding 3D content.
Calibrate the camera using ARToolKit tools.
Track markers in real-time and overlay 3D models.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 4

 VR Systems
Virtual Reality (VR) is a technology that creates a simulated environment, allowing
users to interact with and immerse themselves in a three-dimensional space using
specialized equipment. It combines computer-generated graphics, sensors, and user
input to create the illusion of being in a different world.
VR as a Discipline
As a discipline, VR is interdisciplinary, involving:
Computer Science: For simulation, graphics rendering, and user interaction.
Psychology: To understand how humans perceive virtual environments and interact
with them.
Engineering: For developing hardware and ensuring the integration of various systems.
Design: For creating realistic and immersive experiences through user interface design
and environments.
Physics: To accurately simulate real-world dynamics, such as movement and lighting.
The primary goal of VR is to create an interactive environment that simulates a real or
imaginary world, providing experiences that may be impossible in the physical world.
COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 5
 VR Systems
Basic Features of VR Systems
The basic features of VR systems include:
Immersion: The sensation of being surrounded by or enveloped in the virtual
environment. This is achieved through high-quality visual and audio output.
Interaction: The ability for the user to interact with the virtual world in real-time,
influencing elements like objects, movement, and the environment itself.
Presence: The feeling that the user is physically part of the virtual environment,
facilitated by realistic feedback and interfaces.
Real-time Rendering: The ability of the system to render 3D graphics instantly, allowing
for immediate response to user actions.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 6

Architecture of VR Systems

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 7

 VR Systems
The architecture of a typical VR system includes the following layers:
Input Layer: Captures data from the user, including physical movements, gestures, and
actions. It is managed by devices such as sensors, tracking systems, and input
hardware.
Processing Layer: The heart of the system, where computer-generated graphics and
real-time interaction take place. It consists of a powerful computer or server that
handles physics simulations, 3D rendering, and processing user input.
Output Layer: Provides sensory feedback to the user through visual, auditory, and
sometimes haptic displays. This is the layer where VR output hardware, such as
headsets and audio devices, come into play.
Communication Layer: Connects various components of the system (input, processing,
and output) and may include network connections for multiplayer VR experiences or
additional data sharing.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 8

 VR hardware :
VR Input Hardware
Tracking Systems: These are critical for capturing the user's movements and
positioning within the virtual environment.
Head Tracking: Keeps track of the user’s head position and orientation. Often used
in conjunction with a VR headset to control the viewpoint within the virtual world.
Hand Tracking: Tracks the movement of the user’s hands. Can be implemented with
sensors, cameras, or motion detectors.
Body Tracking: Often used for full-body immersion. Motion capture systems (like
optical or infrared tracking) are used to detect and replicate the user's body
movements in the virtual world.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 9

 VR hardware :
•Motion Capture Systems: These are used for detecting the motion of objects or
people. Systems include:
•Inertial Tracking: Uses accelerometers and gyroscopes to detect movement
without requiring external markers, allowing more freedom of movement.
•Data Gloves: These are wearable devices that capture hand and finger movements
with high precision. They use sensors such as flexion sensors, accelerometers, or
even gloves with embedded motion trackers to detect and translate gestures into
virtual actions.
•Optical Tracking: Uses cameras and markers attached to the user or objects to
capture movement. Examples include Vicon and OptiTrack systems.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 10

 VR hardware :
VR Output Hardware
Visual Displays: These include any hardware used to display the virtual environment to the user.
Head-Mounted Displays (HMDs): The most common VR output hardware, HMDs provide immersive
visual experiences by placing screens (usually LCD or OLED) directly in front of the eyes.
Examples: Oculus Rift, HTC Vive, PlayStation VR, and the Microsoft HoloLens (for mixed reality).
Cave Automatic Virtual Environments (CAVE): These are immersive spaces where projections are
displayed on the walls, floors, and ceiling, providing an immersive 3D experience without the need
for a head-mounted display.
Projectors: In some cases, projectors are used in large-scale VR installations, projecting the
environment onto surfaces like walls, floors, or screens.
Auditory Feedback: To enhance the immersive experience, 3D audio systems are often integrated into VR
systems. These systems help in creating spatial awareness, giving the user auditory cues that change with
their position in the virtual environment.
Haptic Feedback: Devices that provide tactile feedback to simulate the sense of touch. Examples include
haptic gloves, vests, and motion platforms that can simulate the sensation of movement, texture, and
resistance.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 11

 Additional Hardware
•Motion Platforms: Some VR systems use motion platforms to simulate
movement. These can tilt, rotate, or move in various directions to simulate
motion in the virtual environment, enhancing the feeling of presence.
•Eye-Tracking: Some advanced VR systems use eye-tracking technology to adjust
the rendering of images based on where the user is looking, making the
experience even more immersive and allowing for user interface control with eye
movements.
•Force Feedback Devices: Devices like force-feedback joysticks or controllers
provide resistance and feedback to simulate physical interactions with objects in
the virtual world.
Hence these elements together make up the complex and multifaceted systems
used to deliver VR experiences. The combination of these technologies allows VR
to provide powerful, interactive, and immersive simulations for a wide range of
applications, including gaming, training, medical simulation, and more.

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 12

COPYRIGHT © 2019 VERITAS TECHNOLOGIES LLC 13