PROCESAMIENTO DIGITAL DE
IMAGENES Y VISION ARTIFICIAL
Semana 15 – sesión 1
Introducción a la Realidad Aumentada
Objetivos de la sesión
• Análisis de imágenes
Trend Towards Invisible Interfaces
Trend from room scale to invisible computing
Making Computers Invisible
hide the computer in the real world
Ubiquitous Computing
put the user inside the computer
Virtual Reality
Making Interfaces Invisible
• Rekimoto, J. and Nagao, K. 1995. The world through the computer: computer augmented
interaction with real world environments. In Proceedings of the 8th Annual ACM Symposium on
User interface and Software Technology. UIST '95. ACM, New York, NY, 29-36.
Graphical User Interfaces
Separation between real and digital worlds
WIMP (Windows, Icons, Menus, Pointer) metaphor
Virtual Reality
• Circa 1985
Ubiquitous Computing
Computing and sensing embedded in real world
Particle devices, RFID, motes, arduino, etc
From Reality to Virtual Reality
Ubiquitous Computing Augmented Reality Virtual Reality
Augmented Reality
• 1977 – Star Wars
AUGMENTED REALITY
•
Defining Characteristics [Azuma 97]
•
• Combines Real and Virtual Images
• Both can be seen at the same time
• Interactive in real-time
• The virtual content can be interacted with
• Registered in 3D
• Virtual objects appear fixed in space
•AZUMA, R. T. (1997). A SURVEY OF AUGMENTED REALITY. PRESENCE, 6(4), 355-385.
AUGMENTED REALITY VS.VIRTUAL REALITY
Augmented Reality
• Virtual Reality
•
• System augments the real • Totally immersive
world scene environment
• User maintains a sense of • Senses are under control of
presence in real world system
• Needs a mechanism to • Need a mechanism to feed
combine virtual/real worlds virtual world to user
• Hard to register real and • Hard to make VR world
virtual interesting
MIXED REALITY
•
A system that combines
•
real and virtual objects and
information.
• MILGRAM’S REALITY-VIRTUALITY CONTINUUM
Mixed Reality (MR)
Real Augmented Augmented Virtual
Environment Reality (AR) Virtuality (AV) Environment
Milgram coined the term “Augmented Virtuality” to identify
•
systems which are mostly synthetic with some real world imagery
added such as texture mapping video onto virtual objects.
•P. MILGRAM AND A. F. KISHINO, TAXONOMY OF MIXED REALITY VISUAL DISPLAYS. IEICE TRANSACTIONS
ON INFORMATION AND SYSTEMS, E77-D(12), PP. 1321-1329, 1994.
AUGMENTED VIRTUALITY
•
VR with windows into the real world
•
AR VS VR
•
OTHER CONTEXT: THE METAVERSE
•
Neal Stephenson’s “SnowCrash”
•
The Metaverse is the convergence of:
•
1. virtually enhanced physical reality
2. physically persistent virtual space
Metaverse Roadmap
•
• http://metaverseroadmap.org/
METAVERSE DIMENSIONS
•
Augmentation technologies that layer information onto
•
our perception of the physical environment
Simulation refers to technologies that model reality
•
Intimate technologies are focused inwardly, on the identity
•
and actions of the individual or object
External technologies are focused outwardly, towards the
•
world at large
FOUR KEY METAVERSE COMPONENTS
•
MIRROR WORLDS
•
Mirror worlds are informationally-
•
enhanced virtual models of the
physical world.
• Google Earth, Street View, etc
• ogle Maps
•LIFELOGGING
Technologies record and report the intimate states and life
•
histories of objects and users
• Nokia LifeBlog, Nike+, FitBit
NARRATIVE CLIP
•
Wearable camera
•
• Automatic picture capture - 2 pics/minute
• http://getnarrative.com
• AR APPLICATION AREAS
• Computer games
• Map-assisted navigation
• Urban planning / Design / construction
• Tourism / Cultural heritage
• Maintenance
• Training
• Battlefield information display
• Robot-assisted surgery
• Virtual tours—real estate, medical facilities, etc.
AUGMENTED REALITY EXAMPLES
•
AR IN TELECONFERENCING
•
Person works at real desk
•
Remote collaborator represented
•
by picture or video or “talking head”
Objects of discussion; e.g. a patient’s
•
brain image, might also be fused into
visual field
•FROM UNIVERSITY OF WASHINGTON HIT LAB
CNN VIRTUAL VIEW (2008)
•
•HTTPS://YOUTU.BE/V7FQ_ESMJMS
POKEMON GO
•
MICROSOFT HOLOLENSE
•
•HTTPS://YOUTU.BE/XGAKDCEZVWG
• EXAMPLE: HAUNTED BOOK/AR BOOK
• Camera hidden in lamp object
• AR content seamlessly integrated into real book
• Natural page turning/manipulation interaction
•SCHERRER, C., PILET, J., FUA, P., & LEPETIT, V. (2008, SEPTEMBER). THE HAUNTED BOOK. IN
PROCEEDINGS OF THE 7TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON MIXED AND
AUGMENTED REALITY (PP. 163-164). IEEE COMPUTER SOCIETY.
AR BOOK DEMO
•
•HTTPS://YOUTU.BE/XBSFAMN78TC
GAMING: ROCK-EM SOCK-EM
•
Shared AR Demo
•
Markerless tracking
•
ROCKEM SOCKEM DEMO
•
•HTTPS://YOUTU.BE/HXTQ1QBMLIW
SALES AND MARKETING
•
Connect with brands and branded objects
•
Location Based Experiences
•
• Lynx Angels
Web based
•
• Rayban glasses
Mobile
•
• Ford Ka campaign
Print based
•
• Red Bull Magazine
PEPSI AR EXPERIENCE (2014)
•
•HTTPS://YOUTU.BE/GO9RF9GMYPM
ASSEMBLY AND MAINTENANCE
•
© 1996 S. Feiner, B. MacIntyre, &
A. Webster, Columbia University
© 1993 S. Feiner, B. MacIntyre, &
D. Seligmann, Columbia University
•MAINTENANCE SYSTEMS
Ngrain
•
• http://www.ngrain.com/
• Training authoring tool
• Model based AR tracking
ScopeAR
•
• http://www.scopear.com/
• Remote assistance
• Image based tracking
NGRAIN EXAMPLE
•
•HTTPS://YOUTU.BE/XTHEAOBGZS4
MEDICAL AR TRIALS
•
Sauer et al. 2000 at Siemens
•
Corporate Research, NJ
Stereo video see through
•
•F. SAUER, ALI KHAMENE, S. VOGT: AN AUGMENTED REALITY NAVIGATION SYSTEM WITH A SINGLE-CAMERA
TRACKER: SYSTEM DESIGN AND NEEDLE BIOPSY PHANTOM TRIAL, MICCAI 2002
MEDICAL DEMO
•
•HTTPS://YOUTU.BE/KFQULGEYH7W
• INTERACTIVE MUSEUM EXPERIENCES
• BlackMagic
• Virtual America’s Cup
• 410,000 people in six months
• MagicPlanet
• TeManawa science museum
• Virtual Astronomy
• Collaborative AR experience
• AR Volcano
• Interactive AR kiosk
• Scienceworks museum, Melbourne
DIGITAL BINOCULAR STATION
•
•HTTP://WWW.DIGITALBINOCULARSTATION.COM/
DIGITAL BINOCULAR STATION
•
•HTTPS://YOUTU.BE/MQLAKBX0OM4
CITYVIEWAR APPLICATION
•
•HTTPS://YOUTU.BE/FDGRXXJX4SE
COLLABORATION EXAMPLE: HOLOPORTATION
•
Augmented Reality + 3D capture + high bandwidth
•
http://research.microsoft.com/en-us/projects/holoportation/
•
HOLOPORTATION VIDEO
•
•HTTPS://YOUTU.BE/7D59O6CFAM0
• HUMAN OPERATING WITH AR
Think of a heads up display on your auto windshield, or on the
•
instrument panel. What could be there to help you navigate?
(Vectors to nearby eating places? Blinking objects we might
•
collide with? Congestion of nearby intersections? Web pages?)
•SPECIAL DEVICES NEEDED TO
FUSE/REGISTER REAL AND GENERATED
IMAGES
Computer Graphics Problem – Human sees
•
graphics generated from 3D/2D models
Tracking/Pose Problem – Graphics system
•
needs to know how the human is viewing
the 3D environment
Optics Problem – Human sees both real and
•
virtual environments
•FROM UNIVERSITY OF WASHINGTON HIT LAB
DIFFICULT AUGMENTATION PROBLEM
•
Human very sensitive to misregistration
•
• Some applications OK – e.g. circuit board
inspection
Tough calibration procedures for average user
•
COMBINING THE REAL AND VIRTUAL WORLDS
•
We need:
•
• Precise models
• Locations and optical properties of the viewer (or
camera) and the display
• Calibration of all devices
• To combine all local coordinate systems centered
on the devices and the objects in the scene in a
global coordinate system
• COMBINING THE REAL AND VIRTUAL WORLDS (CONT)
Register models of all 3D objects of interest with their
•
counterparts in the scene
Track the objects over time when the user moves and interacts
•
with the scene
REALISTIC MERGING
REQUIRES
Objects to behave in physically
•
plausible manner
• Occlusion
• Collision detection
• Shadows
• STRONG VS. WEAK AR
• Weak AR
• Imprecise tracking
• No knowledge of environment
• Limited interactivity
• Handheld AR
• Strong AR
• Very accurate tracking
• Seamless integration into real world
• Natural interaction
• Head mounted AR
• AR TRACKING EXAMPLES
Optical / vision-based tracking
•
• Ensures portability
• Large number of tracked objects
• Registration and low latency are
crucial for AR systems
Sourceless inertial orientation tracking
•
GPS (or other) position tracking
•
• Generally requires high accuracy
• GPS enables mobile (global) outdoor AR
•HTTPS://YOUTU.BE/TQGAQAFLGG0
“MOBILE” OUTDOOR AR
•
1997 “Touring Machine”
•
Backpack systems (Feiner)
User wears/carries:
•
• Computer
• HMD
• Inertial tracker
• GPS unit/antenna
• Input device(s)
•PERFORMANCE ISSUES WITH TRACKING
Accuracy of the registration of the real and virtual image
•
• Noise – position and pose of camera with respect to the
real scene
• Image distortions
Time delays
•
• Update rate for generating the augmenting image
• Can limit registration accuracy – “1 ms = 1mm error”
AR DISPLAY TECHNOLOGIES
•
Monitor Based
•
• Laptops
• Cell phones
• Projectors (more Ubiquitous Computing)
Head Mounted Displays
•
• Video see-through
• Optical see-through
• MONITOR BASED AUGMENTED REALITY
Simplest available
•
Treat laptop/cell phone as a window through
•
which you can see AR world.
Sunglasses demo
•
• MONITOR BASED AR – SUCCESSFUL
COMMERCIALIZATION
Yellow line in football broadcasts
•
Glowing hockey puck
•
Replace times square
•
billboards with own ads
during broadcast
Baseball cards
•
Ad campaigns
•
OPTICAL SEE-THROUGH HMD
•
VIDEO SEE-THROUGH HMD
•
Cierre
