|1

In this issue
3 Foreword: Sinking Deep
by Paul Golata

5 Immersive Technology Penetrates Reality

by Adam Kimmel

11 Immersive Tech Takes Us from Spectator to Participant

by Jon Gabay

19 Immersive Technology is All About the Tech

by Traci Browne

25 Living, Working and Learning in Virtual Spaces

by David Freedman

31 Beyond Zoom: How Immersive Collaboration Works in the Wild

by David Freedman

33 Immersive Tech Changes Product Creation and Purchasing

by Carolyn Mathas and Jon Gabay

39 3D Sound for a Virtual World

by Jon Gabay

45 The Future of VR Collaboration

by David Freedman

Mouser and Mouser Electronics are registered trademarks of Mouser Electronics, Inc. Other products, logos, and
company names mentioned herein may be trademarks of their respective owners. Reference designs, conceptual
illustrations, and other graphics included herein are for informational purposes only.

Copyright © 2021 Mouser Electronics, Inc. — A TTI and Berkshire Hathaway company

1|
Contributing Authors
Adam Kimmel
Jon Gabay
Traci Browne
David Freedman
Carolyn Mathas

Technical Contributors
Paul Golata
Joseph Downing
Christina Unarut

Design & Production

Robert Harper

Special Thanks
Kevin Hess
Sr. VP, Marketing

Russell Rasor
VP, Supplier Marketing

Jack Johnston, Director

Marketing Communication

Raymond Yin, Director

Technical Content

|2
 Foreword: Sinking Deep
By Paul Golata

3 2 1 Go!
augmented reality (AR,) extended Mouser Electronics invites you to
reality (XR), mixed reality (MR), sink in deeply and enjoy reading how
and virtual reality (VR), it merges we have supported our customers’
human sensory experience with design needs by solving the
computers and digital information, challenges that face them. Come
The decision has been made. The
creating new environments to be in and see how Mouser Electronics
point of no return has been reached.
explored. Immersive technology will is the right distributor to work with
I leave the ground, and I feel like I
redefine how we perceive things to satisfy your next-generation
am flying. I see it all. I marvel at the
and significantly transform the immersive technology design needs.
thrill. Suddenly! Splash! I hit! I am
relationships between human and
immersed in a tomb of wet darkness.
machine interactions forever.
Growing pressure. A slow, steady
rise to the surface. I breakthrough. I
To be successful in the future, About
breathe.
companies will need to adjust the author
their processes to implement and
At one time in your life, you have
execute immersive technologies
probably done what many of us
with excellence. This innovative
has. You stood at the edge of a
technology will enable human
pool, deciding whether or not you
creativity and consciousness to
wanted to commit and jump in.
open up new applications and
You finally decided and went for
markets. In the following pages,
it. Next, you experience complete
Mouser Electronics will provide
immersion, going from our normative Paul Golata joined Mouser
the definitions and overview of this
environment into the sinking deep Electronics in 2011. As a Senior
exciting new technology. From there,
wetness of the swimming pool’s Technology Specialist, Paul
the relationship between objective
watery realms. contributes to Mouser’s success
and subjective reality will be
through driving strategic leadership,
explored. The requisite technologies
Mouser Electronics empowers tactical execution, and the overall
that enable its capabilities and
innovation with our customers as product-line and marketing
tremendous impact on work,
the realm of immersive technologies directions for advanced technology
education, and our everyday lives
develops. Immersive technology related products. He provides design
will be explored. Dynamic ways to
provides an altered perception path engineers with the latest information
experience, touch, hear, and sense
for human beings to be mentally and trends in electrical engineering
will be touched on. It will conclude
present in a non-physical world. It is by delivering unique and valuable
by looking at how design engineers
shaping new perceptions of reality technical content that facilitates and
have to adapt themselves to engage
and what can be perceived as real. enhances Mouser Electronics as the
with this technology.
Employing technologies such as preferred distributor of choice.

3|
Image Source: “garykillian / stock.adobe.com”

|4
 Immersive Technology
Penetrates Reality
By Adam Kimmel for Mouser Electronics

How do you define reality? On the • Mixed reality (MR)–The exponential growth as the IoT
surface, the concept of reality is interaction between physical expands.
easy to understand. It is the world and digital objects. MR uses
around us—all we see, all we feel the strengths and benefits of Beyond smartphone cameras, the
in a given space. Reality is simply both AR and VR to optimize the primary technology that delivers
what exists. user experience. an immersive user experience is
an XR headset. Computational and
At least, this definition is how Given the saturation of processing elements, actuators,
we’ve viewed reality historically. smartphones, AR and MR are and sensors are within the XR
However, with the proliferation of gaining popularity quickly. A 2021 devices, among other components.
the Internet of Things (IoT), smart Statista market report covering These components link the two
devices, and 5G, the premise of 2018-2023 provides the following worlds together and enable
reality has changed from constant, insights around immersive tech: transformative applications to
predictable surroundings to a benefit humanity. Some examples
continuously evolving real and • The immersive technology of XR usages include creating
virtual experience. We see and feel industry’s global revenue is human and social benefits such
the physical reality, which is only close to $6.3 billion (USD) as creating safe, simulated social
part of the story. Physical reality is as of 2020 and is forecast to interactions for children, and aid in
balanced by immersive/extended continue growing exponentially the treatment of ailments ranging
reality (XR), which can add to and for the next five to 10 years. from phobias to pain and anxiety.
create new experiences. Along with the clear healthcare
• Virtual reality is the largest benefits, XR also improves human
Immersive realities have three segment, although the growth convenience through smart cities
principal types: rate is asymptotic from 42 and connected streetlights, aids
percent from 2020-21 to 10 in space travel, and thousands of
• Virtual reality (VR)–An entirely percent from 2022-23. other applications in between.
simulated environment that
lets the user feel immersed in a • Mobile augmented reality The intersection of the virtual
digital-only world. combines with augmented/ and physical worlds can change
mixed reality technology to humanity for the better. The
• Augmented reality (AR)–The generate $3.7 billion (USD), following reviews each type of XR
enhancement of the physical with the $2.6 billion (USD) and outlines the roles played by the
environment by adding digital balance coming from VR. relevant signal chain components
images and experiences. that enable this transformative
Users experience AR through • Augmented and mixed technology.
smartphone cameras reality are growing at higher
frequently. rates, with MR experiencing

5|
Image Source: “HQUALITY / stock.adobe.com”

|6
 Immersive navigate the turn. Because
physical reality is constant and not
within the physical environment
before adding digital features. VR
tech frames of subject to digital tolerance, the digitally positions sightlines entirely
reference feedback was instant. In addition, within the digital environment,
the shelf is stationary, so its usually with a head-mounted
position is not questioned at any display (HMD). The technology
Since the invention of the
point in time. reacts to the user’s vision
technology in 1957, extended
movement, so it is critical that the
reality has captivated the
The position and size of physical technology receives, processes,
scientific and consumer spaces.
objects are critical input and analyzes, and returns data logically
Although scientists developed
feedback for creating digital and rapidly. The better job the
the terms virtual reality (1975)
realities, and physical reality technology does, the more natural
and augmented reality (1990)
engages all five senses. With that the experience seems.
much later, the desire to utilize
framework set and the canvas
computers and technology to
enhance human experience was defined, programmers can create Augmented Reality (AR)
digital experiences from and within
already well underway.
the physical environment. This form of extended reality
It is worth defining the frames of is where the advancements in
reference of immersive tech to Virtual Reality (VR) connected technology shine.
understand the signal chain and Already present in our everyday
how relevant processing elements Virtual reality is perhaps the best- lives with Google Maps, NFL’s
help deliver the experience. known immersive experience. It yellow first-down line, and
serves to upgrade the human Pokémon GO (among many other
experience, providing users the applications), AR enhances what
Physical Reality
chance to do things they could users see to add functionality or
never have done otherwise given features.
The simplest to comprehend,
physical, financial, or chronological
physical reality acts as the canvas
limitations. In addition, VR provides The key to this technology is to
on which the technology applies
its operator with a fully immersive meet the user where they are in
digital features. Using all five
experience, simulating as many of terms of the experiential difference
senses, we receive immediate,
the five senses as possible. the digital features add. The
accurate feedback as to the
first-down line should be a subtle
world around us. It is the notion of
feedback that sets the guardrails of Example: Roman Rumble adder when watching a game, so
Because VR creates an entirely the technology does not distract
what the user can expect.
simulated experience, it is possible the audience from human action.
to spend a day watching gladiators Users want the Pokémon or street
Example: Turning the Corner
compete at the Roman Colosseum intersections on the map to be
A simple example of a shopping
2,000 years ago. The technology visible, thus calling for a more
cart running into the corner of a
can ignite the senses to see, hear, disruptive digital feature.
grocery shelf illustrates this. A
smell, and feel the experience.
distracted shopper attempts to turn
Should a user be brave enough, Example: How High the Moon
the corner of an aisle but hits the
they can participate in the games Stargazers can employ AR when
shelf corner. However, they quickly
themselves, with all the historical looking at constellations and
back up the cart, adjust the turning
accuracy of the day. planets in the night sky. The
radius, and complete the turn.
technology provides users
One of the primary differences information about what they
The feedback that the shelf
between VR and AR is the location see, the position, distance away,
provided was that the radius
of the user within the environment. movement patterns of objects, and
was incorrect. This information
AR uses smartphone (or other) context around the origins and
provided the instant, accurate
cameras to orient the user’s sight movement patterns of the stars.
response the user needed to

7|
AR provides a real-life planetarium reducing cost. MR can also direct Visual Display/
visit from the comfort of home, and mechanics on a detailed repair
Illumination
it answers many of the common process remotely.
questions asked by space fans of MR has the chance to be massively
The eyes are the first sense to
all experience levels. disruptive, finally bridging the gap
engage with immersive tech, so the
between humans and machines
display quality is critical. Having
AR offers helpful and entertaining like never before. As in VR and AR,
a display with an illumination
features and gains popularity with physical object positional accuracy
strategy that lets the user’s
each new connected application by and digital feature response times
vision transition from real to
improving the experience. Image are critical to achieving the desired
virtual (and back) provides the
capture and mapping are essential results with MR.
transformative experience the user
to how well AR works. When done
seeks. Increasingly high-resolution
correctly, the digital features feel
as if they are there in real life,
Technical signal cameras offer more realistic

behaving as a physically present chain components experiences. Balancing real-life

and digital illumination can smooth
object.
the boundary between the two
Immersive technology has far-
realms. Smartphones and display
Mixed Reality (MR) reaching benefits for humankind.
makers pace the development of
These advantages are only as
this component.
Even experienced developers good as the technology can deliver,
and users of immersive tech however. Like the supply chain
for sourcing materials, the signal Connectivity
interchange merge the definitions
of MR and AR. Where they combine chain comprises the network and
order of components that provide 5G will be the most transformative
physical and virtual features, MR
the XR experience. Processing in connectivity. For the processing
refers to the interaction of the two
power/thermal management, visual power to access the data for rapid
instead of the digital elements
display/illumination, connectivity, analysis, connectivity cannot be
overlaying physical. The utility of
and motion tracking are the limiting a question. The amount of data
this interaction is driving the mixed
technologies to unlock immersive providing realistic environments
reality to the highest market growth
tech’s full power. will only increase as the system
year-over-year, according to the
transmits more data to the user.
Statista report. Therefore, it is
The high speed and decreased
helpful to consider the example of Processing Power/
lag of 5G will propel XR into
engineering CAD design to denote Thermal Management applications where technology’s
the primary difference between MR
real-time response to user behavior
and AR. The more intensive the display,
is non-negotiable. The widescale
the higher the power requirement
rollout of 5G enables decouples
Example: A Winning Quality needed to supply the digital
data analysis from a central hub
With a focus on Industrial IoT, features. Integrating power-
and processes more of the data at
applying MR to a production dense thermal management and
the device level (the edge). Edge
process can determine whether processors ensures the technology
processing reduces data travel
a machine tolerance is sufficient, will not distract from the immersive
validating its specification. Quality time and distance.
experience. In addition, the
engineers can project a finished system must cool the increasing
part geometry over a partial processing load sufficiently to Motion Tracking
physical part to predict whether protect the user from equipment
the components are dimensionally failure, especially during medical With minor shifts in vision patterns
accurate. Achieving this level and human-care applications. The and motion, immersion technology
of predictive accuracy early in data center cooling market is must be highly sensitive to user
the process can decrease scrap helping to drive processing and movement. Sensors collect
rate and increase throughput, chip cooling solutions that can help data from the environment, and
improving operating efficiency and XR. actuators record and transmit the
|8
 human response. Autonomous of education. Developments in combined efforts to develop and
vehicle technology is pulling the the areas below will fuel this grow each segment to realize
development of both of these, integration: the exponential gains predicted
especially sensors. For accurate by the market reports. But with
interaction of the physical and • Processing power and cooling the benefit available for humanity
digital environments, sensors must from the data center industry through improving extended reality,
create an accurate map of the the reward is clearly worth the
physical space, and the actuators • Display and illumination led investment.
must transmit the human action as by smartphones and display
intended. manufacturers

Conclusion • Connectivity led by 5G

manufacturers and network
service providers
The benefits of seamless
integration of immersive
• Motion tracking led by sensor
technology are clear. This
and actuator electronics
transformative technology
manufacturing
movement improves entertainment
and helps society by improving
Each of these technologies carries
the products we develop, medical
the risk of limiting the growth of
care and procedures, safety when
immersive technology. It will take
traveling, and quality and depth

About
AD-FXTOF1-EBZ 3D ToF the author
Development Kit

Adam Kimmel has nearly 20

years as a practicing engineer,
R&D manager, and engineering
content writer. He creates white
papers, website copy, case
studies, and blog posts in vertical
markets including automotive,
industrial/manufacturing,
technology, and electronics.
Adam has degrees in Chemical
and Mechanical Engineering and
is the founder and Principal at
ASK Consulting Solutions, LLC,
mouser.com/adi-ad-fxtof1-ebz-kit an engineering and technology
content writing firm.

9|
 Image Source: “HQUALITY / stock.adobe.com”

PowerHap Piezo 11th Gen Core™ U-Series

Actuators Processors

mouser.com/tdk-powerhap mouser.com/intel-u-series

| 10
 Immersive Tech Takes Us
from Spectator to Participant
By Jon Gabay for Mouser Electronics

No longer Overcoming becoming a reality. The real-world

and everyday use for humans
distanced by challenges immersed in a technologically-
reality The video headset was the first
created environment can benefit
individuals and society if used
attempt at visual immersion. wisely.
Even before the spoken word,
Early products such as the Stunt
humans used and understood
Master were nothing more than
gestures to convey information.
low-resolution video screens worn
Charcoal, and eventually pens and
around the eyes (Figure 1). With
pencils, provided more clarity and
only ¼ VGA resolution (320 x 240),
specificity, and for the first time,
the device was just a personal
we could encapsulate knowledge,
video display worn on the user’s
news, and experiences in a written
head. Head tracking, interactivity,
and illustrated form. But our
or built-in audio was not a part
saga of creativity and expression
of the experience, but it did help
certainly did not end there. Think
pioneer the ergonomic and optical
about what exists at our disposal
aspects of wearable technology.
to share our experiences and
Early user Interfaces were clever
knowledge—from audio and video
but not very effective. An example
capabilities, to more interactive
is infrared-sensing gloves that
types of communication.
would allow a computer with a
receiver to detect and attempt
The idea of immersing oneself in a
to understand hand movements
technologically-rendered personal
(Figure 1).
environment is not new. We could
argue that the use of stereo
Another stumbling block was
headphones is a form of immersive
low-speed and -density memory
technology. Closing your eyes, you
chips and modules—by today’s
can almost feel yourself in a nature
standards—without the depth to Figure 1: Early attempts at immersive
setting, for example, with forest technology were minimal but clever. The
store 3D images. Head tracking
sounds and creatures all around. Stunt Master headset (left) featured
was not practical, and energy
Although video and movies provide ¼ VGA resolution and low-fidelity
management was not what it is speakers. The IR glove controller (right)
a partially-immersive environment,
today. used strain technology to measure
the experiencer is still distanced by
finger position and IR LEDs to transmit
reality. But advancements
This technology is finally advanced hand motion and button presses.
enough to be on the cusp of (Source: Mouser Electronics)

11 |
Image Source: “Tran / stock.adobe.com”

| 12
 Figure 2: Virtual machines allow technicians and mechanics to study the intricate specifics of a complex assembly before starting a
job. This is useful for training and actual repairs. (Source: Gorodenkoff/stock.adobe.com)

Real-world uses for redundant memory blocks so that

one block of memory was playing
of seeing a flat picture, or an
isometric or parametric drawing, a
the next plateau back the real-time video while the virtual floating engine can come
other block was updated for the to life in front of a technician’s
following sequence. eyes. The technician can rotate
As is with many technologies
it, virtually disassemble it, look
we take for granted, military
Although many flight simulator right into it, and see where release
applications often pioneer the
enthusiasts enjoyed this clamps, bolts, and assemblies must
underlying technologies that find
technology for fun, the real-world be accessed (Figure 2). Of course,
their way into the public domain.
capabilities of what immersive automotive repair, appliance repair,
The cathode ray tube, television,
technology can do for society and virtually any machine repair
radio, and microwave are just a few
are just now becoming evident. procedure can be enhanced and
examples of this. With immersive
Although gaming is pushing made more understandable when
technology, military training such
immersive technology into the the technician has a resolute
as flight simulators spawned the
affordable and popular realms of virtual projection with which they
immersive environments we are
society, important uses will actually can interact.
building upon.
save lives as the technology
merges with other modern It gets even more interesting when
Although simple by today’s
mainstream capabilities. sensor data is combined with
expectations, this work pioneered
the virtual image so the repair
3D mapping and representation,
For example, consider an aircraft technician can see into the engine
perspective processing, memory
engine in need of repair. Looking and find the faulty bearing, then
management and indexing
through a service manual has determine how to reach that buried
techniques, shading and light
always been the mainstay, but this bolt that can’t be seen.
source processing, and image
can be enhanced using immersive
stitching and scrolling in real-time.
virtual technology. Instead
In addition, it used multiport or
13 |
One of the most anticipated and remote surgery. For example, on Coupled with sensors and accurate
likely impactful areas of use a mission to Mars, a surgeon on GPS, an immersive virtual reality
is within the medical field. For Earth can perform a specialized display can superimpose the actual
example, with high-resolution scan procedure on an astronaut a road over the real-time video, so
data from magnetic resonance million miles away. Even simple we know where and where not to
imaging (MRI), a doctor can procedures such as cleaning out drive. Two things to note from this
journey through a patient’s body an ear canal or clogged artery can example: The addition of sensor
and examine a tumor from every be performed without adjusting data beyond human’s senses
angle. Then, the doctor can scale it oneself to zero gravity. Signal added to the immersive experience
to make it so large that its insides lag time is an issue, especially to enhance abilities beyond what
become visible. when millions of miles away. This they were capable of without the
is where artificial intelligence technology. This is where virtual
When combined with interactive will help temper movements and reality coupled with augmented
manipulators (such as a higher actions for the delay times or if a reality makes a difference.
resolution of our control glove from signal is lost.
Figure 1), a doctor can perform Another benefit virtual and
delicate surgery using robotic tools As an ordinary person grounded augmented technology brings
more precisely than a shaky human here on Earth, immersive to society is the life-saving jobs
hand. Coupled with sensors—in this technology can also save my life. performed remotely or virtually.
case, haptic feedback—the skilled Think about a time GPS routed you For example, when defusing a
surgeon can grab, pull, move, through a blizzard. It is impossible bomb, a virtual presence device
and stitch flesh without causing to know where the road and such as an articulated robot can
damage (Figure 3). the lake are when everything is journey to a dangerous location
covered with snow. and perform delicate operations
This technology even allows for without endangering a human. The

Figure 3: Virtual and augmented reality can let doctors perform delicate surgeries with more precision and control than humanly
possible without the technology—even remotely. (Source: Gorodenkoff/stock.adobe.com)

| 14
 3D depth perception will allow haptic feedback without a conditions, such as a balance
the bomb-squad operator to have glove. Strategically placed video adjustment on a stereo. High glare
much better depth perception than cameras are used to monitor hand or white-out blizzard conditions do
any 2D system could. movements, and a clever ultrasonic not have to be distractions when a
waveform interference technology clear and concise virtual world is
actually lets the operator feel available.
Other dangerous jobs such as tactile feedback—kind of like a
manually shutting off a valve in sonic force field (Figure 5). The ability to detect and warn of
a nuclear reactor can also be potential danger before the human
accomplished more safely and with senses do is a definite safety
higher skill than any 2D system benefit and is not limited to just the
alone could perform. Key in any of human range of perception. Using
these precision remote and virtual thermal technology, heat can be
operations is haptic feedback. seen and recolored to become
The ability of an operator to feel visible. Such technology will let a
the objects he is touching with driver know if a person or animal is
force feedback technology is a in the road ahead well before his
must for remote surgery and any limited visual spectrum can pick it
Figure 5: Although high-def video
up. Ultraviolet can bring out details
cameras capture hand movements,
the human eye can’t.
ultrasonic emitters using precise phase
control let the operator feel a movement.
(Source: Mouser Electronics) Extending the human vision
range from radio frequency (RF)
to X-rays could be a substantial
Balancing diagnostic and troubleshooting
augmented reality
with virtual reality
Like Arnold Schwarzenegger
as The Terminator cyborg,
witnessing reality with information
superimposed can set us apart.
Both augmented reality and virtual
Figure 4: Highly accurate sensors reality have their place, though.
precisely detect finger and hand
motions and gestures, and haptic force
feedback technology dramatically
Typically, with augmented
increases tactile dexterity and force reality, video screens are clear
control. (Source: HaptX). or transparent. In modern terms,
organic light-emitting diode (OLED)
technology is ideally suited to fill
other critical operation. A simple
this need. Each pixel is a separate
infrared (IR) glove has given way
point of emissivity, so it doesn’t Figure 6: Augmented reality allows
to real multi-sensor and multi-
need a backlight or scanning laser information to be superimposed over
actuator gloves such as the HaptX
to render an image. the reality of our senses. This can even
industrial-grade models (Figure 4).
extend the range of our senses by
Combined with stereoscopic video, using coloration techniques to make
An interesting recent development heat visible, for example. (Source:
a virtual headset can function well
is the video-based gesture HQUALITY/adobe.stock.com)
as a virtual reality and augmented
detection, recognition, and
reality experience. In this case,
control technology that features
reality can be tuned out in adverse

15 |
technology. Whether in space, a sound 5.1, it will sound best when for virtual reality primarily and
nuclear reactor, or a submarine, all played purely on a 5.1 system. For augmented reality when combined
these environments depend on the augmented reality, the ability to with a stereoscopic camera.
ability to detect and fix a potential sense and reproduce in 3-D is
problem before it becomes a real important. A soldier in a stealth OLED technology might be the
problem. mission might be able to use best choice here. It is flexible, can
thermal technology to identify an be bent and deformed, and is clear

Not just imaging enemy, and filtered, directional

surround sound can enhance
so you can see right through it.
But best of all, embedded in the
accuracy. If an enemy is behind clear film are microscopic LEDs
Here is where it gets fun. Vision
a wall, the 3-D sound could still that create a lit display without
isn’t our only important survival
detect breathing or a heartbeat. a backlight. OLED displays exist
sense. Many argue that the sense
but are not as readily available
of hearing is just as important in
Modern systems such as Dolby as TFTs. Scanning laser heads-up
some cases. We exist as a species
Atmos are designed for true technology is another option.
because our sense of stereoscopic
surround-sound immersive
hearing allows us to detect dangers
experiences such as in movie Multiple manufacturers also make
and locate an approximate position,
theaters. These systems can have ultra-small and high-resolution
all without seeing.
up to 64 speakers, which would be video cameras available thanks
too many for a wearable headset. to the smartphone revolution
Hearing plays a significant role
It also introduces processing and selfie movements. The small
in immersive technology and the
resource needs because audio footprints make them ideal for
immersive experience. Although
is no longer audio. It is a digitally mounting on glasses frames as
stereo has been satisfying for
modeled sound object whose well as totally immersive opaque
most listeners for a long time, it
characteristics, distance, motion, headsets.
is not an accurate 3-D rendering
and direction cause an audio track
of the soundscape. Even though
to be synthesized and distributed Audio processors and small but
several flavors of surround-sound
to multiple speakers. Although this full-range speakers can also be
technology have been designed
is cool in a theater, it is likely not purchased from reputable suppliers
and implemented, they all depend
feasible for wearable immersive with proven track records. Again,
on additional speakers and
systems. the mobile device industry has
subwoofers. It’s easy to implement
. pushed this technology forward
this in a room but much more
difficult on a headset. We possess the quickly because of the need to
listen to music.

Digital signal processing algorithms

puzzle pieces
Perhaps the most important
take stereo sound information
Designing and building immersive advancements in technology
and extract specific channels
technology headsets are within making these designs feasible
for surround-sound standards.
reach, though. The pieces of are the advanced multi-axis
These synthesized tracks can
the puzzle are ready to design accelerometers we have at our
then be amplified and applied
into your device. This is true for disposal. These are fast enough
to the surround-sound speakers.
augmented, virtual, and combined- to monitor head movements and,
Translators also go back and forth
reality systems. when coupled with a quick and
between stereo and 5.1, 7.1, 7.2, 9.2
responsive video processing and
surround-sound standards.
Thin Film Technology (TFT) rendering system, provide clear
displays are used everywhere and nausea-free use. (Note: When
Here is where the technology
are inexpensive, multiple-sourced, displays lag head tracking, that
needs to diverge. For
and available off the shelf. These separation of perceptions can
entertainment, reproduction is
require LED backlights and are cause nausea.)
most important. For example, if
opaque, meaning they can be used
something is recorded in surround-

| 16
 Other advancements such as Development tools are also
ultra-dense and fast memory, available to allow scenescape
energy management chips, and and multi-character interaction. About
high-density battery technology Headsets with graphics engines the author
are all falling into place to make and CPUs are now in place, and
this technology cost-effective and communities are developing to
efficient enough to become the help support this brave new virtual
preferred user interface going world. After completing his studies
forward. In addition, the high- in electrical engineering, Jon
Gabay has worked with defense,
bandwidth, high-frequency wireless
communications technologies also
Forging forward commercial, industrial, consumer,
allow these immersive devices to into this new world energy, and medical companies
be completely free of any tethering. as a design engineer, firmware
. Virtual and augmented reality coder, system designer, research
scientist, and product developer.
Design options and let users immerse themselves
into artificial realms. With virtual As an alternative energy
considerations reality, the immersive experience researcher and inventor, he has
can be a different world than the been involved with automation
Factors to consider when deciding one outside the headset, while technology since he founded
how to move forward with your augmented reality enhances and ran Dedicated Devices Corp.
augmented and/or virtual-reality information and extends the range up until 2004. Since then, he
applications are cost, time to of human senses. Luckily, we has been doing research and
market, reliability, and special possess the technology to bring it development, writing articles,
functions. Cost and time to together and create these worlds and developing technologies for
market are very high on the list capable of enhancing education, next-generation engineers and
for commercial and consumer machinery repair, surgery, students.
applications. Ruggedness, navigation, and extending the
reliability, and special functions range of senses.
are a high priority for military and
aerospace designs. Everything else
falls in between.

In either case, feasibility studies

and prototyping can take place FiT Family Connectors
by OEMing an existing headset.
Several players have offerings
with varying prices and levels of
performance. These are available
in cabled as well as cable-free
configurations. The most talked-
about is the Occulus Quest 2 for
around $300 (USD). It features
a self-contained compute
and graphics engine, wireless
communications, BLUETOOTH®
interface for peripherals, and good
head tracking. Sony PlayStation®
has a competitive model as well
that is even in the ballpark price-
wise. HTC and Valve Index VR kits
mouser.com/molex-fit-connectors
are also worthy of mention.

17 |
 Image Source: “Gorodenkoff / stock.adobe.com”

DRV2605L Haptic IMP23ABSU MEMS

Drivers Microphone

mouser.com/ti-drv2605l-driver mouser.com/stm-mems-mic

| 18
 Immersive Technology is
All About the Tech
By Traci Browne for Mouser Electronics

Whether you are talking about machine vision, 3D scanning,

extended reality (XR), spatial power management, and hundreds
computing, holograms, or of different components in the
volumetric data, people tend to user device itself. In some cases,
have expectations as to what the like volumetric video, hundreds of
experience should be. The first sensors and cameras surround the
time I put on a VR headset a few action. So, is technology itself the
years ago, I’m not sure what I was challenge?
expecting, but I do know I was
disappointed. Let’s face it: Even Co-founder of HaptX and Professor
today. we’re not quite at “Avatar” of Biomedical Engineering at Cal
level immersion. Poly, Dr. Robert Crockett, says,
not exactly. Instead, he thinks the
Still, people have not given engineers who will do well in the
It is that authentic
up on immersive technology’s immersive technology space are
experience you want to strive
promise, and they are counting the systems thinkers. With readily
for in terms of performance.
on engineers to deliver the goods. available, high-quality, off-the-shelf
Yes, everything you need is likely
Almost every industry you can components, engineers can focus
widely available, but choosing
think of is optimistic about the on how all those components work
which part to use will often depend
benefits immersive technology together as a system.
on the application.
will bring, making the market for
this technology as expansive as That means putting much of your
For example, when we think about
an engineer’s imagination. Yes, effort into predesign work, such as
a realistic visual experience in VR,
delivering on their expectations is thinking about the requirements
does that mean you need to supply
a challenge, but then, that’s the life and what’s really important to the
the highest resolution available for
of an engineer: Solving challenges. customer—then laying out the
your VR headset? Well, it depends.
But what particular challenge are system and identifying where it
we dealing with? could fail. For HaptX, that meant
A virtual reality game or an
lots and lots of prototypes to help
engineering walkthrough of a
think through the engineering
Creating a truly challenge of putting all the pieces digital twin may not need the
highest resolution. In those
immersive together. Then, lots and lots of
cases, thanks to The Law of
testing and iteration until they felt
experience they had a reliable system. As their Closure—the gestalt law that says
if there is a small-enough break
name suggests, that system is a
To create a truly immersive in an object, we will perceive the
glove that adds a realistic “touch”
experience, you need sensors, thing as continuing in a smooth
experience to virtual reality.

19 |
(Image source: HaptX)

pattern—lower resolution may barriers engineers are trying

be just fine. But if we’re being to break. Ideally, you want the
wheeled into an operating user’s spatial experience to be
room to undergo intricate brain indistinguishable from reality. To
surgery, it’s probably better if our do that, experts say you need a
doctors practiced on the most delay of under 15ms. Anything
realistic virtual cranium possible. above that will be frustrating
Yet, even if high resolution isn’t for a user, but it can also cause
necessary, it could be the thing nausea or potential danger
that helps gain market share. depending on the use case. If
you cannot resolve problematic
In many cases, resolution is not latency issues, your hardware
as important as latency. Motion- will likely find its way to the
to-photon latency is one of the landfill.

| 20
 While off-the-shelf sensor systems Oculus tells us on their developer For example, force feedback, the
are not quite at that 15ms rate yet, site that a governor process on feeling of force on the wheel in an
we continue to see incremental their device “monitors an internal automobile simulation, is closer to
improvements. Oculus decided temperature sensor and tries to natural haptics. But what do you
not to wait for component take corrective action when the do when you are not physically
manufacturers to figure it out temperature rises above certain touching an object, but it’s
and built a sensor that supports levels to prevent malfunctioning essential that you feel like you are?
sampling rates up to 1000hz and or scalding surface temperatures.
has just a 2ms delay. They’ve This corrective action consists of When Jake Ruben, Crockett’s
proven it can be done, so hopefully, lowering clock rates.” Oculus’s partner, came to him with his vision
theirs is the tide that raises all advice is that no amount of of creating natural haptics, they
boats. optimization is ever wasted, as started picking at the problem
“every operation drains the battery and came to a realization: Realism
As Crockett said, there are and heats the device.” in the hands was crucial to true
hundreds of components going immersion, and it would also be the
into these systems, and you have
very little space in which to work.
The missing piece: most difficult to achieve.

So, for example, if you are working Sensations Why focus on creating a realistic
with Extended Reality (XR), what sensation of something on your
embedded vision interface you While all these features are chest if humans do most daily
choose is an important decision, important to the immersive activities with their hands? It
but you’ll want to keep it simple experience, one key piece is makes sense to crack that code
because you’re going to have a still missing: haptics. That key first, then move on to other haptic
minuscule amount of room for component to creating a believable experiences.
cabling. interaction with computers brings To demonstrate the reliance on
us full circle and back to being touch, Crockett used the example
While USB 3.0 with a good able to sense it—and haptics allows of an aircraft simulator. Pilots must
5GB/s bandwidth is an excellent this to happen. Haptics simulates flip many switches when getting
multipurpose interface and sensations someone would a plane off the ground, staying
basically plug-and-play, its large experience in the real world when airborne, and, ultimately, landing.
connectors and rigid cabling will they interact with a physical object. They have their eyes focused on
take up much room. On the other That’s why your game controller multiple gauges and panels, and
hand, it will also likely reduce costs rumbles when you receive damage they typically do not look up to
and development time. That could and your phone vibrates when you flip a switch. Instead, they know
be why the MIPI CSI-2 (Mobile get a text message. That vibration where it is through muscle memory.
Industry Processor Interface) is achieved through eccentric Building muscle memory is a
standard is one of the most rotating mass (ERM) actuator, common use case for immersive
common interfaces used in head- linear resonant actuator (LRA), and technology, and in this case,
mounted VR devices. Also, MIPI’s even piezoelectric actuators. natural haptics is needed. But,
6GB/s high bandwidth is faster just as you need to put the virtual
than USB 3.0. Finally, CSI-2’s multi- However, rumble and vibration switch in the exact right spot, you
core processors mean it uses few are what Crockett calls symbolic also want the flip to feel exactly like
resources from the CPU. haptics. They are just a cue. the real thing. To say this isn’t easy
Engineers strive for an authentic to pull off is an understatement.
Power regulation is also something sensation, or as exact as virtual
you will need to pay attention to can get, which he calls natural HaptX decided to use pneumatics
when making devices that users haptics. for their haptic feedback. Each
wear. Increasingly higher screen glove has more than 130 tiny little
resolutions and refresh rates lead Natural haptics makes the user balloons, which they call tactors,
to high display power consumption. feel they are holding the real thing. that inflate with air and push into
your skin. Those tactors replicate

21 |
the exact pattern you would feel certainly helpful to understand wear eyeglasses, hearing aids, or
when touching, say, a switch in a computer science, optics, and cochlear implants. Then there is
cockpit. You also want the user to even human perception. But many the range of motion and dexterity
feel they are touching something experts believe an even broader required just to tighten the headset.
solid. To physically stop the user skill set is needed. These factors can restrict use in a
from putting the hand right through large number of potential users.
the object, they’ve placed a series A working knowledge of
of tendons on the back of the glove.
That ability to control the hand and
psychology, physiology, biology,
kinesiology--well, let’s just say a
Conclusion
feel the correct pattern with the lot of ‘ologies—will come into play
Engineers are in a position to
proper displacement into the skin when designing for user-specific
make immersive technology more
gives the user a natural feeling experiences. That’s when things
inclusive by simply including
of interaction with an object that will get tricky, because no two
universal design into their system
doesn’t exist. realities are the same. Age, gender,
thinking. Immersive technology
body type, nationality, health,
is at that critical point in time
Overcoming economic status, abilities, and
more inform our lived experience.
many technologies encounter

the engineering before becoming widely adopted

game-changers. It is approaching
challenge As crucial as that inclusive
maturity, but it is not yet widely
experience is, universal design
adopted. Users still have hope
principles should be considered
Even Crockett admits that no that their experience will
when creating the hardware
engineer in their right mind would match their expectations. That
and devices that enable those
devise a system that required makes now the perfect time to
experiences. Originally a concept
hundreds of tiny individual tubes, integrate accessibility into the
applied to architecture, universal
each leading to a balloon and each technology.
design has spread into almost
connected to a set of proportional every discipline. It is designing to
control valves. It was an incredibly accommodate everyone regardless
tough engineering challenge that of their age, size, ability, or
has taken almost a decade to solve. cognitive experience.
Nevertheless, they eventually came
up with a practical system that In a paper presented at a workshop About
didn’t involve any exotic technology. on mixed reality and accessibility,
Microsoft researchers posited
the author
Their solution is an excellent that “considering accessibility as
example of that systems thinking a core part of a system’s iterative
mentioned earlier, and that is why design process is valuable not only
a company like HaptX looks for for the more than a billion people
engineers who understand systems worldwide who have some type
and think at a high level. That of disability but for all users since
is, they think not just about what everyone experiences situational
works, but what works together disabilities dependent on their
to form a more extensive design Traci Browne is a freelance writer
context.”
that’s reliable, robust, and meets specializing in manufacturing
the requirements. and industrial applications with a
An example the researchers
focus on emerging technology,
gave of a device that is ripe for
If you are getting the sense that engineering, robotics, and IIoT.
improvement is the head-mounted
immersive technology involves She is driven by her curiosity and
display. Not only are many of
more than just good engineering inspired by the people who dream
them quite heavy, but they are
skills, you may be right. It is up and make stuff.
often not ideal for people who

| 22
 Living, Working and Learning
in Virtual Spaces
By David Freedman for Mouser Electronics

Immersive technology is already blurring the line

between in-person and virtual collaboration

Walking together for the first time headsets by key members of the head of business for immersive-
through the top floors of what design team, who were actually in reality software company Spatial
was soon to become Boston’s their offices in different cities. Systems. “But in virtual reality you
newest major residential tower, the can connect in a way that you just
building project’s top architects Virtual reality and other Immersive- can’t on a flat screen.”
and engineers entered one of the reality technologies can make
spacious condominiums and made people feel as if they’re physically It’s only in the past few years
a horrifying discovery: Some of inside and interacting with that immersive-reality hardware
the unit’s large windows provided computer-generated spaces and and software have become
a clear, direct, unimpeded view— objects—a bit like stepping through sophisticated enough to get
into neighboring apartments. your computer screen into a world beyond crude, cartoonish
That meant, of course, that the behind it. These sometimes very imitations of reality to achieve more
neighbors would just as easily be real-seeming 3D simulations, life-like collaborative experiences.
able to see in. It was an intrusion usually experienced via special And as the technology becomes
on privacy that would never fly high-tech goggles and other gear, more affordable, it becomes
with the fussy prospective buyers can provide highly compelling, increasingly available to almost any
of these high-end dwellings. The even astonishing adventures in employee or even consumer. As
windows on the top floors would the sorts of artificial worlds that the gear gets even better and less
have to be re-angled, which would have long been the stuff of science expensive, and more people and
entail changes to the very structure fiction. organizations become familiar with
of the 32-floor building. the technology, the results could
Increasingly, though, immersive be a huge change in how people
Having to make big changes reality is being harnessed for a work, learn, and relax together.
to a building well along into more down-to-Earth purpose: Immersive reality is already
construction is every builder’s Bringing people who are at a impacting business, education,
nightmare. Such alterations can physical distance together into health, and many social spheres,
run millions of dollars and entail artificial rooms and other synthetic and it may in the not-too-distant
months of delays. But in this case, environments where they can future even become a medium
fixing the building was the work collaboratively learn, share ideas, where many of us spend much of
of a few hours, at relatively trivial make discoveries, solve problems, our day.
cost. That’s because ground hadn’t and simply catch each other up,
even been broken on the project
yet. The group walk-through of
often in ways far more engaging
and illuminating than email,
The different faces
the top floors had taken place phone conversations, or Zoom of immersion
entirely in virtual reality, via a 3D meetings. “You can communicate
computer-generated simulation of in a Zoom meeting,” says Jacob Immersive reality comes in two
the planned structure, shared over Loewenstein, vice president and main forms: Augmented Reality

23 |
 Image Source: “wacomka / stock.adobe.com”

(AR) and Virtual Reality (VR). AR positions of fellow attendees or key versions of people who are across
users continue to see the real products at a conference. Having the world, and it will feel like you’re
world, except with virtual objects to look at all that through a phone all together.”
and information superimposed can be awkward, but in the coming
on those real surroundings. It’s a years we’ll be able to experience VR, on the other hand, is a more
technology that’s become familiar AR through “smart glasses” that encompassing experience that
to hundreds of millions of people are already emerging in prototypes entirely replaces a user’s view of
around the world through the from Facebook and Lenovo, among the surrounding real world with a
Pokémon Go app, in which players other companies. “You can easily simulated one, usually projected
peer at their surroundings through make any object you can imagine onto the user’s eyes via tiny display
their camera-fed phone screens to appear in front of you, and yet screens mounted in headset-like
see cartoon characters who appear you can still see the room around goggles. The headsets also carry
to be lurking nearby. you, so you can walk around the sensors that track changes in the
object, and take a sip from the can user’s head position, altering the
AR has more practical uses, too. of Coke on your desk,” says Nikhil view to match those movements
It can annotate and illustrate Balram, CEO of EyeWay Vision, and provide a realistic sense of
the world around us with helpful which is developing an advanced looking around at a complete
information, pointing out the AR system. “Eventually you’ll be 360-degree environment. Adding
locations of nearby restaurants and able to have meetings where some to the sense of fully inhabiting a
friends, the parts of a machine that of the people are there in the room shared, simulated world are hand-
need servicing, or the names and with you, and others are digital held sensors that enable users to

| 24
 reach out in the scene to “grab”
virtual objects, press buttons, or
Organizations offices or homes. “VR dramatically
reduces geography,” says EyeWay’s
gesture to fellow visitors in the go 3D Balram.
virtual environment; while micro-
speakers built into the headset VR interactions can even feel richer That’s why major design and
provide rich, lifelike, directional than in-person meetings, thanks construction companies around the
sound. to the ability to share 3D views of world, from Ennead in Shanghai to
settings, objects, and even abstract Mecanoo in Delft, the Netherlands,
The pandemic drove home the data that couldn’t be summoned have been turning to immersive
potential value of virtual meetings, up in a real-life room. “You can sit reality for technical and other
as tens of millions of people cut across from each other in a VR collaboration. For example, Valiant
off from their offices turned to room and bring models, videos TMS, an intelligent automation
computer screens to collaborate. and white boards into it, sharing systems company with facilities
But it also introduced millions everything in a way that is super- around the world, conducts
to the drawbacks of trying to collaborative compared to regular review sessions of system designs
keep people engaged during an screen-sharing,” says Spatial’s that bring together engineers
awkward, panel-of-talking-heads Loewenstein. “Whether it’s a in Austria, India, Canada, and
get-together, where participants business meeting, a social hour, or Mexico, using VR software from
may struggle to stay focused on a conference, it feels much more Theorem Solutions in Tamworth in
a small screen while trying to engaging.” the U.K. “That proved especially
ignore what’s going on around useful during the pandemic,
them. “The pandemic was a That’s why the business market for when travel was banned between
catalyst for virtual technology, as VR hardware and software, tiny many countries,” notes Katharine
well as for Zoom fatigue,” says compared to the video-gaming VR Edmonds, a Theorem executive.
Anthony Rowe, an electrical and market just a few years ago, is on
computer engineering professor track to account for half the total VR is also how members of the
at Carnegie Mellon University VR market by 2024, according team working on the Boston
who is at the forefront of VR to industry research firm IDC. A residential tower project,
research. “Now technology needs survey by market research firm composed of design and
to step in and get us past virtual Forrester Consulting found that engineering professionals from
meetings that are just a flat grid half of all companies plan to get Odeh Engineers in Rhode Island
of squares with faces. Interacting into immersive reality within two and Stantec in Boston, found the
in two dimensions isn’t natural for years. privacy problem with the windows.
humans.” The architecture, engineering, and One member of the team also
construction (AEC) industry has discovered when “walking” into an
Collaborating in VR, in contrast, taken a strong lead in embracing electrical services room in the VR
can provide a convincing sense of VR collaboration. It’s not quite as model that he hit his head—well,
fully interacting in a shared space big a jump for professionals in that his avatar’s head—on a low beam,
with other people. To be sure, the world because the industry has providing an invaluable heads-
VR meeting environment can long relied on 3D virtual models up that the room would violate
have a slightly cartoonish feeling of planned construction projects. building code.
to it, with people represented But now VR meetings enable a
by “avatars” that may not seem team to jointly inhabit the model Stories of AEC teams realizing
realistic, but because the avatars and see it in true 3D, as well as to enormous savings with VR abound.
are in 3D, can be made to resemble walk through it together, point out One team saved $32,000 in a
the people they represent, and potential problems, and experiment Florida high-school construction
can mimic their gestures and in with possible changes, all while project when VR meetings allowed
some cases facial expressions, speaking to one another—even if quick resolution of a few dozen
most participants get past those the team members are scattered potential problems that would
limitations quickly. around the world in their respective have otherwise required delays

25 |
to research the problems, share
the details, and get everyone on
Beyond on-boarding. Financial services
company Societe Generale Group
board with the solutions. The architecture brings financial analysts and
British public-utility firm Anglian clients together in virtual spaces.
Water likewise saved thousands Product designers and engineers One company, the San Francisco-
collaborating in VR on an in industries outside of AEC based biodata-analysis firm Larvol,
infrastructure project. And the aren’t far behind in embracing VR already encourages its entire work
supervisors of a new, $2 billion collaboration. Toy company Mattel force of hundreds of employees
subway-line project in Norway were regularly brings distant designers across 14 countries to do any or all
able to bring professionals together together into meeting spaces of their collaborative work in VR,
from four different companies in virtual reality to work out the maintaining only minimal office
across two countries into details of new toy designs with facilities.
coordination and visual-inspection 3D models. And when the design
VR meetings that spotted multiple is ready for manufacturing, the Creating new types of office space
problems early on, preventing big team is joined in virtual rooms by in VR will be liberating for many
cost-overruns and delays. manufacturing engineers at the organizations, insists Loewenstein.
factories that will produce it, often “The office spaces we go to every
So productive are the VR meetings located in China, so the engineers day contribute to our sense of
at the global architectural firm can suggest alterations that will identity, branding, and connection,”
Gensler that now all 28 offices reduce manufacturing costs and he says. “But it’s hard to change
join in on a weekly get-together boost production rate and quality. physical spaces into something
to review projects. The firm uses really stimulating, like an art gallery,
VR in its sales meetings, too, and Aerospace company Lockheed or a camping site, or a meadow. It’s
enlists its VR models to enable Martin runs one of the largest easy to do in VR, and sharing those
designers and construction virtual-reality design labs in the spaces reinforces our connections
engineers to see how different world in its Denver facility, where to the people who work there with
lighting and materials can impact engineers and managers work out us.”
the look of a building, as well as the designs of everything from
to get a better sense of how the satellites to spacecraft. They can Conference-holders are also
building and its elements appear even run simulated performance turning to VR to remove physical
at full scale—something that’s hard tests on the models to optimize distance as a barrier to attending
to do with a conventional virtual the crafts’ aerodynamics and heat while enabling virtual attendees
model or a miniature physical and stress resistance, all before to experience the same levels
model. “There are lots of people physically building so much as a of engagement as in-person
now who use these tools as a core tiny model. attendees. Fittingly, the VR-
part of how they work, and they focused branch of the IEEE, the
wouldn’t let you take it away from The advantages of VR collaboration world’s largest organization of
them,” says Gabe Paez, founder aren’t limited to designers and technical professionals, held its
and CEO of architectural VR tool engineers. Having discovered the 2020 conference online, providing
vendor The Wild. “Five years from power of virtual meetings during full VR access to all keynotes,
now it will seem ridiculous to the pandemic, many organizations sessions, and even social get-
newcomers that the industry used are now looking for ways to wring togethers.
to work on a 2D screen.” more applications and benefits
out of virtual interactions—and Training, too, is already a big
increasingly VR fills the bill. Lloyds application for businesses. State
Banking Group and Ikea are among Farm Insurance enlists VR to
the global companies that are train assessors on how to do an
bringing job candidates and new insurance inspection of a flooded
hires into VR for interviews and home, allowing trainees to open

| 26
 virtual cabinet doors and look
under furniture in order to spot
Enabling the come from higher-speed online
connections and more powerful
potential damage. Walmart lets its transformative processors to boost the resolution
employees virtually face a surging of the avatars, smooth out their
crowd of shoppers during a big motions, and reduce the time lag
The next big leap for the field
sale, so they won’t be overwhelmed between a user’s movement and
will be more realistic avatars that
when it happens in real life. And when that movement is reflected
enable those meeting up in VR to
management consultancy PWC on the avatar’s actions.
look more like their real selves,
does high-level decision-making complete with their changing
training for managers around the As immersive technology improves,
expressions. Rowe’s Carnegie
U.S., leaving those who graduate and more people embrace it, the
Mellon lab is already working with
the VR program testing 35 percent ways in which it will transform our
headsets that include cameras that
higher on the learned material than jobs and lives are nearly impossible
point back at the wearer’s face,
those who took the course in an to predict, says Balram. “Mostly
so an image of the face can be
ordinary online form. The United what we’ve been doing is force-
projected onto the avatar.
Nations, UPS, and Tyson Foods fitting the old ways of doing things
are all experimenting with training into these new environments,” he
Future headsets, Rowe adds, will
applications of VR. “You can learn explains. “Now we need to start
include sensors that pick up facial
on a flat screen, but it’s boring,” completely rethinking how we do
muscle movement to further
says Balram. “Learning is as much things to truly take advantage of
refine avatar expressions, and eye
about engaging interactions these tools.”
trackers to detect where the user
with other people as it is about is looking. “It feels much more real
information, and immersive when you can tell the direction of
technology is much better for that.” someone’s gaze, especially when
you’re looking at each other,” he
says. Other big improvements
in the realism of avatars will

About
Lynx™ Board-to-Board & Mezzanine the author
Connectors

David H. Freedman is a Boston-

based science writer. His articles
appear in The Atlantic, Newsweek,
Discover, Marker by Medium,
and Wired, among many other
publications.

mouser.com/amphenol-intercon-lynx-connectors

27 |
Image Source: “HQUALITY / stock.adobe.com”

| 28
 Beyond Zoom: How Immersive
Collaboration Works in the Wild
By David Freedman for Mouser Electronics

How long should it take to go from Paez notes he and his colleagues The Wild, the group sees massing
one person’s sudden idea for a at The Wild are constantly meeting together in real time, and anyone
design to sharing a 3D mockup and collaborating with teams from can try their hand at improvements
of the design with teammates in these and other companies, which or alternatives.
different offices? are scattered around the U.S. and
world. But no one has to get on a Perhaps most important, says
No time at all, insists Gabe Paez, plane to do it. The collaborators Paez, VR collaboration enables
Founder and CEO of The Wild, a simply don virtual-reality headsets team members to walk around
virtual-reality software company and meet in VR space. “There’ll designs together, looking at the
in Portland, OR. “The goal is to never be a jet that travels fast design from different angles and
shorten the time from idea to enough to bring people together as at different scales, discussing
shared experience, until it’s as quickly as VR does,” says Paez. their impressions and ideas. The
close to instantaneous as possible,” ability to do that as a team inside
says Paez. When a VR meeting is aimed at a shared space—even if it’s a
coming up with a new design, be virtual one—allows the interaction
Paez and his team have aimed it for a skyscraper or a shoe, the to transcend discussions that
to make the design process as collaborators gather virtually in a take place over a Zoom meeting.
collaborative and intuitive as computer-generated space called “When you can move around and
sharing a Google Doc. “Space is the workshop. There, each person see other people who are there
content for us,” he says. appears as a cartoon-like “avatar,” with you sharing the experience,
and the group gathers around a you feel physically activated
The Wild works with a range of virtual table. Anyone in the group and emotionally stimulated,” he
companies to bring them into can then take a try at an initial explains. “It’s a much more
the world of collaborative Virtual “massing”—that is, creating a shape dynamic, rich interaction that can
Reality (VR) design. Many are intended to be a first, rough idea form more meaningful insights.
major players in the architectural, for a design. You just can’t get that when you’re
engineering and construction trapped behind the glass of a
industries, including Perkins & Normally designers do massing computer screen.”
Will and PAE Engineers. But the by themselves with easily-shaped
company also helps giant home- materials such as foam, clay, or The impact is especially large in
improvement retailer Lowe’s, balsa wood. Then, when they feel designing buildings and spaces
footwear manufacturer Adidas, and they have a good start, they can that can be entered and explored
telecommunications leader Verizon, show it to other team members together, he notes. “You’re not just
among others, to come up with for feedback. Gathering feedback talking about the space,” he says.
new products, as well as retailing from all involved might take days “You’re experiencing it as a group
concepts. or even weeks. But in meetings in while all being fully connected in it

29 |
and moving through it to see how remodeling in their homes knows in repeated meetings, these sorts
it changes. Those sorts of spatial that you don’t really appreciate of problems gradually come to
perceptions and connections help what works and what doesn’t until light. So, by the time a building is
define our experiences in real life.” you’ve lived in it for 30 days,” he actually under construction, or a
says. “It’s hard to imagine how the product is rolling off the assembly
Often a big benefit to come nuances of a design will change, line, the results feel familiar to
out of the VR design sessions whether it’s the basic shape of the the team, and there are few if any
is the gradual recognition by design, or just the placement of a surprises. “By then, the team feels
collaborators that a design that at light switch.” it already has a relationship to
first seemed like a winner actually the finished product,” says Paez.
has flaws that need to be ironed With the ability to spend time in “That’s something that’s hard to do
out. “Anyone who has done any VR as a team exploring the design with foam.”

Image Source: “blacksalmon / stock.adobe.com”

| 30
 Immersive Tech Changes Product
Creation and Purchasing
By Carolyn Mathas and Jon Gabay for Mouser Electronics

Immersive technology will to approach immersive technology creating multi-component designs,

increasingly provide design in this way. In that case, it will is in the realm of the possible. We
engineers with the feeling of significantly impact product design have sophisticated companies and
“presence” in a simulated and and, as we will see, eventually be teams pursuing this surprisingly
virtual world. While today’s virtual used to revamp human purchasing coherent and common vision with
reality (VR) technology is already experiences. a spectacular level of investment.
used in design and prototyping, it Proof points for the power of these
will eventually guide users down
a complete path, alerting them to
Virtual reality, intersecting technologies now exist
in virtually every industry, from
what’s expected and what they augmented reality, medicine to energy to electronics,”
may experience before making and our current Solotko added.
decisions, building, and even
buying. reality As Solotko notes, immersive
technology currently reaches its
Immersive technology is just a “Today, immersive design and tentacles into sectors impacting
part of an even bigger puzzle. simulation employ VR as a safety, efficiency, and businesses’
When it comes to design thinking natural extension of computer- bottom lines.
in product design, for example, aided design and engineering.
the focus is on human needs On the horizon, simulation and For example, the University of
to create a desirable, feasible, visualization powered by machine Liverpool’s Virtual Engineering
and viable end product. Taking learning will take immersive Centre has already helped support
design thinking and the concept design into the realm of science over 900 companies in adopting
of presence even further, using fiction, creating a foundation of and developing immersive digital
empathy enables designers to engineering tools to propel the tools for business impact in
stand “in a user’s shoes,” ultimately next generations of innovation,” nuclear, aerospace, energy, health,
enhancing design and product says Simon Solotko, Senior Analyst and the manufacturing sectors.
quality, and empowering the at Tirias Research. The Centre’s haptic technology
complete creative process. employs vibrations from tracked
“We are seeing VR achieve high- gloves that signal when two or
By harnessing the concepts of resolution and precise visual more virtual entities come into
design thinking, empathy, and control necessary to bring an virtual contact. The vibrations raise
immersion, the result is storytelling engineer’s entire day, their immersion levels above normal
that effectively and honestly entire workspace, into VR. Every when only visual feedback is
communicates the precise way a element, from initial design, to delivered to the user. This current
product addresses a customer’s selecting components, evaluating technology is poised to be the
needs. Now suppose we continue functionality, and ultimately basis of future, deeper immersion.

31 |
 Image Source: “Gorodenkoff / stock.adobe.com”

While VR creates a different

environment, AR overlays images
Product design adjacent parts. One mechanical
example of this is how well a drive
over the real world, interactively and testing shaft couples to a gear and keyway.
enabling virtual seeing and An electronics example is seeing
manipulating without having Virtual and augmented reality how well a component’s thermal
an actual model. VR/AR is 3D systems are currently incorporated relief area couples to a heat sink
personified, and a user interface into product design and buried inside an otherwise not
(UI) isn’t dependent on the tactile development, the supply chain, visible assembly.
interaction of a mouse, touchpad, and life cycle management. This
or touchscreen. Instead, the tactile begins at the conceptual stage The key here is that teams can be
interaction is mimicked. and represents a new way of distributed globally, receive and
approaching, executing, testing, share data in real-time, and have
The impact on UI design will be and producing a new design. their own areas of expertise to
that gestures, eye tracking, voice, contribute, while observing any
controlled, and haptic technology Once a virtual representation of possible concerns that may arise
take over. What will remain of a product that does not exist is long before costly prototypes or
traditional UI design is the critical created, designers and teams parts are developed.
component of visual design. benefit in ways they would
otherwise have been unable to. The Thanks to material science, we
VR enables an open environment ability to view to scale and see it in can now work with a lot of new
where actual product quality three dimensions is the first benefit. materials. The ability to virtually
matches a rendered model, even Even experienced mechanical characterize and test these
when there is no room for error. engineers may not be able to materials—which may be scarce
As a result, design engineers see determine the 3D version of a 2D or expensive—allows anyone
their models from different angles, drawing. The ability to zoom inside to participate in a design that
which better provides the ability to a part allows a more thorough employs them.
refine and improve them. examination of its interaction with

| 32
 Each component can carry Once the details of mechanical While virtual technology mildly
attributes such as modulus of parts are captured, prototypes may impacts disciplines like PCB design,
elasticity, melting point, sheer force be fabricated using automated cable harness manufacturing,
limits, and compression limits. The CNC techniques, 3D printing, or documentation, and software
simulated world can show stress good old-fashioned handcraft. But, engineering, it’s the system-level
points and danger zones for design before costly molds and die tools designers, engineering team
iteration improvements. Assembly are made for volume production, leaders, and managers who get
fits and frictions can be determined, the thoroughly scrutinized virtual the streamlining benefits of this
and as 3D modeling and simulation product would have to be put technology. Being responsible for
tools improve, entire complex through the hoops mechanically all aspects of a design project is
assemblies can be rendered and electrically to ensure readiness. hard when you can’t see it, feel it,
virtually. or hold it in your hands.
Often, electronics production
Not only can you see how is delayed until the mechanical Virtual technology will enable
components fit and interact at constraints are known. You can’t design managers to examine their
nominal conditions, but design design a printed circuit board, for designs in detail before committing
teams everywhere can also example, if you don’t know its size to fabrication. This new reality can
view thermal simulations of or where the connectors need to significantly streamline a project
friction, temperature rise, fit, and poke through a chassis. As a result, since costly iterations will decline if
interferences as a gear heats up. mechanical designed constraints not be completely eliminated. The
This vision allows experimentation are typically defined first. result is quicker time to market with
with different materials, all lower costs and higher assurance
before a prototype part is ever When it comes to electronics, at every stage.
fabricated. The time and expense package spacing and heights
saved are apparent. Whatever the can be assessed using immersive As we integrate artificial
case, access to the fundamental technology. Electro-physical issues intelligence (AI) into this
elements of a finished design is can also be addressed. Component technology, AR may also allow
now available. temperatures, heat flow and heat troubleshooting and maintenance
sink effectiveness, capacitor and at a much higher level. Zoomed
With the same data used as a inductor spacing, and alternate ocular video streams, for example,
source, distributed teams can sourced components can all be can detect broken solder joints
work together, yet independently, verified. The alignment of an LED that a human eye could easily
anywhere around the world to with an indicator or light pipe is miss. AI learned performance can
verify and test, assure component beneficial when viewed as a 3D quickly identify common failure
placement and fit, and simulate construct since 2D drawings don’t characteristics and quickly lead
worst-case stresses and failure always tell the best story, and not repair technicians to the problems.
modes. In addition, an empowered everybody can turn a 2D picture Thus, the virtual world can precede
design team can raise concerns, into a 3D mental image. This view the natural world in creating
even about things that are not in effectively allows user interface higher-quality real-world items.
their area of expertise. Companies testing, potentially eliminating
that have followed the Deming
philosophy, a holistic approach to
costly iterations.
The future of
leadership and management, have However, it is not just mechanical purchasing:
discovered this. and electrical issues that Mouser 3.0
immersive technologies will bring

The prototype – to the forefront. Electro-optics,

electromechanical, electrochemical,
As we begin to grasp the concept

Moving from and sensor technologies are also

of design and development in the
immersive world, it’s also essential
virtual to physical beneficiaries of virtual design. to think about a distributor’s role.
Think Mouser 3.0, for example.

33 |
What will Mouser 3.0 look like
as its designers continue to
Challenges immersive environments extending
to the service life on engineered
harness immersive technology? products and systems.”
While we may be closer than
For one, there will be new rules of
we think to immersive buying,
engagement, and one of the most
exciting is product development
designing, and producing, Leaning into a new
and distribution in real-time.
challenges remain to arrive at reality
the customer interaction level of
immersion that we seek. Just a few
Potentially, a user could visit As we examine our current reality in
include:
the Mouser website and select terms of AR and VR, it’s clear we’re
different components to add to a on the brink of a new development
• Improving basic web experience. Engineers will be even
design and test it. He or she could
technology to sufficiently
try various parts and see how more present in their designs,
support both design
they will work using an immersive testing and touching in a virtual
engineering and e-commerce
environment. Perhaps, initially environment, and eventually
using goggles and vibration, the even gaining access to physical
• Optimizing stereoscopic vision production techniques that bring
designer would “touch,” “feel,” and
and perception for individuals
rotate the parts, while a Mouser the virtual to life right before their
and teams
expert assists in real-time. eyes. In addition, as processes
become more flexible and costs
• Providing the user-friendly are reduced, the entire buying and
Designers could scrutinize
element of 2D and 3D
designs from a holistic point of building process will contain more
visualization environments
view, enabling them to observe room for innovations to come to
the future product in real-world life, further changing the way we
• Providing sufficient immersive live.
settings. This not only saves time
analytics that fully support
but also increases the overall
e-commerce platforms
quality of the end product. Using
simulation tools will speed up
• Enabling adequate immersive
innovation in engineering, reduce
comparison for this level of
cost, and improve product quality,
virtual shopping
as we’ve discussed.
About
From concept to detail, Mouser
There will be substantial
advantages within manufacturing
the author
could exponentially extend the
and supply chains beyond the
partnership approach they already
digital prototype, including testing,
have today. As a result, engineers
streamlined processes, and Carolyn Mathas is a freelance
globally will work together and with
the ability to truly bring virtual writer/site editor for United
Mouser as if they are in the same
products to life. Business Media’s EDN and EE
room.
Times, IHS 360, and AspenCore,
According to Solotko, “The as well as individual companies.
When could this happen? Maybe
e-commerce of design components Mathas was Director of Marketing
sooner than we think. A next-
will evolve into this immersed for Securealink and Micrium, Inc.,
generation design platform will
workflow. Steps we take on the web and provided public relations,
support integration, shared
today will need to integrate more marketing and writing services
data, and improved intelligence.
seamlessly into the immersive, VR- to Philips, Altera, Boulder
Integration across design
powered engineering workflows of Creek Engineering and Lucent
processes and disciplines
the future. This can extend through Technologies. She holds an
optimizes resources to reduce
manufacturing, quality assurance, MBA from New York Institute of
development time and cost.
and ultimately customer support— Technology and a BS in Marketing
with online integration and from University of Phoenix.

| 34
 3D Sound for a Virtual World
By Jon Gabay for Mouser Electronics

Introduction Getting here and dynamic processing to create

affordable implementations that
fit more budgets. More home
The world is constantly changing. We exist today in large part
theaters exist today than ever
This evolving state means because of our focused ability to
before, especially in a pandemic
the way we interact with the listen. For example, our ancestors
world, and immersive audio is sure
world, equipment, and people who hunted would use sound to
to be in game and home theater
is also changing. Other than track and find food for a family
locations everywhere.
profit motives, two forces are or village. Using sound to locate
dictating this change. One is that potentially lethal sources in a 3D,
Although stereo allows a basic
technology is pushing us forward immersive way makes avoiding
surround-sound capability, the
and enticing us with must-have predators possible. We could
most popular surround sound and
devices and apps. The other sense where something was, how
3D audio in use today is Dolby
driving force is pandemic-driven far away it was, and how fast it was
Digital 5.1 technology. These
concern for health. approaching.
systems are called Dolby Digital,
Dolby Pro Logic II, DTS, SDDS, and
With more time on people’s hands Immersive audio is the most
THX. They all feature a six-speaker
to be entertained, immersive advanced audio-processing and
configuration (five total bandwidth,
technology has had an opportunity -delivery system yet. To get it right
one subwoofer) surrounding the
to make inroads. Virtual- and requires audio processing and an
listener(s) (Figure 1, p.41). These
augmented-reality headsets are array of well-characterized and
surround-sound technologies were
now readily available from several -calibrated woofers, speakers, and
first used in movie theaters, which
manufacturers. Development tools tweeters strategically placed to
helped advance these systems and
let hobbyists and professionals provide a dynamic full-spectrum
make them more cost-effective
render worlds and scenescapes. listening sensation.
and available to the masses.
While the visuals get all the press,
the audio is moving forward as well. Although implementations like this
Multiple speakers are driven with
have been beyond the average
unique individual audio streams
Of the five acknowledged senses, audiophile of the past, movie
so that the perceived location
sound is perhaps the most theaters and performance venues
of virtual sound surrounds the
appreciated. Whether from the have taken advantage of this
listener. Here, rear left and right
richness of nature, the fullness technology for years. Like all
channels are used for spatial depth.
of an orchestra, or the stirring of pioneered technology, it eventually
The front left, right, and center
spiritual music, sound as much finds its way to the rest of the
channels are used for lateral depth,
as any other sense profoundly world.
and a single subwoofer distributes
impacts us emotionally and
the low-frequency bass for the
physiologically. Modern entertainment systems
entire room.
utilize many specialized filters
35 |
Although ideal for a single We should note that source
listener centrally located in a converters from stereo capture
listening zone (or couch), every can process the stereo audio
listener will experience slight signal to create synthesized multi-
differences everywhere else. The speaker surround-sound signals.
relatively homogeneous sound This demonstrates how digital
will let everyone in the listening signal processing can separate
zone experience audio in motion. source locations mostly from a
What’s more, recording artists stereo source. The best solution Figure 1: Surround-sound 6.1 provides
are advertising their latest CDs would be to capture sound in a lateral speakers to enhance the audio in
as immersive by providing 5.1 3D microphone configuration motion as an audio object moves from
surround-sound tracks. then play it back in the same 3D front to side and back. Again, woofer
speaker configuration. However, placement is arbitrary. Here, it’s not
about the bass. (Source: Zern Liew/
Interestingly, the center-front this is cumbersome and difficult,
stock.adobe.com)
channel is optimized for speech- and most will not go to these levels
range signals. This helps listeners when signal processing makes a
discern conversations while good approximation.
immersed in 3D sound. As the
bandwidth and fullness of sound Is this always the best approach?
became popular, the ability to Can signal processing fool our
distinguish speech became more keenly-developed sense of hearing
challenging, so center-channel using fewer speakers, or will
filtering and amplification can we continue to create walls and
make conversations easier to ceilings of sound?
understand.

The addition of one more

Object-oriented Figure 2: Surround-sound 6.1 provides
rear-center channel ups the audio lateral speakers to enhance the audio in
specification to 6.1 surround- motion as an audio object moves from
The most up-to-date front to side and back. Again, woofer
sound (Figure 2), and 7.1 standard
placement is arbitrary. Here, it’s not
systems eliminate the rear-center implementation of immersive audio
about the bass. (Source: Zern Liew/
channel but add left and right mid- comes from Dolby Atmos, and it is stock.adobe.com)
channels (Figure 3). designed for theater applications.
So far, almost 5,000 theaters have
The 7.1 surround-sound technology been retrofitted to use 64 speakers
adds more speakers and unique to take advantage of this latest
channels. The 2.5-D cube or audio listening experience. As such,
polygon that can be created it supports an extensive array of up
can extend to more speakers, to 128 channels and can be fitted
tweeters, and woofers at strategic with full bandwidth, low-frequency
locations, immersing the listener woofers and subwoofers, as well as
in 2D and limited 3D audio high-frequency tweeters.
(Figure 3). Something directly on
top or bottom can be somewhat Unlike regular audio, Atmos
(and the competing Sony 360 Figure 3: More speakers placed at
approximated through signal
lesser angles help eliminate audio
processing, but it will never be standards) uses the concept of
hotspots, especially if the tracks aren’t
perfect unless confirmed speakers audio objects. An Audio-Visual
mixed or processed correctly or if the
are above and below. Receiver (AVR) will automatically audio converters don’t process the
know the number of speakers, surround sound properly. (Source: Zern
their type, and their location Liew/stock.adobe.com)

| 36
 and perform processing on each At one time, Atmos was much too cause sound to reflect on wall
audio object’s spectral makeup, elaborate and expensive for the and ceiling surfaces, appearing
amplitude location, speed, and average audiophile, but it is now to be coming from above or
direction. However, it is not moving into the realm of got-to- behind the listener. The modern-
just audio. The objects contain have for enthusiasts who have the day Tesla Model 3 uses front
metadata that helps an Object space and budget to wow their soundbar technology as part of its
Audio Renderer (OAR) put the friends. It is also rather high on the 15-speaker audio system to tout
object in motion. Of the 128 gee-whiz index. surround and immersive audio
channels, ten are used for ambient capability. Turn off a Model 3’s rear
stems, and the other 118 are If you have already bitten the speakers and engage immersive
available for audio objects. bullet for other surround-sound audio mode with signal processing
technologies, you can get a and reverb to show its capabilities.
Not every channel is a speaker. Dolby Atmos converter and still Those who’ve tried this swear
Channel information corresponds use your existing speakers and sound is coming from behind.
to objects, and object audio can amplifiers. However, you will Feedback is mixed, and many don’t
be processed and combined with want more, including ceiling like the effect. Reviewers praise
other object audio to be directed speakers. Converters will take and criticize the technology, and
to each speaker at the appropriate Dolby 5.1 and convert it with a many reviewers have mentioned
level. It is up to the AVR to process 17-speaker surround-sound 7.4.1 that different types of music work
the signals in real-time using implementation. and don’t work with sound-bar-
metadata to perform real-time style immersive implementations.
mixing and distribution of sound. It is worth noting that an alternative This makes sense because the
approach to surround speakers quality reproduced will depend on
As you can imagine, it is not like is the soundbar. Soundbar the recording engineers’ mix-down
stereo, where you simply place a technology in various forms is techniques. Advances here will
couple of speakers and are then gaining popularity. The obvious mean that accurate above-and-
ready to listen. With Atmos and benefits of cost reduction, setup below immersive-sound technology
many surround-sound and 3D simplicity, lower power, fewer is almost achievable without floor
sound systems, speakers must be cables, and smaller size drive this and ceiling speakers.
placed then calibrated to be an technology forward, even as we
accurate part of the soundscape.
The average home will not use all
drive forward.
Capturing vs.
128 channels. The standard seems Phased-array vertical soundbars rendering
to be a 34-speaker arrangement for have demonstrated their ability
home theater implementations. to emulate a full audio spectrum Immersive video experiences such
with good clarity and separation. as gaming, for the most part, use
Atmos is not brand-new. It was Musicians who use them will tell created environments. These
first used in 2012 in a theater in you that soundbar columns with are 3D structures with surface
Los Angeles for a Disney movie six-inch speakers produce an renderings and assigned physical
premiere. Since then, large 18-inch speaker’s sound clarity properties. Real video swaths can
theaters, IMAX, planetariums, for subwoofer applications. That be captured and digitally stitched
musicals, plays, and other sound should turn a few heads. As a together to make a panoramic view
applications have propelled it result, horizontal soundbars and that includes the above and below
into the de facto standard used soundbar-based hybrid systems imagery.
to capture audio for new movies (including remote speakers) are
and events. Atmos also uses popular for many home theaters An immersive experience such as
ceiling speakers to create a full and studios. a walk through a national park can
hemisphere of sound, making it integrate rich visuals, and audio
easier to process in real-time while The up-and-down enhances this, can be synthesized or created
providing sound from above. and sideways-pointing speakers through a composite of pre-

37 |
recorded clips. They can be audio- Fortunately, you don’t have to product design engineering can
captured in a 3D sound system invent your own 3D audio capture benefit from immersive technology,
and used as part of the immersive for immersive purposes. Audio both video and audio. From a
experience. Like a video controlled leaders like Sennheiser make video perspective, the mechanical
by head tracking, the audio must specialty truly omnidirectional design of complex assemblies can
also be controlled by head tracking. microphones using segmented be virtually constructed, rendered,
For example, facing a babbling axis and digital tools to capture the and examined. An immersively-
brook will sound much different highly directional sound (Figure 4). generated assembly like a jet
than facing away from it, and if the The AMBEO VR Mic contains engine can be constructed, pushed
sound didn’t track, the immersive several susceptible wideband into, and examined to see if
experience would be lacking. microphone elements in a gears and turbines align. A repair
surround-sound configuration. The technician on the other side of the
DearVR microprocessing software world can be shown what to do
can render directional audio to by a factory expert immersed in a
feed a standard surround sound fabricated environment.
configuration.
Even immersive audio can
For this to work, the audio be helpful in engineering
engine needs to know your head applications. An engineering team
orientation and motion. With a designing a car can listen to a
headset, that’s easy by today’s rendered simulation of engine
standards. Head tracking is built and transmission noise. Internal
in for video rendering. But how environmental controls like airflow,
do you create an immersive head vibrations, and oscillations can
position-based audio system using be extracted from a virtual design.
headphones that are limited to two Windows can be designed and
ears? Tiny speakers can be placed tested to eliminate the thumping
in the headphones around the ears oscillations that still occur on new
and mimic the surround-sound cars when we roll the windows
experience. For most applications, down in just the right position at
stereo will suffice, but it will not just the right speed.
be at the same level as authentic
surround sound. In all cases, an immersive
experience includes audio.

Non-entertainment However, not every case requires

surround sound, and simulated
applications surround may be adequate, at least
until someone solves the problem
While most immersive audio and of creating true surround sound in
visual technology will be used a binaural headset.
for entertainment, there are also
professional uses. For example,

Figure 4: Immersive Audio Capture technologies like the Sennheiser AMBEO VR

Mic allow digital audio engines to render soundscapes based on polar magnitude
and direction orientations. Digital summing can create composite audio combining
multiple sources of sound at different distances. (Source: https://en-us.sennheiser.
com/ambeo#segment-teaser-new-24223)

| 38
 The Future of VR Collaboration
By David Freeman for Mouser Electronics

Coming a project here [at Carnegie Mellon

University] called the Panoptic
the images, so that you become a
virtual presence in a reality that’s
improvements Studio, which can capture people partly physical and partly digital.
in immersive in motion from different angles
and create photorealistic avatars A lot of these improvements will
technology will (Figure 1). I’m not sure you could come the same way GPS did: From
make collaboration say they’ve gotten all the way past the defense industry. No one is
the uncanny valley, but the avatars more interested in new and better
much richer don’t look creepy. ways to collaborate and gain
situational awareness than the
Anthony Rowe, Professor People complain about “Zoom military is. They have the funding
of Electrical and Computer fatigue,” which comes from to develop it, and then it will come
Engineering, Carnegie Mellon interacting with people on a to the rest of the world.
University flat screen. But VR brings on a

When I look at virtual reality, I

different type of fatigue, and right
now it’s worse than Zoom fatigue.
VR will eventually
think, wow, that’s the future of It happens because of “motion-to- be compelling
collaboration. But right now we’re
far off from the sort of technology
photon latency,” which is the delay for most people–
between when you physically move
that we’ll want to use on a daily and when that movement is visible maybe too
basis. The main place you see
people using VR every day right
in VR, and because of “the screen compelling
door effect,” which is due to the
now is in research labs like ours. space between image pixels. Some Jacob Loewenstein, vice
people get motion sickness from president and head of business
There are a lot of problems to the delay. It can all be fixed with for immersive-reality software
solve. One is the “uncanny valley” higher resolution, variable-focus company Spatial Systems
problem with avatars, which images, faster frame rates, and
is when an avatar looks more more powerful processing, but all VR collaboration was a highly niche
realistic than a cartoon, but isn’t that is expensive right now. market that ran only on hardware
completely realistic. That just
that few people could afford. Then
looks creepy. Because of that, it Another big improvement will the Oculus Quest (VR headset)
doesn’t work well right now to use be the creation of hi-fi VR made it affordable and accessible.
a live video feed of the user as environments, where it looks Now the numbers are growing like
the avatar. The feeds just aren’t exactly like you’re walking through crazy.
good enough yet, and to avoid a real world. One way to do that
the creepy effect you have to use is to embed high-quality sensors But we ended up realizing that
filters to make it cartoony. There’s in the real world and link them to most people aren’t

39 |
ready yet to work in VR. Most gets better and people get more than addictive use. We already
people still don’t have any VR comfortable with it. And it will require users to be at least 18
hardware yet. So instead of making definitely change with a younger years old, but we see VR apps from
the meetings all or nothing in terms generation that will grow up with other companies that skew toward
of being in VR, which is how most VR, just like the kids who grew up a younger audience, and that could
VR applications are, we let people with PCs became the big users, become a problem.
participate in a web browser if and then the generation that
that’s all they have. Now more than grew up with smartphones. We’re When VR use starts to get heavier,
half of the people who attend VR already seeing a lot of use in we may want to start putting in
meetings with our software do it in education, with several universities design features that encourage
a browser. hosting seminars in VR, including users to get out of VR every so
MIT, the University of Arizona, often and go outside for some
Eventually most people will have and the School of Social Work at physical activity. Although, using
access to some sort of device Columbia University. VR doesn’t mean you’re just sitting
that will let them participate in VR on a couch the whole time. We
(Figure 2). But part of the problem Longer term, the problem won’t already see ways people can stay
is that today most people wouldn’t be getting people to use VR; it will physically active even while they’re
feel comfortable being in VR for be people who become addicted in VR. There are special shoes
a long period of time. For most of to it. We already have superusers and platforms that let people walk
the people who use our software who stay in VR up to eight hours a through a VR space by physically
for VR meetings, they can stay day. That’s the exception right now, using their legs in a walking motion,
creative and productive for maybe but I think we’ll see more and more though they’re staying in the same
30 minutes or so in VR. Even most of it. It will be incumbent both on place in real life. We’ll see a lot
hard-core VR gamers don’t stay in users to develop good practices in more of that.
it for much more than an hour. using VR, and on VR companies to
That will change as hardware encourage reasonable use rather

| 40
 The biggest request we get from The dream virtual meetings will be AR will be useful for anyone
the work world is to give people those that can combine organic who wants to see some of their
avatars that look like their real humans with digital humans, all surroundings while they’re also
selves instead of cartoons. You collaborating. The digital humans looking at virtual images (Figure
can’t do that with streaming video will be attending in virtual reality, 3). In many cases, people will
right now, but it may be possible and the organic humans will see prefer full virtual reality and won’t
to do it with machine learning live scans of them projected in 3D need to see what’s around them.
programs that generate a realistic through AR glasses. The biggest If you want to walk on the moon,
avatar from photos. But there’s still challenge is letting the organic you don’t want to see your real
a lot to figure out in terms of how humans in the room see each surroundings. But if you want to
avatars interact in VR. other’s eyes, because AR glasses collaborate in virtual reality while
tend to obscure the eyes. Eye you’re sitting in a coffee shop,

Augmented reality contact and following others’ gazes

is a big part of communicating.
you might want to keep an eye on
what’s going on around you. You
will surpass don’t want to emerge from virtual
virtual reality as a AR glasses good enough for reality and discover your laptop is
collaboration aren’t available yet. gone.
collaboration tool They could be developed now, but
the purchase and maintenance There will be real sweet spots
Nikhil Balram, CEO of EyeWay costs would be prohibitively high. for both VR and AR, where it
Vision, an augmented-reality The way to get around that is to will be very clear which one you
technology company, and former use the same business model cable prefer. They’re complimentary
visiting professor of vision science TV companies use. You don’t buy technologies, not competing ones.
at the University of California, your cable TV equipment—you rent
Berkeley it with a monthly fee, and the cable
TV company fixes any problems
you have.

i.MX 8QuadMax Application

Multisensory Enablement Board 3.0
Kit (MEK)

mouser.com/nxp-mek mouser.com/bosch-app-board

41 |
| 42