UNIT – 1

The Components The Human: Human Memory – Thinking – Emotion –

Individual Preferences – Psychology and Design of Interactive Systems. The
Computer: Text Entry Devices – Pointing Devices – Display Devices – Devices
for Virtual Reality and 3D interaction – Physical Controls, Sensors and Special
Devices – Memory – Processing and Networks.

Human-Computer Interaction (HCI), human memory refers to the psychological

processes that allow users to encode, store, and retrieve information related to
interactions with computer systems and digital interfaces. Understanding human
memory is crucial in HCI as it helps designers create interfaces that align with users’
cognitive abilities, enhancing user experience and usability.

Input-Output Channels

Human interaction with the world, including computers, involves receiving information through
various sensory channels and sending information through motor actions. The key input-output
channels relevant to human-computer interaction (HCI) are:

1. Visual Channel:
o Reception: The primary means by which humans receive information from the
environment. It involves the eyes detecting light and colors.
o Components:
 Eyes: The organs of vision that capture light and convert it into neural
signals.
 Cornea and Lens: Focus light onto the retina.
 Retina: Contains photoreceptor cells (rods and cones) that detect light and
color.
 Rods and Cones: Rods are sensitive to low light levels, while cones detect
color and are concentrated in the fovea (the area of the retina with the
highest visual acuity).
o Processing:
 Visual Perception: The brain interprets the neural signals from the eyes to
understand the environment.
  Size and Depth Perception: Humans perceive the size and depth of objects
based on various visual cues.
 Brightness and Color Perception: Interpretation of light intensity and
color, essential for designing interfaces that are easily readable and visually
appealing.
2. Auditory Channel:
o Reception: Involves hearing sounds through the ears.
o Components:
 Ears: Capture sound waves and convert them into neural signals.
 Outer Ear: Collects sound waves.
 Middle Ear: Transmits sound waves to the inner ear.
 Inner Ear: Contains the cochlea, which converts sound waves into neural
signals.
o Processing:
 Auditory Perception: The brain interprets the neural signals from the ears
to understand and localize sounds.
 Frequency and Volume: Recognition of pitch and loudness, important for
designing auditory alerts and feedback in interfaces.
3. Haptic Channel:
o Reception: Involves the sense of touch, which allows humans to perceive and
manipulate objects through physical contact.
o Components:
 Skin: Contains various receptors that detect pressure, temperature, and pain.
 Hands and Fingers: Primary organs for tactile interaction.
o Processing:
 Tactile Perception: The brain interprets signals from the skin to understand
the texture, shape, and temperature of objects.
 Proprioception: The sense of body position and movement, crucial for
coordinating physical actions.
4. Movement:
o Input: Humans interact with computer systems through physical movements, such
as using a mouse, typing on a keyboard, or touching a screen.
o Components:
 Limbs and Body: Provide the means for physical interaction.
  Hands: Essential for fine motor skills required in precise actions (e.g.,
clicking a mouse, typing).
o Processing:
 Motor Skills: Coordination of muscles and limbs to perform actions.
 Feedback Mechanisms: Visual, auditory, and haptic feedback help users
understand the outcomes of their actions.

Vision
Vision is one of the most critical input channels in human-computer interaction (HCI). It involves
two main processes: physical reception of visual stimuli and processing/interpretation of this
information by the brain.

Physical Reception

1. The Human Eye:

o Cornea: The eye's outermost layer that focuses incoming light.
o Lens: Adjusts its shape to further focus light onto the retina.
o Retina: The light-sensitive layer at the back of the eye containing photoreceptor
cells (rods and cones).
2. Photoreceptor Cells:
o Rods:
 Located mainly around the periphery of the retina.
 Highly sensitive to low light levels.
 Responsible for night vision and peripheral vision.
o Cones:
 Concentrated in the fovea (the central part of the retina).
 Function best in bright light.
 Responsible for color vision and high-acuity vision (sharp, detailed vision).
3. Processing in the Retina:
o Light hitting the retina is converted into electrical signals by the photoreceptors.
o These signals are then transmitted via the optic nerve to the brain for further
processing.

Visual Processing

1. Visual Perception:
 o The brain interprets signals from the eyes to construct an understanding of the
visual environment.
o Key aspects of visual perception include size, depth, brightness, and color.
2. Size and Depth Perception:
o Size Perception: The brain estimates the size of objects based on visual cues and
prior knowledge.
o Depth Perception: The ability to judge the distance of objects using visual cues
such as:
 Binocular Cues: Require both eyes, e.g., stereopsis (each eye sees a slightly
different image).
 Monocular Cues: Work with one eye, e.g., relative size, interposition
(overlapping objects), linear perspective (convergence of parallel lines), and
texture gradient.
3. Brightness Perception:
o The ability to perceive different levels of light intensity.
o Adaptation: The eye can adjust to various lighting conditions, enabling vision in
both bright and dim environments.
4. Color Perception:
o The ability to distinguish different wavelengths of light, interpreted as colors.
o Trichromatic Theory: There are three types of cones, each sensitive to different
wavelengths (red, green, blue).
o Color Consistency: The brain maintains stable color perception under varying
lighting conditions.

Implications for HCI

1. Interface Design:
o Use clear, high-contrast visuals to improve readability and reduce eye strain.
o Ensure sufficient spacing between elements to prevent visual clutter.
o Use color effectively to convey information but avoid relying solely on color to
differentiate elements (consider colorblind users).
2. Visual Hierarchy:
o Organize content to guide the user's attention naturally, using techniques such as
size, color, and positioning.
3. Accessibility:
 o Design for users with visual impairments by providing alternative text for images
and ensuring compatibility with screen readers.
o Consider font size, text color, and background color to ensure readability.
4. Feedback and Response:
o Provide visual feedback for user actions (e.g., highlighting selected items, progress
indicators).

Understanding the principles of vision and visual perception allows designers to create interfaces
that are more intuitive, accessible, and effective in conveying information. By leveraging these
principles, HCI can enhance user experience and interaction quality.

Human memory:
Human memory in HCI can be classified into several types:

A model of the structure of memory

1. Sensory Memory: The initial stage where information from the environment
is briefly registered in sensory systems (e.g., visual or auditory stimuli). It
provides a brief buffer that allows the perception of the world to be continuous.
A sensory memory exists for each sensory channel:
 iconic memory for visual stimuli,
 echoic memory for aural stimuli
 haptic memory for touch.
These memories are constantly overwritten by new information coming in on these
channels.
Information is passed from sensory memory into short-term memory by attention,
thereby filtering the stimuli to only those which are of interest at a given time.
Attention is the concentration of the mind on one out of a number of competing
stimuli or thoughts. It is clear that we are able to focus our attention selectively,
choosing to attend to one thing rather than another. This is due to the limited capacity
of our sensory and mental processes. If we did not selectively attend to the stimuli
coming into our senses, we would be overloaded. We can choose which stimuli to
attend to, and this choice is governed to an extent by our arousal, our level of interest
or need.
2. Short-term Memory (STM) or Working Memory: Information that is
consciously processed and temporarily held in the mind. Working memory is
limited in capacity and duration, typically holding information for a few
seconds to a minute. In HCI, this is essential for tasks requiring immediate
attention and processing.
3. Long-term Memory (LTM): The repository of information that is more
permanently stored. Information from short-term memory is transferred to
long-term memory through processes like rehearsal and elaboration. In the
context of HCI, designing interfaces that facilitate the transfer of information
from working memory to long-term memory is important for users to
remember how to use the system effectively over time.
There are two types of long-term memory:
1. Episodic memory represents our memory of events and experiences in a serial
form. It is from this memory that we can reconstruct the actual events that took
place at a given point in our lives.
2. Semantic memory, is a structured record of facts, concepts and skills that we
have acquired. The information in semantic memory is derived from that in
our episodic memory, such that we can learn new facts or concepts from our
experiences. Semantic memory is structured in some way to allow access to
information, representation of relationships between pieces of information,
and inference
There are three main activities related to long-term memory:
1. storage or remembering of information,
2. forgetting and
3. information retrieval.
Storage or remembering of information :
There are two main theories of forgetting: decay and interference.
1. Decay Theory :
The theory suggests that the information held in long-term memory may eventually
be forgotten.
Ebbinghaus concluded from his experiments with nonsense syllables that information
in memory decayed logarithmically, that is that it was lost rapidly to begin with, and
then more slowly.
Jost’s law, which follows from this, states that if two memory traces are equally
strong at a given time the older one will be more durable.
2. Interference Theory :
The theory is that information is lost from memory through interference. If we
acquire new information it causes the loss of old information. This is termed
retroactive interference.
A common example of this is the fact that if you change telephone numbers, learning
your new number makes it more difficult to remember your old number. This is
because the new association masks the old.
Sometimes the old memory trace breaks through and interferes with new information.
This is called proactive inhibition.
An example of this is when you find yourself driving to your old house rather than
your new one.
Forgetting :
Forgetting is also affected by emotional factors. In experiments, subjects
given emotive words and non-emotive words found the former harder to remember
in the short term but easier in the long term.
Indeed, this observation tallies with our experience of selective memory. We tend to
remember positive information rather than negative (hence nostalgia for the ‘good
old days’), and highly emotive events rather than mundane.
Information Retrieval:
There are two types of information retrieval:
Recall: In recall the information is reproduced from memory. Recall can be assisted
by the provision of retrieval cues, which enable the subject quickly to access the
information in memory. One such cue is the use of categories. In an experiment
subjects were asked to recall lists of words, some of which were organized into
categories and some of which were randomly organized. The words that were related
to a category were easier to recall than the others
Recognition: In recognition, the presentation of the information provides the
knowledge that the information has been seen before. Recognition is the less complex
cognitive activity since the information is provided as a cue.

Thinking:
Animals receive and store information, there is little evidence to suggest that they
can use it in quite the same way as humans. Similarly, artificial intelligence has
produced machines which can see (albeit in a limited way)and store information. But
their ability to use that information is limited to small domains.
Humans, on the other hand, are able to use information to reason and solve problems,
and indeed do these activities when the information is partial or unavailable. Human
thought is conscious and self-aware: while we may not always be able to identify the
processes we use, we can identify the products of these processes, our thoughts. This
is done via Thinking.
Thinking can require different amounts of knowledge. Some thinking activities are
very directed and the knowledge required is constrained. Others require vast
amounts of knowledge from different domains. For example, performing a
subtraction calculation requires a relatively small amount of knowledge, from a
constrained domain, whereas understanding newspaper headlines demands
knowledge of politics, social structures, public figures and world events.
There are two categories of thinking:
  reasoning
 problem solving.
Reasoning is the process by which we use the knowledge we have to draw
conclusions or infer something new about the domain of interest. There are a
number of different types of reasoning: deductive, inductive and abductive.
Deductive reasoning (If-then condition)
Deductive reasoning derives the logically necessary conclusion from the given
premises.
Example 1: (Valid Deduction)
If it is Friday then she will go to work
 It is Friday
 Therefore she will go to work.
Example 2: (In Valid Deduction)
If it is raining then the ground is dry
 It is raining
 Therefore the ground is dry.
Is a perfectly a valid deduction, even though it conflicts with our knowledge of what
is true in the world. Deductive reasoning is therefore often misapplied.

Inductive reasoning
Induction is generalizing from cases we have seen to infer information about cases
we have not seen.
For example, if every elephant we have ever seen has a trunk, we infer that all
elephants have trunks. Of course, this inference is unreliable and cannot be proved to
be true; it can only be proved to be false. We can disprove the inference simply by
producing an elephant without a trunk. However, we can never prove it true because,
no matter how many elephants with trunks we have seen or are known to exist, the
next one we see may be trunk less. The best that we can do is gather evidence to
support our inductive inference.
Abductive reasoning
The third type of reasoning is abduction. Abduction reasons from a fact to the action
or state that caused it. This is the method we use to derive explanations for the events
we observe.
For example, suppose we know that Sam always drives too fast when she has been
drinking. If we see Sam driving too fast we may infer that she has been drinking. Of
course, this too is unreliable since there may be another reason why she is driving
fast: she may have been called to an emergency, for example. In spite of its
unreliability, it is clear that people do infer explanations in this way, and hold onto
them until they have evidence to support an alternative theory or explanation. This
can lead to problems in using interactive systems. If an event always follows an
action, the user will infer that the event is caused by the action unless evidence to the
contrary is made available. If, in fact, the event and the action are unrelated, confusion
and even error often result.

Problem solving
If reasoning is a means of inferring new information from what is already known,
problem solving is the process of finding a solution to an unfamiliar task, using
the knowledge we have. Human problem solving is characterized by the ability to
adapt the information we have to deal with new situations. However, often solutions
seem to be original and creative. There are a number of different views of how
people solve problems.

Gestalt theory
Gestalt psychologists problem solving is a matter of reproducing known responses or
trial and error.
Problem solving is both productive and reproductive.
Reproductive problem solving draws on previous experience as the behaviorists
claimed.
Productive problem solving involves insight and restructuring of the problem.
Indeed, reproductive problem solving could be a hindrance to finding a solution,
since a person may ‘fixate’ on the known aspects of the problem and so be unable to
see novel interpretations that might lead to a solution.

Problem space theory

Newell and Simon proposed that problem solving centers on the problem space. The
problem space comprises problem states, and problem solving involves generating
these states using legal state transition operators. The problem has an initial state and
a goal state and people use the operators to move from the former to the latter.

Such problem spaces may be huge, and so heuristics are employed to select
appropriate operators to reach the goal. One such heuristic is means–ends analysis.
In means–ends analysis the initial state is compared with the goal state and an
operator chosen to reduce the difference between the two.
For example, imagine you are reorganizing your office and you want to move your
desk from the north wall of the room to the window.
Your initial state is that the desk is at the north wall.
The goal state is that the desk is by the window.
The main difference between these two is the location of your desk.
You have a number of operators which you can apply to moving things: you can carry
them or push them or drag them, etc. However, you know that to carry something it
must be light and that your desk is heavy. You therefore have a new subgoal: to make
the desk light. Your operators for this may involve removing drawers, and so on.

Analogy in problem solving

Here, people solve novel problems. One suggestion is that this is done by mapping
knowledge relating to a similar known domain to the new problem – called
analogical mapping.
Similarities between the known domain and the new one are noted and operators from
the known domain are transferred to the new one.
Example:
A doctor is treating a malignant tumor. In order to destroy it he needs to blast it with
high-intensity rays. However, these will also destroy the healthy tissue surrounding
the tumor. If he lessens the rays’ intensity the tumor will remain. How does he destroy
the tumor?
The solution to this problem is to fire low-intensity rays from different directions
converging on the tumor. That way, the healthy tissue receives harmless low intensity
rays while the tumor receives the rays combined, making a high-intensity dose. The
investigators found that only 10% of subjects reached this solution without help.

Emotions:
Our emotional response to situations affects how we perform.
For example, positive emotions enable us to think more creatively, to solve complex
problems, whereas negative emotion pushes us into narrow, focussed thinking. A
problem that may be easy to solve when we are relaxed, will become difficult if we
are frustrated or afraid.
INDIVIDUAL DIFFERENCES :
We have made the assumption that everyone has similar capabilities and limitations
and that we can therefore make generalizations. To an extent this is true: the
psychological principles and properties that we have discussed apply to the majority
of people.
Not with standing this, we should remember that, although we share processes in
common, humans, and therefore users, are not all the same.
We should be aware of individual differences so that we can account for them as far
as possible within our designs. These differences may be long term, such as sex,
physical capabilities and intellectual capabilities. Others are shorter term and
include the effect of stress or fatigue on the user. Still others change through time,
such as age.
These differences should be taken into account in our designs. It is useful to consider,
for any design decision, if there are likely to be users within the target group who will
be adversely affected by our decision. At the extremes a decision may exclude a
section of the user population.

A typical computer system

Consider a typical computer setup as shown in the diagram. There is the computer
‘box’ itself, a keyboard, a mouse and a color screen. The screen layout is shown
alongside it. If we examine the interface, we can see how its various characteristics
are related to the devices used. The details of the interface itself, its underlying
principles and design. As we shall see there are variants on these basic devices. Some
of this variation is driven by different hardware configurations: desktop use, laptop
computers, PDAs (personal digital assistants). Partly the diversity of devices reflects
the fact that there are many different types of data that may have to be entered into
and obtained from a system, and there are also many different types of user, each
with their own unique requirements.
Levels of interaction – batch processing
In the early days of computing, information was entered into the computer in a large
mass – batch data entry. There was minimal interaction with the machine: the user
would simply dump a pile of punched cards onto a reader, press the start button, and
then return a few hours later. This still continues today although now with pre-
prepared electronic files or possibly machine-read forms. It is clearly the most
appropriate mode for certain kinds of application, for example printing pay checks or
entering the results from a questionnaire.
With batch processing the interactions take place over hours or days. In contrast the
typical desktop computer system has interactions taking seconds or fractions of a
second (or with slow web pages sometimes minutes!). The field of Human–Computer
Interaction largely grew due to this change in interactive pace. It is easy to assume
that faster means better, but some of the paper-based technology suggests that
sometimes slower paced interaction may be better.
Richer interaction – everywhere, everywhen
Computers are coming out of the box! Information appliances are putting internet
access or dedicated systems onto the fridge, microwave and washing machine: to
automate shopping, give you email in your kitchen or simply call for maintenance
when needed. We carry with us WAP phones and smartcards, have security systems
that monitor us and web cams that show our homes to the world.

TEXT ENTRY DEVICES

A text entry interface or text entry device is an interface that is used to enter text
information in an electronic device. A commonly used device is a mechanical
computer keyboard. Most laptop computers have an integrated mechanical keyboard,
and desktop computers are usually operated primarily using a keyboard and mouse.
Devices such as smartphones and tablets mean that interfaces such as virtual
keyboards and voice recognition are becoming more popular as text entry systems.

Computer keyboard text entry

A computer keyboard is a typewriter-style device which uses an arrangement of
buttons or keys to act as mechanical levers or electronic switches. Each button
typically represents one character, but some symbols may only be accessible via a
combination of buttons. The layout of the keyboard is like that of a traditional
typewriter, although there are some additional keys provided for performing further
functions. There are a number of different keyboard layouts available:
 QWERTY is the standard English-language keyboard layout, as the first six
keys on the row of letters are Q, W, E, R, T and Y. Other keyboards layouts
include AZERTY and Dvorak.
 The AZERTY keyboard is a variation of the standard QWERTY keyboard
adapted for French-language input. The AZERTY layout is optimised for
French language use. Some QWERTY and AZERTY keyboards have
explicitly labelled keycaps for diacritics (accents) but, given the appropriate
keyboard mapping, any keyboard may be used where these letters are needed.
 The Dvorak keyboard is designed so that the middle row of keys includes the
most common letters, with the goal of allowing greater efficiency and comfort
while typing.

These keyboards usually contains letters, numbers, currency sign(s), punctuation,

function and control keys, arrow keys, a keypad, and may include a wristpad.

ITU-T text entry (phone keyboard)

With the popularity of text messaging, text entry with mobile phones has gained use.
Each key contains multiple characters, and these are reached through multiple key
presses. This is frequently used in conjunction with predictive text (also known as
T9) entry. Although once popular, this system has been mostly displaced with the
widespread use of touchscreens on smartphones and is now mostly found on budget
feature phones.

Virtual keyboards
Virtual keyboards are similar to mechanical keyboards, but do not make use of
physical keys. These may be implemented on systems using a screen or projected
onto a surface. The individual letters may be selected by touching them as on a touch
screen or surface, or by clicking on them with a classical pointing device (a mouse or
touchpad), like in the case of virtual computer keyboards. Multi-touch screens even
support virtual chorded keyboards.

On-screen keyboards can be used to type and enter data without using the physical
keyboard. An on-screen keyboard may contain all the standard keys including all
letters, numbers, symbols, and system keys like Home, End, Insert, Page Up and Page
Down, Ctrl, Alt, Caps, and Shift, and can even extend the set of characters available
by simulating alternative layouts. These keys can be selected using the mouse or
another pointing device, or a single key or small group of keys can be used to cycle
through the keys on the screen. The on-screen keyboard is the most common type of
virtual keyboard. The accuracy of this keyboard depends only on hitting the right key.
The main purpose of an on-screen keyboard is to provide an alternative mechanism
for disabled users who cannot use a physical keyboard, or to provide text input on
devices lacking a physical keyboard, such as smartphones and tablets.

Virtual keyboards also allow users to enter characters not available on their physical
keyboard, enabling support for a number of languages with only one hardware
keyboard.

Devices such as smartphones and tablets come with touchscreens and make use of
virtual keyboards. Keyboards vary between operating systems but many third-party
applications are available to replace the system keyboard. Mobile virtual keyboards
are often used alongside predictive text.

Voice recognition
Voice recognition is a system that allows using one's voice in order to send messages,
make phone calls and more. The most powerful voice entry systems can recognize
thousands of words. It usually requires the speaker to speak slowly, distinctly and to
separate each word with a short pause. This system can replace or supplement other
input devices such as keyboards and different pointing devices. The software has
been developed to provide a fast method of writing without using a keyboard and can
help people with various disabilities. The system works by analysing sounds and
converting them to text. It knows how the language is usually spoken and decides
what the speaker is most probably saying. The most powerful systems should
recognise around 95% of clear speech correctly. Several voice recognition
applications are available. Some of the most well-known systems are Apple Inc.'s
Siri and Cortana which is developed by Microsoft. Many voice-recognition programs
offer the ability to start and control programs through spoken commands.

Individual letter selection

This is commonly used on electronic systems where text entry is not important.
Examples include television channel naming and text entry in video game systems
such as the Sony PSP. Usually directional input devices (arrow keys, joysticks) are
used to highlight a letter or number, then an enter key used to select the letter.

Handwriting recognition
Handwriting recognition (or HWR) is the ability of a computer to receive and
interpret intelligible handwritten input from sources such as paper documents,
photographs, touch-screens and other devices. It allows users to use a touch screen
device much like a notepad on which they can write without the need for a keyboard,
and the software finds the closest match in its symbol database to replace the
handwritten letters with. Handwriting recognition primarily uses either optical
character recognition which uses an optical scanner to scan the words written by the
user to determine the best suitable match, or by using a pen-based computer interface
to track the movements of the tip of the pen as the user is writing.

Light pen
Photo of the Hypertext Editing System (HES) console in use at Brown University,
circa October 1969. The photo shows HES on an IBM 2250 Mod 4 display station,
including lightpen and programmed function keyboard, channel coupled to Brown's
IBM 360 mainframe.
A light pen is a computer input device used in conjunction with a computer's CRT
display. It is used to select a displayed menu item. A light pen can also allow users
draw on the screen with great positional accuracy. It consists of a photocell and an
optical system placed in a small tube. When the tip of a light pen is moved over the
monitor screen and pen button is pressed, its photocell sensing element detects the
screen location and sends the corresponding signal to the CPU. The first light pen
was created around 1952 as part of the Whirlwind project at MIT. Because the user
was required to hold his/her arm in front of the screen for long periods of time, the
light pen fell out of use as a general purpose input device. Nevertheless, modern touch
screens on desktop computers require similar ergonomic behaviour from users.

Touch screen
A touchscreen (or touch screen) is a both input and output device and normally
layered on an electronic visual display of an information processing system. A user
can give input or control the information processing system through simple or multi-
touch gestures by touching the screen with a special stylus or one or more fingers.[1]

Digital pen
A digital pen is an input device which captures the handwriting or brush strokes of a
user, converts handwritten analog information created using "pen and paper" into
digital data, enabling the data to be utilized in various applications. For example, the
writing data can be digitized and uploaded to a computer and displayed on its
monitor. The data can then be interpreted by handwriting software (OCR) to allow
the digital pen to act as a text entry interface and be used in different applications or
just as graphics.

A digital pen is generally larger and has more features than a stylus. Digital pens
typically contain internal electronics and have features such as touch sensitivity, input
buttons, memory, writing data transmission capabilities, and electronic erasers

Graphics tablet
A graphics tablet or digitizer is a computer input device that enables a user to hand-
draw images, animations and graphics, similar to the way a person draws images with
a pencil and paper, typically using a stylus. These tablets may also be used to capture
data or handwritten signatures. It can also be used to trace an image from a piece of
paper which is taped or otherwise secured to the surface. Capturing data in this way,
by tracing or entering the corners of linear poly-lines or shapes, is called digitizing.
The image is displayed on the computer monitor, although some graphics tablets also
have a screen.

Some tablets are intended as a replacement for the mouse as the primary pointing and
navigation device.

Graphics tablets can be use with handwriting recognition software to input text, using
the graphics tablet to write on the handwriting recognition detects the letters and
converts it to digital information.

Punched card input

Main article: Punched card input/output
One of the earliest text entry interfaces was the punched card reader. Text (which
could be data or source code) was punched off-line using a keypunch machine. Most
early computers used punched cards as their main input medium and usually also had
an online card puncher for output.

military) and for those suffering RSI. This is discussed in greater detail in Chapter
10, but we can see that it offers three possibilities. The first is as an alternative text
entry device to replace the keyboard within an environment and using software
originally designed for keyboard use. The second is to redesign a system, taking full
advantage of the benefits of the technique whilst minimizing the potential problems.
Finally, it can be used in areas where keyboard-based input is impractical or
impossible. It is in the latter, more radical areas that speech technology is currently
achieving success.

POSITIONING, POINTING DEVICES

Pointing devices allow the user to point, position and select items, either directly or
by manipulating a pointer on the screen. Many pointing devices can also be used for
free-hand drawing although the skill of drawing with a mouse is very different from
using a pencil. The mouse is still most common for desktop computers, but is facing
challenges as laptop and hand- held computing increase their market share. Indeed,
these words are being typed on a laptop with a touchpad and no mouse.

Mouse
A mouse is computer hardware used for navigation through files displayed on the
computer screen. The standard mouse has two buttons and a scroll ball. The two
buttons are used to select actions on the screen by clicking on either of the buttons
for different functions. Most computer user interfaces allow users to customise the
functions of the two buttons on the mouse. One of the advantages of using a mouse
is its ability to simplify navigation through files and folders on a computer by simply
pointing and clicking. The main disadvantage and limitation of using a mouse is its
space requirement. A mouse cannot be used in an area without ample physical space
to drag and scroll.

Foot mouse
A foot mouse is a specialised type of a mouse that is operated by the feet. Users with
disabilities mainly use the foot mouse. The design of a foot mouse depicts a footpad
with several buttons that serve different purposes. The footpad also has a navigation
ball that is rolled to the foot to move the cursor on a computer screen. There is also
afoot mouse for computer experts who prefer using their feet to free their hands for
higher performance in different activities on their computers.

The advantage of using a foot mouse is that it enables the disabled people to enjoy
the easier navigation on their computers. Its disadvantage is that it takes a lot of time
for the user to master the manipulation of a foot mouse. Its main limitation is its
requirement for larger working spaces because it is larger than a normal mouse .

Joystick
Joysticks are input devices used to control characters in computer games. A joystick
comprises of a control column that is attached to a base that allows easy rotation of
the stick. The stick normally has other support buttons used for different purposes
like executing moves in the game. Joysticks work under the principle of motion
whereby the characters on the screen move in a similar manner as the motion
executed by the user on the joystick. The advantage of using a joystick is that it acts
as a central control centre for gaming users. The buttons are attached to a stick,
making it easy for the user to execute moves easily. The disadvantage of using a
joystick that becomes tiring because the user has to keep moving his or her hand. Its
limit is the availability of working space.

Trackball
A trackball operates like a mouse. The standard trackball is made of a socket with a
rolling ball and several buttons. The trackball looks like an inverted mouse, and it is
very easy to navigate using the device. A trackball user utilises his palm and fingers
to get through files on a computer screen. A person using a computer rolls the ball,
and the cursor on the screen move to the required place while clicking on the buttons
where required. The main advantage of using a trackball over a mouse is its ability
to eliminate the large working space requirement. A trackball operates on a static
point; hence, it does not require large working spaces. Its disadvantage is it requires
more accuracy and expertise in navigation because a slight touch on the ball moves
the cursor.
Touch screens
Touch screens are modern interactive screens that act both as input and output
devices. Touch screens are specialised to allow users to control computers by
touching the screen to execute commands. Some touch screens use a special stylus,
whereas others respond to the touch of a finger. Technological growth has seen the
development of multi-touch touch screens that allow users to use several fingers
simultaneously to operate computers. Touch screens operate through screen sensors
that manipulate touch signals through special firmware. The main advantage of using
touch screen technology is that it eliminates the need for other input hardware
devices. There is no need for the user to purchase a keyboard or a mouse if a computer
has a touch screen. The disadvantage of using touch screens is that they are quite
fragile, and they can easily break and lose their sensitivity.

Light pen
A light pen is an interactive computer input hardware that utilises light and the CRT
display technology. A wand with light-sensitivity is used to input data through a CRT
display screen. The advantage of using a light pen is that it allows users to enter data
freely into the computer. It is also easy to use. The limitation is that computers may
fail to recognise some patterns when a user is entering data.
Touchpad
A touchpad is an alternative to a mouse in laptops and other computer devices. It is
an input device that controls a cursor on a computer screen. Computer users operating
a touchpad move their fingers across the touch-sensitive pad, and the movement
translates onto the output screen. The advantage of using a touchpad is the
elimination of the requirement for large spaces to operate a mouse. The main
disadvantage is that just like a mouse, users have to keep shifting the position of their
fingers to scroll down long files.

Display Devices :
Virtually all computer displays are based on some sort of bitmap. That is the display
is made of vast numbers of colored dots or pixels in a rectangular grid. These pixels
may be limited to black and white, in grayscale, or full color.
Cathode ray tube
The cathode ray tube is the television-like computer screen still most common as we
write this, but rapidly being displaced by flat LCD screens. It works in a similar way
to a standard television screen. A stream of electrons is emitted from an electron gun,
which is then focused and directed by magnetic fields. As the beam hits the phosphor-
coated screen, the phosphor is excited by the electrons and glows (see Figure 2.10).
The electron beam is scanned from left to right, and then flicked back to rescan the
next line, from top to bottom. This is repeated, at about 30 Hz (that is, 30 times a
second), per frame, although higher scan rates are sometimes used to reduce the
flicker on the screen. Another way of reducing flicker is to use interlacing, in which
the odd lines on the screen are all scanned first, followed by the even lines. Using a
high-persistence phosphor, which glows for a longer time when excited, also reduces
flicker, but causes image smearing especially if there is significant animation. Black
and white screens are able to display grayscale by varying the intensity of the electron
beam; color is achieved using more complex means. Three electron guns are used,
one each to hit red, green and blue phosphors. Combining these colors can produce
many others, including white, when they are all fully on. These three phosphor dots
are focussed to make a single point using a shadow mask, which is imprecise and
gives color screens a lower resolution than equivalent monochrome screens.
An alternative approach to producing color on the screen is to use beam penetration.
A special phosphor glows a different color depending on the intensity of the beam
hitting it.
The CRT is a cheap display device and has fast enough response times for rapid
animation coupled with a high color capability. Note that animation does not
necessarily mean little creatures and figures running about on the screen, but refers
in a more general sense to the use of motion in displays: moving the cursor, opening
windows, indicating processor-intensive calculations, or whatever. As screen
resolution increases, however, the price rises. Because of the electron gun and
focusing components behind the screen, CRTs are fairly bulky, though recent
innovations have led to flatter displays in which the electron gun is not placed so that
it fires directly at the screen, but fires parallel to the screen plane with the resulting
beam bent through 90 degrees to hit the screen.

Figure 2.10 CRT screen

Liquid crystal display

LCD is a flat display technology, stands for "Liquid Crystal Display," which is
generally used in computer monitors, instrument panels, cell phones, digital
cameras, TVs, laptops, tablets, and calculators. It is a thin display device that offers
support for large resolutions and better picture quality. The older CRT display
technology has replaced by LCDs, and new display technologies like OLEDs have
started to replace LCDs. An LCD display is most commonly found with Dell laptop
computers and is available as an active-matrix, passive-matrix, or dual-scan display.
The picture is an example of an LCD computer monitor.

The LCD displays are not only different in terms of heavy than CRT monitors; even
the process of working of them is also different. An LCD contains a backlight rather
than the firing electrons at a glass screen, which offers light to individual pixels
arranged in a rectangular grid. All pixels have a sub-pixel, red, green, and blue,
which can be turned on or off. The display appears black if all of a pixel's sub-pixels
are turned off and appears white if all the sub-pixels are turned on 100%. The
millions of color combinations can be possible with the help of adjusting the
individual levels of red, green, and blue light.

As compared to CRT technology, LCD consumed much less power and allowed
displays to be much thinner that also made them very less heavy. Instead of emitting
light, LCDs work on the principle of blocking light. In an LCD, where an LED ejects
light, the liquid crystals produce a picture with the help of using a backlight. Also,
while comparing with gas-display and LED displays, LCDs consume less energy.

The CRT monitors and TVs have a refresh rate, but LCD screens do not have a
refresh rate. If you feel a problem with eye strain with the CRT monitor, you might
need to change the monitor's refresh rate setting on your CRT screen. But with the
new LCD screen, you do not need to adjust the refresh rate setting. Some LCD
computer monitors provide support for VGA cables, and most have a connection for
HDMI and DVI cables. But offering support for VGA cables is much less common.

Special displays
There are a number of other display technologies used in markets. The gas plasma
display is used in large screens. The random scan display, also known as the directed
beam refresh, or vector display, works differently from the bitmap display, also
known as raster scan. Instead of scanning the whole screen sequentially and
horizontally, the random scan draws the lines to be displayed directly. By updating
the screen at at least 30 Hz to reduce flicker, the direct drawing of lines at any angle
means that jaggies are not created, and higher resolutions are possible, up to
4096 × 4096 pixels. Color on such displays is achieved using beam penetration
technology, and is generally of a poorer quality. Eyestrain and fatigue are still a
problem, and these displays are more expensive than raster scan ones.
The direct view storage tube is used extensively as the display for an analog storage
oscilloscope, which is probably the only place that these displays are used in any
great numbers. They are similar in operation to the random scan CRT but the image
is maintained by flood guns which have the advantage of producing a stable display
with no flicker. The screen image can be incrementally updated but not selectively
erased; removing items has to be done by redrawing the new image on a completely
erased screen. The screens have a high resolution, typically about 4096 × 3120
pixels, but suffer from low contrast, low brightness and a difficulty in displaying
color.

Large displays and situated displays

Displays are no longer just things you have on your desktop or laptop. There are
several types of large screen display. Some use gas plasma technology to create large
flat bitmap displays. These behave just like a normal screen except they are big and
usually have the HDTV (high definition television) wide screen format which has
an aspect ratio of 16:9 instead of the 4:3 on traditional TV and monitors.
Where very large screen areas are required, several smaller screens, either LCD or
CRT, can be placed together in a video wall. These can display separate images, or
a single TV or computer image can be split up by software or hardware so that each
screen displays a portion of the whole and the result is an enormous image. This is
the technique often used in large concerts to display the artists or video images
during the performance.
Possibly the large display you are most likely to have encountered is some sort of
projector. There are two variants of these. In very large lecture theatres, especially
older ones, you see projectors with large red, green and blue lenses. These each
scan light across the screen to build a full color image. In smaller lecture theatres
and in small meetings you are likely to see LCD projectors. Usually the size of a
large book, these are like ordinary slide projectors except that where the slide would
be there is a small LCD screen instead. The light from the projector passes through
the tiny screen and is then focussed by the lens onto the screen.
The disadvantage of projected displays is that the presenter’s shadow can often
fall across the screen. Sometimes this is avoided in fixed lecture halls by using back
projection.
In a small room behind the screen of the lecture theatre there is a projector producing
a right/left reversed image. The screen itself is a semi-frosted glass so that the image
projected on the back can be seen in the lecture theatre. Because there are limits on
how wide an angle the projector can manage without distortion, the size of the image
is limited by the depth of the projection room behind, so these are less heavily used
than front projection.
Digital paper
A new form of ‘display’ that is still in its infancy is the various forms of digital
paper. These are thin flexible materials that can be written to electronically, just like
a computer screen, but which keep their contents even when removed from any
electrical supply.
There are various technologies being investigated for this. One involves the whole
surface being covered with tiny spheres, black one side, white the other. Electronics
embedded into the material allow each tiny sphere to be rotated to make it black or
white. When the electronic signal is removed the ball stays in its last orientation. A
different technique has tiny tubes laid side by side. In each tube is light-absorbing
liquid and a small reflective sphere. The sphere can be made to move to the top
surface or away from it making the pixel white or black. Again the sphere stays in
its last position once the electronic signal is removed.
Probably the first uses of these will be for large banners that can be reprogrammed
or slowly animated. This is an ideal application, as it does not require very rapid
updates and does not require the pixels to be small. As the technology matures, the
aim is to have programmable sheets of paper that you attach to your computer to get
a ‘soft’ printout that can later be changed. Perhaps one day you may be able to have
a ‘soft’ book that appears just like a current book with soft pages that can be turned
and skimmed, but where the contents and cover can be changed when you decide to
download a new book from the net

DEVICES FOR VIRTUAL REALITY AND 3D INTERACTION

Positioning in 3D space
Virtual reality systems present a 3D virtual world. Users need to navigate through
these spaces and manipulate the virtual objects they find there. Navigation is not
simply a matter of moving to a particular location, but also of choosing a particular
orientation. In addition, when you grab an object in real space, you don’t simply
move it around, but also twist and turn it, for example when opening a door. Thus
the move from mice to 3D devices usually involves a change from two degrees of
freedom to six degrees of freedom, not just three.
Cockpit and virtual controls
Helicopter and aircraft pilots already have to navigate in real space. Many arcade
games and also more serious applications use controls modeled on an aircraft cockpit
to ‘fly’ through virtual space. However, helicopter pilots are very skilled and it takes
a lot of practice for users to be able to work easily in such environments.

In many PC games and desktop virtual reality (where the output is shown on an
ordinary computer screen), the controls are themselves virtual. This may be a
simulated form of the cockpit controls or more prosaic up/down left/right buttons.
The user manipulates these virtual controls using an ordinary mouse (or other 2D
device). Note that this means there are two levels of indirection. It is a tribute to the
flexibility of the human mind that people can not only use such systems but also
rapidly become proficient.
The 3D mouse
There are a variety of devices that act as 3D versions of a mouse. Rather than just
moving the mouse on a tabletop, you can pick it up, move it in three dimensions,
rotate the mouse and tip it forward and backward. The 3D mouse has a full six
degrees of freedom as its position can be tracked (three degrees), and also its
up/down angle (called pitch), its left/right orientation (called yaw) and the amount it
is twisted about its own axis (called roll). Various sensors are used to track the mouse
position and orientation: magnetic coils, ultrasound or even mechanical joints where
the mouse is mounted rather like an angle-poise lamp.
With the 3D mouse, and indeed most 3D positioning devices, users may experience
strain from having to hold the mouse in the air for a long period. Putting the 3D
mouse down may even be treated as an action in the virtual environment, that is
taking a nose dive.

Dataglove
The dataglove is a 3D input device. Consisting of a lycra glove with optical fibers
laid along the fingers, it detects the joint angles of the fingers and thumb. As the
fingers are bent, the fiber optic cable bends too; increasing bend causes more light
to leak from the fiber, and the reduction in intensity is detected by the glove and
related to the degree of bend in the joint. Attached to the top of the glove are two
sensors that use ultrasound to determine 3D positional information as well as the
angle of roll, that is the degree of wrist rotation. Such rich multi-dimensional input
is currently a solution in search of a problem, in that most of the applications in use
do not require such a comprehensive form of data input, whilst those that do cannot
afford it.
Virtual reality helmets
The helmets or goggles worn in some VR systems have two purposes: (i) they
display the 3D world to each eye and (ii) they allow the user’s head position to be
tracked.
The head tracking is used primarily to feed into the output side. As the user’s head
moves around the user ought to see different parts of the scene. However, some
systems also use the user’s head direction to determine the direction of movement
within the space and even which objects to manipulate (rather like the eyegaze
systems). You can think of this rather like leading a horse in reverse. If you want a
horse to go in a particular direction, you use the reins to pull its head in the desired
direction and the horse follows its head.

Whole-body tracking
Some VR systems aim to be immersive, that is to make the users feel as if they are
really in the virtual world. In the real world it is possible (although not usually wise)
to walk without looking in the direction you are going. If you are driving down the
road and glance at something on the roadside you do not want the car to do a sudden
90-degree turn! Some VR systems therefore attempt to track different kinds of body
movement. The user can literally surf through virtual space. In the extreme the
movement of the whole body may be tracked using devices similar to the dataglove,
or using image-processing techniques.