Design of interactive software systems Adriana Guran
HCI BASICS
Lecture 1
Agenda
The Human
The Computer
The Interaction
THE HUMAN
HCI Basics
The Human
Information i/o
visual, auditory, haptic, movement
Information stored in memory
sensory, short-term, long-term
Information processed and applied
reasoning, problem solving, skill, error
Emotion influences human capabilities
Each person is different
Model Human Processor
Vision
Two stages in vision:
physical reception of stimulus
processing and interpretation of stimulus
Designing with blue
Optical center
The Eye - physical reception
mechanism for receiving light and transforming it into
electrical energy
light reflects from objects
images are focused upside-down on retina
retina contains rods for low light vision and cones for
colour vision
ganglion cells (brain!) detect pattern and movement
Vision
Vision - compensation
Vision
Optical Illusions
the Ponzo illusion
the Muller Lyer illusion
Optical Illusions
Optical Illusions
Optical Illusions
Colors
Major impact in UI design
Beyond pure aesthetics, color has associated meanings (Cultural
differences) and elicits emotional responses.
To prevent confusion in meaning, color must be used consistently
An UI must be designed without colors in the beginning
Magic number: 5+2 colors in an interface
For different concepts use different colors
Accessibility
The primary colors as seen with
normal color vision.
The primary colors as seen with
Protanopia (1% of male population).
The primary colors as seen with
Deuteranopia (6% of male population).
The primary colors as seen with
Tritanopia (1% of male population).
Designing with blue
There are special usability issues to be considered in using blue and
yellow in graphics.
Legibility, temporal response, spatial localization and perception of
geometrical shapes are all somewhat compromised in patterns in
which the only differences are in the short-wave-sensitive (SWS,
"blue") photoreceptors.
In graphics terms this mostly applies to color pairs that differ only in
the blue primary.
recommendation: Pure blue should not be used for fine detail or
background
Legibility
Difficult to read in any font size
Text differ from their backgrounds only in the blue
primary:
Solutions
Blue can be used in most contexts if care is taken to achieve
adequate luminance contrast. T
Reading
Several stages:
visual pattern perceived
decoded using internal representation of language
interpreted using knowledge of syntax, semantics, pragmatics
Reading involves saccades and fixations
Perception occurs during fixations
Word shape is important to recognition
Negative contrast improves reading from computer screen
Reading
Can you read this?
I cnduo't bvleiee taht I culod aulaclty uesdtannrd
waht I was rdnaieg. Unisg the icndeblire pweor of
the hmuan mnid, aocdcrnig to rseecrah at
Cmabrigde Uinervtisy, it dseno't mttaer in waht
oderr the lterets in a wrod are, the olny irpoamtnt
tihng is taht the frsit and lsat ltteer be in the rhgit
pclae. The rset can be a taotl mses and you can
sitll raed it whoutit a pboerlm. Tihs is bucseae the
huamn mnid deos not raed ervey ltteer by istlef,
but the wrod as a wlohe. Aaznmig, huh?
Correct paragraph
I couldn't believe that I could actually understand
what I was reading. Using the incredible power of
the human brain, according to research at
Cambridge University, it doesn't matter in what
order the letters in a word are, the only important
thing is that the first and last letter be in the right
place. The rest can be a total, mess and you can
read it without a problem. This is because the
human mind does not read every letter by itself,
but the word as a whole. Amazing, huh?
Hearing
Provides information about environment:
distances, directions, objects etc.
Physical apparatus:
outer ear
protects inner and amplifies sound
middle ear
transmits sound waves as
inner ear
Sound
pitch
loudness
timbre
vibrations to inner ear
chemical transmitters are released
and cause impulses in auditory nerve
sound frequency
amplitude
type or quality
Hearing
Humans can hear frequencies from 20Hz to 15kHz
less accurate distinguishing high frequencies than low.
Auditory system filters sounds
can attend to sounds over background noise.
for example, the cocktail party phenomenon - when one may
immediately detect words of importance originating from
unattended stimuli, for instance hearing one's name in another
conversation
Touch
Provides important feedback about environment.
May be key sense for someone who is visually impaired.
Stimulus received via receptors in the skin:
thermoreceptors
nociceptors
mechanoreceptors
heat and cold
pain
pressure
(some instant, some continuous)
Some areas more sensitive than others e.g. fingers.
Kinethesis - awareness of body position
affects comfort and performance.
Movement
Time taken to respond to stimulus:
reaction time + movement time
Movement time dependent on age, fitness etc.
Reaction time - dependent on stimulus type:
visual
~ 200ms
auditory
~ 150 ms
pain
~ 700ms
Increasing reaction time decreases accuracy in the
unskilled operator but not in the skilled operator.
Movement
Fitts' Law describes the time taken to hit a screen
target:
Mt = a + b log2(D/S + 1)
where:
a and b are empirically determined constants
Mt is movement time
D is Distance
S is Size of target
targets as large as possible
distances as small as possible
Memory
There are three types of memory function:
Sensory memories
Short-term memory or working memory
Long-term memory
Selection of stimuli governed by level of arousal.
sensory memory
Buffers for stimuli received through senses
iconic memory: visual stimuli
echoic memory: aural stimuli
haptic memory: tactile stimuli
Examples
sparkler trail
stereo sound
Continuously overwritten
Short-term memory (STM)
Scratch-pad for temporary recall
rapid access ~ 70ms
rapid decay ~ 200ms
limited capacity - 7 2 chunks
STM
Try to memorize as much as possible from the following sequence
265397620853
Try to memorize as much as possible from the following sequence
0040 732 215 754
Examples
212348278493202
0121 414 2626
HEC ATR ANU PTH ETR EET
Long-term memory (LTM)
Repository for all our knowledge
slow access ~ 1/10 second
slow decay, if any
huge or unlimited capacity
Two types
episodic
semantic
serial memory of events
structured memory of facts, concepts, skills
semantic LTM derived from episodic LTM
Long-term memory
Semantic memory structure
provides access to information
represents relationships between bits of information
supports inference
Model: semantic network
inheritance child nodes inherit properties of parent nodes
relationships between bits of information explicit
supports inference through inheritance
LTM - Forgetting
decay
information is lost gradually but very slowly
interference
new information replaces old: retroactive interference
old may interfere with new: proactive inhibition
so may not forget at all memory is selective
affected by emotion can subconsciously `choose' to forget
LTM - retrieval
recall
information reproduced from memory can be assisted by cues, e.g.
categories, imagery
recognition
information gives knowledge that it has been seen before
less complex than recall - information is cue
THINKING
Reasoning
deduction, induction, abduction
Problem solving
Deductive Reasoning
Deduction:
derive logically necessary conclusion from given premises.
e.g. If it is Friday then she will go to work
It is Friday
Therefore she will go to work.
Logical conclusion not necessarily true:
e.g.
If it is raining then the ground is dry
It is raining
Therefore the ground is dry
Deduction
When truth and logical validity clash
e.g. Some people are babies
Some babies cry
Inference - Some people cry
Correct?
People bring world knowledge to bear
Inductive Reasoning
Induction:
generalize from cases seen to cases unseen
e.g.
all elephants we have seen have trunks
therefore all elephants have trunks.
Unreliable:
can only prove false not true
but useful!
Humans not good at using negative evidence
e.g. Wason's cards.
Wason's cards
7 E 4 K
If a card has a vowel on one side it has an even number on the other
Is this true?
How many cards do you need to turn over to find out?
. and which cards?
Abductive reasoning
reasoning from event to cause
e.g.
Sam drives fast when drunk.
If I see Sam driving fast, assume drunk.
Unreliable:
can lead to false explanations
Reasoning
Problem solving
Process of finding solution to unfamiliar task using
knowledge.
Several theories.
Gestalt
problem solving both productive and reproductive
productive draws on insight and restructuring of problem
attractive but not enough evidence to explain `insight' etc.
move away from behaviourism and led towards information processing
theories
Problem solving
Problem space theory
problem space comprises problem states
problem solving involves generating states using legal operators
heuristics may be employed to select operators
e.g. means-ends analysis
operates within human information processing system
e.g. STM limits etc.
largely applied to problem solving in well-defined areas
e.g. puzzles rather than knowledge intensive areas
Problem solving
Analogy
analogical mapping:
novel problems in new domain?
use knowledge of similar problem from similar domain
analogical mapping difficult if domains are semantically different
Skill acquisition
skilled activity characterized by chunking
lot of information is chunked to optimize STM
conceptual rather than superficial grouping of problems
information is structured more effectively
Errors and mental models
Types of error
slips
right intention, but failed to do it right
causes: poor physical skill,inattention etc.
change to aspect of skilled behaviour can cause slip
mistakes
wrong intention
cause: incorrect understanding
humans create mental models to explain behaviour.
if wrong (different from actual system) errors can occur
Mental models
"The image of the world around us, which we carry in our
head, is just a model. Nobody in his head imagines all the
world, government or country. He has only selected
concepts, and relationships between them, and uses
those to represent the real system. (Jay Wright Forrester
MIT professor)
Everyday Life and Mental Models
(a) You arrive home on a cold winters night to a cold
house. How do you get the house to warm up as
quickly as possible? Set the thermostat to be at its
highest or to the desired temperature?
(b) You arrive home starving hungry. You look in the fridge
and find all that is left is an uncooked pizza. You have
an electric oven. Do you warm it up to 375 degrees first
and then put it in (as specified by the instructions) or
turn the oven up higher to try to warm it up quicker?
Exercise: ATMs
How an ATM works
How much money are you allowed to take out?
What denominations?
If you went to another machine and tried the same what would
happen?
What information is on the strip on your card? How is this used?
What happens if you enter the wrong number?
Why are there pauses between the steps of a transaction? What
happens if you try to type during them?
Why does the card stay inside the machine?
Do you count the money? Why?
How did you fare?
Your mental model
How accurate?
How similar?
How shallow?
Payne (1991) did a similar study and found
that people frequently resort to analogies to
explain how they work
Peoples accounts greatly varied and were
often ad hoc
Computers
Same is often true for understanding how
interactive devices and computers work:
Poor, often incomplete, easily confusable, based on
inappropriate analogies and superstition (Norman,
1983)
e.g. frozen cursor/screen - most people will bash all
manner of keys
External cognition
Concerned with explaining how we interact with
external representations (e.g. maps, notes,
diagrams)
What are the cognitive benefits and what
processes involved
How they extend our cognition
What computer-based representations can we
develop to help even more?
Externalizing to reduce memory load
Diaries, reminders,calendars, notes, shopping lists, to-do
lists - written to remind us of what to do
Post-its, piles, marked emails - where placed indicates
priority of what to do
External representations:
Remind us that we need to do something (e.g. to buy something for
mothers day)
Remind us of what to do (e.g. buy a card)
Remind us when to do something (e.g. send a card by a certain
date)
Computational offloading
When a tool is used in conjunction with an external
representation to carry out a computation (e.g. pen
and paper)
Try doing the two sums below (a) in your head, (b) on
a piece of paper and c) with a calculator.
234 x 456 =??
CCXXXIIII x CCCCXXXXXVI = ???
Which is easiest and why? Both are identical sums
Annotation and cognitive tracing
Annotation involves modifying existing
representations through making marks
e.g. crossing off, ticking, underlining
Cognitive tracing involves externally manipulating
items into different orders or structures
e.g. playing scrabble, playing cards
Design implication
Provide external representations at the
interface that reduce memory load and
facilitate computational offloading
e.g. Information
visualizations have
been designed to
allow people to make
sense and rapid
decisions about
masses of data
Mental models & system
design
Notion of mental models has been used as a basis for
conceptual models
Assumption is that if you can understand how people
develop mental models then can help them develop more
appropriate mental models of system functionality
For example, a design principle is to try to make systems
transparent so people can understand them better and
know what to do
The design principle of transparency
useful feedback
easy to understand
intuitive to use
clear & easy to follow instructions
appropriate online help
context sensitive guidance of
how to proceed when stuck
Emotions
Various theories of how emotion works
James-Lange: emotion is our interpretation of a physiological
response to a stimuli
Cannon: emotion is a psychological response to a stimuli
Schacter-Singer: emotion is the result of our evaluation of our
physiological responses, in the light of the whole situation we
are in
Emotion clearly involves both cognitive and physical
responses to stimuli
Emotions
The biological response to physical stimuli is called
affect
Affect influences how we respond to situations
positive creative problem solving
negative narrow thinking
Negative affect can make it harder to do even easy tasks;
positive affect can make it easier to do difficult tasks
(Donald Norman)
Emotion
Implications for interface design
stress will increase the difficulty of problem solving
relaxed users will be more forgiving of shortcomings in design
aesthetically pleasing and rewarding interfaces will increase
positive affect (Attractive things work better?)
Individual differences
long term
sex, physical and intellectual abilities
short term
effect of stress or fatigue
changing
age
Ask yourself:
will design decision exclude section of user population?
Psychology and the Design of Interactive
System
Some direct applications
e.g.
blue acuity is poor
blue should not be used for important detail
However, correct application generally requires understanding of
context in psychology, and an understanding of particular
experimental conditions
A lot of knowledge has been distilled in
guidelines
cognitive models
experimental and analytic evaluation techniques
THE COMPUTER
HCI Basics
The Computer
a computer system is made up of various elements
each of these elements affects the interaction
input devices text entry and pointing
output devices screen (small&large), digital paper
virtual reality special interaction and display devices
physical interaction e.g. sound, haptic, bio-sensing
paper as output (print) and input (scan)
memory RAM & permanent media, capacity & access
processing speed of processing, networks
Interacting with computers
to understand humancomputer interaction
need to understand computers!
what goes in and out
devices, paper,
sensors, etc.
what can it do?
memory, processing,
networks
A typical computer system
screen, or monitor, on which there are windows
keyboard
mouse/trackpad
window 1
variations
desktop
laptop
PDA
window 2
12-37pm
the devices dictate the styles of interaction that the system supports
If we use different devices, then the interface will support a different style of
interaction
How many
computers in your house?
hands up,
none, 1, 2 , 3, more!!
computers in your pockets?
are you thinking
PC, laptop, PDA ??
How many computers
in your house?
in your pockets?
PC
PDA
TV, VCR, DVD, HiFi,
phone, camera
cable/satellite TV
microwave, cooker, washing
machine
central heating
security system
smart card, card with
magnetic strip?
electronic car key
USB memory
try your pockets and bags
can you think of more?
Interactivity?
Long ago in a galaxy far away batch processing
punched card stacks or large data files prepared
long wait .
line printer output
and if it is not right
Now most computing is interactive
rapid feedback
the user in control (most of the time)
doing rather than thinking
Is faster always better?
Richer interaction
sensors
and devices
everywhere
TEXT ENTRY DEVICES
keyboards (QWERTY et al.)
chord keyboards, phone pads
handwriting, speech
Keyboards
Most common text input device
Allows rapid entry of text by experienced users
Keypress closes connection, causing a character code to
be sent
Usually connected by cable, but can be wireless
layout QWERTY
Standardised layout
but
non-alphanumeric keys are placed differently
accented symbols needed for different scripts
minor differences between UK and USA keyboards
QWERTY arrangement not optimal for typing
layout to prevent typewriters jamming!
Alternative designs allow faster typing but large social base of
QWERTY typists produces reluctance to change.
QWERTY
1
Q
3
W
A
Z
5
R
D
X
7
U
SPACE
9
I
J
M
0
O
L
,
alternative keyboard layouts
Alphabetic
keys arranged in alphabetic order
not faster for trained typists
not faster for beginners either!
Dvorak
common letters under dominant fingers
biased towards right hand
common combinations of letters alternate between hands
10-15% improvement in speed and reduction in fatigue
But - large social base of QWERTY typists produce market pressures not
to change
special keyboards
designs to reduce fatigue for RSI
for one handed use
e.g. the Maltron left-handed keyboard
Chord keyboards
only a few keys - four or 5
letters typed as combination of keypresses
compact size
ideal for portable applications
short learning time
keypresses reflect letter shape
fast
once you have trained
BUT - social resistance, plus fatigue after extended use
NEW niche market for some wearables
phone pad and T9 entry
use numeric keys with
multiple presses
2abc
3-def
4-ghi
5-jkl
6-mno
7-pqrs
8-tuv
9-wxyz
hello = 4433555[pause]555666
surprisingly fast!
T9 predictive entry
type as if single key for each letter
use dictionary to guess the right word
hello = 43556
but 26 -> menu am or an
Handwriting recognition
Text can be input into the computer, using a pen and a
digesting tablet
natural interaction
Technical problems:
capturing all useful information - stroke path, pressure, etc. in a
natural manner
segmenting joined up writing into individual letters
interpreting individual letters
coping with different styles of handwriting
Used in PDAs, and tablet computers
leave the keyboard on the desk!
Speech recognition
Improving rapidly
Most successful when:
single user initial training and learns peculiarities
limited vocabulary systems
Problems with
external noise interfering
imprecision of pronunciation
large vocabularies
different speakers
Numeric keypads
for entering numbers quickly:
calculator, PC keyboard
for telephones
not the same!!
ATM like phone
telephone
calculator
POSITIONING, POINTING
AND DRAWING
mouse, touchpad
trackballs, joysticks etc.
touch screens, tablets
eyegaze, cursors
the Mouse
Handheld pointing device
very common
easy to use
Two characteristics
planar movement
buttons
(usually from 1 to 3 buttons on top, used for making a selection,
indicating an option, or to initiate drawing etc.)
the mouse
Mouse located on desktop
requires physical space
no arm fatigue
Relative movement only is detectable.
Movement of mouse moves screen cursor
Screen cursor oriented in (x, y) plane,
mouse movement in (x, z) plane
an indirect manipulation device.
device itself doesnt obscure screen, is accurate and fast.
hand-eye coordination problems for novice users
How does it work?
Two methods for detecting motion
Mechanical
Ball on underside of mouse turns as mouse is moved
Rotates orthogonal potentiometers
Can be used on almost any flat surface
Optical
light emitting diode on underside of mouse
may use special grid-like pad or just on desk
less susceptible to dust and dirt
detects fluctuating alterations in reflected light intensity to calculate relative
motion in (x, z) plane
Even by foot
some experiments with the footmouse
controlling mouse movement with feet
not very common :-)
but foot controls are common elsewhere:
car pedals
sewing machine speed control
organ and piano pedals
Touchpad
small touch sensitive tablets
stroke to move mouse pointer
used mainly in laptop computers
good acceleration settings important
fast stroke
lots of pixels per inch moved
initial movement to the target
slow stroke
less pixels per inch
for accurate positioning
Trackball and thumbwheels
Trackball
ball is rotated inside static housing
like an upsdie down mouse!
relative motion moves cursor
indirect device, fairly accurate
separate buttons for picking
very fast for gaming
used in some portable and notebook computers.
Thumbwheels
for accurate CAD two dials for X-Y cursor position
for fast scrolling single dial on mouse
Joystick and keyboard nipple
Joystick
indirect
pressure of stick = velocity of movement
buttons for selection
on top or on front like a trigger
often used for computer games
aircraft controls and 3D navigation
Keyboard nipple
for laptop computers
miniature joystick in the middle of the keyboard
Touch-sensitive screen
Detect the presence of finger or stylus on the screen.
works by interrupting matrix of light beams, capacitance changes or ultrasonic
reflections
direct pointing device
Advantages:
fast, and requires no specialised pointer
good for menu selection
suitable for use in hostile environment: clean and safe from damage.
Disadvantages:
finger can mark screen
imprecise (finger is a fairly blunt instrument!)
difficult to select small regions or perform accurate drawing
lifting arm can be tiring
Stylus and light pen
Stylus
small pen-like pointer to draw directly on screen
may use touch sensitive surface or magnetic detection
used in PDA, tablets PCs and drawing tables
Light Pen
now rarely used
uses light from screen to detect location
BOTH
very direct and obvious to use
but can obscure screen
Digitizing tablet
Mouse like-device with cross hairs
used on special surface
- rather like stylus
very accurate
- used for digitizing maps
Eyegaze
control interface by eye gaze direction
e.g. look at a menu item to select it
uses laser beam reflected off retina
a very low power laser!
mainly used for evaluation
potential for hands-free control
high accuracy requires headset
cheaper and lower accuracy devices available
sit under the screen like a small webcam
Cursor keys
Four keys (up, down, left, right) on keyboard.
Very, very cheap, but slow.
Useful for not much more than basic motion for text-editing tasks.
No standardised layout, but inverted T, most common
Discrete positioning controls
in phones, TV controls etc.
cursor pads or mini-joysticks
discrete left-right, up-down
mainly for menu selection
DISPLAY DEVICES
bitmap screens (CRT & LCD)
large & situated displays
digital paper
bitmap displays
screen is vast number of coloured dots
resolution and colour depth
Resolution used (inconsistently) for
number of pixels on screen (width x height)
e.g. SVGA 1024 x 768, PDA perhaps 240x400
density of pixels (in pixels or dots per inch - dpi)
typically between 72 and 96 dpi
Aspect ratio
ration between width and height
4:3 for most screens, 16:9 for wide-screen TV
Colour depth:
how many different colours for each pixel?
black/white or greys only
256 from a pallete
8 bits each for red/green/blue = millions of colours
Digital paper
what?
thin flexible sheets
updated electronically
but retain display
how?
small spheres turned
or channels with coloured liquid
and contrasting spheres
rapidly developing area
appearance
cross
section
VIRTUAL REALITY AND
3D INTERACTION
positioning in 3D space
moving and grasping
seeing 3D (helmets and caves)
positioning in 3D space
cockpit and virtual controls
steering wheels, knobs and dials just like real!
the 3D mouse
six-degrees of movement: x, y, z + roll, pitch, yaw
data glove
fibre optics used to detect finger position
VR helmets
detect head motion and possibly eye gaze
whole body tracking
accelerometers strapped to limbs or reflective dots and video
processing
PAPER: PRINTING AND
SCANNING
print technology
fonts, page description, WYSIWYG
scanning, OCR
Printing
image made from small dots
allows any character set or graphic to be printed,
critical features:
resolution
size and spacing of the dots
measured in dots per inch (dpi)
speed
usually measured in pages per minute
cost!!
Types of dot-based printers
dot-matrix printers
use inked ribbon (like a typewriter
line of pins that can strike the ribbon, dotting the paper.
typical resolution 80-120 dpi
ink-jet and bubble-jet printers
tiny blobs of ink sent from print head to paper
typically 300 dpi or better .
laser printer
like photocopier: dots of electrostatic charge deposited on drum, which
picks up toner (black powder form of ink) rolled onto paper which is then
fixed with heat
typically 600 dpi or better.
Printing in the workplace
shop tills
dot matrix
same print head used for several paper rolls
may also print cheques
thermal printers
special heat-sensitive paper
paper heated by pins makes a dot
poor quality, but simple & low maintenance
used in some fax machines
Fonts
Font the particular style of text
Courier font
Helvetica font
Palatino font
Times Roman font
(special symbol)
Size of a font measured in points (1 pt about 1/72)
(vaguely) related to its height
This is ten point Helvetica
This is twelve point
This is fourteen point
This is eighteen point
and this is twenty-four point
Fonts
Pitch
fixed-pitch every character has the same width
e.g. Courier
variable-pitched some characters wider
e.g. Times Roman compare the i and the m
Serif or Sans-serif
sans-serif square-ended strokes
e.g. Helvetica
serif with splayed ends (such as)
e.g. Times Roman or Palatino
Readability of text
lowercase
easy to read shape of words
UPPERCASE
better for individual letters and non-words
e.g. flight numbers: BA793 vs. ba793
serif fonts
helps your eye on long lines of printed text
but sans serif often better on screen
Scanners
Take paper and convert it into a bitmap
Two sorts of scanner
flat-bed: paper placed on a glass plate, whole page converted into bitmap
hand-held: scanner passed over paper, digitising strip typically 3-4 wide
Shines light at paper and note intensity of reflection
colour or greyscale
Typical resolutions from 6002400 dpi
Scanners
Used in
desktop publishing for incorporating photographs and other
images
document storage and retrieval systems, doing away with
paper storage
special scanners for slides and photographic negatives
MEMORY
short term and long term
speed, capacity, compression
formats, access
Short-term Memory - RAM
Random access memory (RAM)
on silicon chips
100 nano-second access time
usually volatile (lose information if power turned off)
data transferred at around 100 Mbytes/sec
Some non-volatile RAM used to store basic set-up
information
Typical desktop computers:
64 to 256 Mbytes RAM
Long-term Memory - disks
magnetic disks
floppy disks store around 1.4 Mbytes
hard disks typically 40 Gbytes to 100s of Gbytes
access time ~10ms, transfer rate 100kbytes/s
optical disks
use lasers to read and sometimes write
more robust that magnetic media
CD-ROM
- same technology as home audio, ~ 600 Gbytes
DVD - for AV applications, or very large files
