Models

Human-Computer Interaction Lecture

Slides adapted from hci-lecture.org (A. Schmidt, N. Henze, K. Wolf, V. Schwind), Image from: https://pxhere.com/de/photo/956874

The following content is licensed under a Creative Commons Attribution 4.0 International license
(CC BY-SA 4.0)
Prof. Dr. Valentin Schwind 1
 Image from: https://pxhere.com/de/photo/956874

Models Prof. Dr. Valentin Schwind 2

Learning Goals
▪ Understand …
what models are and why they are useful
know about their limitations
have a rough overview of models in HCI
▪ Be able to explain …
explain these models and give examples
discuss implications and how models can be used to evaluate UIs

Models Prof. Dr. Valentin Schwind 3

 2
4𝜋²
𝑇 = 𝑟³
𝐺𝑀
From https://www.jpl.nasa.gov/spaceimages/details.php?id=PIA12114 (PD)

Models Prof. Dr. Valentin Schwind 4

Models
▪ Are representations of phenomena that help us to understand how something
works or how it will work.
We need models for humans (e.g., Cognition, Mental Models,…)
We need models for systems (e.g., Regression, Machine Learning,…)
We need models how human interact with systems
▪ Models are never perfect. There will always be one that is better for specific
questions.
▪ A model is only useful for specific phenomena but not is not useful for most
phenomena.

Models Prof. Dr. Valentin Schwind 5

Fitts’ Law
Prediction of Target Selection

Slides adapted from hci-lecture.org (A. Schmidt, N. Henze, K. Wolf, V. Schwind), Image from: https://pxhere.com/de/photo/956874

The following content is licensed under a Creative Commons Attribution 4.0 International license
(CC BY-SA 4.0)
Prof. Dr. Valentin Schwind 6
In- and Output

Fitts’ Law Prof. Dr. Valentin Schwind 7

In- and Output

distance

width

Fitts’ Law Prof. Dr. Valentin Schwind 8

Paul Fitts‘ Experiment

Four distances: 2, 4, 8, 16 inch

Four widths: 0.25, 0.5, 1.0, 2.0 inch

From: U-M Library Digital Collections. Bentley Image Bank, Bentley Historical Library. Accessed: March 27, 2020. CC BY 4.0

Fitts’ Law Prof. Dr. Valentin Schwind 9

Fitts‘ Experiment
The movement time (MT) to select a
target is a function of the target’s
movement time in seconds

width (W) and distance (D). It

depends on the input device.
W
D
start
target

MT: movement time

a & b: input device-dependent constants
D: distance to the target
distance & width W: width of the target
From: Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of experimental psychology, 47(6), 381.

Fitts’ Law Prof. Dr. Valentin Schwind 10

The Index of Difficulty

▪ Index of Difficulty, ID =
MT = a + b ·ID
ID how difficult a task is independent from the input device
▪ Units:
a is measured in seconds
b is measured in seconds per bit
Index of Difficulty (ID) is described in bits

Fitts’ Law Prof. Dr. Valentin Schwind 11

Determining a and b
a = 0.028 s
b = 0.112 s/bit

y = bx + a
y = 0.11 2x + 0.028
R² = 0.987

▪ D = 16, W = 0.25
ID =
▪ ID = log2(1+64) = 6.02

Fitts’ Law Prof. Dr. Valentin Schwind 12

Predicting the Movement Time

▪ a = 0.028s
▪ b = 0.112s/bit
▪ How long does it take to select a target that is 21 inch away and 3 inch wide?
▪ MT = 0.028 + 0.112 * log2(1+7)
= 0.028 + 0.112 * log2(8)
= 0.028 + 0.112 * 3
= 0.364ms

Fitts’ Law Prof. Dr. Valentin Schwind 13

Determine a and b for another device / task
width distance MT MT
0,0625
0,0625
4
8
0,697
0,771
What are a and b for this? 1,2
y = 0,0926x + 0,2234
R² = 0,8455
1
0,0625 16 0,896
0,8
0,0625 32 1,096
0,125 4 0,649

MT
0,6

0,125 8 0,734
0,4
0,125 16 0,844
0,2
0,125 32 1,028
0,25 4 0,607 0
2 4 6 8 10
0,25 8 0,672 ID
0,25 16 0,771
0,25 32 0,975 a = 0,2234
0,5 4 0,535
b = 0,0926
0,5 8 0,623
0,5 16 0,724
0,5 32 0,902 Understanding Fitts' Law. (2024, November 14). Retrieved from https://us.humankinetics.com/blogs/excerpt/understanding-fitts-law

Fitts’ Law Prof. Dr. Valentin Schwind 14

Fitts‘ in 2D

distance

width

Fitts’ Law Prof. Dr. Valentin Schwind 15

 What are width and distance?

Fitts’ Law Prof. Dr. Valentin Schwind 16

Fitts‘ in 2D

Fitts’ Law Prof. Dr. Valentin Schwind 17

Standardized 1D Fitts‘ Task

Commonly using a fixed set of amplitudes

and Widths, e.g.:

▪ Amplitude (A): 64, 128, 256, 512 pixels

▪ Width (W): 8, 16, 32, 64 pixels

Fitts’ Law Prof. Dr. Valentin Schwind 18

Standardized 2D Fitts‘ Task

Commonly using a fixed set of amplitudes

and Widths, e.g.:

▪ Amplitude (A): 64, 128, 256, 512 pixels

▪ Width (W): 8, 16, 32, 64 pixels

Fitts’ Law Prof. Dr. Valentin Schwind 19

Throughput
▪ A single metric for a pointing device
Works with serial (a series of target
selections)
Works with discrete (single target
selections)
▪ Sufficient with six different IDs to
determine the device-specific
constants a and b

Fitts’ Law Prof. Dr. Valentin Schwind 20

1D vs 2D

MacKenzie, I. S. (2018). Fitts' law. In K. L. Norman & J. Kirakowski (Eds.), Handbook of human-computer interaction, pp. 349-370. Hoboken, NJ: Wiley. doi:10.1002/9781118976005

Fitts’ Law Prof. Dr. Valentin Schwind 21

Serial vs Discrete

MacKenzie, I. S., and Isokoski, P. (2008). Fitts' throughput and the speed-accuracy tradeoff. Proceedings of the ACM Conference on Human Factors in Computing Systems – CHI 2008, pp.
1633-1636. New York: ACM.

Fitts’ Law Prof. Dr. Valentin Schwind 22

Fitts‘ in 2D (ISO 9241-9)

Guo, X. (2022). A Fitts' law evaluation and comparison for human and manipulator on touch task. Cognit. Comput. Syst., 4. doi: 10.1049/ccs2.12057

Fitts’ Law Prof. Dr. Valentin Schwind 23

Effective Throughput
▪ Effective measures (like effective distance and effective width) account for the user’s
actual behavior and variability during the task – not on predefined (ideal) parameters
▪ The effective throughput (𝑻𝑷𝒆 ) is a refined metric in Fitts' Law studies that true
measures both the speed and accuracy of pointing tasks. It is defined by:
𝐴
𝐼𝐷𝑒 log2 (𝑊𝑒 +1)
𝑇𝑃𝑒 = = 𝑒
with
𝑀𝑇 𝑀𝑇
the effective amplitude (𝐴𝑒 )
the effective width (𝑊𝑒 )
the effective index of difficulty (𝐼𝐷𝑒 )
the actual movement time (𝑀𝑇)

Fitts’ Law Prof. Dr. Valentin Schwind 25

Effective Distance
▪ Effective Distance (𝑨𝒆 ) is the actual distance covered by the user, combining both the
nominal distance and deviations along the target axis
𝐴𝑒 = 𝑎 + 𝑑𝑥
𝒂 is the distance from the starting point to the target
𝒅𝒙 represents the effective movement variation by accounting for any overshoot or
undershoot of the target along the intended path with the formula:
𝑐 2 − 𝑏 2 − 𝑎²
𝑑𝑥 =
2𝑎
with the distances between
𝑎 = "from“ [x1, y1] and "to" [x2, y2]
𝑏 = "select" [x, y] and "to" [x2, y2]
𝑎 = "from" [x1, y1] and "select" [x, y]

MacKenzie, I. S., and Isokoski, P. (2008). Fitts' throughput and the speed-accuracy tradeoff. Proceedings of the ACM Conference on Human Factors in Computing Systems – CHI 2008, pp.
1633-1636. New York: ACM.

Fitts’ Law Prof. Dr. Valentin Schwind 26

Effective Width
▪ The effective target width (𝑾𝒆 ) captures the endpoint variability
The variability and precision of the user's movements do not align
perfectly with the physical dimensions of the target
𝑊𝑒 reflects the effective accuracy, not just the theoretical difficulty
implied by the nominal width 𝑊
▪ Effective width 𝑊𝑒 ​is derived from the standard deviation of the
endpoint positions along the axis of movement
𝑊𝑒 = 2 2 × 2 ⋅ σ = 2 ⋅ 2.066 ⋅ 𝑆𝐷𝑑𝑥 ≈ 4.133 ⋅ 𝑆𝐷𝑑𝑥
where 𝑺𝑫𝒙 is the standard deviation of effective movement variations
▪ 𝑊𝑒 is roughly equivalent to covering 96% of the data points in a
normal distribution (two-tailed), encompassing nearly all endpoint
variability
MacKenzie, I. S., and Isokoski, P. (2008). Fitts' throughput and the speed-accuracy tradeoff. Proceedings of the ACM Conference on Human Factors in Computing Systems – CHI 2008, pp.
1633-1636. New York: ACM.

Fitts’ Law Prof. Dr. Valentin Schwind 27

Effective Throughput
𝑐 2 − 𝑏 2 − 𝑎²
𝐴𝑒 𝑎+
log ( + 1) log ( 2𝑎 + 1)
𝐼𝐷𝑒 2 𝑊𝑒 2 4.133 ⋅ 𝑆𝐷𝑑𝑥
𝑇𝑃𝑒 = = =
𝑀𝑇 𝑀𝑇 𝑀𝑇
# Variables we need to solve Fitts' equation for 2 D
data$a <- hypot(data$fromX - data$toX, data$fromY - data$toY) # distance between "from" and "to"
data$b <- hypot(data$selectX - data$toX, data$selectY - data$toY) # distance between "select" and "to"
data$c <- hypot(data$fromX - data$selectX, data$fromY - data$selectY) # distance between "from" and "select"
data$dx <- (data$c^2 - data$b^2 - data$a^2 ) / (2 .0 * data$a)
data$AmpE <- data$a + data$dx

# Aggregate to get dxSD

means <- data %>% group_by(SubjectID, ID, Conditions) %>% summarize(
MT = mean(Duration), SD = sd(Duration), Ae = mean(AmpE), dxSD = sd(dx), Amp = mean(Amplitude),
Size = mean(Size), IDs = round(log2 ((mean(Amplitude) / mean(Size)) + 1 ), 2 ), .groups = 'drop‘)

# Aggregate to get TPe

finalTPs <- means %>% group_by(SubjectID, Conditions, IDs) %>% summarize(
MeanTime = mean(MT),
IDe = mean(log2 ((Ae/(4 .1 3 3 * dxSD)) + 1 )),
TPe = mean(log2 ((Ae/(4 .1 3 3 * dxSD)) + 1 ) / (MT/1 0 0 0 )), Code example in R
TPs = mean(log2 (IDs) / (MT/1 0 0 0 )),
.groups = 'drop')

Fitts’ Law Prof. Dr. Valentin Schwind 28

Example
▪ Setup
Target Distance (𝐷) = 100 pixels
Nominal Target Width (𝑊) = 20 pixels 20 px

Standard Deviation of endpoint spread (𝜎) = 5 pixels

Movement Time (𝑀𝑇) = 500 ms 5 px

▪ Effective Throughput Calculation 100 px

𝑊𝑒 = 4.133 × 5 = 20.665 𝑝𝑖𝑥𝑒𝑙𝑠
100
𝐼𝑒 = log 2 + 1 = log 2 5.837 ≈ 2.54 𝑏𝑖𝑡𝑠
20.665
2.54
𝑇𝑃𝑒 = = 5.08 𝑏𝑖𝑡/𝑠
0.5

Fitts’ Law Prof. Dr. Valentin Schwind 29

Regression Trend Examples

Static IDs Effective IDs

Priya, K., & Joshi, A. (2023). Fitts’ Throughput Vs Empirical Throughput: A Comparative Study. Human-Computer Interaction – INTERACT 2023. Springer. doi: 10.1007/978-3-031-42280-
5_28

Fitts’ Law Prof. Dr. Valentin Schwind 30

Throughput Measures Examples

Schwind, V., Halbhuber, D., Fehle, J., Sasse, J., Pfaffelhuber, A., Tögel, C., ...Henze, N. (2020). The Effects of Full-Body Avatar Movement Predictions in Virtual Reality using Neural Networks.
ResearchGate, 1–11. doi: 10.1145/3385956.3418941
Mutasim, A., Batmaz, A., & Stuerzlinger, W. (2021). Pinch, Click, or Dwell: Comparing Different Selection Techniques for Eye-Gaze-Based Pointing in Virtual Reality. ResearchGate. doi:
10.1145/3448018.3457998

Fitts’ Law Prof. Dr. Valentin Schwind 31

Which device has the highest throughput?

3.7 - 4.5 bit/s 2.3 - 3.0 bit/s 2.3 - 2.9 bit/s 2.55 bit/s

Fitts’ Law Prof. Dr. Valentin Schwind 32

Fitts’ Law Prof. Dr. Valentin Schwind 33
Fitts’ Law Prof. Dr. Valentin Schwind 34
Rule of Infinite Edges ∞
▪ Edges and corners are the easiest to reach
by a pointing device
∞
The width of a target edge is infinite large
Only works in full screen
Pages are scrollable
▪ The coordinates of the corners are also
called prime-pixels
∞

Prof. Dr. Valentin Schwind 35

The Dominance of the Mouse

Using Fitts’ Law “was a major factor leading to the

mouse's commercial introduction by Xerox”
http://www2.parc.com/istl/groups/uir/people/stuart/stuart.htm

Image from Card, S. K., English, W. K., & Burr, B. J. (1978). Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a CRT. Ergonomics, 21(8),
601-613.

Fitts’ Law Prof. Dr. Valentin Schwind 36

Complex UIs

Fitts’ Law Prof. Dr. Valentin Schwind 37

Steering Law
Pointing through tunnels

Slides adapted from hci-lecture.org (A. Schmidt, N. Henze, K. Wolf, V. Schwind), Image from: https://pxhere.com/de/photo/956874

The following content is licensed under a Creative Commons Attribution 4.0 International license
(CC BY-SA 4.0)
Prof. Dr. Valentin Schwind 38
Steering Law Prof. Dr. Valentin Schwind 39
 distance
width

Steering Law Prof. Dr. Valentin Schwind 40

Steering Law Prof. Dr. Valentin Schwind 41
Tunnels

distance
width

Steering Law Prof. Dr. Valentin Schwind 42

Changing Fitts’ Law to model steering tasks?

distance
width

Steering Law Prof. Dr. Valentin Schwind 43

Steering Law Definition

ID=

▪ The movement time (MT) to acquire a target through a tunnel is a function of the
length (D) and width (W) of the tunnel. It depends on the input device and the
number of tunnels.
MT: movement time
a and b: constants dependent on the pointing system
D: distance, i.e., the length of the tunnel
W: width of the tunnel (can have variable thickness)
C: the parametrized path (any curvilinear shape)

Steering Law Prof. Dr. Valentin Schwind 44

Steering Law Prof. Dr. Valentin Schwind 45
Hick‘s Law
Visual Search Tasks

Image from: https://www.pexels.com/photo/grayscale-photography-of-assorted-shirts-hanged-on-clothes-rack-1884584/

The following content is licensed under a Creative Commons Attribution 4.0 International license
(CC BY-SA 4.0)
Prof. Dr. Valentin Schwind 46
Visual Search
10s
9s
8s
7s
6s Find
5s Denmark
4s
3s
2s
1s
0s

Hick‘s Law Prof. Dr. Valentin Schwind 47

Visual Search
▪ Russia ▪ Latvia 10s
▪ Ukraine ▪ Croatia
▪ France ▪ Bosnia and Herzegovina
9s
▪ Spain ▪ Slovakia
▪ Sweden ▪ Estonia
▪ Norway ▪ Denmark 8s
▪ Germany ▪ Switzerland
▪ Finland ▪ Netherlands 7s
▪ Poland ▪ Moldova
▪ ▪
▪
Italy
United Kingdom ▪
Belgium
Armenia
6s Find
▪ ▪
Denmark
Romania Albania
▪ Belarus ▪ North Macedonia 5s
▪ Kazakhstan ▪ Turkey
▪ Greece ▪ Slovenia 4s
▪ Bulgaria ▪ Montenegro
▪ Iceland ▪ Kosovo
3s
▪ Hungary ▪ Cyprus
▪ Portugal ▪ Azerbaijan
▪ Austria ▪ Luxembourg 2s
▪ Czechia ▪ Georgia
▪ Serbia ▪ Andorra 1s
▪ Ireland ▪ Malta
▪ Lithuania ▪ Liechtenstein
0s

Hick‘s Law Prof. Dr. Valentin Schwind 48

Time Complexity for Unordered Lists
▪ We have a list with n items in an unknown order
Time obviously increases with n
What is the time complexity for an algorithm in Big O notation?
O(n²)

Hick‘s Law Prof. Dr. Valentin Schwind 49

Visual Search
10s
9s
8s
7s
6s Find
5s Denmark
4s
3s
2s
1s

Hick‘s Law Prof. Dr. Valentin Schwind 50

Visual Search
▪ Albania ▪ Kosovo
▪ Andorra ▪ Latvia 10s
▪ Armenia ▪ Liechtenstein
▪ Austria ▪ Lithuania 9s
▪ Azerbaijan ▪ Luxembourg
▪ Belarus ▪ Malta 8s
▪ Belgium ▪ Moldova
▪ Bosnia and Herzegovina ▪ Montenegro
7s
▪ Bulgaria ▪ Netherlands
▪ Croatia ▪ North Macedonia
▪
▪
Cyprus
Czechia
▪
▪
Norway
Poland
6s Find
▪ Denmark ▪ Portugal 5s Denmark
▪ Estonia ▪ Romania
▪ Finland ▪ Russia
4s
▪ France ▪ Serbia
▪ Georgia ▪ Slovakia
3s
▪ Germany ▪ Slovenia
▪ Greece ▪ Spain
▪ Hungary ▪ Sweden 2s
▪ Iceland ▪ Switzerland
▪ Ireland ▪ Turkey 1s
▪ Italy ▪ Ukraine
▪ Kazakhstan ▪ United Kingdom 0s

Hick‘s Law Prof. Dr. Valentin Schwind 51

Time Complexity for Ordered Lists
▪ We have a list with n items in an known order
▪ Time obviously increases with n
▪ What is the time complexity for an algorithm in Big O notation?
▪ O(log(n))

Hick‘s Law Prof. Dr. Valentin Schwind 52

Hick‘s Law
▪ Given n equally probable choices, the average reaction time T required to choose
among the choices is approximately:
𝑇 = 𝑏 ∗ log2(𝑛 + 1)
▪ Common practical value: 𝑏 = 150 𝑚𝑠/𝑏𝑖𝑡
▪ Hick’s Law is often used to motivate menu designs
In an unordered list, search time is linear
In an ordered list, search time becomes logarithmic

Hick‘s Law Prof. Dr. Valentin Schwind 53

Combining Models

Hick‘s Law Prof. Dr. Valentin Schwind 54

Power Law of Practice
Routine Tasks

Image generated with Midjourney

The following content is licensed under a Creative Commons Attribution 4.0 International license
(CC BY-SA 4.0)
Prof. Dr. Valentin Schwind 55
Cigar Roller in Cuba

Crossman, E. R. F. W. (1959). A theory of the acquisition of speed-skill. Ergonomics, 2(2):153–166.

Power Law of Practice Prof. Dr. Valentin Schwind 56

Practice vs TCT

McLaughlin, A., Simon, D., & Gillan, D. (2010). From Intention to Input: Motor Cognition, Motor Performance, and the Control of Technology. Reviews of Human Factors and Ergonomics, 6,
123–171. doi: 10.1518/155723410X12849346788741

Power Law of Practice Prof. Dr. Valentin Schwind 57

Learning by Doing

task completion time

▪ The more practice
the easier a task becomes
the faster a user becomes
▪ How does a skill improves over time?
General observation: User skills improve as power function
of amount of practice practice trials

▪ General formula:

log(task completion time)

𝑇 = 𝑎 ∙ 𝑃−𝑏 or log(𝑇) = −𝑏 ⋅ log(𝑃) + log(𝑎)
with
T = task completion time
P = practice trials
a,b = device specific constants
log(practice trials)

Power Law of Practice Prof. Dr. Valentin Schwind 58

The Three Stages of Learning and the Resting Debate

Johanson, C., Gutwin, C., Bowey, J., & Mandryk, R. (2019). Press Pause when you Play: Comparing Spaced Practice Intervals for Skill Development in Games. . doi:
10.1145/3311350.3347195

Power Law of Practice Prof. Dr. Valentin Schwind 59

The Power of Practice
▪ Continuous practice lacks rest intervals, which are critical for the brain to
generalize feedback and avoid getting “stuck in a rut“
▪ Rest intervals may help break ineffective learning cycles, particularly in problem-
solving scenarios
▪ Debate exists on whether rest intervals should remain constant or adapt based on
the learner's experience

Johanson, C., Gutwin, C., Bowey, J., & Mandryk, R. (2019). Press Pause when you Play: Comparing Spaced Practice Intervals for Skill Development in Games. . doi:
10.1145/3311350.3347195

Power Law of Practice Prof. Dr. Valentin Schwind 60

Keystroke-Level Model (KLM)
Learning and Reacting

Image from: https://pxhere.com/de/photo/779902

The following content is licensed under a Creative Commons Attribution 4.0 International license
(CC BY-SA 4.0)
Prof. Dr. Valentin Schwind 61
Currency Converter

Sum an amount
Enter From this To this
me
6 currency currency

▪ Task: Convert 12 Euro in US Dollar

▪ one hand on the mouse, nothing selected
▪ What do we need to know?

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 62

Currency Converter

1. select text field

2. delete value
3. enter value
Sum an amount
Enter From this To this
4. select Euro
me
6 currency currency
5. select Dollar
6. select Convert

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 63

Keystroke-Level Model (KLM)
▪ Simplified version of GOMS "Goals, Operators, Methods, and Selections rules"
▪ KLM predicts how long it will take an expert (or trained) user to accomplish a
routine task without errors using an interactive computer system
▪ Execution of a task is decomposed into primitive operators
Physical motor operators
Pressing a button, pointing, drawing a line, …
Mental operator
Preparing for a physical action
System response operator
User waits for the system to do something

Card, Stuart K; Moran, Thomas P; Allen, Newell (1980). "The keystroke-level model for user performance time with interactive systems". Communications of the ACM. 23 (7): 396–
410. doi:10.1145/358886.358895. S2CID 5918086.

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 64

Keystroke-Level Model (KLM)
Operator Description Associated Time
K Keystroke, typing one letter, number, etc. or
function key such as ‘CRTL’ or ‘SHIFT’

H ‘Homing’, moving the hand between mouse and

keyboard
B/BB Pressing (B) or clicking (BB) a button
P Pointing with a mouse to a target

D(nD, lD) Drawing nD straight line segments of length lD

M Subsumed time for mental acts; sometimes used
as ‘look-at’
R(t) System response time, time during which the
user cannot act

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 65

Keystroke-Level Model (KLM)
Operator Description Associated Time
K Keystroke, typing one letter, number, etc. or Expert typist (90 wpm): 0.12s
function key such as ‘CRTL’ or ‘SHIFT’ Averaged skilled typist (55 wpm): 0.20s
Average non-secretarial typist (40 wpm): 0.28
Worst typist (unfamiliar with keyboard): 1.2s
H ‘Homing’, moving the hand between mouse and
keyboard
B/BB Pressing (B) or clicking (BB) a button
P Pointing with a mouse to a target

D(nD, lD) Drawing nD straight line segments of length lD

M Subsumed time for mental acts; sometimes used
as ‘look-at’
R(t) System response time, time during which the
user cannot act

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 66

Keystroke-Level Model (KLM)
Operator Description Associated Time
K Keystroke, typing one letter, number, etc. or Expert typist (90 wpm): 0.12s
function key such as ‘CRTL’ or ‘SHIFT’ Averaged skilled typist (55 wpm): 0.20s
Average non-secretarial typist (40 wpm): 0.28
Worst typist (unfamiliar with keyboard): 1.2s
H ‘Homing’, moving the hand between mouse and 0.4s
keyboard
B/BB Pressing (B) or clicking (BB) a button
P Pointing with a mouse to a target

D(nD, lD) Drawing nD straight line segments of length lD

M Subsumed time for mental acts; sometimes used
as ‘look-at’
R(t) System response time, time during which the
user cannot act

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 67

Keystroke-Level Model (KLM)
Operator Description Associated Time
K Keystroke, typing one letter, number, etc. or Expert typist (90 wpm): 0.12s
function key such as ‘CRTL’ or ‘SHIFT’ Averaged skilled typist (55 wpm): 0.20s
Average non-secretarial typist (40 wpm): 0.28
Worst typist (unfamiliar with keyboard): 1.2s
H ‘Homing’, moving the hand between mouse and 0.4s
keyboard
B/BB Pressing (B) or clicking (BB) a button 0.1s / 2*0.1s
P Pointing with a mouse to a target

D(nD, lD) Drawing nD straight line segments of length lD

M Subsumed time for mental acts; sometimes used
as ‘look-at’
R(t) System response time, time during which the
user cannot act

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 68

Keystroke-Level Model (KLM)
Operator Description Associated Time
K Keystroke, typing one letter, number, etc. or Expert typist (90 wpm): 0.12s
function key such as ‘CRTL’ or ‘SHIFT’ Averaged skilled typist (55 wpm): 0.20s
Average non-secretarial typist (40 wpm): 0.28
Worst typist (unfamiliar with keyboard): 1.2s
H ‘Homing’, moving the hand between mouse and 0.4s
keyboard
B/BB Pressing (B) or clicking (BB) a button 0.1s / 2*0.1s
P Pointing with a mouse to a target 0.8s to 1.5s with an average of 1.1s
Can also use Fitts’ Law
D(nD, lD) Drawing nD straight line segments of length lD
M Subsumed time for mental acts; sometimes used
as ‘look-at’
R(t) System response time, time during which the
user cannot act

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 69

Keystroke-Level Model (KLM)
Operator Description Associated Time
K Keystroke, typing one letter, number, etc. or Expert typist (90 wpm): 0.12s
function key such as ‘CRTL’ or ‘SHIFT’ Averaged skilled typist (55 wpm): 0.20s
Average non-secretarial typist (40 wpm): 0.28
Worst typist (unfamiliar with keyboard): 1.2s
H ‘Homing’, moving the hand between mouse and 0.4s
keyboard
B/BB Pressing (B) or clicking (BB) a button 0.1s / 2*0.1s
P Pointing with a mouse to a target 0.8s to 1.5s with an average of 1.1s
Can also use Fitts’ Law
D(nD, lD) Drawing nD straight line segments of length lD 0.9s*nD + 0.16*lD
M Subsumed time for mental acts; sometimes used
as ‘look-at’
R(t) System response time, time during which the
user cannot act

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 70

Keystroke-Level Model (KLM)
Operator Description Associated Time
K Keystroke, typing one letter, number, etc. or Expert typist (90 wpm): 0.12s
function key such as ‘CRTL’ or ‘SHIFT’ Averaged skilled typist (55 wpm): 0.20s
Average non-secretarial typist (40 wpm): 0.28
Worst typist (unfamiliar with keyboard): 1.2s
H ‘Homing’, moving the hand between mouse and 0.4s
keyboard
B/BB Pressing (B) or clicking (BB) a button 0.1s / 2*0.1s
P Pointing with a mouse to a target 0.8s to 1.5s with an average of 1.1s
Can also use Fitts’ Law
D(nD, lD) Drawing nD straight line segments of length lD 0.9s*nD + 0.16*lD
M Subsumed time for mental acts; sometimes used 1.35s
as ‘look-at’
R(t) System response time, time during which the
user cannot act

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 71

Keystroke-Level Model (KLM)
Operator Description Associated Time
K Keystroke, typing one letter, number, etc. or Expert typist (90 wpm): 0.12s
function key such as ‘CRTL’ or ‘SHIFT’ Averaged skilled typist (55 wpm): 0.20s
Average non-secretarial typist (40 wpm): 0.28
Worst typist (unfamiliar with keyboard): 1.2s
H ‘Homing’, moving the hand between mouse and 0.4s
keyboard
B/BB Pressing (B) or clicking (BB) a button 0.1s / 2*0.1s
P Pointing with a mouse to a target 0.8s to 1.5s with an average of 1.1s
Can also use Fitts’ Law
D(nD, lD) Drawing nD straight line segments of length lD 0.9s*nD + 0.16*lD
M Subsumed time for mental acts; sometimes used 1.35s
as ‘look-at’
R(t) System response time, time during which the Dependent on the system
user cannot act

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 72

Keystroke-Level Model (KLM)

1. select text field P, BB

2. delete value H, K
3. enter value M, K, K
Sum an amount
Enter From this To this
4. select Euro H, M, P, BB
me
6 currency currency
5. select Dollar M, P, BB
6. select Convert P, BB

Operator Times:
P ≈ 1.1s B = 0.1s H = 0.4s
M = 1.35s K = 0.28s
Total time:
4 · 𝑃 + 8 · 𝐵 + 2 · 𝐻 + 3 · 𝑀 + 3 · 𝐾 = 𝟏𝟎, 𝟖𝟗𝒔

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 73

Keystroke-Level Model (KLM)

1. select text field P, BB

2. delete value H, K
3. enter value M, K, K
Sum an amount
Enter From this To this
4. select Euro H, M, P, BB
me
6 currency currency
5. select Dollar M, P, BB
6. select Convert P, BB

Operator Times:
P ≈ 1.1s B = 0.1s H = 0.4s
M = 1.35s K = 0.28s
Total time:
4 · 𝑃 + 8 · 𝐵 + 2 · 𝐻 + 3 · 𝑀 + 3 · 𝐾 = 𝟏𝟎, 𝟖𝟗𝒔

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 74

Keystroke-Level Model (KLM)
Version 1 Version 2
Hand on mouse, nothing selected, go to photo:
▪ Which is the fastest interface?
▪ Which is the slowest?

Version 3 Version 4

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 75

Summary
▪ The Keystroke-Level Model predicts task completion time for simple dialogs
▪ Assumes a trained average user
▪ Especially useful to compare alternatives
▪ Using KLM by hand can become lengthy and complex
▪ KLM is not useful for tasks that require reasoning

Keystroke-Level Model (KLM) Prof. Dr. Valentin Schwind 76

GOMS
A Human Information Processor Model

Image from: https://pxhere.com/de/photo/779902

The following content is licensed under a Creative Commons Attribution 4.0 International license
(CC BY-SA 4.0)
Prof. Dr. Valentin Schwind 77
GOMS
▪ Goals
(Verbal) description of what a user wants to accomplish
Various levels of complexity possible
▪ Operators
Possible actions in the system
Various levels of abstraction possible (sub-goals / ... / keystrokes)
▪ Methods
Sequences of operators that achieve a goal
▪ Selection rules
Rules that define when a user employs which method (among alternatives)

GOMS Prof. Dr. Valentin Schwind 78

GOMS
GOMS

(CMN-)GOMS KLM NGOMSL CPM-GOMS

• Plain GOMS • Keystroke-Level • Natural GOMS • Cognitive

• Pseudo-code Model Language Perceptual Motor
• First introduced by • Simplified version • Stricter version of analysis of activity
Card, Moran and of GOMS GOMS • Critical Path
Newell • Provides more Method
well-defined, • Based on the
structured natural parallel multi-
language processor stage of
• Estimates learning human information
time processing

GOMS Prof. Dr. Valentin Schwind 79

GOMS Prof. Dr. Valentin Schwind 80
 ▪ GOAL: GET-MONEY
▪ GOAL: USE-CASH-MACHINE
▪ INSERT-CARD
▪ ENTER-PIN
▪ SELECT-GET-CASH
▪ ENTER-AMOUNT
▪ COLLECT-MONEY
▪ outer goal satisfied

GOMS Prof. Dr. Valentin Schwind 81

 ▪ GOAL: GET-MONEY
▪ GOAL: USE-CASH-MACHINE
▪ INSERT-CARD
▪ ENTER-PIN
▪ SELECT-GET-CASH
▪ ENTER-AMOUNT
▪ COLLECT-CARD
▪ COLLECT-MONEY
▪ outer goal satisfied

GOMS Prof. Dr. Valentin Schwind 82

GOMS Prof. Dr. Valentin Schwind 83
GOMS Prof. Dr. Valentin Schwind 84
GOMS Example: Closing a Window
GOAL: CLOSE-WINDOW
[select
GOAL: USE-MENU-METHOD
MOVE-MOUSE-TO-FILE-MENU
PULL-DOWN-FILE-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-ALT-F4-METHOD
HOLD-ALT-KEY
PRESS-F4-KEY]
VERIFY-CLOSE

For a particular user:

Rule 1: Select USE-MENU-METHOD unless
another rule applies
Rule 2: If the application is GAME,
select ALT-F4-METHOD

GOMS Prof. Dr. Valentin Schwind 85

Goals
GOAL: CLOSE-WINDOW
[select
GOAL: USE-MENU-METHOD
MOVE-MOUSE-TO-FILE-MENU
PULL-DOWN-FILE-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-ALT-F4-METHOD
HOLD-ALT-KEY
PRESS-F4-KEY]
VERIFY-CLOSE

For a particular user:

Rule 1: Select USE-MENU-METHOD unless
another rule applies
Rule 2: If the application is GAME,
select ALT-F4-METHOD

GOMS Prof. Dr. Valentin Schwind 86

Operators
GOAL: CLOSE-WINDOW
[select
GOAL: USE-MENU-METHOD
MOVE-MOUSE-TO-FILE-MENU
PULL-DOWN-FILE-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-ALT-F4-METHOD
HOLD-ALT-KEY
PRESS-F4-KEY]
VERIFY-CLOSE
For a particular user:
Rule 1: Select USE-MENU-METHOD unless
another rule applies
Rule 2: If the application is GAME,
select ALT-F4-METHOD

GOMS Prof. Dr. Valentin Schwind 87

Methods
GOAL: CLOSE-WINDOW
[select
GOAL: USE-MENU-METHOD
MOVE-MOUSE-TO-FILE-MENU
PULL-DOWN-FILE-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-ALT-F4-METHOD
HOLD-ALT-KEY
PRESS-F4-KEY]
VERIFY-CLOSE
For a particular user:
Rule 1: Select USE-MENU-METHOD unless
another rule applies
Rule 2: If the application is GAME,
select ALT-F4-METHOD

GOMS Prof. Dr. Valentin Schwind 88

Selection Rules
GOAL: CLOSE-WINDOW
[select
GOAL: USE-MENU-METHOD
MOVE-MOUSE-TO-FILE-MENU
PULL-DOWN-FILE-MENU
CLICK-OVER-CLOSE-OPTION
GOAL: USE-ALT-F4-METHOD
HOLD-ALT-KEY
PRESS-F4-KEY]
VERIFY-CLOSE
For a particular user:
Rule 1: Select USE-MENU-METHOD unless
another rule applies
Rule 2: If the application is a GAME,
select USE-ALT-F4-METHOD

GOMS Prof. Dr. Valentin Schwind 89

GOMS Summary
▪ Characteristics
Can be used to model complex tasks
Clearly sets inner and outer goals
Cannot predict completion times
But the simpler KLM can
▪ Predictions
More operators, longer completion
Deep depth of goal structure → high short term-memory load
Users stop when goals are satisfied

GOMS Prof. Dr. Valentin Schwind 90

Seven Stages of Action
Human-Computer Interaction Lecture

Image from: Photo by Paolo Sacchi / Meet the media Guru from https://www.flickr.com/photos/meetthemediaguru/5553249364/ (CC BY-SA 2.0)

The following content is licensed under a Creative Commons Attribution 4.0 International license
(CC BY-SA 4.0)
Prof. Dr. Valentin Schwind 91
 Image from: Photo by Paolo Sacchi / Meet the media Guru from https://www.flickr.com/photos/meetthemediaguru/5553249364/ (CC BY-SA 2.0)

Seven Stages of Action Prof. Dr. Valentin Schwind 92

Bridging the Gulfs

there should be light!

the gulf of execution the gulf of evaluation

refers to unclear mapping refers to the degree to which an artifact
between intention and provides representations that can be
execution directly perceived and interpreted

a brighter world a dark world

Seven Stages of Action Prof. Dr. Valentin Schwind 93

Overcoming the Gulfs
▪ Question: ”How to bridge the gulfs, especially the gulf of evaluation?”
The solution to this problem is precisely the task of the designer of an interface – the
cognitive effort of the user must be minimized.
A good design should therefore assist the steps of action cycles and allow “a
comfortable transition between the stages”.
▪ Examples:
Is it possible for the user to recognize the system status?
Does the UI provide sufficient feedback on the consequences of an action that could be
executed?
Is the user able to understand the system feedback?
Does the UI provide sufficient feedback for all interpretations that are possible?
Can the user match his/her goal with the (changed) status of the system?

Seven Stages of Action Prof. Dr. Valentin Schwind 94

Seven Stages of Action
goals

evaluate the
intend to act
interpretation

plan sequence of interpret the

actions perception

execute the perceive the state

action sequence of the world

Seven Stages of Action Prof. Dr. Valentin Schwind 95

Seven Stages of Action
record long
goals
videos

evaluate
cannot the
record
clean upto
intend phone
act
interpretation
long videos

delete
plan trash,
sequence interpret
clips the
are very
cache, videos
of actions perception
short

perceive the
perform actions
execute the some videos are
state of the
on mysequence
action phone not recording
world

Seven Stages of Action Prof. Dr. Valentin Schwind 96

Evaluation and Design Questions
▪ Avoid the gulf of evaluation
• Can the user tell what state the system is in?
• Can the user tell if the system is in the desired state?
• Can the user map from the system state to an interpretation?
▪ Avoid the gulf of execution
• Can the user tell what actions are possible?
• Does the device easily support required actions?
• Does the interface help with mapping from intention to physical movement?

Seven Stages of Action Prof. Dr. Valentin Schwind 97

Implications on Design
▪ Critical points
Forming inadequate goal
Not knowing the appropriate action
Not finding the correct action
Receiving inappropriate feedback
▪ Principles of good design
System state and actions are always visible
Good conceptual model with a consistent system image
Interfaces include good mappings that show the relationship between stages
Continuous feedback to the user

Seven Stages of Action Prof. Dr. Valentin Schwind 98

Watch
▪ https://www.youtube.com/watch?v=ahtOCfyRbRg
▪ https://www.youtube.com/watch?v=n4fCHYbRcKw

Seven Stages of Action Prof. Dr. Valentin Schwind 99