User Interface
Evaluation
Prepared By: Dawn Menor, MSIT
Evaluation Criteria
❏ When evaluating the interaction model and interface, there are
largely two criteria.
❏ Usability
❏ User Experience (UX)
Usability and User experience (UX)
❏ Usability refers to the ease of use and learnability of the user
interface.
❏ Usability can be measured in two ways, quantitatively or
qualitatively.
❏ User experience (UX) is the other important aspect of interface
evaluation. There is no precise definition for UX.
❏ UX is generally accepted that the notion of user experience is
“total” in the sense that it is not just about the interface, but also
something about the whole product/application and even
extends to the product family (such as the Apple® products or
MS Office).
Quantitatively or Qualitatively Assessment
❏ Quantitative assessment often involves task-performance
measurements. That is, we assume that an interface is “easy to use
and learn” (good usability) if the subject (or a reasonable pool of
subjects) is able to show some minimum user performance on
typical application tasks(task completion time/task completion
amount).
❏ Qualitative assessment often includes questions involving the
ease of use, ease of learning, fatigue, simple preference, and
other questions specific to the given interface.
Excerpts from the NASA TLX
Usability Questionnaire. The NASA
Task Load Index method assesses the
workload on a seven-point scale.
Increments of high, medium, and low
estimates for each point result in 21
gradations on the scale.
Excerpts from the IBM
Usability Questionnaire
for computer systems.
Various aspects to be considered in totality for assessing user experience (UX).
Evaluation Methods
Evaluation Methods
❏ Overall, an evaluation method can be characterized by the
following factors:
❏ Timing of analysis (e.g., throughout the application
development stage: early, middle, late/after)
❏ Type and number of evaluators (e.g., several HCI experts vs.
hundreds of domain users)
❏ Formality (e.g., controlled experiment or quick and informal
assessment)
❏ Place of the evaluation (laboratory vs. in situ field testing)
Focus Interview/Enactment/Observation
Study
❏ One of the easiest and most
straightforward evaluation
methods is to simply interview
the actual/potential users and
observe their interaction
behavior, either with the
finished product or through a
simulated run.
A cognitive walkthrough with the
interviewer.
Expert Heuristic
Evaluation
❏ Expert heuristic evaluation is
very similar to the interview
method.
❏ The difference is that the
evaluators are HCI experts and
that the analysis is carried out
against a pre prepared HCI
guideline, called heuristics.
❏ Heuristics are mental shortcuts
that can facilitate
problem-solving and probability
judgments or or self-discovery.
Measurement
❏ Measurement methods
attempt to indirectly quantify
the goodness of the
interaction/interface design
with a score through
representative task
performance (quantitative) or
quantified answers from
carefully prepared subjective
surveys (qualitative). A case of a task-performance measurement: (1)
nominal: a game interface using a keyboard, and
(2) new: a game interface using a new controller.
Safety and Ethics in Evaluation
❏ Most HCI evaluation involves simple interviews and or carrying out simple
tasks using paper mock-ups, simulation systems, or prototypes. Thus,
safety problems rarely occur. However, precautions are still needed.
❏ Some seemingly harmless tasks may bring about unexpected harmful
effects, both physically and mentally.
❏ Evaluations must be conducted on volunteers who have signed consent
forms.
Summary of the Measurement Method
Note: More reliable results, but generally time
consuming to prepare and conduct the process.
