UX-Research Methods

Glossary

Attitudinal Methods Analytics

A class of research methods that collects A class of research methods that involve
self-reported data about users’ perceptions collecting real-time usage data for a product.
and attitudes. Attitudinal data is based on Examples of collected metrics include the
“what users say.” Surveys, user interviews, number of user visits, the number of clicks on
and focus groups are attitudinal methods. a particular element, percentage of users who
Attitudinal methods are often contrasted with took a particular action (e.g., checkout, scroll)
behavioral methods, which collect data about on a web page. Analytics research methods
user actions and behaviors. are not controlled: the data collected reflects
users’ behaviors in their natural environment.
Using such methods requires that the product
is instrumented with analytics software such
A/B Test (A/B Testing)
as Adobe Analytics or Google Analytics.
An analytics method that involves randomly
deploying two different versions of a product
to two different user groups in order to Behavioral Methods
identify which works best. The winning
version is usually selected based on metrics A class of research methods that collect data
such as conversion rate or click-through rate. reflecting users’ actions and behaviors.
To conduct an A/B test, you will need to install Unlike attitudinal methods, which are based
specialized analytics software. on “what users say,” behavioral methods are
based on “what users do.” Examples of
behavioral methods include usability testing
and analytics methods.

nngroup.com/articles/research-methods-glossary Page 1 of 6
Card Sorting Contextual Inquiry
A research method in which study participants A type of field study in which the researcher
group individual labels according to criteria watches the user as they naturally perform
that make sense to them. This method helps their task in their normal environment (e.g.,
designers to group items into categories and home, office) and asks for information to
create an information architecture of a site or understand how and why the user did what
application. Card sorting can be “open” (if the they did. Contextual inquiry is a combination
categories are not defined in advance of the of in-depth observation and interview and
study and participants group similar items into aims to gain a robust understanding of work
clusters) or “closed” (if a predefined set of practices and behaviors. Contextual inquiry is
categories is given to participants and they typically conducted in the early discovery
are asked to assign items to these categories). stages of a project.

Clickstream Analytics Desirability Study

An analytics method that involves analyzing A research method that involves exposing
the sequence of pages that users visit as they participants to a product and then asking
use a site or application. It can provide them to select a few words that describe their
insights about potential issues, typical reaction to that product out of a list of
navigation routes, and the content that users possible options. This type of method can
interact with right before completing key bedone qualitatively or quantitatively, to
actions on a site or in an application. assess the aesthetic properties of an image
and whether they are consistent with a desired
brand image.
Concept Testing
An attitudinal research method that involves Diary Study
collecting users’ thoughts and attitudes about
a product idea (“concept”) in its incipient A research method used to collect self-report-
stages, usually through a qualitative survey. ed data about user behaviors, activities, and
It is used very early in the discovery phase of experiences over an extended period that can
the design process to understand whether range from a few days to months. During that
a specific product idea meets users’ needs period, study participants are asked to keep
and expectations. a diary and log specific information about the
activities of interest.

Context Methods
Ethnographic Study
A class of research methods that involves
capturing the context in which users naturally A class of qualitative research methods that
engage in a specific behavior. The context involves observing users in their natural
can be captured directly, by observing users habitat. In UX, the term is used as a synonym
in their natural environment (field studies), for “field study.” However, in social sciences,
or indirectly, by asking them to log context ethnographic studies involve immersion in a
details whenever that behavior occurs particular culture or community, to observe
(diary studies). the behaviors and rules of that community.

nngroup.com/articles/research-methods-glossary Page 2 of 6
Eyetracking
Five-Second Test
A behavioral research method that involves
tracking users’ eye movements as they An attitudinal research method in which a
interact with a product or perform a specific study participant is shown a design for five
activity, to determine where they focus their seconds and then asked to describe what they
attention. Eyetracking studies require special saw. A five-second test is meant to gather
equipment to capture participants’ eye move- users’ first reactions to the aesthetic qualities
ments. Eyetracking data can be used to of a design.
understand which design elements attract
users’ attention and which are ignored.
Focus Group
A qualitative, attitudinal research method in
Field Study which a facilitator conducts a meeting with
A research method that involves observing the a group of 6–9 people to discuss their
users in their natural context. Field studies are experiences with a product or service.
a type of context method. Field studies vary in The term “focus” relates to the role of the
the amount of interaction between the facilitator, who maintains the group’s focus
researcher and the participant. Some field on certain topics during discussions. Focus
studies are purely observational, while others groups are used in the early discovery stages
involve active participation from the of product development to gauge users’
researcher in the form of interviews or mental models and expectations.
tasks given to the user.

Formative Method
First-Click Test Any method that focuses on determining
A behavioral, task-based research method in which aspects of the design work well or not,
which participants are given a specific task and why. Formative evaluations occur
before they are exposed to a design and then throughout a design or redesign and provide
stopped immediately after their first click on information to incrementally improve the
the corresponding screen. This test works best interface. Formative methods are often
when users have a specific goal in mind the contrasted with summative methods, which
first time they encounter the site. The are used after a design has been finalized to
first-click test is similar to the five-second test provide an overall assessment of its usability.
in that they are both very quick; however, in
the first-click test, participants are likely to
ignore most page elements that are not Moderated Usability Testing
directly related to their goal.
The traditional type of usability testing which
involves a facilitator (moderator) and a
participant interacting synchronously. The
facilitator gives participants tasks to do one
at a time and may probe with further questions
and clarifications; they can also ask the
participant to stop when they consider
it necessary.

nngroup.com/articles/research-methods-glossary Page 3 of 6
Multivariate Test Prototype Testing
A design-optimization method in which A type of usability testing in which the
multiple variants of specified design elements interface being tested is a design prototype
are tested in a user interface, with the goal of rather than a live product. The prototype can
maximizing an analytics metric such as be presented to the participant on paper
conversion. This method determines which (paper prototyping) or using interactive
combination of the variants results in the prototyping software. Prototype testing is
highest-performing design. For example, used before a design is implemented to
a multivariate test could be used to determine identify potential usability issues and fix
whether a button should be labeled Place them, or to explore how alternative design
Order or Submit and whether it should solutions fare with users.
be blue or red.

Qualitative Method
Participatory Design
A type of research method that aims to collect
A research method in which one or several observational data about users’ behaviors and
users are invited to offer their own solution interactions. Such data may identify whether
to a particular design problem. Participants particular aspects of the interface are easy or
may be provided with some basic building hard to use. Focus groups and user interviews
blocks that they could use to create their are examples of qualitative methods. Usability
designs. The resulting designs are not testing can also be qualitative when used to
usually implemented, but rather used to uncover issues in a design.
get an understanding of users’ needs
and expectations.
Qualitative Usability Testing

Paper Prototyping A type of usability testing that aims to collect

observational data about user behaviors and
A type of usability testing that is related to interactions and that is often used to identify
the Wizard of Oz method and in which an problems in an interface. Qualitative usability
early-stage design is shown to the participant testing can be moderated or unmoderated.
on paper. The piece of paper schematically
represents the prototype of a specific page.
The participant indicates to the facilitator
Quantitative Method
which action(s) they would take on that page
in order to complete the assigned task. The A type of research method that collects
facilitator (or another person present in the metrics such as success, satisfaction,
room who plays the role of the computer) then conversion, task time, or number of user
changes the page to reflect the new state of visits. Quantitative methods focus on
the system. Paper prototyping is a type of numbers. Examples of quantitative methods
prototype testing. include analytics-based methods and
quantitative usability testing. In quantitative
usability testing, metrics such as task time
and success are gathered in order to assess
whether particular tasks are easy to perform.

nngroup.com/articles/research-methods-glossary Page 4 of 6
Quantitative Usability Testing Summative Method
A type of usability testing that collects A research method that focuses on evaluating
metrics such as task success, user the user experience of a product, usually by
satisfaction, and time on task to understand comparing it with that of a competitor or of
whether particular tasks were easy to one of its own prior versions. Quantitative
perform. Quantitative usability testing is usability testing is usually used as a
usually used to assess the overall experience summative method.
of a product and make inferences about user
behavior in the target user population.
Survey

Remote Research Method A research method in which a participant

responds to multiple-choice or open-ended
Any type of research method that involves a questions that are presented to them online,
facilitator and one or more participants and in on paper, or by phone. Surveys are an
which the participant is in a different location attitudinal research method that collects
than the facilitator. Usability testing, focus participants’ self-reported perceptions and
groups, user interviews, and contextual attitudes. Surveys can be used to collect
inquiries can be done remotely, with various qualitative or quantitative data.
degrees of success.

Task Analysis
Remote Moderated Usability Testing
A research method that studies how users
A type of user testing in which the facilitator perform a specific task: their goals, the
and the participant are not in the same room different steps they take, the order in which
and interact through video-conferencing they do them, when and where they do it, and
software. what information they need during the task.
Task analysis often involves a mix of interviews
and context methods such as contextual
inquiry, field studies, or diary studies. It is
Retrospective Think-Aloud Protocol
used to inform the design of complex
A variation of the think-aloud protocol in workflows for a product.
which participants are asked to explain their
behavior immediately after they interact with
a product. In a variation of the protocol, Think-Aloud Protocol
participants may be shown a replay of their
interaction and asked to explain out loud what The practice of asking a usability-testing
they were thinking when they were doing participant to “think out loud” as they are
those actions. Retrospective think-aloud is interacting with an interface – in other words,
used in situations where thinking out loud to verbalize their actions and thoughts. It is
would be too distracting for participants widely used in qualitative usability testing to
(e.g., during complex activities) or would supplement the behavioral data provided by
contaminate data (e.g., in eyetracking or the participant’s actions.
quantitative usability testing).

nngroup.com/articles/research-methods-glossary Page 5 of 6
Tree Testing User Interview
A task-based research method that evaluates A one-on-one attitudinal research method in
a hierarchical category structure (or tree) by which an interviewer asks a participant ques-
having users find the locations in the tree tions about a topic, listens to their responses,
where specific resources or features can be and follows up with further questions to learn
found. It is an information-architecture more details. User interviews can be used by
method used to assess how well the themselves in discoveries to inform the early
navigational hierarchy of a site matches stages of product design or can be combined
users’ expectations. with other methods such as contextual inquiry
and usability testing.

Unmoderated Usability Testing

Usability (User) Testing
A type of usability testing in which there is no
facilitator (moderator) and in which the tasks A research method in which a researcher
are presented to the participant by special (called a “facilitator” or a “moderator”) asks a
software. Unmoderated testing is usually participant to perform tasks, usually using
remote, with the participant testing the one or more specific user interfaces. While
software at their location of choice, through the participant completes each task, the
an Internet connection. However, it is possible researcher observes the participant’s behavior
to run unmoderated usability tests in person, and listens for feedback. Usability testing can
by having the participant interact by be qualitative or quantitative. Qualitative
themselves with the interface of interest. usability testing is used to identify problems
in an interface, whereas quantitative usability
testing focuses on collecting metrics that
help assess the overall user experience
Usability (UX) Benchmarking
of the product.
A type of summative research method that
involves evaluating a product’s user
experience by comparing its performance Wizard of Oz
against a competitor or a previous version
of the same product. Many UX benchmarking A moderated research method in which
studies are quantitative usability-testing a participant interacts with a prototype
studies in which participants perform a set manned by a human who controls the system
of top tasks; metrics such as task success, responses. This method is used to test early
task time, and satisfaction are collected stages of the design, before investing
and compared across product versions resources in developing it. It is also used
or competitors. to understand users’ expectations, mental
models, and vocabularies.

nngroup.com/articles/research-methods-glossary Page 6 of 6