Course

Innovation Implementation

Innovation Implementation

Design Innovation
Experiments
Test your innovation and then make it even
better

19 Minutes

Put your innovation to

the test
You—or you and your group—have a promising
opportunity to explore.

But before you launch your new or improved

product, service, or process—stop. Find out if it
works the way you intended and meets the
anticipated need.

How? By testing it.

Testing doesn’t have to be a long,

drawn-out process. Small,
inexpensive experiments can help
you figure out whether an
innovation will be successful.

By stating—and then testing—one or more

hypotheses about your project, you can make sure
you’re on the right track.

What’s a hypothesis?

The information you gather is also key for

refining your innovation before you launch it
into the world.

Here’s how Southwest Airlines tested innovations

for redesigning their customer experience. Take
note of the steps they went through as they
designed their experiment—you’ll see them in
action as you move through this lesson.

Experiments: Small, Bigger, Biggest

About Bio Transcript

Heather Figallo — Senior Director,

Southwest Airlines

Lee Moreau — Principal, Continuum

Learn how Southwest Airlines used

experimentation to create a new airport
experience for their passengers.

Have an innovation project in mind?

Fill in the Experimentation Plan Worksheet below

as you move through this lesson.

TOOL

Experimentation Plan Worksheet

Download Tool

How does your team stack up?

Make a Connection

LESSON KIT

Before we launch into something new, we test our

idea to gain feedback and insights...

A All the time

B Occasionally

C Never

Decide what you want to

learn
Testing your innovations means you don’t have
to rely on intuition or guesswork to make smart
decisions.

But first, you need to decide what you want to

learn through your experiment. That means
clarifying:

Your core hypothesis

Any related assumptions

Key success criteria and metrics

Core hypothesis

We believe that redesigning the customer

experience from the time customers exit TSA
security to the moment they board the plane will
enable us to improve customer and employee
satisfaction. The new experience should also
reflect the attributes our brand is famous for—a
fun-loving attitude and warm human touch. We
will know our hypothesis is true when we
achieve the success criteria listed below.

Assumptions

Success criteria/metrics

Design your experiment

Experiments come in all shapes and sizes.
Southwest had access to considerable resources to
redesign their customers’ airport experience. But
you can design experiments that will give you
valuable information with far fewer resources.

Consider these four interrelated questions as you

design an experiment:

1. What format will be most effective?

For a product enhancement: You could

create a simple sketch, and have prospective
customers review it and comment on it.

For a new process: You could design a pilot

to have potential users test how easy it is for
them to learn and use.

“If you are not embarrassed of

your prototype, then you are
showing it off too late.”
—Michael Burtov, founder & CEO of
GeoOrbital

TOOL

Pick a Format For Your Experiment

Download Tool

2. Who will your test subjects be—and how

many do you need?
Ideally, test subjects are the people most affected
by your innovation. For example, in Southwest’s
case, test subjects included customers and
employees.

Select the smallest number of users you’ll need to

get maximum insights from the experiment.

3. What data will you collect?

At its simplest, data falls into two categories:

Quantitative data

Is expressed as numbers—like the percent

increase in sales during a specific time period,
or the amount of time it takes someone to
perform a new process.
Helps you answer the questions, “How many?”
and “How much?”
Can generate statistically significant results
that you can compare from experiment to
experiment.

Qualitative data

For example, the Southwest team collected:

Quantitative data through customer and

employee satisfaction surveys

Qualitative data by observing and

interviewing customers

Use both quantitative and qualitative data to

build a complete picture of the feasibility or value
of your innovation.

When deciding what data to collect:

Select the kind that will help you answer

your most important questions.

Resist the urge to rely on whatever data is

easiest to collect or access.

Be sure you’re not looking to confirm what

you already think to be true.

TOOL

Gather Testing Data

Download Tool

4. How will you run your experiment and

collect data?
How you experiment and collect data will vary
widely, depending on what you’re testing.

You could:

Have customers evaluate a physical model

of a redesigned product and complete a
survey to rate their satisfaction.

Bring users of a new process together to role

play the process, then conduct a group
interview to get their impressions.

Watch users interact with your prototype

and make notes of your observations.

Paper prototypes

Developed paper prototypes of new mobile

experience and signage, and tested them with
employees and customers.

Full prototype

Pilot test

Take a Moment

What can your project team test in a rough,

early form? How can you test it?

Due to privacy considerations, please

record your answers separately and mark
when you've completed the task.

Capture what you

learned
After you complete your experiment, analyze the
data you’ve collected. Then, look back at your
original hypothesis and assumptions, and ask:

What did your experiment show? Were your

hypotheses correct? Were your assumptions
proved or disproved?

What do you need to refine based on what

you learned?

Do you need to conduct further

experiments?

Were there unexpected learnings or surprises

that you should document and share with
others in your organization?

Core hypothesis

The team’s core hypothesis was correct:

Customer and employee satisfaction scores did
rise as a result of the redesigned customer
experience.

Satisfaction scores

Qualitative feedback

Employee response

Refine—and test again

As you learn from the results of an experiment,
you may want to make changes to your innovation
—and then test again. *

“Test to improve rather than

prove.”
—Cyril Bouquet, Jean-Louis Barsoux, and
Michael Wade

It’s common to run a series of experiments,

starting with low-risk, low-cost efforts and
increasing risk and costs with each iteration.
That’s what the Southwest team did: They started
out small and moved to progressively more
expensive and more complex experiments.

What does this look like when you‘re operating at

a smaller scale? Let’s see how GeoOrbital, a
Boston-based startup, used iterative design to
invent and reinvent the electric wheel.

Reinventing the Wheel—Literally

About Bio Transcript

Dakota Decker — CTO, GeoOrbital

Michael Burtov — Founder & CEO,

GeoOrbital

Embarrassed by your early prototypes?

Don’t be—it means you’re in an excellent
position to iterate and make them better.

Make a Connection

The GeoOrbital team continually iterated as

they refined their electric wheel. What
iterative experiments have you seen—or
been part of—in your organization?

Due to privacy considerations, please

record your answers separately and mark
when you've completed the task.

Keep experimenting,
launch, or end
At a certain point, your experimenting will be
done. You and your team will need to decide if
you’re ready to launch your innovation.

It may be sooner than you think. Many

organizations are releasing innovations that are
far from perfect. They then use hands-on
experience or customer feedback to drive
continuous improvements.

Sometimes, experiments show your

innovation opportunity isn’t what
you thought.

Maybe your customers don’t see the benefits you

thought they would. Or you discover the costs of
bringing your idea to market are too high.

If your project falls flat, remember that

innovation is about learning. The insights—and
the experience—you gained along the way are
certain to be invaluable as you look toward your
next innovation.

poll

What’s your organization’s approach to

launching an innovation?

Expects new products and services to

be thoroughly tested and refined prior
to launch

Encourages releasing “roughly right”

innovations

Depends on the risk level and

importance of the new innovation

Vote

30-Second Takeaway

Use small, inexpensive experiments to

test whether an innovation will succeed.

Begin by defining what you want to learn.

That means documenting your core
hypothesis, assumptions, and success
criteria.

Experiments come in all shapes and sizes.

Design yours to meet your needs.

Capture what you learned. Analyze

results and document key findings.

Decide what’s next. Keep experimenting

and refining, launch your innovation, or
end the project and focus your innovation
efforts elsewhere.

Take Action

What will you try on the job to design

innovation experiments?

Due to privacy considerations, please

record your answers separately and mark
when you've completed the task.

Lesson Kit
Includes all of your responses, key
concepts of the lesson, handouts, and
worksheets

Download Lesson Kit

Email Me

Ready to Move On?

When you are finished exploring the above
material, mark this portion of the lesson as
complete.

Complete Lesson

Up Next: Build Support for Innovation