2014 experiments, tokyo

Learn-Measure-Build Craft Useful Experiments By Running Backwards

Janice FraserFebruary 2014

LUXR.CO JUNE 2012

LUXR.CO JUNE 2012

TWEET!

#LeanStartup

@clevergirl@luxrco

LUXR.CO JUNE 2012

• Learn the difference between an experiment and a solution test.• Discover the seven steps to crafting an experiment with a clear

result. • Practice identifying assumptions and defining testable

hypotheses. • Understand the anatomy of an experiment. • Design an experiment to test a real-life hypothesis.

• "Get out of the classroom" to run a real-world experiment on your fellow conference-goers.

• Define a result and make a go-forward decision. • Refine the experiment based on cost, time, and manpower. • Identify the four characteristics of successful experiments.

© 2013 LUXr Incorporated, www.luxr.co

June & Pete’s Wedding Cake


http://saptstrength.com/

Pete June


April

http://saptstrength.com/

May

June

July

August


sedonaweddingcakes.com


freebestpictures.blogspot.com


thekitchn.com


Zoe Clark, The Cake Parlour


sedonaweddingcakes.com

Zoe Clark, The Cake Parlour


LEAN Star tup

© LUXR.CO APRIL 2013

LEAN Star tup


Build measure

learn!

© LUXR.CO 2013

Lean Startup is NOT

Cheap Startup

Fast Startup

Shortcut Startup


An approach for building companies that are creating new products and services in situations of extreme uncertainty.

The approach advocates creating small products that test the entrepreneurʼs assumptions, and using customer feedback to evolve the product, thereby reducing waste.

Lean Startup is...


1. List your assumptions.2. Understand your customers.3. Experiment efficiently.4. Adjust direction based on evidence.


Go backward to go forward.

Learn

Measure

Build

© LUXR.CO 2013

Plot the difference

THINK

release

MAKErelease

MAKErelease

MAKE

RISK

= U

NVAL

IDAT

ED E

FFOR

T

TIME

© LUXR.CO 2013

Lots of little wigglesRI

SK =

UNV

ALID

ATED

EFF

ORT

TIME

© LUXR.CO 2013

RISK

= U

NVAL

IDAT

ED E

FFOR

T

TIME

Each wiggle is a learning cycle.

MAKE

releaseBUILD

LEARN

BUILD

MEASURE

LUXR.CO JUNE 2012

There’s a pattern to startup work:

ideation decisionmaking

ideation etc.

LUXR.CO JUNE 2012

In foolproof experiments, the

results should be self-evident.

LUXR.CO JUNE 2012

1. Choose the problem to tackle. Clarify what’s uncertain, what you need to learn.

2. Understand the possible answers.3. Brainstorm “what you could do to prove it”.4. Choose the most broadly differentiated set of

indicators.5. Run the experiment. 6. Write down the results...on a sheet of paper. 7. Project the results into your future.

7 steps to foolproof experiments

LUXR.CO JUNE 2012

The Workshop

LUXR.CO JUNE 2012

Hypothesis

We believe that modern families would benefit from having better ways to store their

belongings, and are willing to pay for a service that will pick up and store boxes, then deliver

them back home on demand.

Today’s Challenge

LUXR.CO JUNE 2012

Box Me UpProvides you with plastic storage boxesTransports your stuff to storagePay to get your stuff back Per-box per year2800¥ per box per yearoutside of the city (less expensive place)box size: 30 gallon boxservice area is in Tokyoinsurance is 20,000 yen per boxtemperature and humidity are normally conditionedno limit for storage perioe24 hours access to the boxprivately storagepayment method: paypal, square, credit card, no cashdelivery will be by carnothing illegal, perishable, waste will be stored3 employees in the companymain customer will be familiesgot 70,000,000 yen for initial investment

LUXR.CO JUNE 2012

Experiment Framework

For Example:

We believe people like [customer type] have a need for(or problem doing) [need/action/behavior].

The smallest thing we can do to prove that need is [experiment].

We will know we have succeeded when [quantitative/measurable outcome] or [qualitative/observable outcome].

Every experiment has three parts:

1. Hypothesis that is provable/disprovable

2. The experiment itself; the thing you build

3. An indicator of result

LUXR.CO JUNE 2012

{Activity}

Identify assumptions10 assumptions, 1 per sticky

LUXR.CO JUNE 2012

{Activity}

Divide into 2 pilesWill kill the company in the next 6 months

if weʼre wrong

Everything else

LUXR.CO JUNE 2012

{Activity}

Review & de-dupe

LUXR.CO JUNE 2012

{Activity}

Stack rank the urgents

LUXR.CO JUNE 2012

{Activity}

Pick the top one

LUXR.CO JUNE 2012

{Activity}

Design an experiment to learn if this is true.

we believe that ________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

__________________________

Briefly describe it.

State how you will know if the hypothesis is valid or invalid. This can be quantitative evidence or qualitative.

How much time/money/effort will it take?

LUXR.CO JUNE 2012

{Activity}

Discuss, then pick the experiment to run.


__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____________ in 2 days.

Decide


__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____________ in 2 weeks.


__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

__________________________________

____________ in 2 months.

✔

LUXR.CO JUNE 2012

Experiments that were designed

assumption experiment measurable/observable outcome

busy people need a service to help them getmenial stuff done

fb ad, neighborhood targeting presents an offer, landing page. Phone number or tweet. No out

of pocket cost.

10% ctr on ad, 10% ctr on landing page. 2 days


find busy people (in offices). Set up a “shop”. Sit in their reception and see if we can get jobs. Prevail upon friend

who runs 300-person company

5% of employees will make a hire. 1 repeat booking

2 wks


ad in 10 different offices, diff types of companies 2 weeks.

LUXR.CO JUNE 2012

What did you learn?

Top 5 things

{Debrief}

LUXR.CO JUNE 2012

Design the next experiment:

2 days, 2 weeks, 2 months

{Next Steps}

LUXR.CO JUNE 2012

What’s the right size for the experiment?

{Choose}

LUXR.CO JUNE 2012

Techniques Checklist• Brainstorm quietly• Use Sticky Notes for ideas• Self-Edit• Go to the wall for team sharing• Stack-ranking• Roman Voting• Right-Sizing the work

LUXR.CO JUNE 2012

FISHBOWL

2014 experiments, tokyo

Business