FAKE DOOR TESTING How to Test Product Ideas Quickly
Jess Lee, CEO & Co-founder, Polyvore
Jess Lee HONG KONG /
STANFORD CS 2004 /
GOOGLE MAPS PM /
POLYVORE CEO /
DRAWING / BACON /
SCI FI & ACTION MOVIES /
jessyoko.com @jesskah jess.polyvore.com
The Evolution of Shopping INFORMATION
SPECS / FACTUAL REVIEWS / PRICE INSPIRATION
TASTE / TRENDS / ASPIRATION
Mix & match products you love • Shop looks created by tastemakers
POLYVORE.COM
POLYVORE APP
The Great Startup Rollercoaster
TIME
EFFO
RT
/ A
NX
IETY
1ST HIRE
FUNDRAISING FIRST N USERS
PRODUCT-MARKET FIT
LIKELIHOOD OF HEAVY DRINKING
SOURCE: HTTP://BLOG.ELADGIL.COM/2010/09/ARE-YOU-ALCOHOLIC-YET-OR-GREAT-STARTUP.HTML
Product-Market Fit Being in a good market with a product that can satisfy that market
Market Whose problem are you trying to solve? How many people have this problem?
Product What solution will best serve this market?
How can I build this solution?
Ask friends / family
CONVENTIONAL WISDOM
Talk to Users
Ask Industry Experts / Entrepreneurs / VCs
CONVENTIONAL WISDOM
Talk to Users
THE UNFORTUNATE TRUTH
People are bad at predicting what they want
People are good at reacting to things that exist
THE SAVING GRACE
THE ONLY SOLUTION
Build & test it quickly
HOW TO BUILD & TEST QUICKLY
Fake Door Testing
Only build the important bits. Fake the rest. Test on real users.
HOW TO BUILD & TEST QUICKLY
Fake Door Testing
EXAMPLE #1
App Notifications
EXAMPLE #2
Universal Shopping Cart
EXAMPLE #3
App Internationalization
EXAMPLE #4
Outfit Sales
List key assumptions
Identify the biggest unknowns
Build the unknowns, fake the rest
Measure your success
Steps for Fake Doors
• People will buy more items if they get a bigger discount
• We can establish partnerships with retailers for this idea
List Key Assumptions IDEA: Shop user-generated outfits and get a sliding discount based on the # of items you buy
• People are interested in shopping outfits
• We can scalably create outfits with products from partners
ASSUMPTIONS:
• We can handle payment, fulfillment, shipping, returns
Identify Biggest Unknowns
• We can establish partnerships with retailers for this idea
• We can scalably create outfits with products from partners
• People will buy more items if they get a bigger discount
• People are interested in shopping outfits
BIGGEST UNKNOWNS:
• We can handle payment, fulfillment, shipping, returns
SEEMS LIKE A SOLVABLE PROBLEM:
• We can establish partnerships with retailers for this idea
• We can scalably create outfits with products from partners
• People will buy more items if they get a bigger discount
• People are interested in shopping outfits
BIGGEST UNKNOWNS:
• We can handle payment, fulfillment, shipping, returns
SEEMS LIKE A SOLVABLE PROBLEM:
Build the Unknowns, Fake the Rest
Build outfit sale UI
Handle shipping ourselves
Show it to real users
Fake the partners
Make the outfits ourselves
Build outfit sales UI
Show it to users
Fake the Partners
Make the outfits ourselves
Handle shipping, fulfillment, returns ourselves
Measure Your Success RESULTS AFTER 3 MONTHS, 63 OUTFITS, $3722.16: • 1.89% conversion rate
• 2.93% checkout rate
• 2.32 items / order
• 21% return rate
LESSONS LEARNED
• People will buy more if the discount goes up
• Our checkout flow with Amazon Payments is terrible
• Needs further improvements to be highly profitable
Need to be tolerant of failure
Define clear metrics & constraints
Don’t betray your users’ trust
Not a replacement for ideation
Tips for Fake Doors
Tolerance for Failure
Clear Metrics & Constraints
Don’t Betray Users’ Trust
Not a Replacement for Ideation
Thank You!