eewei@emoti.vu / @emotivu / Emoti.vu

Metrics Driven DesignBetter define success, deliver impact, make money

Eewei Chen@Ultraman

http://www.ml.sun.ac.za/wp-content/uploads/2009/08/social_graph.png

“Efforts and courage are not enough without purpose and direction.”John F. Kennedy

20 years

What is success?

Stories, workflows, wireframes, interactions

My Flights

ALERT:Manila to Cebu

Book A Flight

Get The Best Exchange Rates

Flight Status

CP100 - OK

Check-in

Available (1)

Destination Guides CEB Club

Commercial Partners About Cebu Pacific

Sign-inRegister

Shake Iphone to view more Special Flight Offers

MANILA TO

BORACAYOne way from

P780

DISRUPTION: Manila to Cebu delayed

Abstract Concrete

UX Assets?

Working software

Burn down

Collaboration

Refactoring

Agile

good enough

Disruption

no longer exist

no one can live without

Survival?

Can UX success be measured?

Pack selection

+ 213%Checkout completion

+ 252%User satisfaction*

+ 237%

Sky Upgrades

Free Trial uptake

+ 137%Subscriptions uptake

+ 187%User satisfaction*

+ 223%

Now TV Subscriptions

Query resolution

+ 120%Info retrieval speed

+ 239%User satisfaction*

+ 174%

Sky.com Nav

Problem: Poor trial and subs uptake

Solution: • New marketing animation• More effective brand messages• Timed marketing triggers• Remote usability test (2K users)• 4 week POC, 12 weeks

Problem: Too many choices, low upgrade conversion

Solution: • Usage based recommendations • 9 packs down to 3• Quick check out• MVT Testing to 1% traffic• 3 week POC, 8 weeks delivery

Problem: Inaccurate FAQs and MySKY tasks

Solution: • Super nav drop down• Quick tasks• Simplified IA• Remote usability test (2K users)• 2 week POC, 6 weeks delivery

*In App customer survey feedback form

BUT what if there isn’t much to go on?

(Case study)

Digital Airline Experiences are fragmented

Fear & Punishment works??

Apple wants it all

Many platforms

Stay connected with travellers

• Data: Passenger info• Need: More connected mobile digital experience• ROI: Mobile Ad wallet share• UX: Useful traveller engagement throughout lifecycle

Airline

Airline

Airline

Advertiser

Advertiser

Advertiser

Opportunity

Mobile Advertising

US$11.4 Billion (i) Mobile Ad spend in 2013

£500 million UK spend in 2012 is most per user (ii)

(i)  -­‐  h'p://www.gartner.com/newsroom/id/2306215(ii)  -­‐  h'p://www.guardian.co.uk/media/2012/oct/09/mobile-­‐adverDsing-­‐500-­‐million-­‐pounds-­‐2012

“Connecting

with Travellers”

TrendGap

Channel

Vision

Data (any)

Insight

Business Traveller

going on a Business Trip

buys a ticket 7 Days or less before travel

52/41 = (127) index

Business Traveller

• 23% of all travellers, Frequent flyer (175)• male (123), 50 Years Old (139), Company Chairman (233)• Earns 90K / month (133)• Android smartphone / tablet user (NA)• Unaware of Airline Loyalty Program (109)

Behaviours (unique insight)

• Books travel & hotel < 7 days in advance (127)• Pays for tickets at airline ticket office (115), airport (117) airline call centre (111) and travel agent (115). • Checks-in at airline desk (106)• Buys advertised goods at airport (128), in-flight (113), later (117). Spends over 250 (191) on perfume, electronics, watches, sunglasses, toys, DVDs/Music• Research In-Flight magazine (114) and travel sites (132). • Plans next trip going to airport (129), in airport lounge (129) and on the flight (111)• Uses laptop on the plane (124)• Doesn’t value airline membership (67% of business)

Personality• Last minute business trip planner • Appreciates quality• Will use something if it makes his life easier• Wants respect and a personalised service. • Prefers human face-to-face contact• Impulse buyer, influenced by advertising• Likes to stay informed and up-to-date • Wants to maximise work time

Barriers• Does not trust social media recommendations• Does have time to do much research• Money not an issue but wants to know it is well spent

Output, Outcome, Impact

• Deal alerts <7 days before travel to increase Deal

Section Visits by 25% & Ad CPM by 25%• Improve mobile usage (095) and mobile boarding pass value (101) to reduce non-web and non-mobile bookings (all of above)• Faster mobile check-in (092) to reduce physical in-person check-ins (by all of above) and check-in time by 25%.• Product and service recommendations (200): business (1.35), food (1.06), hotel (1.03) and consumer products(1.04), entertainment (1.05) to improve Ancillary CPM by 200%• Personalised Ads: Automotive (1.31), Corporate (1.29), Conference (1.19), Electronics (1.10), Tourism (1.01) and in-flight retail choices (0.81) to increase online/mob CPM and conv (TBD).• Travel updates (0.94) and info to increase recommend to friend (092) and repeat booking purchase (101)• In-flight wifi to increase laptop usage (124)•(numbers)  represent  indexed  data  analyDcs

Deal Alerts <7 Days in advance of travel

Data

Empathy

Books travel & hotel <7 Days in advance (127)

Deal Alerts <7 Days in advance of travel

Increase Ancillary CPM rate by 25%

Does not have time to do research

Increase PVs by 25%

Last minute planner

OUTPUT

OUTCOME

IMPACTBarriers

Personality

Behaviour

(numbers)  represent  indexed  GPS  data

Success

OutputSoftware we build. The materials we produce.

OutcomeThe immediate change due to delivering output.

ImpactThe larger change we see over time

Outcome

Impact

Output

Assumptions

We believe by Output, we will Outcome* leading to Impact*

We believe by creating deal alerts, we will increase visits to deals section by 25% leading to an increase in Ad CPM by 25%

*QualitaDve  or  quanDtaDve  measurement

Outcome

Impact

Output

What do we do first?

Value (relative)

EASY

HIGH VALUE

DIFFICULT

Quick wins Differentiator

Parity / building blocks Bin

LOW VALUE

Deal Alerts

Home screen advert

Basic geo-location

*if  we  had  to  build  the  capability  ourselves.  Therefore...  Get  client  to  build  and  enable

Mobile CMS

In-flight wifi*iOS Push Alerts

ROI:Customer value x PersonasBusiness value ABusiness value BEase of implementationDependency level

WireframesDeal Alerts <7 Days in advance of travel

My Flights

ALERT:Manila to Cebu

Book A Flight

ALERT / ADVERT

Flight Status

CP100 - OK

Check-in

Available (1)

Destination Guides CEB Club

Commercial Partners About Cebu Pacific

Sign-inRegister

Shake Iphone to view more Special Flight Offers

MANILA TO

BORACAYOne way from

P780

DISRUPTION: Manila to Cebu delayed

Flight notifications

Ancillary service notifications

OFF

Connect with Facebook

Disconnect Twitter

Connect with LinkedIn

Settings

Share this app

Contact us

Edit your personal details

CEB Club profile

Get in touch

Social Networks

Push notifications

ON

CEBU PACIFIC: You've just arrived. Book a taxi

and save 15% today

CEBU PACIFIC: Discounted fares to Boracay this

week starting from PHP 780

SPECIAL OFFERS

Discount Airtime Tariffs

SPECIAL OFFERS

ALERT / ADVERT

MANILA TO

BORACAY

One way from

P780.....Terms & ConditionsAdditional info on Special offers

BOOK NOW

Done

Outcome

Set explicit notifications (time, location, price etc.)

Receive well timed push notification

See on page alerts on key pages* (e.g. HP)

Click to view target page (Destination Page + AD)

*Assume  persistent  signed-­‐in  state

Impact

Output

Output

Output

What more can we measure?

Passenger  info  from  Airline

Origina'onDes'na'onGenderDeparture  dateReturn  dateLanguageCabin  classAgeAirline  loyalty  levelTicket  type  (open  or  return)Na'onality

Implicit

Click  path  analysisPage  view  /  PreviewTime  on  pageShared  on  FB  and  TwiHerEmailGeo-­‐locate  /  Check-­‐inTime  of  day

Implicit  but  heavy  AI,  Machine  Learning*

Sentence  analysisMood  analysisOther  behavioural  paHern  analysis

Explicit

WizardsQues'onnairesBookingSearchSaveSeSngs  /  PreferencesLikes

Good Start Differentiation

Track it

What about making money?

Success can be Everywhere*

Triggers*

Holistic

AARRR!**

*BJ  Fogg,  Behavioral  Analysis:  h'p://www.behaviormodel.org/**  Dave  McClure:  Pirate  Metrics:  h'p://www.slideshare.net/dmc500hats/startup-­‐metrics-­‐for-­‐pirates-­‐long-­‐version

Don’t forget to experiment

*In App customer survey feedback form

Validation

• Guess

• Consequence

• Experiment

Cornell U, 1964

• Assumption

• Validate

• MVP

September 2011

Richard Fenyman Eric Ries

If it disagrees with experiment, it’s WRONG!

In Conclusion

• Problem / Goal• Opportunity• Data• Success= Outputs, Outcomes, Impact• Assumptions• Empathy• Value Map• Wireframes• Track & Capture• Make Money• Experiment

Metrics Driven DesignHow to define success and deliver more impact

Eewei Chen@Ultraman

http://101DesignIngredients.com