the value of experimentation: a guest panel discussion

THE VALUE OF EXPERIMENTATIONA GUEST PANEL DISCUSSION

Host: Tremis Skeete, Principal Product Manager, nexTier InnovationsSponsor: Alpha

PRODUCT THINKING NYC3.29.2017

Ellen Chisa VP of Product Lola

Daniella Patrick Innovation Lab Product Manager Accenture

Jordan Bergtraum Management Consultant (formerly VP of Product Management at ServiceChannel)

Nis Frome Co-Founder, Product Manager & Content Lead Alpha

It’s a future the CEO of Netflix says he couldn’t even have predicted five years ago, when the company was still primarily shipping DVDs to customers in the U.S. while grappling with its emerging video streaming service. Even now, the Internet continues to scramble the game so fast that Hastings said his company is racing to keep up with all the changes.

When asked, he didn’t even want to hazard a guess as to where Netflix might be five years from now.

“We don’t really know,” Hastings said. “It’s not Netflix that’s making the changes. It’s the Internet. We’re figuring out every year how to use the Internet to make a great consumer experience. Every year is an experiment.”

Reference:https://venturebeat.com/2016/05/07/netflixs-reed-hastings-and-ted-sarandos-every-year-is-an-experiment

1. What does it mean when you perform an experiment?

2. How can we communicate the value of experiments?

3. What performing experiments can do for you as designer?

4. What will users get out of being a part of an experiment?

What does it mean when you perform an experiment?

“The objective of a Product Manager is to take an idea and break it down into its core assumptions, and

find ways to test as cheap as possible.”

Experiments enable product managers to have autonomy. They can more effectively push back against feature-creep and design by committee by simply arguing that they are running an experiment rather than launching a product.

Experiments drastically limit brand exposure to prospective customers. A startup can launch a failed product and go back to the drawing board without missing a beat. A major credit card company cannot do the same without risking its reputation and damaging its ability to launch trusted products in the future.

Experiments can transport user feedback from the end of the product lifecycle to the beginning.

Reference: What Product Managers Need to Know About Minimum Viable ExperimentsBy Mike Fishbein, Alpha(http://insights.alphahq.com/minimum-viable-experiments)

“In the past few months, I've been trying to embrace experiments. Instead of waiting until I had a bank of 30 [youtube] videos in place, I just wanted to do a test, see how people responded, and then go from there.

Now, you're probably thinking ... don't you tell your clients to start small, to launch fast, and to experiment? Yes, I do. But, as many of you can probably relate, sometimes we are our own worst client. Because as designers, we want to make sure it's really well done.

So if you are struggling to act on your ideas then figure out how you can start small, learn, and then go from there.”

Sarah DoodyUser experience designer, consultant, and writer. http://www.sarahdoody.com

“User Experience Design is a way to reduce risk, improve productivity, and reduce cost.”

Reference: UX Briefing For Executives & Managers (https://blog.prototypr.io/ux-briefing-for-executives-managers-8331160747f1)

[Product] Pricing is both the crudest, and most subtle, signal that you can send to your customers, and they don’t really

think about their reaction to that signal – they feel it. The only way to really understand your customers and design an effective pricing strategy is to run experiments to work

out exactly how they’ll behave.

Reference: Designing Pricing Strategies: Don’t Ask – Experiment!http://www.mindtheproduct.com/2016/11/designing-pricing-strategies-dont-ask-experiment/

A/B TestingComparing two versions of a web page to see which one performs

better. You compare two web pages by showing the two variants (let's call them A and B) to similar visitors at the same time. The one that

gives a better conversion rate, wins!

Reference: The Complete Guide to A/B Testing (https://vwo.com/ab-testing/)

Reference: RankScience.com

Reference: Casey Winters on Pinterest's Retention Wins, and Why 90% of SEO Advice is Wrong(http://reforge.com/blog/casey-winters-pinterest-retention-seo)

A special thanks to our guest panelists:

A special thanks to Thor Ernstsson, Nis Frome, and the team at…

Alphahttp://alphahq.com

NexTierInnovations.com

AlphaHQ.com

[email protected]