how to beta test hardware products in an increasingly complex world
TRANSCRIPT
1
How to Beta Test Hardwarein an Increasingly Complex World
2
Meet the Speaker
Emily Hossellman
Director of Marketing @ Centercode
@ehossellman
Agenda
1. The current hardware landscape
2. The role of customer validation
3. Running a successful hardware beta test
4. Using the data post-beta
5. Q&A
3
4
Who is Centercode?
Services Fully managed tests and programs
Software Complete test management platform
Testers Access to 150k+ deeply profiled global candidates
Content Free tools and best practices
We've been running beta tests since 2001
Beta testing and customer validation are our core competency
5
Current Landscape
6
Current Landscape
1. Tech environments are more complex
2. Customers have higher expectations
3. The spotlight is bright
7
Tech Environments Are More Complex
More devices talking to each other
Products themselves are more complex
Hardware products do not function in a silo anymore
Constant growth and increased fragmentation
You're building for an ecosystem
8
Customers Have Higher Expectations
Decreasing tolerance for friction
Do everything they want
Be intuitive and easy to set up
Instant answers to questions
Personalization and flexibility
Customers are vocal (with the good and the bad)
Many markets don't have established expectations
9
The Spotlight is Bright
Press speculation about who will be the big players
Buzz around product announcements can build hype
A failed launch can be catastrophic to a brand
Tons of companies trying to get into the space
This creates a perfect storm for hardware companies
10
Role of Customer Validation
11
Beta Testing =Customer Validation
Targeted customers using your product in real environments
Increases quality, improves user experience, validates performance
Many names: CAT, UAT, FUT, Customer Evaluation, Field Trials, Pre-release
12
What Beta Brings to the Table
1. Introduces you to your target audience
2. Facilitates a focused, meaningful conversation
3. Gives you the opportunity to really listen
13
What Beta Brings to the Table
Not just technical check → Customer validation
● Sanity check / will they use it
● Out-of-box experience
● Test for interoperability/compatibility
● Prepping support processes
● Marketing insights
● Diversity - more eyeballs, more environments, more use cases
14
Running Your Beta
15
The Process
Planning
Recruiting
Shipping
Managing
Retrieving Units
Closing
16
Planning
Set your objectives / get stakeholders involved early
Visualize your tester team -- core criteria and segmentations
Determine whether you need to do phases
Anticipate unit cannibalism
17
Recruiting
Recruit targeted testers
Build a detailed qualification process
Look for enthusiasm and good writing
Fill the toughest segmentations first
Be efficient with your selections to get a wide variety
18
Shipping
Create a process for shipping/tracking
Mark units and record serial numbers
Plan for failed units
Plan for international shipping
Do 2-day shipping
19
Shipping
Keep your testers in the loop
Confirm shipping addresses
Make testers sign for packages
Include chargers, cords, accessories
Complete the out-of-the-box experience
20
Managing
Map out your test to align with your objectives
Have a support process in place
Triage feedback you receive
Check alignment from the beginning
Consider collecting video
Collect detailed bug reports
21
Managing
Create a great testing experience
Solicit feedback about comfort/usability/appearance
Ask for feature requests
Mimic real world feedback
Test your firmware update process
Regress fixes / get feedback on changes
22
Retrieving Units
Decide whether you need the units back (or if testers can keep them)
Set the return expectations early
Include return shipping labels and return instructions in the initial package
Hold onto rewards until you have the unit back
Process returned units quickly (and tell the tester you received it)
23
Closing
Say thank you!
Identify rockstars and evangelists
Keep testers informed
Get prioritized data to your stakeholders
24
Leveraging the Data
25
Improve Quality
Prioritize your bugs using scoring
Attach test platforms
Use the data to justify changes
26
Prepare Support
Improve documentation based on hurdles testers faced
Prepare for known issues
Build in more customer feedback mechanisms
27
Fuel Marketing & PR
Build a trove of user generated content (quotes, videos, testimonials)
Find what testers loved about your product
Use data to back up marketing claims
Find new use cases
Identify/cultivate evangelists
Publicize your beta success
28
Final Thoughts
Beta testing is your chance to get true customer validation
Plan out your test: priorities and stakeholders
Set up your test well and the feedback will come naturally
Trends will emerge, look for alignment
Listen to the feedback
29
Q&A
Emily Hossellman
Director of Marketing @ Centercode
@ehossellman
Sign up for our newsletter
For more resources, visit our library
30
Thank You!