insight types that influence enterprise decision makers (christian rohrer at enterprise ux 2015)
TRANSCRIPT
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Christian P. Rohrer, PhD | Chief Design Officer, Intel Security, Consumer Products
Insight at Scale: Insight Types That Influence Enterprise Decision Makers
Presented at UX Enterprise Conference, San Antonio, May, 2015
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Topics and Overview 1. Context
• Me: My background and perspective • You: Internal or External to the “Enterprise”
• Them: Customers, Administrators, End Users
2. UX Model and Research Methods
• User Needs are Central to UX
• User Research Methods • Research Methods Across Product Development
3. Insight Challenges and Opportunities
• Access to Insights • Insights that Drive Decision Making
• The Golden Trapezoid
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Context
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 4
1987 1999 1992 2004
Chief Design Officer VP Experience Design
Consumer, Mobile Enterprise
Senior Director of User Experience Design
Director of UER and Product Research
Director of User Experience Research
Unix/Xenix Tech Support Engineer & Trainer
UC Santa Cruz Computer Science
(Honors BA)
Stanford University Cognitive Science,
Education, HCI (PhD)
1998
Network Computer, Inc. (aka Liberate)
The Santa Cruz Operation (aka Caldera)
2011
A bit about me and my perspective
2015
Ethnographic Field Researcher
2008
*
* Intel acquired McAfee, March, 2011. How many of you knew?
*
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Intel Security Experience Design Team ≈ 50 Amazing people | 6 offices around the world
5
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
“Security” Great User Experiences
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Enterprise Employee
Enterprise Employee
Now let’s talk about YOU… Are you an Internal or External Enterprise UX team?
Enterprise Employee
Enterprise Employee
Enterprise Employee
Enterprise Employee
Enterprise Employee
Enterprise Employee
An “Enterprise” (company)
2. A B2B business that
builds products for external “enterprise” customers?
Design User Research
Product Mgmt
1. An IT or internal department, building for
internal customers?
Design User Research
IT Develop-
ment
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
THEM: these Enterprise Employees, who are “THEY?”
8
VP/Director Tech Ops CISO CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Enterprise Employee
They could be many types of people – for example:
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
In the world of enterprise security, THEY might be classified into something like this:
9
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
“Users” “Choosers”
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Customer
Customer
Or more simply: { End Users | Admins | Customers }
Customer
Customer Admin
Admin
End User
End User
An “Enterprise” (company)
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 11
What do “They” need to do to succeed? Potential examples from an Enterprise context:
Buy Deploy Control Report Use Not Get Fired
Customer
Admin
End User
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 12
UX Model and Research Methods
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 13 See also, Jesse James Garrett’s The Elements of User Experience
Christian Rohrer (2006-2015)
User Needs
A Simple Model of User Experience
ms secs mins Insights on these layers?
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
User Research Methods • Many methods available • Many pros & cons
• But what to use and when?
• Methods on a 3D landscape: o Qualitative vs. Quantitative o Behavioral vs. Attitudinal o Context of Product Use
14
?
?
?
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 15
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 16
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 17
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 18
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 19
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 20
• Natural: Examine natural behavior and attitudes • Scripted: Focus insight topics or enforce consistency • De-contextualized: Issues under study are broader than product usage • Hybrid: Creatively use limited form of product use to meet research goals
Context of Use
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 21
Usability Lab Studies
• Single-user using product/prototype
• Lab-based • Usually scripted • Behavior > Attitude
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 22
Ethnographic Field Studies
• A technique inspired by the field method used by sociocultural anthropologists
• Observation of work or natural use of products
• Goal: understand through the eyes of the observed
• Of all qualitative methods, the most powerful/flexible
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 23
Focus Groups & Interviews
• Group or one-on-one discussions about specific topics (brand, marketing, or products/services)
• By definition, only talking – not using product/prototype
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 24
Clickstream Analysis & A/B
Testing • Analysis of data
stored in logs (web or SW telemetry) on what users click
• A/B Tests: give a random sample of users alternative version of website design; compare logs of behavior to current site design Image source: TechCrunch
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 25
Surveys • Asking large
numbers of users what they think in a structured way
• Email surveys: invite participants via email
• Intercept surveys: randomly invite a percentage of users on a site or using an app for their opinion about the site/app
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 26
• Do we have the right idea? • If not, what is better? • Who is the target? • What makes it work? • Can be Qual or Quant
Concept Testing
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 27
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 28
Google “user research methods” for more info on the landscape:
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 29
Insight Challenges and Opportunities
Soooooo many potential sources of information….
30
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Enterprise Users and Customers
Security/ Systems Architect
Let’s start with an Internal UX team, developing solutions inside of an organization
31
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
An “Enterprise” (company)
The Good News: • Easy access to
users and customers
and • No competition!
(Wait, is this good?)
1. An IT department with a UX team, building for
internal customers
UX/Design
Prog Mgmt
IT Develop-
ment
The challenges for an Internal UX team, developing solutions inside of an organization
32
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
An “Enterprise” (company)
The Bad News Challenge: • No competition therefore • No pressure to
deliver a high quality UX
1. An IT department with a UX team, building for
internal customers
UX/Design
Prog Mgmt
IT Develop-
ment
The challenges for an Internal UX team, developing solutions inside of an organization
33
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
An “Enterprise” (company)
Challenge: • Internal
customers: rarely held to “good UX” as a measure of success
1. An IT department with a UX team, building for
internal customers
UX/Design
Prog Mgmt
IT Develop-
ment
Internal Customer
“Good UX? Meh.”
The challenges for an Internal UX team, developing solutions inside of an organization
34
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
An “Enterprise” (company)
Challenge: • No time to do
User Centered Design 1. An IT department with
a UX team, building for internal customers
UX/Design
Prog Mgmt
IT Develop-
ment
The challenges for an External UX team, developing solutions for an external enterprise customer
35
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Enterprise Users and Customers
2. A business that builds
products for external
“enterprise” customers
UX/Design
Develop-ment
Product Mgmt
Challenge: There may be a specific department (e.g., Sales) that owns the relationship with the enterprise customer…
36
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Enterprise Users and Customers
2. A business that builds
products for external
“enterprise” customers
UX/Design
Develop-ment
Product Mgmt
Challenge: You may be blocked from accessing the enterprise customers and their users if the benefits of doing so aren’t clear
37
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Enterprise Users and Customers
2. A business that builds
products for external
“enterprise” customers
UX/Design
Develop-ment
Product Mgmt
Challenge: If you DO manage to get access to enterprise customer information, it will often be about BUYING DECISIONS
38
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Business Users and Customers
2. An business that builds
products for external
enterprise customers
UX/Design
Develop-ment
Product Mgmt
What to BUY
How are BUYING DECISIONS typically made?
39
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Business Users and Customers
2. An business that builds
products for external
enterprise customers
UX/Design
Develop-ment
Product Mgmt
CUSTOMERS (deciding to buy)
• How much does it cost? • Does it “check all the
boxes” I’m supposed to check?
• Will it get me fired if it doesn’t work?
“What to Buy” leads to “What to Build”
40
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Business Users and Customers
2. An business that builds
products for external
enterprise customers
UX/Design
Develop-ment
Product Mgmt
Decision: What to BUILD What to BUY
Challenge2: How good is this “data”? Will this serve all in the ecosystem?
41
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Business Users and Customers
An business that builds
products for external
enterprise customers
UX/Design
Develop-ment
Product Mgmt
Anecdotal
Third-hand
Self-Reported
Conflicting
As with Consumer, there are many types of enterprise “Users”
42
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Business Users and Customers
2. An business that builds
products for external
enterprise customers
UX/Design
Develop-ment
Product Mgmt
USERS
End Users: “If I have to use it, can it at least be USABLE?” Better yet: USEFUL
43
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Business Users and Customers
2. An business that builds
products for external
enterprise customers
UX/Design
Develop-ment
Product Mgmt
“Make the
product USABLE”
Admin users: “I have so little time - don’t make me think. Fit into my workflow. Most importantly, don’t let me screw up.”
44
Decision Makers
Tech
nica
l Sop
hist
icat
ion
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Business Users and Customers
An business that builds
products for external
enterprise customers
UX/Design
Develop-ment
Product Mgmt
“Make it EASY TO DEPLOY,
MAINTAIN, and USEFUL in my
workflow”
Challenge: Insights from different sources from different perspectives to different decision makers = CONFLICT
45
VP/Director Tech Ops
CISO
CIO
Security/ Systems Architect
Network Ops
Manager
Desktop Ops
Manager
Employee End User
Employee End User
Business Users and Customers
An business that builds
products for external
enterprise customers
UX/Design
Develop-ment
Product Mgmt
Insights on:
product USABILITY
Insights on: what makes these
decision-makers BUY Inevitable
Conflict !
Insights on: DEPLOY,
MAINTAIN, and workflow USE
Customer
Customer
It’s a complex ecosystem… Q: How do we make sense of it?
Internal Customer
Customer Admin
Admin
End User
End User
B2B
UX/Design
Develop-ment
Product Mgmt
Internal/IT UX/
Design Prog
Mgmt
IT Develop-
ment
SALES
A: In the context of TIME…
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Understand Conceive Design Develop Launch
Strategize Optimize Assess
Key Q: How well did we do?
Key Q: How shall we do it?
Key Q:
What shall we do? &
Why
?Qualitative & Quantitative § Ethnographic Field Studies § Interviews/Focus Groups § Clickstream Analysis § Survey results (NPS) § Concept Testing
Mostly Qualitative § Participatory design § Concept Studies § Prototype Evaluations § Usability (lab) studies § Desirability studies
Mostly Quantitative § A/B Testing § Usability benchmarks § Online UX Assessments § Surveys (CSAT, NPS) § Clickstream Analysis
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Strategize
Key Q:
What shall we do? &
Why
User Needs
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Strategize
Key Q:
What shall we do? &
Why
Qualitative & Quantitative § Ethnographic Field Studies § Interviews/Focus Groups § Clickstream Analysis § Survey results (NPS) § Concept Testing
User Needs
Customer
End User
UX/Design
Product Mgmt
Admin
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Optimize
Key Q: How shall we do it?
User Needs
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Optimize
Key Q: How shall we do it?
Mostly Qualitative § Participatory design § Concept Studies § Prototype Evaluations § Usability (lab) studies § Desirability studies
End User
UX/Design
Admin
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Assess
Key Q: How well did we do?
?User
Needs
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
Assess
Key Q: How well did we do?
?Mostly Quantitative § A/B Testing § Usability benchmarks § Online UX Assessments § Surveys (CSAT, NPS) § Clickstream Analysis
User Needs
Customer
End User UX/
Design
Develop-ment
Product Mgmt
Admin
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential
You Can’t Do It All
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 56
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 57
Title Line
Subtitle Line
Top of Content Box Line
Top of Footer Line
Left Margin Line Right Margin Line
Top of Footer Line
Top of Content Box Line
Subtitle Line
Title Line
Right Margin Line Left Margin Line
.
Intel Confidential 58
Thanks! @christianrohrer