Bilgisayar Mühendisliği Bölümü

GYTE - Bilgisayar Mühendisliği Bölümü

Measuring the User Experience on a Large Scale:

User-Centered Metrics for Web Applications

BIL 763

Melik Yücel

GYTE - Bilgisayar Mühendisliği Bölümü 2

• Introduction• Related Work• PULSE Metrics• HEART Metrics• Goal-Signals-Metrics• Conclusion

CONTENT

GYTE - Bilgisayar Mühendisliği Bölümü 3

IntroductionMeasuring User Experience

• More applications and services become web based

• Abilty to use large-scale user log

• The established mesaurements are mostly small scale and

checklists as part of lab expirement

• Lack of user behavioral data and too generic to address

user experience questions

• This work proposes new framework and process

GYTE - Bilgisayar Mühendisliği Bölümü

Related Work• Analysis of user surveys, large-scale remote

usability and benchmarking tests

• Tracking and analysis of metrics for websites

• Custom analysis of large-scale log data

4

GYTE - Bilgisayar Mühendisliği Bölümü

PULSE Metrics• Page views

• Uptime

• Latency

• Seven-day active users

• Earnings

5

GYTE - Bilgisayar Mühendisliği Bölümü

HEART Metrics• Happiness (satisfaction)

• Engagement

• Adoption

• Retention

• Task Success (effectiveness & efficiency)

6

GYTE - Bilgisayar Mühendisliği Bölümü

Goals-Signals-Metrics

7

A simple process that steps teams through

articulating the goals of a product or feature, then

identifying signals that indicate success, and finally

building specific metrics to track on a dashboard.

GYTE - Bilgisayar Mühendisliği Bölümü

Goals• Opportunity to collect all the different ideas from

different team members

• Goals of different a particular project or feature may be different from product in general

• Do not worry about signals and metrics at this stage

8

GYTE - Bilgisayar Mühendisliği Bölümü

Signals

• Choose signals that are sensetive and specific to the goal

• Sometimes failure is easier to identify than success

9

GYTE - Bilgisayar Mühendisliği Bölümü

Metrics

10

• Raw counts need to be normalized

• There are many challenges in ensuring accuarcy of metrics based on web

• To compare your product to others,you may need track additional metrics from the standart set used bye those products

GYTE - Bilgisayar Mühendisliği Bölümü

Conclusion• Developed new framework and process for

large-scale user experience• Applied to more than 20 different products

and projects of Goolge.• Can be used by different organisations easily

11

GYTE - Bilgisayar Mühendisliği Bölümü

Thanks...

12