testing content effectiveness - penn state 2015

It’s What’s Inside That Counts.

Testing Content Effectiveness

#psuweb @misb

Hello! My name is…

Melissa Beaver Director of User Experience NewCity, insidenewcity.com

#psuweb @misb

Is our content doing what we think it’s doing?

#psuweb @misb

Is anyone using/reading it?

#psuweb @misb

CrazyEgg Heatmap

Wrong place?

#psuweb @misb

Source: Think Education with Google, “Education Trends through the Eyes of Your Customer (April 2012), http://www.thinkwithgoogle.com/research-studies/education-trends-through-the-eyes-of-your-customer.html

Web content is free!

#psuweb @misbPhoto courtesy Jesslee Cuizon, Flickr.

The Hidden Cost

#psuweb @misb

Editing Writing

1/2 –1 hour

1–3 hours

2–5 hours

10–20 hours

to copy edit to edit or rewrite existing content

to research & write original

content

to research & write 1 feature

article

50 articles 500-1,000

Hours

https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot-a32c0b3353d5dff3aba41c5a2a7dcd33.png

#psuweb @misb

Every page has a job to do

Support conversions

Answer user questions

Leave users with the intended impression

Quantifiable Effectiveness

#psuweb @misb


#psuweb @misb

Where do I look?

Identify outliers

Examine top entrances

Look at recurring content types

Map to stakeholder & project goals

#psuweb @misb

Establish your baseline

‣ Unique sessions, users, pageviews

‣ Traffic sources

‣ Bounce rate

‣ Session duration

‣ Entrances

Make fair comparisons

#psuweb @misb

#psuweb @misb

Define clear goals

Go to this report

Jun 1, 2014 Aug 4, 2014Content DrilldownALL » PAGE PATH LEVEL 1: www.pomona.edu/ » PAGE PATH LEVEL 2: /academics/ » PAGE PATH LEVEL 3: /majorsandminors/

This report is based on 214,736 sessions (90.07% of sessions). Learn more

Page path level 4

All Sessions

From New York

From Not US

All Sessions

From New York

From Not US

All Sessions

From New York

From Not US

All Sessions

From New York

Explorer

Pageviews Unique Pageviews Avg. Time on Page Bounce Rate % Exit

All Sessions15,355

% of Total: 1.97%

(780,172)

13,081% of Total: 2.25%

(581,869)

00:01:06Site Avg: 00:01:05 (1.13%)

71.44%Site Avg: 49.19% (45.22%)

50.59%Site Avg: 30.52% (65.77%)

From New York553

% of Total: 0.07%

(780,172)

467% of Total: 0.08%

(581,869)

00:01:25Site Avg: 00:01:05 (29.64%)

63.53%Site Avg: 49.19% (29.15%)

51.72%Site Avg: 30.52% (69.47%)

From Not US1,915

% of Total: 0.25%

(780,172)

1,647% of Total: 0.28%

(581,869)

00:01:20Site Avg: 00:01:05 (21.80%)

77.18%Site Avg: 49.19% (56.90%)

51.12%Site Avg: 30.52% (67.52%)

1. /index.aspx

14,225 (92.64%) 12,116 (92.62%) 00:01:08 71.41% 52.11%

524 (94.76%) 439 (94.00%) 00:01:27 67.09% 51.91%

1,744 (91.07%) 1,493 (90.65%) 00:01:23 77.45% 52.41%

2. /naturalsciences.aspx

433 (2.82%) 369 (2.82%) 00:00:57 77.78% 34.64%

10 (1.81%) 10 (2.14%) 00:00:55 33.33% 40.00%

53 (2.77%) 46 (2.79%) 00:00:59 81.25% 49.06%

3. /humanitiesfinearts.aspx

288 (1.88%) 248 (1.90%) 00:00:50 66.67% 32.29%

9 (1.63%) 8 (1.71%) 00:00:43 0.00% 66.67%

52 (2.72%) 46 (2.79%) 00:00:31 75.00% 36.54%

4. /interdisciplinary.aspx

224 (1.46%) 194 (1.48%) 00:00:51 57.14% 28.12%

6 (1.08%) 6 (1.28%) 00:00:20 0.00% 33.33%

+

Pageviews ( )All Sessions Pageviews ( )From New York Pageviews ( )From Not US

Jun 15 Jun 29 Jul 13 Jul 27

500500500

1,0001,0001,000

All Sessions1.97%

From New York0.07%

From Not US0.25%

Increase…

‣Quality?

‣Quantity?

‣Awareness/Visibility?

For…

‣Which programs?

‣Which regions?

‣Which audiences?

#psuweb @misb

Be specific

By Site Section/Page

‣ Navigation paths

‣ Page elements

‣ Conversions

‣ Unique pageviews by site section

‣ Bounce rate

‣ Time on page

‣ Scrolling

Segment by…

‣ location

‣ device

‣ traffic source

‣ referrer

‣ custom segments

Content groups

#psuweb @misb

Go to this report

Jun 25, 2014 Aug 24, 2014Pages

Rows 1 8 of 8

Explorer

365,569% of Total: 100.00% (365,569)

365,569% of Total: 100.00%

(365,569)

1. 96,900 26.51%

2. 77,120 21.10%

3. 63,282 17.31%

4. 50,873 13.92%

5. 30,970 8.47%

6. 30,829 8.43%

7. 8,819 2.41%

8. 6,776 1.85%

+ Add Segment

Pageviews

Jun 29 Jul 6 Jul 13 Jul 20 Jul 27 Aug 3 Aug 10 Aug 17

5,0005,0005,000

10,00010,00010,000

Main site sections (ContentGroup) Pageviews Pageviews Contribution to total: Pageviews

Resources

26.5%

21.1%

8.4%

8.5%

13.9%

17.3%

Homepage

Research

About

Work & Study

Sanger directory

Search results

(not set)

© 2014 Google

All Sessions100.00%

• Navigate to Behavior » Site Content » All Pages for an overview. (Also available in other Site Content reports.)

• Select “Main site sections (Content Group)” in the “Content Grouping:” drop down, just to the right of the “Primary Dimension:” field, in between the explorer graph and the data table.

Segments

#psuweb @misb

• Available in most reports, at the top underneath the gray bar, series of boxes showing the applied segments. Click “+ Add Segment”

• To define a new segment, click the red “+ New Segment” button. • To share your custom segment with other analytics users, select “Share” from the “Actions” menu.

Entrances by traffic typeSegments

#psuweb @misb

Tracking impact after big news story in national media

#psuweb @misb

Any increase in traffic?

#psuweb @misb

Sessions

Unique Pageviews

Other impacts?

#psuweb @misb

• Navigate to: Audience » Mobile » Overview. • In visualization: Explorer Tab » Motion Chart View (upper right corner). • Change to line graph view (far right tab in chart). • Change options to: “Sessions” for y, “Bounce Rate” for color.

Mobile & tablet traffic changed

#psuweb @misb

• Navigate to: Audience » Mobile » Overview. • In visualization: Explorer Tab » Motion Chart View (upper right corner). • Change to line graph view (far right tab in chart). • Change options to: “Sessions” for y, “Bounce Rate” for color. • Uncheck desktop.

Where did mobile traffic enter?

#psuweb @misb

• Navigate to: Behavior » Site Content » All Pages. • Change segment if desired (Mobile in this example). • In visualization: Explorer Tab » Motion Chart View (upper right corner). • Change view options to y=“Unique Pageviews”, Color = “Bounce Rate, Size = “Entrances” • Drag timeline (at bottom) to appropriate date.

… and where did they go?

#psuweb @misb

Go to this report

Jun 25, 2014 Sep 24, 2014PagesALL » PAGE: /about/press/2014/140801.html

Previous Page Path Pageviews % Pageviews

19 82.61%

2 8.70%

1 4.35%

1 4.35%

Next Page Path Pageviews % Pageviews

5 38.46%

1 7.69%

1 7.69%

1 7.69%

1 7.69%

1 7.69%

1 7.69%

1 7.69%

1 7.69%

Entrances Jun 25, 2014 Sep 24, 2014: 71.14%

Previous Pages Jun 25, 2014 Sep 24, 2014: 28.86%

Exits Jun 25, 2014 Sep 24, 2014: 77.85%

Next Pages Jun 25, 2014 Sep 24, 2014: 22.15%

Navigation Summary

Group pages by: Current Selection: /about/press/2014/140801.html Show rows: 10

+ Add Segment

Pageviews

July 2014 August 2014 September 2014

303030

606060

Ungrouped

/

/about/press/

/about/people/

/about/press/2014/140721.html

/

/about/engagement/

/about/history/

/about/how/

/about/press/

/about/press/2014/



/workstudy/

© 2014 Google

Mobile Traffic0.03%

• Navigate to Behavior » Site Content » All Pages. • Change segment if desired; (mobile in this example). • In visualization, click “Navigation Summary” tab. • Change “Current Selection:” to page of interest.

Did they click on the accordion, click on the tabs, watch a video, interact with forms … ?

#psuweb @misb

Go to this report

Jul 5, 2014 Aug 4, 2014Top Events

Event Category

Rows 1 12 of 12

Explorer

Event

Total Events Unique Events Event Value Avg. Value

201,402% of Total: 100.00% (201,402)

51,269% of Total: 100.00% (51,269)

11,647,434% of Total: 100.00% (11,647,434)

57.83Site Avg: 57.83 (0.00%)

1. Reading 75,421 (37.45%) 10,590 (20.66%) 11,613,031 (99.70%) 153.98

2. Accordion 63,538 (31.55%) 12,629 (24.63%) 0 (0.00%) 0.00

3. External Link 19,508 (9.69%) 14,991 (29.24%) 0 (0.00%) 0.00

4. Interest Form 13,460 (6.68%) 1,121 (2.19%) 4,727 (0.04%) 0.35

5. Engagement Module 11,324 (5.62%) 8,506 (16.59%) 0 (0.00%) 0.00

6. Event Registration Form 10,803 (5.36%) 1,853 (3.61%) 7,412 (0.06%) 0.69

7. Video 6,157 (3.06%) 631 (1.23%) 0 (0.00%) 0.00

8. Download 632 (0.31%) 595 (1.16%) 0 (0.00%) 0.00

9. Mailto 263 (0.13%) 195 (0.38%) 0 (0.00%) 0.00

10. Application 173 (0.09%) 67 (0.13%) 22,264 (0.19%) 128.69

11. Campus Visit Registration Form 110 (0.05%) 79 (0.15%) 0 (0.00%) 0.00

12. Error 13 (0.01%) 12 (0.02%) 0 (0.00%) 0.00

+ Add Segment

Total Events Pages / Session

Jul 8 Jul 15 Jul 22 Jul 29

5,0005,0005,000

10,00010,00010,000

555

101010

© 2014 Google

All Sessions100.00%

Google Analytics

Events

Is anyone watching?

#psuweb @misb

But are users really watching it?

#psuweb @misb

Go to this report

Jul 5, 2014 Aug 4, 2014Top EventsALL » EVENT CATEGORY: Video

Event Action

Rows 1 10 of 10

Explorer

Event

Total Events Unique Events Event Value Avg. Value

6,157% of Total: 3.06% (201,402)

631% of Total: 1.23% (51,269)

0% of Total: 0.00% (11,647,434)

0.00Site Avg: 57.83 (100.00%)

1. Start 919 (14.93%) 627 (13.85%) 0 (0.00%) 0.00

2. 1% 794 (12.90%) 561 (12.39%) 0 (0.00%) 0.00

3. 5% 725 (11.78%) 523 (11.55%) 0 (0.00%) 0.00

4. 10% 704 (11.43%) 513 (11.33%) 0 (0.00%) 0.00

5. 20% 663 (10.77%) 490 (10.82%) 0 (0.00%) 0.00

6. 33% 622 (10.10%) 462 (10.21%) 0 (0.00%) 0.00

7. 50% 563 (9.14%) 433 (9.56%) 0 (0.00%) 0.00

8. 67% 504 (8.19%) 393 (8.68%) 0 (0.00%) 0.00

9. 90% 437 (7.10%) 345 (7.62%) 0 (0.00%) 0.00

10. Ended 226 (3.67%) 180 (3.98%) 0 (0.00%) 0.00

+ Add Segment

Total Events Pages / Session

Jul 8 Jul 15 Jul 22 Jul 29

200200200

400400400

151515

303030

© 2014 Google

All Sessions1.23%

Where are users clicking?

#psuweb @misb

CrazyEgg Heatmaps Note;

Recommend cross-referencing data with Google Analytics.

Relative sizes of clickable elements influence intensity. E.g., buttons with a proportionally smaller clickable area can seem more clicked than a slideshow or hero image with a large clickable area.

Did they scroll that far?

#psuweb @misb

CrazyEgg Scrollmaps

Qualitative Effectiveness

#psuweb @misb

#psuweb @misb

Perceptions Is our website making the right impression?


#psuweb @misb

Use to…

See how you might be influencing users’ impressions

Figure out why traffic is good but conversions are poor

Include a user’s voice in design decisions

Inform disagreement among stakeholders


#psuweb @misb

How to ask…

After completing some information seeking tasks

Impressions about the peopleDescribe what kind of [customer, student, etc.] you think is a good fit for this [organization, product, program, etc.]

Impressions about the institutionWhat 3 adjectives or phrases would you use to describe [organization or company] based on your visit to this website?

What students are a good fit?

#psuweb @misb

“The polar opposite of me - i.e., someone who loves the outdoors and wants to be

part of a tight knit community…”

Describe what kind of student you think would be a good fit for this college.

#psuweb @misb

Which design makes the right impression?

#psuweb @misb

“Seems clearer,

less like a business.”

“This one feels more academic”

Which design makes our institution feel like the right fit?

#psuweb @misb


#psuweb @misb

Impressive

“Top 3%—that’s pretty awesome.”

“I now know that the Science Center is part of the University of Texas. I didn’t even notice that on the others... I did see UT but wasn’t sure what that stood for. Nice to know who they are affiliated with.”


#psuweb @misb

Too much research

“He looks very scientific... I don’t get a health care impression... Seems more focused on research aspect—doesn’t make me want to choose this.”

#psuweb @misb

What is a Health Science Center?

Google Consumer

Survey

#psuweb @misb

What does UT Health Science Center do?

‣ “It provides healthcare, is a medical school, and research center, and these 3 aspects combined make for an excellent healthcare experience.”

‣ “It’s a teaching medical center that offers all sorts of resources—medical and dental care, a learning environment, and research facilities.”

#psuweb @misb

Rating Helpfulness Which content do users find

helpful, useful?


#psuweb @misb

Use to…

Determine if content answers questions completely

Gain insight into how your users answer fuzzy questions that don’t always have clear answers

Understand placement in the user’s journey


#psuweb @misb

How to ask…

Specific task-based scenario Part 1. You want to find out [this amorphous topic]. What information do you look for on this website? Part 2. How do you evaluate if this [program/product/service] fits your needs?


#psuweb @misb

How to ask…

General browsing scenarioPart 1. You’d like to check out [organization]. Think about what qualities you look for in [that type of organization]. Part 2. Exploring this website, how do you find out if [this organization] is a good fit for you?

“So, what did they do, exactly?”

Unmoderated User Testing

#psuweb @misb

I want examples…

#psuweb @misb

Receptiveness

#psuweb @misb

“Eyesore.” Seems made up, “fake.”

“Impressive…Meeting their patient’ needs—not

just making money.”

#psuweb @misb

Comparison Which one do users “pick” and why?


#psuweb @misb

Use to…

See how your site compares to your peers or competitors in terms of usefulness (not just look and feel)

Observe more realistic behavior

Understand less tangible influences


#psuweb @misb

How to ask…

Same question as before… Part 1. You want to find out [this]. What information do you look for on this website? Part 2. How do you evaluate if this [college or program] fits your needs?

But repeat with 3–4 other competitors.

Randomize or place your site somewhere in the middle.


#psuweb @misb

How to ask…

Follow-up with some ranking questions…Which [organization or company] seemed the most [welcoming, innovative, goal adjective] based on their website? Why?

Ask up to 3 of these.

Comparing schools

#psuweb @misb

Cheerful

Depressing

Comparing doctors

#psuweb @misb

Publications…huh?

#psuweb @misb

What are these?

Impressive

Specialist content could be more helpful than you may think

#psuweb @misb

Show me what kind of

surgeon he is

“Love/hate relationship with pictures because they were

helpful but disturbing.”

Unmoderated User Test

#psuweb @misb

Where do I start?

#psuweb @misb

Experiment

Image courtesy BigdogLHR, Flickr.


#psuweb @misb

Some tools in your lab kit

UserTesting.com (More qualitative)

‣ Try peek.com for a free, lightweight version

Try My UI

Typeform

VerifyApp Testing Suite

Optimal Workshop

Crazy Egg

Heap Analytics

Google Analytics Try Amazon’s Mechanical Turk to recruit participants inexpensively.

Albert Einstein

“No amount of experimentation

can ever prove me right; a single

experiment can prove me wrong.

#psuweb @misb Image Universiteit Leiden, Flickr.

I don’t know. Let’s test it.

Said by you

#psuweb @misb

#psuweb @misb

Thank you! @misb

What do you think about the future of digital communications in higher education?

http://bit.ly/future-he

testing content effectiveness - penn state 2015

Data & Analytics