Cronkite News & Parse.lyCronkite News’ favorite Parse.ly experiments of Fall 2016

Cronkite News is the news division of AZ-PBS, part of the Walter Cronkite School of Journalism & Mass Communication at Arizona State University.

We used Parse.ly to measure...

Newsroom performance

Our giant Parse.ly screen shows all of our 100+ reporters and producers how stories are performing RIGHT NOW, and how we can/should react in order to grow in a meaningful way.

Weekly and semester reports of our site help us understand long(er) term performance and evergreen-ness.

Reporter performance

All reporters created their own author reports (sent to their email each week) to track their personal performance.

Special experiments

We used Parse.ly to help us answer three questions.

Let’s explore them!

Question 1Does placement on

our home page affect story engagement?

Our website’s home page elevates three stories each day: The story that appears in the top hero position, and two just below it; seeing other content requires a scroll. We sought to know whether appearing prominently on our home page affects engagement. Were users more likely to spend time on our stories or share them if they had prime home page real estate?

Working with 40 stories across three weeks, we tagged stories “hp” if they were placed in one of the top three positions, and “nohp” if they were not.

What we learned

Home page placement and social interactions

are correlated.

HP NO HP

AVG. TIME 0:55 0:52

SOCIAL INTERACTIONS (MEAN)

222 58

SOCIAL INTERACTIONS (MEDIAN)

31 22

Average engaged time for stories with home page placement and those without were relatively similar. Social interactions (likes, shares, etc.) saw a bigger divide: Average interactions for “hp” stories were much greater than those without. However, the mean may have been skewed by viral-ish, highly newsworthy election stories, so the median may be more notable … and the “hp” median was higher, too. We want to continue to examine this relationship in the post-election news climate.

Question 2Does video placement within stories affect engaged time?

Depending on available multimedia assets, we place videos in various locations throughout our stories: sometimes at the top, sometimes throughout the body, sometimes at the bottom - and sometimes, stories have no videos at all. We sought to learn whether we could keep audiences engaged longer simply by identifying the best position for a video and optimizing toward that.

We added these tags to posts over the course of three weeks: no video, top video (videos that appear at the top of the page), middle video (within the body of the page), and bottom video (at the very bottom of the page).

What we learned

Video placement didn’t significantly affect

engagement time.(Plus some hard lessons

about re-tagging and re-crawling posts.)

AVG. TIME

NO VIDEO 1:24

BOTTOM 1:24

TOP 1:12

MIDDLE 1:06

The “time” metric may not be the best way to explore this. Users spent more time reading “no video” stories than stories with videos in any position. “Bottom” video stories had the same average time, though we published less of those … plus, logic follows that users who reach the bottom of a story at all spend more time on it; they may not watch the video at all. High engaged time tends to more strongly correlate with beat.

We also learned the hard way how to use Parse.ly to re-crawl for new tags. We’ll try this again in spring!

Question 3Do stock photos or original

photos perform better?

Cronkite News doesn’t have dedicated photojournalists; reporters take their own photos. And like many newsrooms, we can’t always capture an idyllic image to draw readers into every story, so we sometimes turn to stock (or Creative Commons) photography. We sought to understand whether the photos we took (or the ones we didn’t) performed better.

To evaluate performance, we used tags to mark stories that contained original photos or stock photos, then measured the tags in Parse.ly.

What we learned

Stock photos perform better.(But that doesn’t mean we

should use them.)

ORIGINAL STOCK

AVG. VIEWS 132 185

AVG. USERS 108 165

AVG. TIME 0:49 0:40

Stock photos did a better job of bringing users to our site; average views and users were higher for stories that led with a stock image than stories that used our own photography. This suggests not that we should stop shooting, but rather that we must stress higher-quality photography.

Indeed, once they’re actually on our site, the audience seems to prefer original photography; stories with our own photos (which obviously speak more to the news we’re covering!) kept users engaged longer.

Question 4Do our Friday shows perform better when we use Parse.ly metrics to pick the content?

Fact: You can use Parse.ly to inform your TV content! We’re a converged newsroom; we air a news show on PBS throughout Arizona every day at 5 with repeats at 11. Monday through Thursday, the show is live; Friday’s show is our “Social Refresh” - a compilation of the stories that performed best across social media that week.

We typically choose Friday’s content based on stories that earn the most likes in Twitter and/or Facebook.

This fall, we used Parse.ly to pick the content: We chose the week’s top performers by Parse.ly’s “social referrers” and “shares, likes, tweets & pins” metrics.

We used our daily Nielsen reports to analyze performance of this semester’s Friday shows against the performance of the spring semester’s Friday shows.

What we learned

Spring Friday shows perform better

… and we should examine this question differently.

SPRING ’16 FALL ’16

AVG. HOUSEHOLDS

4,772 4,248

AVG. RATING .25 .22

AVG. SHARE .62 .56

Spring Friday shows performed better: They reached more households, earned higher ratings and higher ratings shares (among our timeslot competitors).

But reach, of course, is not the same as engagement; instead, we should assess drop-off between quarter hours (the difference between viewers of the top and bottom halves of the show), which suggests whether viewers like what they see! This requires some different Nielsen data collection; we’ll initiate that in spring.

Moving forward...

1

We’ll tag better

Now that we have a handle on Parse.ly, we’ll be smarter about tagging our content and be diligent about re-crawling pages we’ve tweaked to ensure more precise sampling.

2

We’ll measure dynamically

Often, there’s not just one perfect metric that gives you “the answer.” When exploring a single question, we’ll aim to look at groups of metrics and the stories they tell together.

3

We’ll focus on social

We’re particularly curious about our social media audiences; in spring, we want to focus our Parse.ly experiments on uncovering the behaviors of our social referrals and leveraging that knowledge for growth.

Above all, we LEARNED!

Here’s to measuring smarter in 2017!