Transmedia Metrics Model

Download Transmedia Metrics Model

Post on 28-Nov-2014




1 download

Embed Size (px)


How do you measure transmedia? What types of metrics will help transmedia producers to better understand and subsequently compare and contrast the impact of a transmedia story? These are milliondollar questions begging for answers. We propose a model that can be used to create and give direction to a transmedia production team of writers and performers. The key benefit of our conceptual model is that the behavior of an individual user can be compared to others. In doing so, we can interpret the relative engagement of an individual user compared to others (as a ratio) at certain points of interaction (touchpoints, chapters, scenes, beats). By tracking the user journey the storyteller gets actionable insights on the behavior of that individual, but also on the behavior of groups of users.


<ul><li> 1. 1 vtooowrwarodosr ad new metrics model introduction How do you measure transmedia? What types of metrics will help transmedia producers to better understand and subsequently compare and contrast the impact of a transmedia story? These are million-dollar questions begging for answers. Measuring engagement is important because it contextualizes what an audience size means and how a transmedia production is performing. It helps producers and storytellers to see what actually gets fans to engage. Measuring engagement, however, can be desirable from different perspectives. A producer or marketer will be interested in metrics to evaluate the campaign in terms of return on investment. A storyteller specifically wants to know how the audience responds to the narrative. We believe that measuring engagement means placing audience size into the broader context of how the transmedia production as a whole is actually performing. Stakeholders in the production can see where, how and when fans engage, which allow them to make refinements. We aim to present an engagement model that can easily be integrated into the daily activities of a transmedia storyteller. In this research, we choose to focus on the goals of the storyteller. We propose a model that can be used to create and give direction to a transmedia production team of writers and performers. MOtivation At the Fontys ACI Transmedia Storytelling Lab, students learn to design, develop and execute transmedia productions. In this process, the students experience all facets of transmedia production. We discovered that the teams needed meaningful measuring tools to determine whether their campaigns performed as expected. They needed insights on audience behaviour to be able to give direction to their campaign and tap into the user need. Our goal is to design a model that can be used to create and give direction to a transmedia production. A review of current models User engagement defined In their work, Attfield et al.[1] define user engagement from a broad perspective. Their definition reads: User engagement is the emotional, cognitive and behavioural connection that exists, at any point in time and possibly over time, between a user and a resource.This approach emphasises the emotional, cognitive and behavioural factors that, as a whole, define user engagement. All three factors are open to measurement. Figure 1 The four components of Engagement (based on Forrester Research) According to Forrester Research[2] cognitive engagement can be measured through awareness, interest and intention. It is important to measure the likelihood of a user advocating, for instance by forwarding content or inviting friends. Besides measuring influence, Forrestor Research stresses involvement and interaction. This behavioural engagement focuses on the presence and action of the user, which can be measured by the number of visitors or the amount of time spent on a website. How to measure transmedia experiences </li> <li> 2. 2 Additionally, one could look at the click through ratio, the number of online transactions or the number of uploaded videos. Finally, emotional engagement attempts to measure the affection or aversion of a user. Intimacy is considered an emotional engagement factor and is measured by satisfaction rating or by sentiment analysis in blogs, comments, surveys and questionnaires. What are points of interest for a transmedia storyteller? To Robert Bole, director of Broadcasting Board of Governors (US), it is clear that creators of the new narratives need these analytics. He notes, Rather than the old way of more is better as a proxy for quality (more audience = bigger success), the digital world allows storytellers to hone their craft through nearly real-time feedback across multiple channels that can result in quick shifts, additions and changes to narratives depending on pockets of audience behaviour and interests[3]. In his talk at the DC Digital Analytics Association Symposium in 2013, Bole defined six points of interest that need to be addressed to help understand engagement. The metrics and analytic tools that help understand the impact of stories that unfold across multiple platforms are, according to Bole, Narrative attribution Conversion and pathways Segmentation format feedback Quantifiable impacts Engaged loyalty Audience-created enhancement By measuring the narrative attribution, a storyteller can identify which story element (or media mix) converted the audience member into an engaged, multi-channel user. Measuring the conversion and pathways helps determine optimal paths for constructing narrative arcs. Segmentation format feedback measures the success of the format in engaging and propelling a narrative arc by audience type. Quantifiable impact is needed to measure audience participation across the narrative arc (The What do people do? problem). Engaged loyalty means finding out if additional context and information leads to more informed, loyal and frequent users. Audience-created enhancement involves measuring how audiences add value through contributions to the narrative arc. From points of view to actionable insights Although we found several works on the needs of a storyteller, we found less information about transforming those needs into practical, actionable insights. We did, however, find a variety of models on the desired outcome of transmedia analysis. Mayfields Power Law of Participation model[4] is such an example. The model shows twelve stages of participation, ranging from the rather inactive reader to that of a lead, a highly engaged and active audience member (see figure 2). Figure 2. Power law of Participation model (Mayfield) a review of current models </li> <li> 3. 3 vtohoer nweeodo frodr actionable insights Mayfields work relates to the roles that Henry Jenkins describes in his work[5]. There are at least two significant roles that users can play to push a campaign forward. Jenkins mentions two of those roles: lurking and spoiling. Lurking could be defined as a stage where users look at the content produced but are not actually interacting with it or sharing it. Lurking plays an important part in the evaluation phase. Lurkers are those who are interested but not ready (or may never be ready) to start interacting with the transmedia campaign. Another important role is the spoiler. Spoilers are a very active group of people that seek to know everything that can be known about a campaign. Often, they group together on forums or Facebook groups, out of sight of the campaign creators They also create blogs related to a campaign. Pratten[6] defines five levels of increasing engagement: attention, evaluation, affection, advocacy and contribution. In his work, Pratten proposes measurements aiming specifically at measuring engagement in a transmedia world. His five levels of increasing engagement expand Forrester Researchs earlier-mentioned four measures of engagement, by adding affection, meaning affinity towards the world. Figure 3. Measuring Engagement (based on Pratten) Having this kind of information about the segmentation of the participants in a transmedia production is very useful for a storyteller. He or she will be able to measure and to adjust his story accordingly, based on these insights. Conclusion on current models Our students found various models and methods to measure engagement (not all of which were mentioned in this paper). A range of models is available to either classify levels of engagement or to create typologies of the engaged. We need to fill the gap between the existing models to find a way of gathering meaningful insights that can be used to give direction to a transmedia production in a practical way. Based on the research performed by our students on existing models, we want to make sure that our new metrics model: Measures engagement and classifies the user Transforms raw data into meaningful insights Tracks the user journey Tracks the connection between different channels (on-and offline) </li> <li> 4. 4 introducing our new metrics model We see three important aspects of a transmedia production: A storyworld that is to be discovered An audience that is involved A period of time We consider audience members who interact with the world to be engaged users. By tracking the behaviour of individuals, we can map how they discover the world and how they interact with it over time. We record each time a user touches something in the storyworld. By listing all of these points of interaction and structuring them into chapters, scenes and beats, we can track the journey and the hotspots of engagement for individual users as they progress through the story. Figure 4. The new metrics model: users interact with the storyworld over a period of time How does the tracking work? Points of interaction are predefined by the creator (the storyteller) of a transmedia production in the form of acts, scenes, plot points, beats, and so on. An audience member, referred to as a user, progresses in the storyworld by discovering these scenes and beats, interacting with characters, solving puzzles, sharing content with friends or viewing content on specific channels like YouTube or blogs. By tracking this individual user journey through the storyworld, the storyteller receives insights into the behaviour of that individual. With this information, the storyteller is able to analyse whether the transmedia production is performing the way it should. Is all the content discovered? Does the user progress? Does s/he need help? Now, actionable insights are at the storytellers disposal. Interactions are tracked using the concept of toggle switches. If a user interacts with a certain point, the toggle switch changes from OFF to ON. Picture taking a tour through a house and switching on the light when entering a room, leaving a trail of lights in all the rooms visited. Figure 4. Toggle switches change from OFF to ON at a point of interaction Key benefit: comparing behaviour of individuals The key benefit of our conceptual model is that the behaviour of an individual audience member can be compared to the behaviour of others. In doing so, we can interpret the relative engagement of individuals compared to some or all others at certain points of interaction as a ratio. Tracking all of the behaviours of all individuals allows us to group them on different levels. introducing our new metrics model </li> <li> 5. 5 voorwoord Examples of relative engagement measurement: Compare the amount of interactions of User A in [Scene 5] to all interactions in [Scene 5] Show the progress of User B compared to the most progressed user List the number of users that interacted with [Scene 7, Beat 2] compared to all users Interpreting gathered data The gathered data about the users journey through the world (points of interest touched by a user at specific moments in time) is combined with that of other users. This also opens up the possibility for ways of segmentation and classification that the reviewed models offer. Segmentation is based on actual data of individual users, which is an asset over previous models. Now, users can be classified as lurkers, contributors or cognitively engaged relative to all the users in the storyworld. expert evaluation We asked leading experts to evaluate the clarity, completeness, effectivity, applicability and benefit of our model. Amongst them are Sam Ford (Peppercomm, New York, NY), Pamela Rutledge (Media Psychology Research Center, Boston, MA), Bart Robben (Elastique, Hilversum, NL), Egbert van Wyngaarden (Transmedia Desk, Munich, DE) and Soraia Ferreira (UT Austin, Porto, PT). Since we focus on content engagement (story) rather than on return on investment (ROI) in the development of our conceptual model, we first wanted to know what respondents sought in a new metrics model for storytellers. As a cross-section of the various models, we selected (1) loyalty measurement, (2) participation ratio, (3) target characteristics, (4) media channel effectiveness (5) content effectiveness and also (6) ROI measurement as conventional metrics used in defining the success of transmedia productions. Therefore, we asked our respondents to rank the importance of these metrics. Our respondents chose content effectiveness as the most important focus of a storytellers measurement model. The second most important was the measurement of ROI. Loyalty measurement, participation ratio, target characteristics and media channel effectiveness seemed to be less important. These results are in line with our focus on the storyteller but they show the relative importance of ROI as well. We also asked the respondents to evaluate our model. First, we asked them to rate the following items on a scale from 1 to 10: applicability, effectiveness, completeness and clarity. Next, we asked our respondents to explain this numerical evaluation using semi-structured interviews. In general, we found that our respondents were positive about the applicability and clarity of the model. In addition, they liked the focus on the ability to measure user journeys. At the same time, they were sceptical about the effectiveness and completeness of our model.e model expert evaluation What youve designed here is very clear. It will work very well, especially in a system like Conducttr. Pamela Rutledge </li> <li> 6. 6 voorwoord Positive aspects of the model Our respondents feedback on the positive elements of the new metrics model: Most models process the use, engagement or participation of one medium, but this model combines all different media and their results (Welhuis). It measures the journey, and also the overall journeys (Ferreira). The model would be helpful for producers who are interested in understanding how each element of their story was successful, compared to the others, or to find out in what way each audience member wants to travel through their story and engage with som...</li></ul>