esomar 2010 - digital culture
DESCRIPTION
Presentation by Ellen Bird & Christian Kugel at ESOMAR Global 2010TRANSCRIPT
©2010 VivaKi. All Rights Reserved. Page 1
What online behavior reveals about digital cultureAn assessment of techniques for identifying themes and interpreting them for insightsChristian Kugel & Ellen Bird December 2010The contemporary online experience is a participatory one. This relatively recent development has had significant impact on the nature of digital culture. Based on a belief that an understanding of digital culture is key in developing digital marketing ef-forts, VivaKi created Reflecteur, an original cross-agency initiative. In this paper the authors describe a theoretical framework for interpreting digital culture and compare and contrast two methods—user-generated content monitoring and crowdsourcing—for uncovering digital cultural insights.
The Internet was invented to be a participative medium. Without individuals participating, its value plummets. The Internet sans people influencing it simply makes no sense. Over the past five years, however, this attribute has been magnified by the birth and adoption of social media. Social platforms such as video sharing, blogging, networking and photo sharing (just to name just a few) have truly revolution-ized people’s digital experience. Social tools are built into nearly every site—people have come to expect the ubiquity of the upload button, the comment field, and the “share this” icon.
Digital marketing trends have clearly followed the migration to social experiences. It is the rare brand that does not have presence in multiple social en-vironments or infuse its digital assets with social functionality. As content diversified to include a richer mix of user-generated material, and as social platform-enabled sharing became com-mon, a distinct digital culture emerged—a culture rooted in the act of participation.
And increasingly, we believe that understanding, internal-izing and aligning with digital culture will be a price of entry for crafting successful digital marketing initiatives. Just as conventional advertising must recognize and address cultural cues, digital marketing will
increasingly be compelled to do so as well. The initiative described here was designed to help an agency network keep its finger on the pulse of digital culture and arm every em-ployee with an understanding of the themes, characteristics and nuances of this ever evolving, dynamic subject.
We christened this initiative with the project name “Reflec-teur,” meant to convey our goal of being a voyeur of digital culture while reflecting the underlying cultural insights back to our constituents. This initiative began in Q4 2007 and
continues to this day. Over the course of three years, we have leveraged user-generated content (UGC) monitoring, crowdsourcing, ethnographic techniques and investigative methods to understand digital culture, paint a picture of it for our constituents and ultimately enable them to incorporate these insights into digital marketing initiatives. And in that time, Reflecteur has grown from a small project incubated inside Denuo, a boutique VivaKi agency, to all of the agency families within the entire VivaKi network.
VivaKi comprises the digital and media agencies of Publi-cis Groupe. Two media agency
families, Starcom MediaVest and ZenithOptimedia, as well as two digital agency networks, Razorfish and Digitas, are all
Figure 1: VivaKi Agency Network
ZenithOptimedia
©2010 VivaKi. All Rights Reserved. Page 2
members of the VivaKi network. All VivaKi agencies believe that the ability to win in the future requires comprehensive digital expertise and capabilities. Understanding, internalizing and applying digital culture, for the reasons stated above, is a necessary element of that exhaustive expertise.
Theoretical FrameworkPrecious little research has been published on the topic of digital culture. The subject has not been extensively exam-ined in academia, due in large part to the fact that it does not fit neatly into any one academic discipline. It certainly is a relevant sub-topic in sociological cultural studies, but it also touches fields as diverse as cultural anthropology, communi-cations and media theory, marketing, psychology and aspects of natural sciences. Without an existing guiding framework that we could readily use, we were left to build our own. Fortunately, cultural theorists commonly incorporate aspects of media influence in their work, and many of the ideas pub-lished in the last twenty years on the topic were relevant to our cause.
In our attempt to identify a working theoretical framework, we examined a number of established, competing theories. The framework that we ultimately embraced was postmod-ernism. Most of the tenets of this theory of culture coincided with our early observations of the new participatory digital experience. Barker articulates postmodern culture quite succinctly by claiming that it is “marked by a sense of the fragmentary, ambiguous and uncertain quality of the world along with high levels of personal and social reflexivity” (2008). He further states that “postmodernism argues that knowledge is: specific to language-games; [and] local, plural and diverse” (2008). When each of these embedded concepts is examined independently, it describes aspects of the digital experience almost perfectly. One has to look no further than status updates on Facebook or comments to videos posted to YouTube to see the proliferation of language games, such as jargon (e.g., noob, pwnage), acronyms (e.g., LOL, FTW) and emoticons. The notion of local/plural/diverse knowledge is best exemplified by Wikipedia, which relies on millions of editors, each with their own type of expertise, to create, edit and police the site’s content. The idea of fragmentation is one that keeps brand managers and media planners up at night and is the significant contributing factor for the formation of online ad networks. And social reflexivity describes what we see in action any time a new online meme catches on: online users simultaneously are influenced by digital culture (by consuming it) and directly influence it (by sharing or contrib-uting to new content).
Harvey echoes these points and perhaps goes even further by saying that “postmodernism swims, even wallows, in the fragmentary and the chaotic currents of change as if that is all there is” (1990). It is almost as if he is describing not a
theory of culture, but the Internet itself. Such characteristics are precisely why we have a need for search engines, and they describe the design philosophy of a site like Face-book—which is never the same for even a second as people continually add photos and post new status updates to it. In 1985, well before Twitter, Flickr or Blogger, Hassan described postmodernism in opposition to modernism in a simple table. He contrasted the postmodern attribute of “anarchy” to mod-ernism’s “hierarchy” and “idiolect” to “master code” and even mentioned the postmodernist trait of “participation” com-pared to the modernist trait of “distance” (Hassan, 1985).
Based on a dozen more supporting examples, we concluded that postmodern cultural theory very naturally described how people interact with content, with brands and with each other via the Internet.
Initial approachIn Q4 2007, VivaKi contracted with a UGC monitoring vendor to provide the source data for the Reflecteur initiative. We are not naming the vendor here. The purpose of this paper is not to evaluate or assess any particular UGC monitor-ing firm but rather to describe the characteristics of UCG monitoring as they relate to understanding digital culture. We believe that the challenges detailed here are endemic to all UGC monitoring vendors in this particular context. The plan called for an initial launch of Reflecteur in January, 2008. Two months prior, the vendor switched on the data feeds, at which point the team began exploring the data and engaging in mock analytical exercises to ensure that the launch date could me met. For six months, the team worked with the data every week, first in anticipation of the launch and then while producing weekly issues of the Reflecteur publication.
Determining the methodThe methods for capturing, aggregating and analyzing UGC data are varied. The specific method recommended by the UGC vendor deviated meaningfully from a “typical” project.
Table 1: Schematic differences between moderism and postmodernism modernism postmodernism
purposeart object/finished workdistancecreation/totalizationgenre/boundryhypotaxismetaphorselectiontypeparanoia
playprocess/performance/happening
participationdecreation/deconstruction
text/intertextparataxis
metonymycombination
mutantschizophrenia
Source: Hassan (1985)
©2010 VivaKi. All Rights Reserved. Page 3
Most UGC monitoring initiatives are focused on brands—counting the number of brand mentions, analyzing brand sentiment, identifying keywords that are frequently associat-ed with brands, and so on. For these applications, a com-monly-accepted best practice is to cast as wide of a “data eli-gibility” net as possible. When trying to capture the entirety of brand mentions in UGC, there is little sense in purposefully restricting or blocking entire streams of data. Any UGC data should be eligible for inclusion, whether in the form of blog posts, uploaded videos, message boards, comments, tweets, social network status updates and anything else.
For the Reflecteur initiative, the UCG vendor recommended a blog-oriented approach, meaning that the eligible user-generated content was restricted to just blog posts. Because we were interested in cultural concepts and not pre-defined keywords (as in the cases of brand monitoring projects), we were compelled to cast a narrower net. Including arbitrary message boards would prove problematic precisely because of the lack of a keyword list. Without an a priori analyti-cal framework, data from such broad and varied sources becomes meaningless. Keyword lists are the backbone of the brand-oriented analytical framework. For the purpose of uncovering insights in digital culture, there are no meaningful analogues.
The blog-oriented approach to data eligibility effectively meant that we deliberately chose to ignore large parts of the UGC landscape. For the reasons described above, the vendor and the team did not believe that there was a feasible alternative. Collectively, we all agreed that the major tradeoff associated with the blog approach (altogether ignoring most UGC data—everything that was not a blog) was necessary in order to achieve clarity in the data.
Data architectureHaving settled on a blog-oriented approach, the next step was to apply sub-parameters to the blogosphere. For the purposes of ensuring a successful launch, Reflecteur initially focused on four aspects of digital culture: parenting, con-sumer technology, celebrity gossip and timewasters. Digital culture is, of course, much bigger than these four categories. But these four—which were chosen in consultation with our colleagues at VivaKi agencies—represented a nice cross-section of both target audiences as well as content types. Therefore, they represented a reasonable proof of concept. If the process worked across these topics, the initiative could be expanded to other relevant topics and subsequently grow in scope.
After identifying these four categories, the vendor and the Reflecteur team assembled a list of core blogs which would power the data feeds. Each topic required its own sepa-rate data feed; the four were effectively treated as unique
projects with no built-in overlap. We deliberately varied the number of core blogs for each of the four topics. Knowing if there was a “right” number of core blogs would have been an important finding in expanding the initiative to include other topics. All the core blogs were in English. The identified source blogs were distributed as follows:
• Parenting: 250 blogs
• Celebrity gossip: 89
• Consumer technology: 43
• Timewasters: 25
User-generated content eligible for inclusion in each of the data streams originated with these source blogs. Every week, the software pulled all the posts from each of the blogs in these four categories. These blogs, therefore, were referred to as “G0.” Any blogs that linked to G0 blogs in any given week were also eligible for inclusion. These were called the “G1” blogs. And finally, any blogs linking to the G1 blogs were also included (in any given week). These were called the “G2” blogs (See figure 2).
The G0 blogs were chosen through a collaborative process between the team and the vendor. Having worked on projects for brands that intersected the four categories (parenting for brands targeting moms, for example), the vendor had al-ready established lists of relevant blogs. The team examined each blog on the lists, removed the ones that were less rel-evant or inactive and supplemented the lists with additional blogs. This was a very manual process, but it needed to be.
Syndicated data does not exist for the parameters in which we were working (blogs and specific areas of cultural focus). And the team was creating a process—we did not have a di-rect precedent. We certainly borrowed from the vendor’s past experience with similar subjects, but that experience was
Figure 2: UGC blog distinctions
G0 Blog
G1 Blogs
G2 Blogs
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
©2010 VivaKi. All Rights Reserved. Page 4
directionally applicable, not directly applicable. The manual sorting only existed for the G0 blogs, however. The software automatically generated the rest of the eligible blogs.
The resulting data feeds, one for each of the four topics, sourced the data from a pool of blogs; the pool being derived based on the presence of links. Any blog post originating from any blog in the pool (G0, G1 or G2) was included in that week’s analysis and data feed for that topic. Any blog post originating outside of that pool was never sourced—it was never eligible for inclusion in the analysis.
It is important to note the nature of the actual data included in the analyses. In this method, the “raw data” comprised blog posts. Whether using Blogger, Wordpress or any other blog publishing platform, posts share certain attributes, which the vendor’s software was tuned to recognize. At the core of the analytical process were these individual posts. To attribute a relative value to each post, the software then analyzed links—it looked for other blogs and blog posts that linked back to any individual post in the core blogs.
Imagine that SiteA has been identified as a core blog. SiteA/Post1 is an original post appearing on that blog. SiteB/Post1 (a different blog) links to SiteA/Post1. That link is captured as a measure of relative value. SiteA/Post1 is credited with one link (See figure 3).
This process continues until the system exhausts all the links to the core blog post that it can find. In doing so, the soft-ware builds the lists for the G1 and G2 blogs, for which the link-tracking process repeats. The result is a ranking of blog posts, one list for each of the four culture topics, based on the number of links each post receives.
The reason for this analytical framework was rooted in pragmatism. We absolutely needed a measure of value; we needed to know where to look to identify digital culture phenomena. We never envisioned the software uncover-ing any actual insights—that would occur through human interpretation. But we needed a compass—a tool to help us focus on the most salient items each week. The vendor and we assumed that links represented a reasonable proxy for that compass. After all, if SiteA/Post1 garners ten times the number of links that SiteC/Post3 garners, it is reasonable to expect that the former is likely to be much more important. It touches a nerve of some sort, or it inspires people’s desire to share, or it generates a new online meme. It is more likely to contain some insight into digital culture.
Data reportingLike most UGC monitoring projects (including brand-focused projects), data reporting for this initiative occurred through an interactive dashboard. The primary mechanism for navi-gating the data was centered on the number of links any given blog post amassed during a given week. The reporting system ranked each post based on the total number of links and presented basic information about each post, including the title of the post and its URL. The “raw” data (the original blog posts) was therefore available for examination via the reporting interface. The interface was designed to add value to the raw data by focusing on “tabbed” data—the informa-tion about link backs. An effective way to think about raw and tabbed data for UGC monitoring is to compare it to conventional survey research. Raw data in survey research are the respondent-level records, and tabbed data would take the form of a set of cross tabs. In this example of UGC monitoring, raw data are individual blog posts, and tabbed data takes the form of the link information (number of link backs) found in the reporting interface. In both cases, the tabbed data serves only as a tool to analyze and interpret the raw data.
Additional functionality was built into the reporting system. A “hot topic” section displayed a ranking of specific words that recurred in the pool of blogs for that week. Similar to the information presented in “hot topics” and “hot searches” found in Google Trends, or in the “trending topics” that Twit-ter compiles, this report simply counted the occurrences of words and short phrases found in the pool of blogs. And because the pool changed every week— G1 and G2 blogs would migrate in and out of the pool from week to week—the reporting interface also captured the sites comprising the pool for each week.
Interpretation and insight disseminationReflecteur was initially a weekly publication (it is now bi-weekly), so the team constructed a weekly process for
Figure 3: Blog post rankingBlog 1
Post
Post
Post
Post1
Post
Post
Post
Post
Post
Post
Post
Post
Post
Blog 2
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post
Post1
8 links 3 links
©2010 VivaKi. All Rights Reserved. Page 5
accessing the reporting interface and interpreting the data for insights. We relied heavily on the main reporting feature (the ranking of link backs) to determine which posts seemed to generate broad interest. The team would examine the top 10-15 posts directly, which began with visiting the blogs and reading the ranked posts.
We relied on ethnographic techniques to arrive at the under-lying insights. Robert Kozinets (2002) described a framework for online ethnography which he called “netnography.” Simply put, netnography is the extension of ethnography to the digi-tal space. Ethnography is particularly suited for adaptation to online research due to its inherent flexibility and open-endedness – traits well-suited to the dynamic nature of the Internet. Relying on online communities, Kozinets outlined five steps for conducting netnographic research: 1. Make cul-ture entrée, 2. Gather and analyze data, 3. Ensure trustwor-thy interpretation, 4. Conduct ethical research and 5. Provide opportunities for culture member feedback (2002).
In addition to applying aspects of the netnography frame-work, we found that conventional ethnographic techniques translated quite nicely to UGC environments such as blogs. Rather than observing people in their homes interacting with a website, however, we looked for evidence of interac-tion that people left. These came in two primary forms: the comments that site visitors submit on the blog post itself and the commentary that other bloggers include when they link to the post. These user reactions have proven to be the most fertile ground for assessing the nature of people’s response. In UGC environments, people are increasingly comfortable posting comments and engaging in mini debates with other commenters. Because the string of comments is cumula-tive and archived, it serves as a collective record of reader response. For this reason, comments became the keystone for understanding the underlying insight behind a piece of user-generated content.
Interpretation did not stop with examining comment threads, however. As it gained more ex-perience interpreting user-gen-erated content, the Reflecteur team developed a list of ques-tions to ask when inspecting a piece of content. This checklist served as a guide to ensure that the team approached each piece of content consistently (see table 2).
The question that we ultimately try to answer is this: what, if anything, does a given piece of user-generated content reveal about digital culture? Is it rep-
resentative of some established or emerging trend, behavior or aspect of the digital collective consciousness? We focus on qualitative insights, just as conventional ethnography does. We do not attempt to address the extent of cultural themes that we identify because ethnography cannot adequately achieve a quantitative task such as this. But we do note re-curring themes over time; and, in doing so, can directionally gauge if a theme is particularly pervasive, if it evolves or if it fades away.
Once we were comfortable that we have a good grasp of the digital cultural insights from any given week, the next chal-lenge was disseminating those insights to our constituents. At launch, the constituent-facing part of Reflecteur took the form of a weekly publication comprising three elements:
• Four original articles describing pieces of user-generated content and the accompanying digital cultural insight. These articles also linked to the content so that the reader could experience it for themselves.
• Nine additional links to pieces of content that are note-worthy but not necessarily worthy of a full article. These links either attain some level of virality or they support a digital culture theme which we had previously uncov-ered.
• Top Reflecteur sites. Because the pool of blogs var-ied from week to week, it was important to highlight a sample of the sites comprising the pool. Each week, the publication linked to 25 sites that contributed to the source data in the reporting system.
The publication was designed to calibrate the awareness of and sensitivity to digital culture among Denuo’s employee base. It was not designed to address any specific marketing challenge or target audience research objective. Reflecteur was initiated to help the organization transform itself to align with trends in digital culture—a topic that until then had been invisible at the institutional knowledge level. Individually, Denuo’s employees were tuned into digital culture at various
levels. Organizationally, it was not; Reflecteur filled that gap.
Over time, the consciousness of the organization was re-orient-ed to be more aligned with dig-ital culture. Individuals would reference insights uncovered by Reflecteur through the course of their day-to-day work, com-pare their interpretations of a new meme with one another and connect the dots between their client deliverables and meaningful recurring themes found within digital culture.
Table 2: Checklist for Reflecteur content
Ellicit high levels of participation
Have cross community appeal
Have global appeal
Show a larger understanding of digital culture
Spawn a new digital meme or activity
Does it: As illustrated by:
Dviersity and amount of interactions
Diversity in linking sites
Links from sites around the world/in various languages
References to other digital culture memes/themes
Creation of derivative works by others
©2010 VivaKi. All Rights Reserved. Page 6
Problems with UGC monitoringThe first issue of the Reflecteur publication launched as scheduled. As the team continued working with the UGC monitoring data over the course of the next several months, however, we began to notice a series of problems. These problems can be classified into two categories: substand-ard data quality and a disconnect between blog posts that are merely rich in link backs with posts that illustrate some greater digital culture insight.
Poor data quality was not a problem that the team initially anticipated. This particular vendor was well-respected, and various agencies in the network had commissioned reports from it in the past. A key problem with vendor-produced reports, however, is that the underlying data (the “raw” and “tabbed data” as described earlier) are obfuscated. The raw and tabbed UGC data tend not to be displayed in a slide pres-entation report to clients. The vendor’s analysts obviously work with it, but the agency or client tends not to experience the data firsthand—the reports focus on heavily summarized data, implications of the findings and recommendations for action. This was the first occasion where we experienced the data itself without the perversion of summarization. Even standard UGC monitoring dashboards tend to conceal the underlying data—not as much as a report does, but certainly more so than we experienced with the data interface for this project.
The most disturbing example of this was the sudden visibility into the effect that aggregator and spam blogs had on the results. These computer-generated sites exist to tap into the paid search market. Software is written to automatically copy entire blog posts and entire blogs, repost them to new sites, and repeat the process. Because search engine rankings are based partly on cross-linking, these counterfeit sites link to each other, thus creating the appearance of a healthy blog community. None of the content is original, the sites do not have an established audience base, and there is no real rea-son why a person would want to visit them. But because of the mini networks that the cross-linking creates, these sites can garner quite attractive search rankings. Such placement naturally attracts free traffic, and the sites monetize the unsuspecting visitors through paid search ads on the sites as well as banner ads from lower-tier ad networks. Setting aside the ethical or search marketing problems associated with these sites, this reality did introduce a very considerable problem into our UGC data streams.
The UGC monitoring software was frequently unable to discern if a particular post was original or whether it was a result of the automated scripts described above. To a hu-man, the difference is immediately very clear. But because the software was often confounded by these sites, the data interface tended to be clogged up with counterfeit sites. After all, the primary measure of value for any given blog post (in
the UGC monitoring system) was the link back. These sites all linked to each other, and they appeared to the monitoring system as blogs. So their rankings relative to the legitimate blogs increased. Over time, the team spent more of its time sorting through the aggregator and spam blogs. It certainly could be done, but the problem diminished the value of the data interface immensely because it reduced the power of automation.
A second problem centered on attribution of value. On the surface, attributing value based on the number of link backs seems reasonable. If one post garners more link backs, it is presumably more important and more likely to represent an insight into digital culture. True blog-to-blog link backs are not that common, however. Increasingly, people share content they like through Twitter (via any number of URL shorteners), Facebook, instant messaging, Digg and dozens of other social media applications. To be sure, blogs are an important part of the user-generated landscape, but to as-sign all the value of an individual post based on how many other blogs link to it is problematic. It simply fails to capture the majority of the social links.
This omission results in problems in the data feed. The blog post ranked first might only have a handful more link backs than the post ranked fifth, or tenth. Because blog link backs are relatively rarer than sharing links in other social media environments, the distinction between ranked posts is quite negligible. The other problem with attributing value to link backs is that it ignores the importance of traffic and expo-sure. If a post has ten link backs but only 100 visits/views, it is probably less important than a post which has three links but 10,000 views. The team would have ideally incorporated exposure metrics to help quantify the value of individual blog posts, but that data simply did not exist—at least in a form that could be integrated into the reporting system.
In addition to the attribution of value, the type of content proved to be problematic as well. Automated monitoring systems work great for analyzing text, but for anything multimedia the systems tend to be less-than-capable. So much of digital culture is defined by videos, photos, graphics, animations and other images (e.g., cartoon drawings). Yet, the UGC systems can really only handle text well. The rea-sons for this limitation are obvious—text comprises common symbols easily recognizable by software. But the negative impact to the Reflecteur initiative was unmistakable. In order to surface multimedia content, it had to originate with some text commentary on a blog post. This certainly happens, but it essentially places an artificial handicap on multimedia con-tent. Because this content is so crucial to the fabric of digital culture, such a limitation became unacceptable.
The final problem revolved around user reaction. In our at-tempts to uncover digital culture insights, we often turned to
©2010 VivaKi. All Rights Reserved. Page 7
user reactions to get a sense of how people responded to any particular piece of content. Comments posted to a YouTube video, tweets, comments posted to a message board, etc. all are tremendously valuable when trying to discern the insight. The reporting system accounted for none of these.
Search for alternativesDue to the problems with the UGC data detailed above, a new data collection process was called for. After examin-ing various possibilities we settled on using a crowdsourcing approach. Crowdsourcing, which was first coined in Wired magazine in 2006, delegates a specific task to a group, with each individual often performing the same general role. It is typically discussed as one route for cost effectively solving marketing and business problems. However, in the case of Reflecteur, crowdsourcing could open up a wider swath of the digital world for our examination. Much like the blind men and the elephant, each individual only sees a small portion of everything created and shared within the digital universe. We each have our own concept of what digital culture is.
In order to make this process work, a tool was required to collect the various crowdsourced items. We chose Diigo, a social bookmarking site, for a number of reasons. First, it had the ability to create closed groups to share links. Though other sites also have this functionality, Diigo also had a clean and easy to use interface with an additional bonus of a web browser toolbar plugin that allowed contributors to bookmark a site without leaving the page. This toolbar meant that the entire contribution process took a total of a few seconds, helping to alleviate any burdens on users and subsequently increase participation.
The team tested this process along side the UGC listening system for six months to assess whether the sites we col-lected as a group were better, worse or the same as the sites and content the UGC software surfaced. Of particular concern was whether or not we’d see a dramatic shift in the types of sites and items tagged in the crowdsourcing process. Would we find that the crowd was too homogeneous and therefore missing important items across the rest of digital culture, invalidating our results? We actually found the opposite. The crowdsourced data not only tended to overlap much of the usable UGC data, but it included items far beyond the original UGC areas of focus (parenting, consumer technology, celebrity gossip and timewasters.)
The employees of Denuo are a digitally tuned-in group. But each individual was tuned-in to a different area of digital culture. Popular items in areas such as food, gaming, fashion and design were captured within our Diigo group, but not by the UGC software due to the required restrictions on subject areas. The content formats were also more diverse. While the UGC software could only capture blog posts, our contribu-
tors captured tweets, websites, videos, photo galleries, new platforms and news articles. While many of these items had been discussed on blogs, they were often not the items that popped as the majority of user interaction had happened on the photo itself, for example – interactions the UGC monitor-ing could not capture. The Internet is not as constrained by traditional demographics – age, race, location, etc. Though the crowd was almost completely composed of individuals in their 20s and 30s living in the United States, we found that individual’s hobbies and interests had a much greater impact on which sub-communities of digital culture were repre-sented.
The one area that remained a bias was around language, as the vast majority of contributors spoke English as their first language. However, this too ended up being a much smaller issue than had been expected, as so much of what becomes popular within digital culture is visual and therefore able to cross-language boundaries. And we continued to refine our source pool by expanding it even further. One benefit of this extension has been an increase in the diversity of backgrounds and demographics of our contributors, which is already leading to higher quality data.
Though we were pleased with the quality of data we were getting via the crowdsourcing process, it did require some changes in our interpretation process. We were no longer given a list of link backs to see (as a proxy) how popular an item was, as we had been with the UGC data. Instead, this assessment of popularity and virality was left to the Reflec-teur team to determine. In order to do so in a consistent fashion, our criteria for inclusion in the Reflecteur publication, listed above, became more formalized. Once this process was set into place we moved fully to the crowdsourced approach. This shift was actually a six month migration as we tested and assessed which process worked best for our goals.
With this shift, a number of incremental benefits to the crowdsourced approach emerged. The first was the unex-pected usefulness of the database of links created within Diigo. The manner of sharing links – via a simple web browser toolbar – also allowed expanded participation to non-researchers within the agency. Moreover, all links were never deleted from the Diigo group, remaining accessible to all members. The simplicity of this new database of links, compared to the reporting interface of the UGC software, al-lowed this same group to peruse and access the data easily. This database quickly became a core source for examples and information for client presentations, pitches and various agency projects.
At the same time, everyone in the agency expanded their understanding and awareness of digital culture beyond their own experiences. RSS feeds of the link database and e-mail alerts were used by a majority of contributors to consistently
©2010 VivaKi. All Rights Reserved. Page 8
remain aware of what was being talked about and shared online—allowing everyone to recalibrate digital efforts in real-time. As a side benefit, the process of participating in Reflecteur promoted collaboration and new employee connec-tions across the entire agency.
Because we did not ask contributors to go outside their usual routine to look for Reflecteur links, there is obvious bias in the group of sourced sites. They are reflective of the group contributing. We began to combat this bias by inviting every-one within Denuo to participate in the crowdsourcing process. By adding these additional perspectives we hoped to piece together a more complete image of what our elephant—digi-tal culture—looked like as a whole.
Taken from the perspective of conventional research, whose foundation is built on the idea of representative sampling, this bias would be glaring, and to some, too subjective. It is important to note, however, that this bias is absolutely inherent in any crowdsourcing project. The two cannot be separated. Crowdsourcing values expertise and perspective over representativeness. Because Reflecteur was designed to be a qualitative initiative, the team was comfortable with the migration to the new sourcing method and the accompanying trade-offs.
Migration to VivaKiAt the beginning of 2010 we made the decision to expand Reflecteur beyond Denuo, the original incubator agency. After successfully using Reflecteur to calibrate this agency’s sen-sitivity to digital culture and refining our new crowdsourcing process for over 12 months, it was clear that other agencies within the VivaKi network could benefit from participating in the initiative. We had seen within Denuo that when more individuals participate, the result is broader visibility into the digital culture landscape and, therefore, better insights. By expanding to all VivaKi agencies, each of which had a vested interest in better understanding digital culture, we would be able to improve the effectiveness of Reflecteur.
Over a period of almost six months, we worked with the leadership at each major agency to identify new contributors, who were often junior members of the organization, to join the collaboration. These contributors were exposed to a short orientation and overview of Reflecteur. They were then given access to the social bookmarking group and encouraged to post links. As expected, actual participation varies dramati-cally, with some individuals posting regularly and some very infrequently. Every agency does have a number of active contributors adding to the Reflecteur data pool every week. With this additional participation, it took very little time to see an increase in the diversity and quality of links.
Concurrently, Reflecteur’s new role within the wider VivaKi organization required another design change. Though the core elements of the publication – four articles, nine addition-al links and top sites – remained the same, each participating agency received its own white-labeled version of the publica-tion. As participants in the data sourcing process, all could legitimately claim collaboration in the publication’s creation. They were also all given the opportunity to market their agency on the third page of the publication, often showcasing their top digital work. Finally, they were given free reign to share the publication with clients, partners, and anyone else outside the agency.
Many chose to also post the publication to their intranets and company blogs. However, it is agency employees who are the core distribution base for Reflecteur. This focus is due to the fact that it is the agencies that are first and foremost the cli-ents of Reflecteur. It is a tool for them to expand their knowl-edge of and calibrate their digital efforts to what is happening within digital culture. Applied to the larger VivaKi network, Reflecteur has become a powerful tool for improving overall organizational intelligence within the digital culture space. Recently, the initiative has expanded to include offices out-side the United States. The global rollout will further diversify the content as well as the impact of Reflecteur in influencing digital marketing.
Extending the value of the dataAlong with the bi-weekly publications, the team also cre-ates semi-annual presentations describing the recent trends within digital culture. For these presentations we re-examine everything published in the most recent six months, tak-ing note of the core insights of each item. After selecting the most prevalent, pertinent and recent themes, we craft a presentation that articulates each of these themes with a series of supporting examples. Examples of marketers aligning with these themes are also included to show the “real world” application of digital culture insights. While the Reflecteur publication focuses on the timeliest items, these presentations allow us to step back and examine a larger period within digital culture – the point being to find recur-ring patterns.
These semi-annual summary presentations do not cover new content; the material is sourced strictly from past publica-tions. But the perspective is somewhat different. The publica-tions are topical; the presentations, on the other hand, are designed to provide a broader view of the overall digital culture landscape. What we have seen over time is that there are consistent, transitory and evolutionary themes within digital culture. All of these are useful for the participating agencies as inspiration for larger approaches and positioning within the digital space as well as for short term campaigns and initiatives.
©2010 VivaKi. All Rights Reserved. Page 9
Like the publication, the presentations are shared with the employee base at each agency and with those agencies’ clients. But because the two are designed to share different perspectives on the same topic, they are used differently. The publication is regularly shared, along with the agencies’ other publications and thought leadership materials. The presentation is shared as well, but usually in a different con-text. VivaKi agencies use the presentation to inspire brain-storms, kick off working sessions to address key marketing challenges and as special events to freshen the thinking of both the agencies and the client teams.
Concluding thoughtsReflecteur has now been in existence for three years. In that time we have changed data sources, enhanced the design of the user-facing product and expanded both our contributor pool as well as our distribution. The primary lesson we have learned is that, like any culture, digital culture is continually evolving. The digital experience is increasingly about par-ticipation in the form of sharing and creating—hallmarks of social media. Because of this characteristic, people are more empowered than ever to not just consume digital culture, but to also actively contribute to and shape it. This is precisely why the ability to internalize digital culture will be a prereq-uisite to crafting effective digital marketing initiatives moving forward. Any organization focused on digital marketing will need to create institutional knowledge initiatives in order to build this capability.
Barker states that “culture is not ‘out there’ waiting to be correctly described…rather, the concept of culture is a tool that is of more or less usefulness to us” (2008). A grasp of digital culture ultimately helps our constituents—our col-leagues in the VivaKi agencies—better understand human motivations, behaviors, desires and attitudes. Reflecteur is the mechanism that helps us properly use the tool that is culture.
At the commencement of the Reflecteur initiative, we un-derestimated the degree to which automation could play a role in understanding digital culture. Our reliance on UGC monitoring as the initial data collection mechanism demon-strated this. While many brands have success incorporating UGC monitoring into their business and market intelligence work streams, it simply did not work for this initiative. Our conclusion is that the subject of digital culture is just not appropriate for software-generated, automated data col-lection. Keyword lists, which tend to work well in brand-oriented UGC monitoring projects, do not translate to the topic of culture. Additionally, culture, particularly postmodern digital culture, is too nuanced and multifaceted. The level of complexity within the subject matter greatly outpaces the level of sophistication of the UGC monitoring systems. At the
present, this gap is too large. As UGC monitoring technology progresses, perhaps this gap will narrow, in which case we will be willing to re-examine our conclusions.
While UGC monitoring can not fulfill a role in the Reflecteur initiative, technology certainly does. Crowdsourcing, the technique with which we have seen much success, relies on technology platforms to be efficient. The expertise is decid-edly human, but the social bookmarking platform greatly facilitates the collaboration of people who have that exper-tise. Without the enabling technology, crowdsourcing as an effective data sourcing method is not possible. An additional benefit of this method is its ability to quickly and easily incorporate new aspects of digital culture. This adaptability extends to future platform innovations and will allow us to evolve Reflecteur in response to new innovations and facets of digital culture – we are relying on human collaboration and expertise and not over-relying on technology to source data.
Through our three years, 70+ issues, 5000+ data items and 50+ contributors, we have witnessed firsthand that the digi-tal space has a distinct culture (and subcultures) all its own. As an arena of cultural studies, examining and understanding digital culture relies on established techniques. We rely on classic ethnographic techniques and the lens of postmodern-ist cultural theory as our guide to recognizing salient insights. We also rely on the collective wisdom of VivaKi’s people, who are wired into digital culture because of their personal pas-sion for it. And finally, we rely on technology to facilitate the collaboration between these people. This combination, es-tablished observational techniques, well-documented theory, crowdsourcing and a facilitating technology platform, is the secret to how VivaKi has been able to create a network-wide institutional knowledge base of the very important, yet still emerging topic of digital culture.
ReferencesHarvey, David. (1990). The Condition of Postmodernity: An
Enquiry into the Origins of Cultural Change. Malden, MA: Blackwell Publishing Ltd.
Kozinets, Robert. (2002). “The Field Behind the Screen: Using Netography for Marketing Research in Online Communities”. Journal of Marketing Research. Vol 39, No.1, pp 61-72.
Barker, Chris. (2008). Cultural Studies. London: Sage
Hassan, Ihab. (1985). “The Culture of Postmodernism.” Theory, Culture and Society. Vol 2, No. 3, pp 119-32.
©2010 VivaKi. All Rights Reserved. Page 10
Authors’ noteThough our goal in creating Reflecteur is to uncover insights about digital culture, we did not focus on that here. The focus of this paper was on the process of examining digital culture and, more specifically, the benefits and pitfalls of a UGC monitoring versus a crowdsourcing method. We have collected a number of digital culture insights over the last three years. The bi-weekly Reflect-eur publication is freely available for download at vivaki.com.
For more informationContact Ellen Bird at [email protected] or Christian Kugel at [email protected]
Christian KugelAs a veteran of Denuo, Millward Brown and Starcom, Christian has developed a unique and expansive perspective on how people interact with brands, each other and media. In his current role, Christian is SVP at Publicis Groupe’s VivaKi unit, where he works with senior leadership in the corporate strategy group. He created Reflecteur, a first-of-its kind initiative that deconstructs, analyzes and explains digital culture. Prior to VivaKi, Kugel was a member of the management team in Denuo. There, he led the HP account and managed Denuo’s proprietary toolset. In 2007, Kugel created Socialight, Denuo’s propri-etary consulting tool that measures the degree and influence of conversations and recom-mendations among groups of consumers. Deployed internationally, Socialight has been used to uncover insights on over 100 brands. Before Denuo, Kugel was an Account Group Director at Millward Brown, where he managed multi-national research initiatives for blue chip technology clients. Prior to that, Kugel was Director of Insights & Analytics at Starcom IP. He was named an ‘Agency Innovator’ by Internationalist. His work with Socia-light resulted in the ‘Most Innovative Research’ award by WOMMA, and he was nominated for the prestigious Goodyear Award for best international research.
Ellen BirdIn the past two years, Ellen has established herself as a leading expert in digital culture within Publicis Groupe’s VivaKi unit. She currently manages Reflecteur, which she trans-formed from being an internal Denuo initiative into one that reaches all 15,000 VivaKi employees. Prior to her current role in the VivaKi corporate strategy group, Ellen man-aged emerging media initiatives at Denuo, such as leading the strategy and activation for HP’s first formal presence on social networking platforms. She also managed a pioneering marketing program for HP which brought the power of unbiased, third party product re-views to shoppers’ hands—via their mobile devices in the store, at the point of purchase. Prior to Denuo, Ellen worked at Starcom Worldwide, where she managed new media as well as traditional marketing efforts for blue chip accounts including Bank of America and Disney Parks. She has been a featured speaker at events for the Word of Mouth Market-ing Association and Digital Hollywood. Although Ellen is a native of the East Coast, she graduated with a degree in Sociology from Northwestern University and is currently based in Chicago.