Comparing filmand video game reviews:

By Ben GiffordCleveland State University

December 7, 2011

A report using computer-aided text analysis

“Sometimes 'dead' is better”

E.T. the movie:Won 4 Oscars, nominated for another 5

E.T. the game:3.5 million unsold copies (out 5 million total) were allegedly buried in aNew Mexico landfill

- Jud Crandall, Pet Semetary (1989)

Background/Rationale

Video games are popular and the industry is wealthy

72 percent of American households play games

Average game player is 37, played for 12 years

$30.3 billion industry in 2006 $46.5 billion industry in 2009

Purchasing decisions

What makes people decide to buy games? Advertising Reviews Friends/Word of Mouth ??

Game reviews

Little-to-no research has been dedicated to video game reviews themselves

Are they “up to snuff?” Do they display critical thought?

Can they impact sales (future research) This project focused on comparing game

reviews to film reviews.

Similarities between media

Origins in “exciting spaces” Movies: kinetoscope parlors,

nickelodeons Games: arcades

Transitioned to home viewing/playing through VHS, television and home consoles/computers for games

Similarities continued

Focus on narratives One goal of both is to induce feelings of

presence Movies: sound, color, widescreen, high-

def surround sound, 3D Games: improved graphics, voice-acting,

motion capture, “natural”/motion controls, as well as 3D, surround sound, etc.

Film Reviews Early movie critics and reviewers came

from a theater background, and many early films resembled plays in the way they were shot and staged

Many notable film critics: Pauline Kael Roger Ebert Leonard Maltin

Game Reviews

Early game reviewers came from?? No truly famous game critics, at least not

by name

Comparing the Two

Scalzi (2006) points to the relatively young age of video games.

Games have only been at the point of telling stories for around 20-30 years.

It's only in the past 10-15 years that mae narratives have really evolved.

Comparing the Two Kael began writing for the New Yorker in

1967. She established a “golden age” for film

criticism. Film had already been around for 50-60+

years at the point Talkies had existed for about 40 years Narratives in films even before that

First Gaming Magazines

Both magazines launched in 1981. Computer and Video Games (U.K., left) still exists today but in a much different format and is primarily web-based.

Electronic Games Magazine

The chart above came from the first issue published in winter 1981 and compares the home consoles available at the time.

First video game reviews

Computer Gaming World also launched in 1981 and regularly reviewed computer games.It became Games For Windows in 2006, and shut down in 2008.

(First issue pictured)

First video game reviews

Famitsu began reviewing games in 1986 and is still well-regarded today.

It gave its first perfect score (40/40) in 1998 and has only given a total of 17 perfect scores in its 25 years of existence.

(First issue pictured)

Methodology

Using the popular review aggregator Metacritic (metacritic.com), 94 video game and 100 film reviews were gathered.

Reviews were selected using systematic random sampling and random.org

Sample Example

Make sure the proper filters are in place. This will show all of the reviews for PlayStation 3 games in descending order of Metascore

Pick a random starting point using random.org, based on sample size.

Sample ExampleNeed to see all the reviews that Metacritic has gathered

Pay attention the the Metascore – an average of all critics' scores

Sample ExampleScroll down and count all the reviews that are in line with the Metascore for that game.

If there are none, than count the ones closest distance wise (e.g. Metascore is 76, but no 76 reviews. Count all 77s and 75s if they exist. If not, count 78s and 74s, etc. until at least one review is counted)

Sample ExampleVisit random.org and enter the range with “1” as the minium and the number of reviews as the maximum. Red Dead Redemption had 11 reviews scoring a “95,” so enter “11” as the maximum.

Click “Generate.” Count that many down from the first review that was counted. That will be the data collected.

Sixth review from top scoring RDR at “95”

Click

Note: If a review is not written in English, or if a link is broken and that review cannot be found by searching the site and google, resample until a valid review is found.

Sample Example

Copy/paste whole review into a text file. Discard titles, subtitles, image captions, advertisements, ratings, and conclusions separate from review if any are present.

Note that some reviews may span multiple pages, requiring researcher to copy/paste each page in turn.

Sample ExampleReturn to master list of game or movie reviews

Count “X” reviews down from the previous item sampled, where “X” is “9” for PS3 games and “70” for movies.

Repeat the process over and over until the population lists are exhausted.

1

2

3

4

5

6

7

8

9

Coding

Reviews were coded using 12 dictionaries (8 Pennebaker, 4 custom) in Yoshikoder

Sample words

Cognitive Mechanics: abandon*, accept, accepted, accepting, accepts, achiev*, acknowledg*, adjust*

Film Genres: Action, Adventure, Biopic, Blaxploitation Chick Flick, Comedy, Crime, Detective, Disaster

Game Genres: 3PS, Action, Adventure, Arcade, Beat 'em Up, Beat em up, Dance, FPS, Fighter, Fighting

Negative Emotions: abandon*, abuse*, abusive, ache*, aching, advers*, afraid, aggravat*, aggress*

Sample Words Cont.Sample words

Nonsense: er, hm*, uh, um, umm*, zz*

Optimism: accept, accepta*, accepted, accepting, accepts, advantage*, adventur*, assur*, award*, best Referencing audience: thee, thine, thou, thoust, thy, Y'all, ya, ye, you, you'd, you'll, you're, you've, your*

Referencing self: I, i'd, i'll, i'm, i've, let's, lets, me, mine, my, myself, our, ours, ourselves, us, we, we'd,

Sample Words Cont.Sample words

Sensory: appear, appeared, appearing, appears, ask, Asked, asking, asks, ate, bitter*, call

Technical film terms: a-list*, a-movie*, accelerated motion, act, actor*, actress*, acts, ad lib, adaptation, Technical game terms: abstract game*, act, acts, adaptation, advergame*, aesthetic, allegor*, allusion*

Vulgarity: (use your imagination; they're all there)

More About Dictionaries

'Technical' Dictionaries Number of Entries

Film 309

Game 204

Overlap between the two

105

Genre Dictionaries Number of Entries

Film 39

Game 41

Overlap 3

Results

Cognitive Mechanics***

Optimism***

Negative Emotions*

Sensory Language

References to Audience***

References to Self**

Nonsense Words

Vulgarity

Film Genres***

Game Genres***

Technical Film Terms***

Technical Game Terms***

.0000 .0100 .0200 .0300 .0400 .0500 .0600

Mean Values For Average Language Use Across All Reviews

Movie Game

Average Language Use

Ca

teg

ori

es

* p < .05 ** p < .01 *** p < .001

Results

Cognitive Mechanics***

Optimism***

Negative Emotions*

Sensory Language

References to Audience***

References to Self**

Nonsense Words

Vulgarity

Film Genres***

Game Genres***

Technical Film Terms***

Technical Game Terms***

-1.000 -.800 -.600 -.400 -.200 .000 .200 .400 .600

Paired Samples Correlations

Between Medium and Dictionaries

Games Movies

Significant Results Tabled(Average language use)

Dictionary (Average) Games Movies Difference

Technical game terms .0420 .0169 .0251

Technical film terms .0154 .0274 .0120

Game genres .0039 .0013 .0026

Film genres .0021 .0049 .0028

References to self .0086 .0053 .0033

References to audience .0285 .0024 .0261

Negative emotions .0156 .0182 .0026

Optimism .0112 .0075 .0037

Cognitive mechanics .0491 .0405 .0086

MANCOVA Movie reviews collected were much shorter

than game reviews (word count)

Medium N Mean Median Std. Dev Min Max

Game 94 998.94 994.50 374.144 274 2052

Movie 100 464.61 419.50 291.621 52 1942

What if the word count restraints placed on movie reviewers affected writing style?

MANCOVA Tests of Significance

Only medium is significant

MANCOVA

Comparing new significant results to old ones

Dictionary (Average) Correlation squared (T-test)

Eta Squared (MANCOVA)

Technical game terms .608 .126

Technical film terms .219 .023

Film genres .096 .023

References to audience .659 .200

Using word count as a covariate really reduced variance explained by the significant variables

Discussion Problems

Yoshikoder Sampling

Future directions Financial impact of critical reviews Quanitfy differences between critcal

essays and reviews More dictionaries and improving validity

Fin