SGM302 – Research Workshop Module – “Media Content Analysis”

DR. CAROLINA MATOSLECTURER IN MEDIA AND SOCIOLOGYDEPARTMENT OF SOCIOLOGYCITY UNIVERSITY LONDON

Set readings Core reading: Fairclough, N. (2015, 1989) "Critical discourse analysis in practice: interpretation, explanation and the position of the analyst" in Language and Power, p. 154-177

Krippendorff, K. (2012). Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage

Additional: Bignell, J. (2002) Media semiotics: an introduction Carpentier, N. (2008) Discourse theory and cultural analysis: media, arts and literature Fairclough, N. (2003) Analysing discourse: textual analysis for social research Grimmer, J., & Stewart, B. M. (2013). Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis, 21(3), 267-297. doi: 10.1093/pan/mps028

Core points * Media content analysis in historical perspective

* Content analysis: some definitions

* Quantitative versus qualitative content analysis

* Objectivity and intercoder reliability

*Code book and tips on how to conduct a content analysis

* Methods of sampling for media content analysis

* Discourse analysis (and CDA)

* Contemporary media examples: refugee crisis and Brexit coverage

* Conclusions and seminar activities

Quantitative versus qualitative research: merits and limits

* Both methods (qualitative and quantitative) have their strengths and weaknesses. It is up to the individual researcher to decide which method to use, or to combine both

* The trend in any good, serious quality research has been the combination of both quantitative and qualitative methods (i.e. discourse analysis and quantitative content analysis)

* Quantitative analysis: efficient way of collecting information from a large number of respondents (survey, content analysis), “scientific”, “objective”

* Qualitative research provides depth and detail, is better able to answer questions of why and how than quantitative data.

Disadvantages: small sample and relies on the experience of the researcher

Two approaches to analysing text: positivist and interpretivist

Positivist:

CA: “who says what in which channel to whom with what effect”? (Lasswell, 1948, 50)

Interpretivist:

* Conversation Analysis – The study of talk in interaction. This could include institution or casual conversation (i.e. Silverman, 2006)

* Discourse Analysis – The way versions of the world and of society are produced (and reproduced) in and through discourse

* Critical Discourse Analysis – how political and social domination is reproduced in text and talk (Fairclough, 2003, year; Van Dijk, 1999)

*

Media content analysis Turning words into numbers:

* Content analysis is used to study a range of texts, from interview transcripts to films, TV programmes, advertising, political speeches, magazine and newspaper articles

•Media content analysis became popular as a research method during the 1920s and 1930s (focus was on Propaganda Studies). It grew in the 1950s as a research method with the arrival of television

• Has in the last 20 years become an increasingly popular method in the field of mass communication research (Riffe and Freitag, 1997 in Neuendorf, 2002)

•* Qualitative content analysis includes rhetorical analysis, narrative analysis, semiotic analysis, interpretative and discourse (or critical discourse) analysis.

Definitions of content analysis * “Quantitative content analysis is a statistical technique for obtaining descriptive data on content variables. It offers the possibility of obtaining more precise, objective and reliable observations about the frequency with which given content characteristics occur…” (in George, 1959, 2009)

* Kerlinger (1973): “Content analysis is a method of studying and analysing communications in a systematic, objective and quantitative manner…..”

* Krippendorff (2013, 24) has emphasised reliability and validity: “…. The systematic reading of a body of texts, images and symbolic matter to make replicable and valid inferences from texts to the contexts of their use”

* Manifest versus latent content analysis – the manifest (or denotative or shared) describes what an author has written, as opposed to connotative or latent (“between the lines”), the intention. CA can only be applied on manifest content. (in Riffe et al, 2005)

Quantitative content analysis What are some appropriate communication content for study?

* Content analysis can be an appropriate method to identify words or labels in advertisements; phrases of themes in political speeches; paragraphs of space in newspapers devoted to crime stories and whole editorials in the press endorsing particular candidates

* Content analysis can also be used to address accusations of the underrepresentation of minorities in the media by attempting to measure the number of stories on immigration, asylum seekers, etc

* An important prerequisite for the content analyst is that the investigator knows what he/she is looking for before beginning to count.

* Critics of quantitative CA have argued that the method puts too much emphasis on the comparative frequency of different symbols’ appearance

Content analysis as a scientific method * “Content analysis is a summarising, quantitative analysis of messages that relies on the scientific method (including attention to objectivity, inter-subjectivity, a priori design, reliability, validity, generalisability, replicability, and hypothesis testing) and is not limited as to the types of variables that may be measured or the context in which the messages are created or presented.” (K. A. Neuendorf, 2002, 10)

* Both content and form must be considered

* Neuendorf (2002) alludes to latent content, suggesting that there are no limits to the variables that may be measured (Berelson (1952) imposes a limit on the variables, stating that only manifest content is measurable)

* CA is valuable when the researcher has a question which much be addressed using quantitative data (i.e. “how much?”, “how often” question)

The content analysis tradition * Holsti (1969)defined the uses of content analysis into three basic categories: a) make inferences about the antecedents of a communication; b) describe and make inferences about characteristics of a communication and c) make inferences about the effects of a communication

* The creation of coding frames is an important aspect of content analysis. As K. Krippendorff (1980 and 2004) states, six questions must be addressed in every content analysis:

1) Which data are analysed?; 2) How are they defined?

3) What is the population from which they are drawn?

4) What is the context relative to which the data are analysed?

5) What are the boundaries of the analysis?

6) What is the target of the inferences?

Uses of content analysis by purpose, communication element and question (Holsti,

1969)Purpose

Make inference about the antecedents of communication

Element

Source

Encoding Process

Question

Who?

Why?

Use

Authorship

I.e. Traits of individuals

Describe and make inferences about the characteristics of communications

Channel

Message

Recipient

How?

What?

To whom?

Analyse techniques of persuasionDescribe trends in communicationPatterns of com.

Make inferences about the consequences of communications

Decoding process With what effect? Measure readabilityAnalyse flow of informationAssess responses

Content analysis - example of a Dummy Table (in Riffe et al 2005)

Character is

Minority female

Speaking role

%

Non-speaking role

%

Minority male

White female

%

%

%

%

White male

Total

%

100%

%

100%

How to do a content analysis * Coding – Establish basic categories and themes induced from the text

* Types of codes:

These can be formulated based on a range of elements:

1) Previous research or theory;

2) Research or evaluation questions you are addressing;

3) Questions and topics from your interview

4) What your instinct tells you (i.e. your gut instincts and feelings about the topic)

Codes can thus be: 1) themes, topics; 2) ideas, concepts; 3) terms, phrases, keywords and 4) narratives and stories

How to conduct a content analysis study*

1. Formulate the research question – You begin with a topic and a research question (i.e. your question may be if each candidate has equal coverage)

2. Decide on the units of analysis –

3. Develop a sampling plan -

4. Construct coding categories and a recording sheet -

5. Coding and intercoder reliability – If you have multiple coders

6. Data collection and analysis -

* (Neuman, 2011, 367)

Suggested steps of CA 1) Read through the transcript and make notes when there is something interesting

2) Go through the notes (processing);

3) Read through the list and categorise each item in a way which describes it

4) Identity categories relating to patterns or themes identified (i.e. can compare and contrast, list them as major or minor. Can some be merged?)

5) Coding

6) Discussion

7) Conclusions (go back to the original transcripts and make sure that you have captured all the necessary information)

Developing a coding scheme or framework * Levels of coding:

1) Basic coding -

2) Focused/Theoretical Coding -

a) Themes and content and

b) Broad analytic themes (in relation to broader questions on theory and concepts

* Developing your coding:

May be inductive (theory-generating) and/or deductive (theory-testing)

Codes serve as shorthand devices to label, separate, compile and organise data (Charmaz, 2006)

Starts by being descriptive, end up being analytical

Manual versus electronic coding * Manual: Pen, posts, indexed notebooks

* Electronic:

A) Computer-Assisted Qualitative Data Analysis Software (i.e. Nvivo, AtlasTI, etc)

* Advantages include speed in handling large volumes of data; facilitates team research

* Search function in Word; Excel spreadsheets

* TextSmart by SPSS

Important: It is up to the researcher to do the thinking and conceptual work. Remember the computer will not help you analyse the data!

Coding book: objectivity and intercoder reliability * The coding system:

* Validity

* Reliability

What we measure:

a) Frequency

b) Direction

c) Intensity

d) Space

Intercoder reliability – If you use several coders, you need to check for consistency (i.e. Krippendorff’s alpha)

What is discourse? • Relationship between discourse and power: * Discourse analysis within the Foucaldian perspective refers to institutionalised patterns of

knowledge, to a connection between knowledge and power

* Foucault (1972) in The Archaeology of Knowledge emphasised “the importance of historicizing discourse, seeing a relationship between discourse, representation and knowledge in a way which “truth” is said to only mean something within a specific historical context”.

* Discourse as a site of resistance - “Competing discourses thus represent the struggle of particular groups to apply their own readings of the world”

• Ideology – can be viewed as being a set of values which are either taken for granted or not (today the word “discourse” is preferred to ideology).

Discourse analysis: what is it ?

* Discourses are ideas embedded in what we do, say and think, and these create the terms upon which we know the world

* Discourse analysis is a method committed to challenging common sense thinking. It is critical of taken for granted knowledge and argues that the ways in which we view the world are historically specific and socially constructed (in Matos, 2008)

* “Discourse holds a powerful role in establishing our conceptions of what things are, their nature and how they should be regarded” (Bryman, 2012)

* There are a variety of approaches to discourse analysis (i.e. Fairclough, 1992, 2003; Van Dijk, 1993; Jorgenson and Philips, 2002; Wodak and Meyer, 2009)

Fairclough and DA*(CDA) * Key early text in the tradition was Norman Fairclough’s Language and Power (1989)

* Approach draws from social theory and from the work of authors such as Marx, Gramsci, Habermas, Foucault and Bourdieu to discuss the connection between ideology, power and their reproduction through discourse

* Critical discourse analysis (CDA) is a multidisciplinary approach to the study of discourse which focuses on the ways social and political domination is reproduced through text and talk

* Analysis of text thus focuses on its linguistic features (i.e. vocabulary used, grammar), while the discursive practices relate to the production and consumption of texts (i.e. intertextuality)

* Critical discourse analysis in practice and forms of interpretation:

* Fairclough, 1989; Van Dijk, 2008

Fairclough and DA* * Fairclough (1993) emphasises that there is no set procedure for conducting DA. In his Language and Power, he offered a sophisticated linguistic approach to analysing discourse

* CDA in practice: interpretation, explanation and the position of the analyst

Dimensions of meaning Values of features Structural effects

Contents Experiential Knowledge/beliefs

Relations Relational Social relations

Subjects Expressive Social identities

CDA in practice* * Concerned with the social significance of texts: description needs to be complemented with interpretation and explanation

* “…the values of textual features only become real….if they are embedded in social interaction where texts are produced and interpreted against a background of common sense assumptions which give textual features their values”

* Interpretations are generated through a combination of what is in the text and what is “in” the interpreter, in the sense of the member’s resources (MR), which the latter brings to interpretation”

* Lists six major domains of interpretation: 1) Two on the interpretation of context (social orders and interactional history) and 2) Four on the interpretation of the text (phonology/grammar, semantics/pragmatics (meaning of utterance), cohesion/pragmatics (local coherence) and schemata (text structure and ‘point’)

* (Fairclough, 1989, 142)

Critical discourse analysis (CDA)

* Bryman (2012) states that CDA is a method which emphasises the role of language as a power resource, and which is linked to ideological and socio-cultural change

* Van Dijk (2008, 27) examines the relationship between discourse and social power, and is influenced by Bourdieu’s notion of “symbolic power”, among others

* CDA focuses on the creation and reproduction of unequal power relations in society

* Both Fairclough and Van Dijk are proponents of CDA, but offer slightly different approaches

* Van Dijk (1993) sees social cognition as the link between critical discourse analysis and the social world (which limits the analysis to how discourse is represented in the minds of individuals)

Fairclough’s three dimensional framework for studying critical discourse analysis

1) the analysis of texts (spoken, written);

2) the analysis of discourse practices of text production, distribution and consumption;

3) analysis of social and cultural practices which frame discourse practices and texts.

Micro, meso and macro-level interpretations:

a) The micro-level involves studying the text’s syntax, metaphoric structure and rhetorical devices;

b) The meso-level consists in looking at the text’s production and consumption and the power relations involved and

c) The macro-level is concerned with inter-textual relations between texts, and mainly with how external factors affect the text being studied.

Critical discourse analysis: definitions and challenges (in Van Dijk, 2008)

Fairclough and Wodak (1997: 271-80)summarise the main tenets of CDA as being:

1) CDA addresses social problems

2)Power relations are discursive

3) Discourse constitutes society and culture

4)Discourse does ideological work

5)Discourse is historical

6) The link between text and society is mediated

7)Discourse analysis is interpretative and explanatory

8) Discourse is a form of social action.

Brexit media coverage

Reuters Institute Study on the Brexit coverage

* The findings covered two sample days of coverage a week during the first two months of the referendum campaign immediately after David Cameron's post-summit Cabinet meeting on February 20, 2016

* Of the 928 articles focused on the referendum, 45% were in favour of leaving, with only 27% in favour of staying in the EU

* 19% of articles focused on the referendum were categorised as ‘mixed or undecided’ and 9% as adopting no position.)

http://reutersinstitute.politics.ox.ac.uk/news/study-shows-majority-press-coverage-eu-referendum-campaign-was-heavily-skewed-favour-brexit

Brexit media coverage: findings Positions varied between newspapers:

* “The Daily Mail included the most pro-leave articles followed by The Daily Express, The Daily Star, The Sun and The Daily Telegraph, while the newspapers including the most pro-remain articles where The Daily Mirror, The Guardian and The Financial Times. “

* “The articles examined in The Times were relatively balanced between the two positions, with a slight pre-dominance of pro-leave articles”

*Researchers also tracked what arguments were made for either pro-leave or pro-remain news stories. After removing articles focused on personalities, the campaigns or Brexit in general, the most cited arguments in the remaining 765 articles were:

1) the economy/business (33%); sovereignty (29%), migration (18%), regulations (14%) and terrorism/security (6%).

2. Press Coverage of the Refugee and Migrant Crisis in the EU: a CA of 5 EU

countries * The UNHCR Press commissioned a report by the Cardiff School of Journalism, following the impact of the refugee crisis in Europe since 2014, when more than 200.000 refugees fled

* Content analysis of 5 EU countries (Spain, Italy, Germany, UK and Sweden)

* Researchers looked at articles written in 2014 and early 2015. They found major differences between countries in terms of:

A) the sources journalists used (i.e. domestic politicians, foreign politicians, citizens or NGOs), b) the language they employed; c) reasons given for the rise in refugee flows; d) suggested solutions.

* German and Sweden used the terms “refugee” or “asylum seeker”, while Italy and UK preferred the word “migrant”. Media varied in terms of the predominant themes to their coverage

(see further information in the handout)

Sample: Refugees welcome? An evaluation of the digital media coverage (Canadian press)

Coding Schedule:

1) Research questions: a) How do the Canadian digital news media report on the refugee crisis in Europe? ; 2) Are refugees portrayed negatively in the Canadian digital news media?

V1 – Outlet: 1) Globe and Mail; 2) National Post

V2 - Date

V3 – Content type: 1) News; 2) Feature; 3) Editorial

V4 – Source: 1) Government/officials; 2) Advocacy groups; 3) Refugees/Migrants; 4) Government officials and refugees; 5) General public/polls and 6) The press

V5 – Main theme (primary theme that appears in the news article): 1) Plight of refugees; 2) Conflicts amongst refugees/clashes; 3) Economic and social burden of refugees, etc

Conclusions * Both methods (qualitative and quantitative) have their strengths and weaknesses. It is up to the individual researcher to decide which method to use, or to combine both

* The trend in any good, quality research has been the combination of both quantitative and qualitative methods (i.e. discourse analysis and quantitative content analysis)

* Rise of the qualitative research tradition again and the popularity of methods such as discourse analysis and ethnography

* “Triangulation” method – Multiple method approach that aims to compensate the weaknesses of each method

* What methods do you want to use for your own media analysis research?

Seminar activities Part I – Choose one of the media coverage studies to discuss in groups:

A) What were their core findings?

B) What was the methodology applied, and could it have been done differently?

Part II – Think about your own research and prepare its methodology following the questions:

A) How would you go about conducting a similar CA or DA study?

B) What would your research questions and hypotheses be?

C) How would you go about the coding process? Create a brief coding list.

D) What stories would you select for DA and what would you seek to find?

Seminar activity handout exercise * Use the interview and/or newspaper stories as examples to help you answer the seminar questions:

Can you identify particular themes of the coverage?

What ideas, concepts, words (repetition, slang, metaphors, use of verb and noun)?

Is it negative or positive?

Can you apply Fairclough’s model of DA here?

* Tips on writing it all up:

Create a logical narrative that brings everything together. Identify core findings and patterns, then pick out important aspects of your findings to interpret to the reader in more detail.