Measurement of Public Relations Effects

James E. GrunigJames E. Grunig

Professor EmeritusProfessor Emeritus

Department of CommunicationDepartment of Communication

University of MarylandUniversity of Maryland

Landmarks in Public Relations Metrics 1952: Cutlip & Center discuss the 1952: Cutlip & Center discuss the

importance of both research and evaluation.importance of both research and evaluation. 1960s: Communication researchers begin to 1960s: Communication researchers begin to

evaluate the effects of communication evaluate the effects of communication campaigns.campaigns.

1977: AT&T measurement project 1977: AT&T measurement project culminates in first-ever conference on PR culminates in first-ever conference on PR measurement at the University of Maryland.measurement at the University of Maryland.

Landmarks in Public Relations Metrics 1990: Broom & Dozier publish 1990: Broom & Dozier publish Using Using

Research in Public Relations.Research in Public Relations. Describes the Describes the “scientific management of public relations.”“scientific management of public relations.”

1992: First book from IABC Excellence 1992: First book from IABC Excellence project published. Explains the value of project published. Explains the value of public relations to an organization.public relations to an organization.

Landmarks in Public Relations Metrics 1996: Summit meeting on PR effectiveness 1996: Summit meeting on PR effectiveness

held in New York, sponsored by the U. S. held in New York, sponsored by the U. S. Institute for Public Relations, Institute for Public Relations, Inside PRInside PR, , and Ketchum Public Relations.and Ketchum Public Relations.

1999: Institute for Public Relations forms 1999: Institute for Public Relations forms Commission on PR Measurement and Commission on PR Measurement and Evaluation.Evaluation.

Some Initial Caveats

““Research” is a more appropriate term than Research” is a more appropriate term than “measurement” or “metrics.“measurement” or “metrics.

Research includes conceptualization as well Research includes conceptualization as well as measurement.as measurement.

The lack of conceptualization in public The lack of conceptualization in public relations is a greater problem than the lack relations is a greater problem than the lack of measurement.of measurement.

Types of Public Relations Research Research Research ININ public relations. public relations.

Used by practitioners in their work.Used by practitioners in their work. Research Research ON ON public relations.public relations.

Constructive, critical research by academic Constructive, critical research by academic scholars on the practice of public relations.scholars on the practice of public relations.

Research Research FOR FOR public relations.public relations.

Theoretical research on how to conduct the Theoretical research on how to conduct the practice of public relations.practice of public relations.

The Nature of Conceptualization

The process of thinking logically about The process of thinking logically about concepts, definitions, measures, and the concepts, definitions, measures, and the relationships among them.relationships among them.

Research is a problem-solving process. Research is a problem-solving process. The presence or absence of a dependent The presence or absence of a dependent

variable defines a problem.variable defines a problem. Independent variables affect dependent Independent variables affect dependent

variables; they can be changed to solve a variables; they can be changed to solve a problem.problem.

Levels of Analysis for Research IN Public Relations

Planning and evaluation of communication Planning and evaluation of communication programsprograms..

Auditing the quality of the public relations Auditing the quality of the public relations functionfunction..

Showing the value of public relations to the Showing the value of public relations to the organizationorganization..

Auditing the contribution of public relations Auditing the contribution of public relations to to societysociety..

Segments of the Public Relations Programming Process

Formative research to identify publics with Formative research to identify publics with whom the organization needs a relationship.whom the organization needs a relationship.

Process research to monitor communication/ Process research to monitor communication/ relationship cultivation strategies.relationship cultivation strategies.

Evaluation research to measure the effects of Evaluation research to measure the effects of communication programs and the quality of communication programs and the quality of relationships and organizational reputation.relationships and organizational reputation.

Research at the Program Level

Individual communication programs Individual communication programs such as media relations, community such as media relations, community relations, or customer relations are relations, or customer relations are successful when they affect the successful when they affect the awareness, cognitions, attitudes, and awareness, cognitions, attitudes, and behaviors of both publics and members behaviors of both publics and members of the organization. of the organization.

Formative Research for Programs Observations.Observations. Advisory groups.Advisory groups. Interviews.Interviews. Focus groups.Focus groups. Questionnaires and survey research.Questionnaires and survey research. Content analysis of media.Content analysis of media. Cyber analysis.Cyber analysis. Naturally occurring information.Naturally occurring information. Data bases.Data bases.

Process Objectives for Evaluation of Programs Research Research forfor public relations has identified public relations has identified

cultivation strategies that improve the cultivation strategies that improve the quality of relationships with publics.quality of relationships with publics.

Examples are:Examples are: Disclosure by publics of concerns.Disclosure by publics of concerns. Complaints or inquiries by publics.Complaints or inquiries by publics. Disclosure by management to publics.Disclosure by management to publics.

Outcome Objectives for Evaluation of ProgramsOne-WayOne-Way Communication.Communication. Message retention.Message retention. Cognition.Cognition. Attitude.Attitude. Behavior.Behavior.

Two-WayTwo-Way Disclosure.Disclosure. Accuracy.Accuracy. Understanding.Understanding. Agreement.Agreement. Symbiotic Symbiotic

behavior.behavior.

Research Methods for Evaluation QuantitativeQuantitative

Surveys.Surveys.

Experiments.Experiments. QualitativeQualitative

Observations.Observations.

Interviews.Interviews.

Focus Groups.Focus Groups.

Methods of Limited or No Value Media analysis (except for monitoring Media analysis (except for monitoring

media relations).media relations). Advertising equivalencies.Advertising equivalencies. General surveys of attitudes, images, or General surveys of attitudes, images, or

reputation.reputation.

Research at the Organizational Level

• Over the long-term, successful short-term communication activities and programs contribute to the building of quality, long-term relationships with strategic publics.

• Relationships add value by reducing costs, reducing risks, and increasing revenue.

• The organizational value of public relations can be determined by measuring the type and quality of relationships.

Formative Research at the Organizational

Level: Environmental Scanning Monitoring of management decisions for Monitoring of management decisions for

implications on stakeholders. implications on stakeholders. Segmentation of stakeholders and publics.Segmentation of stakeholders and publics. Qualitative observations of activists, advisory Qualitative observations of activists, advisory

groups, contacts. groups, contacts. Interviews with organizational boundary Interviews with organizational boundary

spanners. spanners. Cyber scanning.Cyber scanning. Electronic databases.Electronic databases. Monitoring of media and political processes. Monitoring of media and political processes.

Relationships Can Be Measured to Evaluate Public Relations

The newest trend The newest trend

in public relations researchin public relations research

Types of Relationships

ExchangeExchangeOne party gives benefits to the other only One party gives benefits to the other only because the other has provided benefits in because the other has provided benefits in the past or is expected to do so in the future.the past or is expected to do so in the future.

CommunalCommunalBoth parties provide benefits to the other Both parties provide benefits to the other because they are concerned for the welfare because they are concerned for the welfare of the other—even when they get nothing in of the other—even when they get nothing in return. return.

Relationship Outcomes

TrustTrustOne party’s level of confidence in and One party’s level of confidence in and willingness to open oneself to the other willingness to open oneself to the other party.party.

Control mutualityControl mutualityThe degree to which parties agree on who The degree to which parties agree on who has rightful power to influence one another.has rightful power to influence one another.

Relationship Outcomes

CommitmentCommitment

The extent to whichThe extent to which each party believes each party believes and feels that the relationship is worth and feels that the relationship is worth spending energy to maintain and promote. spending energy to maintain and promote.

SatisfactionSatisfaction

The extent to whichThe extent to which each party feels each party feels favorably toward the other because favorably toward the other because positive expectations about the positive expectations about the relationship are reinforced. relationship are reinforced.

Three Examples of Relationships

Community relations at a U. S. Community relations at a U. S. Department of Energy laboratory.Department of Energy laboratory.

Media relations.Media relations. Employee relationsEmployee relations..

Example: Indicators of Control Mutuality

1. This organization and people like me are attentive to what each other say.2. This organization believes the opinions of people like me are legitimate. 3. In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)4. This organization really listens to what people like me have to say. 5. The management of this organization gives people like me enough say in the decision-making process.

Indicators for GE

EXCHGECMNALGESATISGECOMMGECMGETRUSTGE

Me

an

6.0

5.8

5.6

5.4

5.2

5.0

4.8

4.6

4.4

Trust Indicators by Organization

TRUSTARCTRUSTNRATRUSTMSTRUSTSSTRUSTGE

Me

an

6.5

6.0

5.5

5.0

4.5

4.0

3.5

Qualitative Research on Relationships

Begin with “grand-tour” questions:Begin with “grand-tour” questions:1. “Do you feel that you have a relationship with 1. “Do you feel that you have a relationship with (organization)(public)? Why or why not?(organization)(public)? Why or why not?2. “Please describe your relationship with 2. “Please describe your relationship with (organization)(public).(organization)(public).

Analyze using the dimensions of Analyze using the dimensions of relationship or new characteristics that relationship or new characteristics that emerge.emerge.

Probe for dimensions of relationships.Probe for dimensions of relationships.

Trust Would you describe any things that (organization) Would you describe any things that (organization)

(public) has done to treat (organization)(public) fairly (public) has done to treat (organization)(public) fairly and justly, or unfairly and unjustly? (and justly, or unfairly and unjustly? (integrityintegrity) ) 

Would you describe things that (organization)(public) Would you describe things that (organization)(public) has done that indicate it can be relied on to keep its has done that indicate it can be relied on to keep its promises, or that it does not keep its promises? promises, or that it does not keep its promises? ((dependability)dependability)

How confident are you that (organization)(public) has How confident are you that (organization)(public) has the ability to accomplish what it says it will do? Can the ability to accomplish what it says it will do? Can you give me examples of why you feel that way? you give me examples of why you feel that way? ((competence)competence)

Relationships and Reputation

The concept of reputation has value when The concept of reputation has value when used in conjunction with relationships.used in conjunction with relationships.

Reputation is a byproduct of organizational Reputation is a byproduct of organizational performance, as evaluated by stakeholders, performance, as evaluated by stakeholders, and of relationships with stakeholders.and of relationships with stakeholders.

Open-End Questions Measure Reputations Open-End Questions Measure Reputations Best (“In a sentence or two, please tell me Best (“In a sentence or two, please tell me what comes to mind when you think of what comes to mind when you think of [organization].”)[organization].”)

Functional Level

The public relations function as a whole can The public relations function as a whole can be audited by comparing the structure and be audited by comparing the structure and processes of the department or departments processes of the department or departments that implement the function with the best that implement the function with the best practices of the public relations function in practices of the public relations function in other organizations or with theoretical other organizations or with theoretical principles derived from scholarly research. principles derived from scholarly research. Evaluation at this level can be called Evaluation at this level can be called theoretical or practical benchmarking.theoretical or practical benchmarking.

The IABC Excellence Study Provides A Theoretical Benchmark

Excellent public relations is:Excellent public relations is: Managerial.Managerial. Strategic.Strategic. Integrated but not sublimated to other Integrated but not sublimated to other

management functions.management functions. Symmetrical.Symmetrical. Diverse.Diverse. Ethical.Ethical. Global.Global.

Societal Level

Organizations have an impact beyond their own bottom line. They also affect other individuals, publics, and organizations in society. As a result, the contributions of public relations to society can be audited by observing and measuring the ethics and social responsibility of organizations.

Public Relations and Ethics

Suggested by research in Slovenia.Suggested by research in Slovenia. Most research has been on personal ethics.Most research has been on personal ethics. A theory of organizational ethics is needed.A theory of organizational ethics is needed. Public relations can serve as the ethics Public relations can serve as the ethics

officer of an organization.officer of an organization.

Auditing Ethics

TeleologyTeleology

What consequences do decisions have on What consequences do decisions have on publics?publics?

DeontologyDeontology

The moral obligation to communicate with The moral obligation to communicate with and disclose our behaviors to publics when and disclose our behaviors to publics when an organization has consequences on them.an organization has consequences on them.