posc 431 – public policy analysis introduction to public policy analysis

POSC 431 – Public Policy Analysis

Introduction to Public Policy Analysis

Introduction

Policy analysis is a social and political activity.– The subject matter concerns the lives and well-being of

large numbers of our fellow citizens.

– The process and results of policy analysis usually involve other professionals and interested parties.

• Done in teams and office-wide settings.

• Immediate consumer is a client, who may be a hierarchical superior.

• Ultimate audience includes diverse subgroups of politically attuned supporters and opponents.

Introduction Policy analysis: More art than science.

– Draws on intuition as much as method. Eightfold path.

– Steps.• Define the problem.• Assemble some evidence.• Construct the alternatives.• Select the criteria.• Project the outcomes.• Confront the tradeoffs.• Decide!• Tell your story.

Introduction

Eightfold path (contd.)– Steps not necessarily taken in precisely this

order, and not all may be significant for each problem. But serves as a starting point.

Iteration is continual.– The problem-solving process – being the

product of trial and error – is iterative, so that you usually must repeat each of these steps, sometimes more than once.

Introduction

Iteration is continual (contd.)– The spirit in which you take any of these steps,

especially the earliest phases of your project, should be highly tentative.

Some of the guidelines are practical, but most are conceptual.– Most concepts are obvious, but some are technical and

some are common terms used in special ways

– All concepts become intelligible through experience and practice.

Introduction

The concepts come embedded in concrete particulars.– In real life, policy problems appear as a

confusing welter of details.– Concepts are formulated in the abstract.– Analyst must learn to see the analytic concepts

in the concrete manifestations.

Introduction

Your final product.– Coherent narrative style.– Steps.

• Describe problem.• Lay out alternatives.• Each course of action has projected outcomes,

supported by evidence.• If no alternative dominates, discuss nature and

magnitude of trade-offs.• State recommendation.

The Eightfold Path

Define the Problem.– Think of deficits and excesses.

– The definition should be evaluative.

– Quantify if possible.

– Conditions that cause problems are also problems.

– Missing an opportunity is a problem.

– Common pitfalls in problem definition.• Defining the solution in the problem.

• Be skeptical about the causal claims implicit in diagnostic problem definitions.

The Eightfold Path Assemble Some Evidence.

– Think before you collect.• The value of evidence.• Self-control.

– Do a literature review.– Survey “best practice.”– Use analogies.– Start early.– Touching base, gaining credibility, brokering

consensus.– Freeing the captive mind.

The Eightfold Path

Construct the alternatives.– Start comprehensive, end up focused.– Model the system in which the problem is

located.– Reduce and simply the list of alternatives.– Design problems.– A linguistic pitfall.

The Eightfold Path

Select the criteria.– Apply evaluative criteria to judging outcomes,

not alternatives.– Criteria selection builds on problem definitions

– and continues.– Evaluative criteria commonly used in policy

analysis• Efficiency.• Equality, equity, fairness, justice.• Freedom, community, and other ideas.

The Eightfold Path

Select the criteria (contd.)– Weight conflicting evaluative criteria.

• The political process takes care of it.

• The analyst imposes a solution.

– Practical criteria.• Legality.

• Political acceptability.

• Robustness and improvability.

– Criteria in optimization models.• Linear programming.

• Improving linguistic clarity.

The Eightfold Path

Project the Outcomes.– Projection = Model + Evidence.– Attach magnitude estimates.– Break-even estimates.– The optimism problem.

• Scenario writing.

• The other guy’s shoes heuristic.

• Undesirable side effects.

• The ethical costs of optimism.

The Eightfold Path

Project the outcomes (contd.)– The outcomes matrix.– Linguistic pitfalls.

Confront the trade-offs.– Commensurability

• Break-even analysis revisited.

– Without projecting outcomes, there is nothing to trade-off.

– Simplify the comparison process.

The Eightfold Path Decide!

– The twenty-dollar-bill test. Tell your story.

– The New York taxi driver test.– You, your client, and your audiences– What medium to use?– Your story should have a logical narrative flow.– Some common pitfalls.

• Following the eightfold path.• Compulsive qualifying.• Showing your work.• Listing without explaining.• Style.

The Eightfold Path

Tell your story (contd.)– Report format.

• Table format.

• References and sources.

– Memo format.– The sound bite and the press release.

Problem-Centered Policy Analysis


Introduction to the Public Policy Process

Policy Analysis in the Policy-Making Process Policy analysis is the activity of creating

knowledge of and in the policy-making process. In creating knowledge of policy-making processes

policy analysts investigate the causes, consequences, and performance of public policies and programs.

Such knowledge remains incomplete, however, unless it is made available to policymakers and the public they are obligated to serve.

The Process of Policy Inquiry

This course provides a methodology for policy analysis.– Methodology is a system of standards, rules and

procedures for creating, critically assessing, and communicating policy-relevant knowledge.

Problem solving is a key element of the methodology of policy analysis

Policy analysis is also a methodology for formulating problems as a part of a search for solutions.

The Methodology of Policy Analysis The methodology of policy analysis draws

from and integrates elements of multiple disciplines– Political science– Sociology– Psychology– Economics– Philosophy

The Methodology of Policy Analysis (contd.) Policy analysis is partly descriptive, seeking

knowledge about causes and consequences of public policies.

But, policy analysis is also normative, creating and critiquing knowledge claims about the value of public policies for past, present,and future generations.

The Methodology of Policy Analysis (contd.) Policy-relevant knowledge involves choosing

among competing values such as health, wealth, security, peace, justice, equality, and freedom.– To choose among competing values is not a technical

judgment, but a moral judgment. Thus, policy analysis is a form of applied ethics.

Policy analysis also seeks to create knowledge that improves the efficiency of choices among alternative policies.

The Methodology of Policy Analysis (contd.) The methodology of policy analysis aims at

creating, critically assessing, and communicating policy-relevant knowledge.

The Methodology of Policy Analysis (contd.) Knowledge in this context refers to

plausibly true beliefs, as distinguished from beliefs that are certainly true, or even true with a particular statistical probability– Statistical evidence performs only a

supplementary or reinforcing role.

The Methodology of Policy Analysis (contd.) In short, policy analysis has developed a

core of moderately coherent underlying theories, a variety of methods that enjoy reasonably broad assent among practitioners, a tradition of criticism directed at political, ideological, and ethical issues raised by policy analysis, and systematic as well as anecdotal evidence of having improved the capacities of clients to solve problems.

The Methodology of Policy Analysis (contd.) A key feature of research and analysis on

social problems over the past 40 or more years is the growing recognition of complexity.– The methodological core of policy analysis can

be broadly characterized as a form of critical multiplism.

The Methodology of Policy Analysis (contd.) For critical multiplism, inductive

plausibility, not certainty, is the defining characteristic of knowledge and a major standard of success in policy inquiry.– The analyst does not enumerate supporting or

confirming cases but identifies, evaluates, and eliminates competing explanations.

The Methodology of Policy Analysis (contd.) The other major standard is policy

relevance.– Knowledge that assists in formulating and

solving problems, as these problems are experienced by policymakers and citizens on whom the policies have an impact, including the disenfranchised.

The Methodology of Policy Analysis (contd.) General guidelines for policy inquiry under

critical multiplism.– Multiple operationalism.

• The use of multiple measures of policy constructs.

– Multimethod research.• The use of multiple methods to observe policy processes and

outcomes.

– Multiple analytic synthesis.• The synthesis and critical assessment of available analyses of

similar policies and programs.

– Multivariate analysis.• The inclusion of multiple variables in policy models.

The Methodology of Policy Analysis (contd.) General guidelines for policy inquiry under

critical multiplism (contd.)– Multiple stakeholder analysis.

• The investigation of interpretive frameworks and perspectives of multiple policy stakeholders.

– Multiple perspective analysis.• The incorporation into policy analysis of multiple perspectives

– ethical, political, organizational, economic, social, cultural, psychological, and technical.

– Multimedia communications.• The use by policy analysts of multiple communications media

is essential for ensuring that knowledge is policy relevant.

The Methodology of Policy Analysis (contd.) Given typical time and financial constraints,

it is usually impossible to observe all of these guidelines.

In addition, depending on the stage of the policy process being analyzed, several guidelines may not be relevant.

Multiple methods makes analysts less likely to commit errors resulting from limited perspectives.

Policy-Relevant Information

The methodology of policy analysis designed to answer five questions:– What is the nature of the problem?

• Policy problems.

– What present and past policies have been established to address the problem, and what are their outcomes?

• Policy outcomes.

– How valuable are these outcomes in solving the problem?

• Policy performance.

Policy-Relevant Information

The methodology of policy analysis designed to answer five questions:– What policy alternatives are available to

address the problem, and what are their likely future outcomes?

• Policy futures.

– What alternatives should be acted on to solve the problems?

• Policy actions.

Five types of policy relevant information.

POLICYPERFORMANCE

POLICYPROBLEMS

POLICYACTIONS

POLICYOUTCOMES

POLICYFUTURES

Policy-Analytic Procedures Methodology of policy analysis incorporates

standards, rules, and procedures.– The standards and rules govern the selection of

procedures and the interpretation of results. The five procedures common to all human

problem solving:– Definition– Prediction– Prescription– Description– Evaluation

Policy-Analytic Procedures

Five procedures as applied to policy analysis:– Problem structuring (definition)

• Yields information about the conditions giving rise to a policy problem.

– Forecasting (prediction)• Supplies information about future consequences of acting on

policy alternatives, including doing nothing.

– Recommendation (prescription)• Provides information about the relative value or worth of these

future consequences in solving or alleviating the problem.


– Monitoring (description)• Yields information about the present or past

consequences of acting on policy alternatives.

– Evaluation (evaluation)• Provides information about the value or worth of

these consequences in solving or alleviating the problem.


The five procedures serve as the means of organizing particular methods and techniques of analysis.– Methods are general procedures for producing and

transforming policy relevant information.

– Each of the methods is supported by several techniques.

The complete framework for problem-centered policy analysis combines policy-relevant information transformed by policy analytic procedures.

Five policy-analytic procedures

Monitoring Recommendation

Evaluation Forecasting

ProblemStructuring

ProblemStructuring

Pro

blem

Stru

ctur

ing

Problem

Structuring

Problem-centered policy analysis

POLICYPERFORMANCE

POLICYPROBLEMS

POLICYACTIONS

POLICYOUTCOMES

POLICYFUTURES

Monitoring Recommendation

Evaluation Forecasting

ProblemStructuring

ProblemStructuring

Pro

blem

Stru

ctur

ing

Problem

Structuring

Phases of the policy-making process

PHASE CHARACTERISTICS ILLUSTRATION

AGENDA SETTING

Elected and appointed officials place problems on the public agenda. Many problems are not acted on at all, while others are addressed only after long delays.

A state legislator and her cosponsor prepare a bill that goes to the Health and Welfare Committee for study and approval. The bill stays in committee and is not voted on.

POLICY FORMULATION

Officials formulate alternative policies to deal with a problem. Alternative policies assume the form of executive orders, court decisions, and legislative acts.

A state court considers the use of standardized achievement tests such as the SAT on grounds that the tests are biased against women and minorities.

POLICY ADOPTIONA policy alternative is adopted with the support of a legislative majority, consensus among agency directors, or a court decision.

In Roe v. Wade Supreme Court justices reach a majority decision that women have the right to terminate pregnancies through abortion.

POLICY IMPLEMENTATION

An adopted policy is carried out by administrative units which mobilize financial and human resources to comply with the policy.

The city treasurer hires additional staff to ensure compliance with a new law which imposes taxes on hospitals that no longer have tax-exempt status.

POLICY ASSESSMENT

Auditing and accounting units in government determine whether executive agencies, legislatures, and courts are in compliance with statutory requirements of a policy and achieving its objectives.

The General Accounting Office monitors social welfare programs such as Temporary Assistance to Needy Families (TANF) to determine the scope of welfare fraud.

Policy analytic procedures and phases of policy-making

POLICY ASSESSMENT

AGENDA SETTING

POLICY FORMULATION

POLICY ADOPTION

POLICY IMPLEMENTATION

ProblemStructuring

Forecasting

Recommendation

Monitoring

Evaluation

Problem Structuring

Problem structuring can supply policy-relevant knowledge that challenges the assumptions underlying the definition of problems reaching the policy-making process through agenda-setting.

Problem structuring can assist in discovering hidden assumptions, diagnosing causes, mapping possible objectives, synthesizing conflicting views, and designing new policy options.

Forecasting Forecasting can provide policy-relevant knowledge

about future states of affairs which are likely to occur as a consequence of adopting alternatives, including doing nothing, that are under consideration at the phase of policy formulation.

Forecasting can examine plausible, potential, and normatively valued futures, estimate the consequences of existing and proposed policies, specify probable future constraints on the achievement of objectives, and estimate the political feasibility (support and opposition) of different options.

Recommendation

Recommendation yields policy-relevant knowledge about the benefits and costs of alternatives the future consequences of which have been estimated through forecasting, thus aiding policymakers in the policy adoption phase.

Recommendation helps estimate levels of risk and uncertainty, identify externalities and spillovers, specify criteria for making choices, and assign administrative responsibility for implementing policies.

Monitoring

Monitoring provides policy-relevant knowledge about the consequences of previously adopted policies, thus assisting policy-makers in the policy implementation phase.

Monitoring helps assess the degree of compliance, discover unintended consequences of policies and programs, identify implementational obstacles and constraints, and locate sources of responsibility for departures from policies.

Evaluation Evaluation yields policy-relevant knowledge

about discrepancies between expected and actual policy performance, thus assisting policymakers in the policy assessment phase of the policy-making process. Evaluation not only results in conclusions about the extent to which problems have been alleviated; it also may contribute to the clarification and critique of values driving a policy, aid in the adjustment or reformulation of policies, and establish a basis for restructuring problems.

POSC 431 Public Policy Analysis

The Functions of Policy Argument

Introduction

Policy argumentation is central to policy analysis and the policy-making process.

The analysis and evaluation of policy argumentation are central to the process of critical thinking.

The Structure of Policy Arguments

A policy argument is the product of argumentation, which is the process.

In real-life policy settings, arguments are complex and prone to misunderstanding.

To minimize misunderstanding, we use the structural model of argument developed by Stephen Toulmin, which is designed to investigate structures and processes of practical reasoning.


The conclusions of practical arguments are always uncertain, as are the reasons and evidence that lead to these conclusions.


Types of knowledge claims.– A knowledge claim is the conclusion of a

policy argument.– Three types of knowledge claims:

• Designative: questions of fact. What are the observed outcomes of a policy and why did they occur?

• Evaluative: questions of value. Was the policy worthwhile?

• Advocative: questions of right action. Which policy should be adopted?


Types of knowledge claims (contd.).– Policy arguments contain six elements:

• Information (I), Claim (C), Warrant (W), Backing (B), Rebuttal (R), Qualifier (Q).

– The first four of these elements are present in every policy argument.

– The claim C is the conclusion or output of an argument, which is supported by policy-relevant information I, which is the beginning or input of the argument.

– The warrant W is the justification, or reason, for concluding C from I.

– The qualifier Q indicates that C has a given truth or plausibility.


Example:– The Senator supports the privatization of the

federal highway system, which have significant gains in efficiency and a reduction in taxes. Considering that the privatization of public services has been successful in other areas, this is definitely a “no brainer”. Besides this is the same conclusion as a panel of experts on privatization.


The underdetermination of conclusions by information.– Policy-relevant information does not fully determine

the conclusions of policy arguments. “Information does not speak for itself.”

– Identical information can and often does lead to different conclusions, which we call policy claims to emphasize the fallible and indeterminate character of arguments.


Example: Policy-relevant information from the Coleman Report “Black students attending primarily black schools had lower achievement test scores than black students attending primarily white schools.”– Designative claim and qualifier: “Since schools in large

urban areas are primarily black, the hopes of blacks for higher education achievement [simply] cannot be realized.”

– Evaluative claim and qualifier: “The Coleman Report is [clearly] a racist document based on ethnically biased achievement tests.”

– Advocative claim and qualifier: “[There is no question] that a national policy of compulsory school busing ought to be adopted to achieve integrated schools.”


Warrants and rebuttals.– Although each of the claims about the Coleman

report begins with the same information, very different conclusions are drawn.

– Differences are due not to the information, but to the role of the warrants in justifying (plausibility or implausibility) the claims on the basis of the information supplied.

Modes of Policy Argumentation Modes of policy argumentation are the characteristic

routes followed by information as it is transformed into policy claims.

The several different modes of argument involve reasoning from authority, method, generalization, classification, intuition, cause, sign, motivation, analogy, parallel case, and ethics.

Each of the eleven modes of argument has a different type of warrant, and multiple modes can be found in any policy argument.

The warrants are the reasons offered by the proponent or opponent of a policy to justify a claim, or inference, based on the information supplied.

Modes of Policy Argumentation

Authority– Reasoning from authority is based on warrants

having to do with the achieved or as cribed statuses of producers of policy-relevant information. For example, experts, insiders, scientists, specialists, gurus, power brokers. Footnotes and references are disguised authoritative arguments.

Argumentation from Authority


Method– Reasoning from method is based on warrants

about the approved status of methods or techniques used to produce information. The focus is on the achieved or ascribed status or "power" of procedures, rather than persons. Examples include approved statistical, econometric, qualitative, ethnographic, and hermeneutic methods and techniques.

Argumentation from Method


Generalization– Reasoning from generalization is based on similarities

between samples and populations from which samples are selected. Although samples can be random, generalizations can also be based on qualitative comparisons. The assumption is that what is true of members of a sample will also be true of members of the population not included in the sample. For example, random samples of n 30 are taken to be representative of the (unobserved and often unobservable) population of elements from which the sample is drawn.

Argumentation from Generalization


Classification– Reasoning from classification has to do with

membership in a defined class. The reason ing is that what is true of the class of persons or events described in the warrant is also true of individuals or groups which are members of the class described in the information. An example is the untenable ideological argument that because a country has a socialist economy it must be undemocratic, because all socialist systems are undemocratic.

Argumentation from Classification


Cause– Reasoning from cause is about the activity of

generative powers ("causes") and their con sequences ("effects"). For example, a claim may be made based on general propositions, or laws, of economics that state invariant relations between cause and effect. Other causal claims are based on observing conditions that must be satisfied to infer that a policy has a specified effect. Most argumentation in the social and natural sciences is based on reasoning from cause.

Argumentation from Cause


Sign– Reasoning from sign is based on signs, or indicators,

and their referents. The presence of a sign indicates the presence of an event or condition, because the sign and what it refers to occur together. Examples are indicators of institutional performance such as "organizational report cards" and "benchmarks," or indicators of economic performance such as "leading economic indicators." Signs are not causes, because causality must satisfy additional requirements not expected of signs.

Argumentation from Sign


Motivation – Reasoning from motivation is based on the

motivating power of goals, values, and inten tions in shaping individual and collective behavior. For example, a claim that citizens will support the strict enforcement of pollution standards might be based on reasoning that, since citizens are motivated by the desire to achieve the goal of clean air and water, they will act to offer their support.

Argumentation from Motivation


Intuition – Reasoning from intuition is based on the

conscious or preconscious cognitive, emotional, or spiritual states of producers of policy-relevant information. For example, the awareness that an advisor has some special insight, feeling, or "tacit knowledge" may serve as a rea son to accept his judgment.

Argumentation from Intuition


Analogy-Metaphor– Reasoning from analogies and metaphors is based on

similarities between relations found in a given case and relations characteristic of a metaphor, analogy, or allegory. For example, the claim that government should "quarantine" a country by interdicting illegal drugs—with the illegal drugs seen as an "infectious disease"—is based on reasoning that, since quarantine has been effective in cases of infectious diseases, interdiction will be ef fective in the case of illegal drugs.

Argumentation from Analogy

Modes of Policy Argument

Parallel Case– Reasoning from parallel case is based on

similarities among two or more cases of policy making. For example, a reason that a local government should strictly enforce pollution standards is that a parallel policy was successfully implemented in a similar local government elsewhere.

Argumentation from Parallel Case

Modes of Policy Argument

Ethics– Reasoning from ethics is based on the rightness or

wrongness, goodness or badness, of policies or their consequences. For example, policy claims are frequently based on moral principles stating the conditions of a "just" or "good" society, or on ethical norms prohibiting lying in public life. Moral principles and ethical norms go beyond the values and norms of particular individuals or groups. In public policy, many arguments about economic benefits and costs involve unstated or implicit moral and ethical reasoning.

Ethical Argumentation

Systems of Argumentation Completeness. Elements of an argument should

comprise a genuine whole that encompasses all appropriate considerations. For example, the plausibility of arguments about the effects of a policy depends on whether such arguments encompass a full range of plausible rival explanations similar in form and content to classes of rival hypotheses (threats to validity) developed in the tradition of quasi-experimentation.

Systems of Argumentation

Consonance. Elements of an argument should be internally consistent and compatible. For example, ethical arguments concerning the justice or fairness of a policy are plausible to the degree that they incorporate a system of internally and externally consistent ethical hypotheses.


Cohesiveness. Elements of an argument should be operationally connected. For example, the plausibility of an ethical argument depends on whether responses to several levels of descriptive and valuative questions – levels ranging from verification and validation to vindication – are operationally linked.


Functional regularity. Elements of an argument should conform to an expected pattern. For example, statistical arguments that offer estimates of parameters of unobserved (and often unobservable) populations are plausible to the degree that patterns in the sample and the population from which it is drawn are functionally regular or uniform, not irregular, based on sample data and background knowledge.

Class Example: Excerpt from Bush Speech on Iraq It is clear that we need to change our strategy in Iraq. So

my national security team, military commanders, and diplomats conducted a comprehensive review. We consulted Members of Congress from both parties, allies abroad, and distinguished outside experts. We benefited from the thoughtful recommendations of the Iraq Study Group a bipartisan panel led by former Secretary of State James Baker and former Congressman Lee Hamilton. In our discussions, we all agreed that there is no magic formula for success in Iraq. And one message came through loud and clear: Failure in Iraq would be a disaster for the United States.

Class Example: Excerpt from Bush Speech on Iraq The consequences of failure are clear: Radical Islamic

extremists would grow in strength and gain new recruits. They would be in a better position to topple moderate governments, create chaos in the region, and use oil revenues to fund their ambitions. Iran would be emboldened in its pursuit of nuclear weapons. Our enemies would have a safe haven from which to plan and launch attacks on the American people. On September the 11th, 2001, we saw what a refuge for extremists on the other side of the world could bring to the streets of our own cities. For the safety of our people, America must succeed in Iraq.

Class Example: Excerpt from Bush Speech on Iraq Many are concerned that the Iraqis are becoming too

dependent on the United States and therefore, our policy should focus on protecting Iraq's borders and hunting down al Qaeda. Their solution is to scale back America's efforts in Baghdad or announce the phased withdrawal of our combat forces. We carefully considered these proposals. And we concluded that to step back now would force a collapse of the Iraqi government, tear that country apart, and result in mass killings on an unimaginable scale. Such a scenario would result in our troops being forced to stay in Iraq even longer, and confront an enemy that is even more lethal.

POSC 431- Public Policy Analysis

Problem Structuring

Nature of Policy Problems

Problem structuring, which is a continuously recurring phase of policy inquiry in which analysts search among competing problem formulations of different stakeholders, is no doubt the most important activity performed by policy analysts.

It is so important because policy analysts seem to fail more often because they solve the wrong problem than because they get the wrong solution to the right problem.


Beyond problem solving.– Policy analysis is a dynamic, multilevel process

in which methods of problem structuring take priority over methods of problem solving (see figure).

Priority of Problem Structuring in Policy Analysis

NO

YES

YES

ProblemSensing

PROBLEMSITUATION

Problem Structuring

POLICYPROBLEM

Problem Unsolving

Problem Dissolving

RIGHT PROBLEM?

Problem Solving

POLICYSOLUTION

POLICY SOLUTION?

Problem Resolving

Nature of Policy Problems Beyond problem solving.

– Distinctions among problem-related processes.• Problem sensing versus problem structuring.• Problem structuring versus problem solving.• Problem resolving versus problem unsolving and problem

dissolving.

Characteristics of problems.– Interdependence of policy problems (policy messes).

• Analytic versus holistic approaches.

– Subjectivity of policy problems.– Artificiality of policy problems.– Dynamics of policy problems.

Nature of Policy Problems Characteristics of problems (contd.)

– In short, systems of problems (messes) cannot be decomposed into independent subsets without running the risk of producing the right solution to the wrong problem.

– The whole is greater (qualitatively different) than the sum of its parts.

– A recognition of the interdependence, subjectivity, artificiality, and dynamics of policy problems alerts us to the possible unanticipated consequences that may follow from policies based on the right solution to the wrong problem.


Problems versus issues.– If policy problems are really systems of

problems, then policy issues are equally complex.

– Policy issues reflect not only conflict over courses of actions, but over definitions of the problem.


Major Issues

Secondary Issues

Functional Issues

Minor Issues


Problems versus issues.– Major issues.

• Those encountered at highest levels of government within and between federal, state, and local jurisdictions.

• Involve questions of agency mission.

– Secondary issues.• Located at the level of agency programs at the

federal, state, and local levels.• The setting of program priorities and the definition

of target groups and beneficiaries.


Problems versus issues.– Functional issues.

• Located at both the program and project levels.

• Involve questions of budget, finance, and procurement.

– Minor issues.• Located at the level of specific projects.

• Involve questions of personnel, staffing, employee benefits, vacation times, working hours, and standard operating procedures.


Problems versus issues.– Strategic policies are policies where the

consequences are relatively irreversible.– Operational polices are policies where the

consequences are relatively reversible.


Three classes of policy problems.– Well-structured.

• Prototype: completely computerized decision problems.

– Moderately structured.• Prototype: prisoner’s dilemma.

– Ill-structured.• Prototype: most important problems.


ELEMENT

STRUCTURE OF PROBLEMWell

structuredModerately Structured Ill Structured

Decision maker(s) One or few One or few Many

Alternatives Limited Limited Unlimited

Utilities (values) Consensus Consensus Conflict

Outcomes Certainty or risk Uncertainty Unknown

Probabilities Calculable Incalculable Incalculable

Problem Structuring in Policy Analysis The requirements for solving ill-structured

problems demand that the analyst take an active role in defining the problem.

Problem Structuring in Policy Analysis Creativity in problem structuring.

– The product of the analysis is sufficiently novel that most people could not or would not have arrived at the same solution;

– The process of analysis is sufficiently unconventional that it involves the modification or rejection of previous accepted ideas;

– The process of analysis requires sufficiently high motivation and persistence that analysis takes place with high intensity or over long periods of time;

Problem Structuring in Policy Analysis Creativity in problem structuring (contd.).

– The product of analysis is regarded as valuable by analysts, policymakers, and other stakeholders, since it provides an appropriate solution to the problem; And.

– The problem initially posed is so ambiguous, value, and ill defined that part of the task is to formulate the problem itself.

Problem Structuring in Policy Analysis Phases of problem structuring.

– Problem search.– Problem definition.– Problem specification.– Problem sensing.

Problem Structuring in Policy Analysis

METAPROBLEM

FORMAL PROBLEM

PROBLEM SITUATION

SUBSTANTIVE PROBLEM

Problem Search

Problem Definition

Problem Sensing

Problem Specification

Problem Structuring in Policy Analysis Errors of the third type (EIII).

– How well do the substantive and formal problems correspond to the original problem situation?

• If most problem situations are messes, then models should reflect complexity.

– Types of errors.• Type I – rejecting the null hypothesis when it is true.

• Type II – accepting the null hypothesis when it is false.

• Type III – solving the wrong problem.

Types of Policy Models

Policy models are simplified representations of selected aspects of a problem situation constructed for specific purposes.– By definition, they are artificial constructs.

Descriptive models.– The purpose of descriptive models is to explain

and/or predict the causes and consequences of policy choices.

• Used to monitor the outcomes of policy actions and to forecast performance.

Types of Policy Models

Normative models.– Explain and predict, but also to provide rules

and recommendations for optimizing some utility or value.

– Example: compound interest.

0)1( SrS nn

Forms of Policy Models

Verbal models.– Expressed in everyday language.

– The equivalent of substantive problems.

– Limitation: the reasons for recommendations and predictions may be hidden.

Symbolic models.– Use mathematical symbols to describe relationships

among key variables believed to characterize a problem. The premises must be made explicit.

bXaY

Forms of Policy ModelsTable 1. Mean education and mean income

Birmingham, Alabama neighborhoods

Proportion with high school or higher

1.0.9.8.7.6.5.4.3.2.10.0

Adju

ste

d in

com

e 2

000

50000

40000

30000

20000

10000

0 Rsq = 0.6369

Forms of Policy Models

Procedural models.– Represent dynamic relationships among

variables believed to characterize a policy problem.

– Example:• Decision tree.

Methods of Problem Structuring Boundary analysis (used to estimate boundaries of

metaproblem).– Saturation sampling.– Elicitation of problem representations.– Boundary estimation.

Classificational analysis.– Uses logical division and logical classification.– Criteria.

• Substantive relevance.• Exhaustiveness.• Disjointness.• Consistency.• Hierarchical distinctiveness.

Methods of Problem Structuring

Hierarchy analysis.– Possible clauses, plausible causes, and

actionable causes.– Uses same rules as classificational analysis.

Synectics.– Personal analogies.– Direct analogies.– Symbolic analogies.– Fantasy analogies.

Methods of Problem Structuring Brainstorming.

– Groups should be composed of knowledgeable subjects.

– Idea generation and idea evaluation should be kept separate.

– Atmosphere should be open and permissive.– Idea-evaluating should only begin after idea-generating

has ceased.– At the end of idea-evaluating, ideas should be

prioritized and incorporated into a proposal that contains a conceptualization of the problem and its potential solutions.

Methods of Problem Structuring Multiple perspective analysis.

– Technical perspective.– Organizational perspective.– Personal perspective.

Assumptional analysis.– Stakeholder identification.– Assumption surfacing.– Assumption challenging.– Assumption pooling.– Assumption synthesis.

Methods of Problem Structuring

Argumentation mapping.– Assessing probability and plausibility of policy

argument warrants.

PPA 503 – The Public Policy-Making Process

Lecture 2a - The Context of Public Policy: Values and Environment

Historical Development of Public Policy The history of American politics and policy

is characterized by considerable change. The policy history of the United States is

characterized by “policy restraint”. The history is divided into four eras:

– A period of divided power,– An era of state activism,– An era of national activism, and– An era of national standards.

Historical Development of Public Policy Divided power (1787-1870).

– Weakness of national government under Articles of Confederation, Shays’ Rebellion.

– The Constitution placed limits on scope of federal government, but was still much stronger than the government under the Articles. Also contained structural impediments to radical policy.

• Division of powers and separation of powers.

• Policy conflicts tend to turn on jurisdictional questions as much as policy questions. (Which level of government as important as what to do).

Historical Development of Public Policy Divided Power (contd.)

– Powers granted to Congress fell into two categories: management of national responsibilities (defense) and commercial responsibilities (interstate commerce).

– Powers broadened by “necessary and proper” clause”.– In general, however, from the perspective of the average citizen,

the federal government was not the most important official participant in policy making.

– Reinforced by structure, Founder’s understanding, rural nature of the population, sparseness and homogeneity, libertarian political philosophy, absence of industrialization.

– Embodied by conflict between Hamilton and Jefferson. History supported Hamilton.

Historical Development of Public Policy State activism (1870 to 1933).

– Industrialization and attendant increases in wealth, poverty, crime, disease.

– Increased attempts at state regulation, but difficult for individual states. Interstate commerce suggested federal intervention, but federal government reluctant.

• Did pass Sherman Anti-Trust Act in 1890. Clayton Act 1914, Pure Food and Drug Act. Many trust breakups. (Now re-merging).

• Federal government supported laissez-faire economics in its court and legislative decisions (Lochner vs. New York 1905).

– Integration of former slaves into society 1865 to 1880s. Declined thereafter (Plessy vs. Ferguson 1896). Turned control of issue over to states.

Historical Development of Public Policy National activism (1933-1961).

– Demands triggered by Great Depression.– Lochner rule influenced Supreme Court actions

on New Deal until 1937.– New Deal created modern system of national

activism. Further enhanced by federal government activity during World War II.

– National Defense system in 1950s.

Historical Development of Public Policy National standards (1961 to 1981).

– Great Society.– Scientific study of public policy impelled by

federal government’s efforts to set standards for states and localities.

– Retrenchment caused by Vietnam and Watergate restrictions on President.

• Budget and Impoundment Control Act.

• War Powers Act.

Historical Development of Public Policy The end of big government? (1981-present).

– The election of Ronald Reagan.– The Reagan Revolution changed the tenor of

American politics and signaled a realignment.– These eras suggest that the American

constitution without substantial change is flexible enough to bend, but not break in the face of major policy upheavals.

Stability in American Politics and Policy Making

Table 2.1. Elements of American StabilityTable 2.1. Elements of American Stability

TypeType What this meansWhat this means Examples in actionExamples in action

Ideological stabilityIdeological stability Americans tend not to stray from Americans tend not to stray from a set of ideological precepts a set of ideological precepts based largely on our national based largely on our national experience.experience.

No labor party because of some No labor party because of some suspicion of class warfare.suspicion of class warfare.

Political stabilityPolitical stability Politics in the U.S. tends to be Politics in the U.S. tends to be fairly stable for extended periods.fairly stable for extended periods.

Our constitutional structure has Our constitutional structure has changed little. 27 amendments.changed little. 27 amendments.

Policy stabilityPolicy stability Policies tend to change very little Policies tend to change very little over time.over time.

The gradual evolution of Social The gradual evolution of Social Security over sixty years.Security over sixty years.

Stability in powerStability in power Changes in power do not cause Changes in power do not cause major policy, political, or social major policy, political, or social upheavals.upheavals.

The transition from one President The transition from one President or Congress to another is or Congress to another is generally very smoothgenerally very smooth

Source: Derived from James A. Anderson, Public Policymaking, 4Source: Derived from James A. Anderson, Public Policymaking, 4 thth ed. ed.

Stability in American Politics and Policy Making Ideological stability means that Americans and their

representatives have not been quick to shift their basic political beliefs. Since the founding and even before, Americans have believed in personal liberty and equality (except for those outside the mainstream), a generally limited government, “popular sovereignty”, the rule of law, and respect for market economics, private property, and free enterprise.– Although attitudes have changed on a multitude of issues including

slavery, racial discrimination, the rights of women and children, and voting rights, the core values have remained the same.

– The acceptable range for ideology is much narrower in the U.S. than in Europe.

• Reinforced by the single member district and the two-party system.– Some beliefs can be contradictory. Equality and capitalism, for

example.

Stability in American Politics and Policy Making Political stability is a key element of our overall

national stability. We have operated under the same constitutional arrangement since 1789 (oldest continuously operating system with a written constitution).– Although many more groups have been included, the

basic rules have remained the same.• Structure that promotes a two-party system, Presidential

election by electoral college, Senate based on state representation, House apportioned to population.

• Reinforced by stable beliefs.

Stability in American Politics and Policy Making Policy stability – Political stability produces

policy stability.– American government and constitutional system not

designed to be quickly responsive to national needs or desires. Checks and balances, separation of powers

• New Deal conflict, and court packing plan example.• Clinton’s health plan examples.

– The federal system also operates to delay policy change.

• Laboratories of innovation versus status quo orientation. In many senses, the constitution a

counterrevolutionary document. The Civil War may have been more revolutionary.

Policy Restraint and Barriers to Change One should not conclude that policy stability is solely the

result of the constitutional structure, or that policy change can happen. It is just very difficult.

Barriers to change.– Ideological and political stability. Shared power, high barriers to

amendment, separation of powers, checks and balances.– Basic rules and norms.

• The Senate filibuster• Seniority• Congressional procedural rules

– Open government and policy restraint.• Open public meeting laws, Administrative Procedure Act, and the

Freedom of Information Act open up the process for public scrutiny and by side effect slow down the policy process.

Policy Restraint and Barriers to Change A rationale for stability.

– Deliberation and public participation are at least as important as rapid and efficient policymaking.

– Rapid policy change can happen with the right combination of political factors, but during normal periods, minorities can block policy change and the system preserves their right to do so.

– Policy does not change rapidly because most of the public does not support such change except under rare circumstances.

Policy Restraint and Barriers to Change Fragmentation.

– Fragmentation is a double-edge sword.• On the one hand, it requires multiple serial

majorities to promote change.

• On the other hand, it offers multiple points of access to the policy process.

– Dimensions of fragmentation.• Separation of powers.

• Division of powers (federalism).

Policy Restraint and Barriers to Change

Table 2.2. The Balance of PowerTable 2.2. The Balance of Power

FunctionFunction CongressCongress PresidentPresident CourtsCourts

LegislativeLegislative Make lawsMake laws Recommend laws; veto Recommend laws; veto laws; make regulations laws; make regulations that have the force of that have the force of law.law.

Review laws to Review laws to determine legislative determine legislative intent, new intent, new interpretationsinterpretations

ExecutiveExecutive Override vetoes; Override vetoes; legislative vetoes of legislative vetoes of regulationsregulations

Enforce and implement Enforce and implement laws.laws.

Review executive acts; Review executive acts; restrain executive restrain executive actions.actions.

JudicialJudicial Impeach judges and the Impeach judges and the president, call witnesses president, call witnesses to hearings, set judicial to hearings, set judicial jurisdiction.jurisdiction.

Pardon criminals, Pardon criminals, nominate judgesnominate judges

Interpret laws.Interpret laws.

Note: The primary function of each branch is indicated in the boxes with the diagonal lines.Note: The primary function of each branch is indicated in the boxes with the diagonal lines.

Core Values and Beliefs in American System

Fundamental Beliefs – Life The individual's right to life should be

considered inviolable except in certain highly restricted and extreme circumstances, such as the use of deadly force to protect one's own or others' lives.


Fundamental Beliefs– Liberty The right to liberty is considered an unalterable aspect of the

human condition. Central to this idea of liberty is the understanding that the political or personal obligations of parents or ancestors cannot be legitimately forced on people. The right to liberty includes personal freedom: the private realm in which the individual is free to act, to think and to believe, and which the government cannot legitimately invade; political freedom: the right to participate freely in the political process, choose and remove public officials, to be governed under a rule of law; the right to a free flow of information and ideas, open debate and right of assembly; and economic freedom: the right to acquire, use, transfer and dispose of private property without unreasonable governmental interference; the right to seek employment wherever one pleases; to change employment at will; and to engage in any lawful economic activity.


Fundamental Beliefs– The Pursuit of Happiness It is the right of citizens

in the American constitutional democracy to attempt to attain--to "pursue"--happiness in their own way, so long as they do not infringe upon rights of others.

– Common Good The public or common good requires that individual citizens have the commitment and motivation--that they accept their obligation--to promote the welfare of the community and to work together with other members for the greater benefit of all.


Fundamental Beliefs

– Justice People should be treated fairly in the distribution of the benefits and burdens of society, the correction of wrongs and injuries, and in the gathering of information and making of decisions.

– Diversity Variety in culture and ethnic background, race, lifestyle, and belief is not only permissible but desirable and beneficial in a pluralist society.


Fundamental Beliefs– Truth Citizens can legitimately demand that truth-telling as

refraining from Iying and full disclosure by government be the rule, since trust in the veracity of government constitutes an essential element of the bond between governors and governed.

– Popular Sovereignty The citizenry is collectively the sovereign of the state and holds ultimate authority over public officials and their policies.

– Patriotism Virtuous citizens display a devotion to their country, including devotion to the fundamental values and principles upon which it depends.

Core Values and Beliefs in American System Constitutional principles.

– Rule of Law and Judicial Review Both government and the governed should be subject to the law.

– Separation of Powers Legislative, executive, and judicial powers should be exercised by different institutions in order to maintain the limitations placed upon them.

– Representative Government The republican form of government established under the Constitution is one in which citizens elect others to represent their interests.

Core Values and Beliefs in American System Constitutional principles.

– Checks and Balances The powers given to the different branches of government should be balanced, that is roughly equal, so that no branch can completely dominate the others. Branches of government are also given powers to check the power of other branches.

– Individual Rights Fundamental to American constitutional democracy is the belief that individuals have certain basic rights that are not created by government but which government should protect. These are the right to life, liberty, economic freedom, and the "pursuit of happiness." It is the purpose of government to protect these rights, and it may not place unfair or unreasonable restraints on their exercise. Many of these rights are enumerated in the Bill of Rights.

Core Values and Beliefs in American System Constitutional Principles

– Limited Government The powers of government may not be used to restrict fundamental freedoms including life, liberty, and property.

– Freedom of Religion There shall be full freedom of conscience for people of all faiths or none. Religious liberty is considered to be a natural inalienable right that must always be beyond the power of the state to confer or remove. Religious liberty includes the right to freely practice any religion or no religion without governmental coercion or control.

Core Values and Beliefs in American System Constitutional Principles

– Federalism Power is shared between two sets of governmental institutions, those of the states and those of the central or federal authorities, as stipulated by the Constitution.

– Civilian Control of the Military Civilian authority should control the military in order to preserve constitutional government.

Information Management in Public Policy The environment of public policy in the United

States has changed dramatically because of the advances in information technology.

In 1813, Congress made provision for the protection of state documents by requiring that copies of important legislative documents be made available to selected universities, state libraries, and historical societies.

The origin of the federal depository library system.

Information Management in Public Policy By 1945, the USGPO was doing eight

mailings per day to each of 418 depository libraries.

Beginning in 1980s and 1990s, the USGPO began the shift to microfiche.

In the 2000s, the USGPO is not the only source of information and most documentation is now electronic.

The Cold War: Origins of Information Technology in the U.S. Colossus computer – 1943. Codebreaker. ENIAC – 1942—1946. Ballistics

calculations. UNIVAC 1 – weighed 8 tons. - Sold to

Census Bureau. First commerical sale.– U.S. Navy and Atomic Energy Commission

bought the next two.– First private sector purchaser – General

Electric.

The Cold War: Origins of Information Technology in the U.S. Mainframe computing – 1970s

– IBM 370.– IBM 7090, first transistorized computer.

Personal computer – 1980s– Apple– IBM PC– Ended the use of mainframe computing by

government.

The Cold War: Origins of Information Technology in the U.S. Advanced Research Projects Agency

(ARPA) within Department of Defense.– Over 12 years, major governmental initiatives

reconstructed telecommunications infrastructure and provided foundation for the Internet.

– Data communications technology (air defense).– Communications satellite development.– ARPAnet 1969 – first cooperative computer

network.

Architecture of Federal Information Policy, 1946-1980 Administrative Procedures Act of 1946.

– Federal Register, electronic 1993. Federal Records Act of 1950.

– Preserve documentation of the organization, functions, policies, decisions, procedures, and transactions of agency.

The Freedom of Information Act of 1966.– Right to public access to government

information.

Architecture of Federal Information Policy, 1946-1980 The Technology Assessment Act of 1972.

– Office of Technology Assessment, later superseded by other technology agencies.

The Federal Advisory Committee Act of 1972.– Timely notice in Federal Register of advisory

committee meetings.

The Privacy Act of 1974.– Protects privacy of individuals identified in information

systems maintained by federal agencies.

Architecture of Federal Information Policy, 1946-1980 The Government in the Sunshine Act of 1976.

– Public entitled to fullest information on decision-making processes of federal agencies.

Presidential Records Act of 1978.– Made presidential records public rather than private.

The Paperwork Reduction Act of 1980.– Mandated an information resource management (IRM)

approach to federal data.

From BITNET to FIRSTGOV: Growth of the Internet 1981-2000 BITNET (1981) – Because It’s Time NETwork.

– Cooperative network situated at CUNY.

NSFNet (1986) – Managed with IBM, MCI, and Merit Network. 10,000 hosts by 1987.

Combination of ARPANet, BITNET, and NSFNET provided infrastructure for the Internet.

1984 – Domain Name System (DNS) and Uniform Resource Locators (URL).

From BITNET to FIRSTGOV: Growth of the Internet 1981-2000 Internet Corporation for Assigned Names and Numbers

responsible for civilian URLs, FTP, lookup files. Military responsible for .mil, Feds responsible for .gov, and foreign countries for the suffixes.

In 1992, federal government repealed acceptable use policy allowing commercialization of the Internet.

1992 – Supreme Court prohibited Internet sales taxation for companies without physical presence in state.

1993 – Federal funding of internet ends. 1995 – Concept of the “Digital Divide.” 2000 – FirstGov.gov, portal to all government information.

Policy Issues in the Information Age, 1986-2005 Public access to information. Public participation in e-government,

including public comments on federal regulations.

Accessibility for disabled. Rehabilitation Act Amendments of 1986.

Individual privacy. Matching regulated but more match since 9/11.

Policy Issues in the Information Age, 1986-2005 Modernizing education. Increased technology

infrastructure. Regulating e-vice.

– Child Online Protection Act of 1998 (unconstitutional).– Child Pornography Prevention Act of 1996

(unconstitutional).– Prosecutorial Remedies and Tools Against the

Exploitation of Children Today Act of 2003 (PROTECT).

– Child Obscenity and Pornography Prevention Act of 2003

– Many more acts.

Policy Issues in the Information Age, 1986-2005 Securing intellectual property.

– Digital Millennium Copyright Act of 1998, extended to digital media. Fair use for universities. Prohibited removal of copyright management information from digital media.

Electronic voting.– Shift from paper ballots, but many security issues.

Regulating the outsourcing of IT jobs.– Raises information security issues.

Securing E-Government, 1986 - 2005 Guaranteeing security of cyberspace key issue. 1980s mostly computer crime. 1990s, architecture for securing cyberspace

against foreign and domestic enemies. Presidential Decision Directive 63 (Protecting

America’s Critical Infrastructure). Government Information and Security Reform Act

of 2000. 9/11.

Securing E-Government, 1986 - 2005 USA Patriot Act of 2001. Cyber Security and Research Act of 2002. Amendments to Fair Credit Reporting Act

to reduce identity theft. OMB – Information Systems Council to

coordinate sharing of terrorist information. 9/11 Commission – separate, secure

network for federal information sharing.


Forecasting Policy Futures

Forecasting in Policy Analysis Forecasting is a procedure for producing factual

information about future states of society on the basis of prior information about policy problems.

Forms of forecasting.– Projection.

• A forecast based on the extrapolation of current and historical trends into the future.

– Prediction.• A forecast based on explicit theoretical assumptions.

– Conjecture.• A forecast based on informed or expert judgments about future

states of society.

Forecasting in Policy Analysis

Aims of forecasting.– Forecasts provide information about future

changes in policies and their consequences.– Forecasting permits greater control through

understanding past policies and their consequences, implying that the future is determined by the past.

– Forecasting also enables us to shape the future in an active manner, irrespective of what has happened in the past.

Forecasting in Policy Analysis Limitations of forecasting.

– Forecast accuracy.• Recent simple forecasting models have had huge errors in recent

years (as much as 50%) in econometric forecasting.

– Comparative yield.• Both simple and complex theoretical models have been no more

accurate than simple extrapolative models and informed expert judgment.

– Context.• Institutional (nonprofit more accurate than business and

government).• Temporal (long-term less accurate than short-term).• Historical (modern complexity reduces accuracy).

– “Assumption drag”.

Forecasting in Policy Analysis Types of futures.

– Potential futures.• Future societal states that may occur.

– Plausible futures.• Future states that, on the basis of assumptions about causation

in nature and society, are believed to be likely if policymakers do not intervene to redirect the course of events.

– Normative futures.• Potential and plausible futures which are consistent with an

analyst’s conception of future needs, values, and opportunities.• The specification of normative futures narrows the range of

potential and plausible futures, thus linking forecasts to specific goals and objectives.


CHARACTERISTIC GOALS OBJECTIVES

Specification of purposes

Broadly stated (. . . To upgrade the quality of health care)

Concrete (. . . Increase the number of physicians by 10 percent)

Definition of terms

Formal ( . . . The quality of health care refers to the accessibility of medical services)

Operational ( . . . The quality of health care refers to the number of physicians per 100,000 persons . . .)

Time periodUnspecified ( . . . In the future)

Specified ( . . . In the period 2006-2016)

Measurement procedure

Nonquantitative (adequate health care insurance)

Frequently quantitative ( . . . The number persons covered per 1,000 persons)

Treatment of target groupsBroadly defined ( . . . People in need of care)

Specifically defined ( . . . Families with incomes below $19,000)

Table 1. Contrast between goals and objectives


Sources of goals, objectives, and alternatives.– Authority.

– Insight.

– Method.

– Scientific theories.

– Motivation.

– Parallel case.

– Analogy.

– Ethical systems.

Approaches to Forecasting

Approaches to forecasting.– Decide what to forecast, or determine the object

of the forecast.– Decide how to make the forecast, or select one

or more bases for the forecast.– Choose techniques that are most appropriate for

the object and base selected.


Objects.– The object of a forecast is the point of reference

of a projection, prediction, or conjecture.– Objects of forecasting.

• Consequences of existing policies.

• Consequences of new policies.

• Contents of new policies.

• Behavior of policy stakeholders.


Bases.– The basis of a forecast is the set of assumptions

or data used to establish the plausibility of estimates of consequences of existing or new policies, the content of new policies, or the behavior of policy stakeholders.

Approaches to Forecasting Bases (contd.).

– Bases for forecasts.• Trend extrapolation (projection).

– The extension into the future of trends observed in the past.– Based on inductive logic.

• Theoretical assumptions (prediction).– Systematically structured and empirically testable sets of laws

or propositions that make predictions about the occurrence of one event based on another.

– Based on deductive logic.• Informed judgments (conjecture).

– Knowledge based on experience and insight, rather than inductive or deductive reasoning.

– Based on retroductive logic.

Methods and Techniques of ForecastingAPPROACH BASIS APPROPRIATE

TECHNIQUESPRODUCT

Extrapolative forecasting

Trend extrapolation Classical time-series analysis

Linear trend estimation

Exponential weighting

Data transformation

Catastrophe methodology

Projections

Theoretical forecasting Theory Theory mapping

Causal modeling

Regression analysis

Point and interval estimation

Correlational analysis

Predictions

Judgmental forecasting Informed judgment Conventional Delphi

Policy Delphi

Cross-impact analysis

Feasibility assessment

Conjectures

Methods and Techniques of Forecasting Classical time-series analysis.

– Time series are made up four components.• Secular trend.• Seasonal variation.• Cyclical fluctuations.• Irregular movements.

– Linear trend estimation.– Nonlinear time series.

• Oscillations.• Cycles.• Growth curves.• Decline curves.• Catastrophes.

Time – Series Example

Methods and Techniques of Forecasting Linear forecasting example.

– Major disaster data set, 1953-2007.• Disaster Declarations = 9.851 + .803 * years; R2 = .592.

– Where 1953 is set to zero.

• 1953 – 1971: Disaster Declarations = 13.516 + 0.340 * years; R2 = .103.

• 1972 - 1988: Disaster Declarations = 71.419 - 1.551 * years; R2 = .497.

• 1989 - 2007: Disaster Declarations = -9.368 + 1.281* years; R2 = .329.

• Projection for 2008 (55) using whole data set: 54.0.

• Projection for 2008 (55) using post-1988 set: 61.1.

Methods and Techniques of Forecasting Theoretical forecasting.

– Theory mapping.• Types of causal arguments.

– Convergent arguments.– Divergent arguments.– Serial arguments.– Cyclic arguments.

• Procedures for uncovering the structure of an argument.– Separate and number each assumption.– Underline the words that indicate claims (“therefore”, “thus”,

“hence”).– When specific words are omitted, but implied, supply the

appropriate logical indicators in brackets.– Arrange number assumptions and claims in an arrow diagram that

illustrates the causal argument or theory.

Methods and Techniques of Forecasting Example for theory mapping from Daniels

and Clark-Daniels.– http://www.csub.edu/~rdaniels/Pages%20from

%20DanielsClarkDanielsIJMED.pdf.

Methods and Techniques of Forecasting Causal modeling and regression analysis.

– The creation of theoretical models on the basis of previous research and logic.

– The testing of these models with causal analysis methods such as path analysis and structural equation modeling.

– Example:• http://www.csub.edu/~rdaniels/Model%20of%20Pre

sidential%20Disaster%20Decision-Making.xls.

Methods and Techniques of Forecasting Judgmental forecasting.

– Delphi technique.• Principles.

– Selective anonymity.

– Iteration.

– Informed multiple advocacy.

– Polarized statistical response.

– Structured conflict.

– Computer conferencing.

Methods and Techniques of Forecasting Judgmental forecasting (contd.).

– Delphi technique (contd.).• Steps.

– Issue specification.– Selection of advocates.– Questionnaire design.

• Forecasting items (probability of occurrence).• Issue items (ranking of importance).• Goal items (desirability and feasibility).• Option items (alternatives).

– Analysis of first-round results.– Development of subsequent questionnaires.– Organization of group meetings.– Preparation of final report.

Methods and Techniques of Forecasting Delphi article.

– http://www.csub.edu/~rdaniels/ch3b1.pdf.


Lecture 3a – Official and Unofficial Actors and Their Roles in Public Policy

Introduction

Political science traditions.– Institutionalism – focus on texts of constitutions, laws,

and other written statements of policies and the relationships between formal government institutions.

– Behaviorism – focus on political motivations of individuals, acting singly and in groups, often through polling, game theory, and statistical techniques.

– Neo-institutionalism – focus on organizations and systems in which individuals interact and achieve political and policy goals through explicit or implicit rules and operating procedures.

Introduction

Main categories of actors in the policy process.– Official actors – statutory or constitutional

responsibilities.• Legislative, executive, and judiciary.

– Unofficial actors – participation with no explicit legal authority.

• Interest groups, media.

Legislatures

First listed branch in the federal and most state constitutions.– Source of considerable research.

Primary function is lawmaking. Number of bills and resolutions gives some idea of how busy legislatures are.

Legislatures

JurisdictionJurisdiction

House or House or Assembly Assembly BillsBills

House or House or Assembly Assembly ConcurrenConcurrent t ResolutionResolutionss

House or House or Assembly Assembly Joint Joint ResolutionResolutionss

Senate Senate BillsBills

Senate Senate ConcurrenConcurrent t ResolutionResolutionss

Senate Senate Joint Joint ResolutionResolutionss

U.S. U.S. Congress Congress (2001-(2001-2002)2002) 5,7675,767 521521 125125 3,1813,181 160160 5353California California Legislature Legislature (2001-(2001-2002)2002) 3.2833.283 264264 6666 2,2772,277 114114 5656

Legislatures

Burden eased by staff. Bills sifted by committee structure at both

the federal and state level.– Committee chairs wield significant power.– Most bills fail to move past their first

committee hurdles because they are largely symbolic gestures.

Legislatures Other critical functions performed by legislators

that affect public policy.– Casework – activities to help constituents with

government agencies or to gain a privilege or benefit.• Supports reelection.

– Oversight – Monitor the implementation of public policy.

• Government Accountability Office – www.gao.gov. Studies public programs and makes recommendations to improve efficiency, effectiveness, and accountability.

• Public hearings.– Help understand issues.– Reveal shortcomings in current policies.– Make political capital.

Legislative Organization

Legislative Organization

California process.– http://www.leginfo.ca.gov/bil2lawx.html.

Legislative Organization What you see on C-SPAN does not represent the

bulk of legislative action on policy. Most of the critical work on public policy is done

in committees, which review legislation, propose and vote on amendments, and, in the end, decide whether a bill will die at the committee level or be elevated for consideration by the full body.

One of the most critical elements of legislative organization is the organization on party lines.

Legislature – Critiques of Public Policy Process Many people argue that legislatures are out

of touch with the people. To understand why legislatures work as

they do, you need to understand two elements of the legislature: the nature of the members of the body and the organization and nature of the branch itself.

Legislature – Critiques of Public Policy Process The primary goal of the typical legislator is

reelection. Casework allows legislators to please voters.– Home style and hill style.

Legislatures are decentralized institutions, especially Congress.– Committees and subcommittees.

– Decentralization and centralization of party leadership.

– Issue networks and policy subsystems.

Legislatures – Implications for Policy Making Decentralization and casework focus makes

complex and change-oriented legislation difficult to pass.

The Executive Branch

For the sake of discussion, the executive branch can be considered in two parts: the administration, staff, and appointees; and the bureaucracy.

Advantages of an elected executive in the policy process.– Veto power.

– Unitary branch of government.

– Media and public attention.

– Informational advantage over the legislature.

The Executive Branch

Elected executive limitations.– “Power to persuade”.– The size of the Executive Office of the

President.

Elected executive’s focus on agenda-setting.

Administrative Agencies and Bureaucrats Characteristics of bureaucracy.

– Fixed and official jurisdictional areas.– Hierarchical organization.– Written documentation.– Expert training of staff.– Career, full-time occupation.– Standard operating procedures.

Key complaints about bureaucracy.– Size.– Red tape.

Administrative Agencies and Bureaucrats What Do Government Agencies Do?

– Government agencies provide services that are uneconomical for the private sector (public goods – free-rider problem).

• Public goods are indivisible and nonexclusive.

– Complaints tend to focus on speed, efficiency, and effectiveness of public service delivery.

Administrative Agencies and Bureaucrats Bureaucracy and the problem of accountability.

– The key problem is the question of accountability. Most public employees are appointed on merit, not accountability to elected officials.

– Early thinking focused on separation of politics and administration.

– Modern thinking: Agency decisions are political and in the realm of administrative discretion.

– Problem: no single, agreed-upon definition of the public interest.

– Administrative discretion: ability to make decisions with minimal interference.

The Courts

The ability to interpret legislative and executive actions: judicial review.

Courts are the weakest because their authority rests on the legitimacy of the law and their ability to argue their case.

Legislatures and executives initiate public policy, while courts react to the practical effects of such policies.

Unofficial Actors and Their Roles in Public Policy Individual citizens.

– Low political participation.• Voting.• Other forms of participation: campaigning,

contacting, etc.

– Despite this, citizens can be mobilized:• Recall election in California.

– Generally speaking, individuals want the most services for ourselves while paying the least taxes for those services.

Unofficial Actors and Their Roles in Public Policy Interest groups.

– Interest groups have been part of the political scene since the founding.

• Madison and the dangers of faction.

– Since the 1960s the number of groups has greatly expanded.

• Transportation, mass communication, expansion of government.

– Few legal barriers to the creation of groups.

Unofficial Actors and Their Roles in Public Policy Interest groups.

– The power of interest groups varies.• Knowledge, money, information.

• Group size, peak associations.

• Intensity, direct economic interest, ideological commitment.

• Social movements (combinations of interest groups).

Unofficial Actors and Their Roles in Public Policy Types of interest groups.

– Institutional versus membership groups.– Economic (private) versus public interest versus

ideological groups.• Benefits, free-rider problems.

Activities of interest groups.– Lobbying.– Campaign contributions.– Access (well-off).– Mass mobilization, protest, and litigation.– Riots and protest marches.

Unofficial Actors and Their Roles in Public Policy Political parties.

– Functions.• Voting cues.

• Transmission of political preferences.

• Creation of packages of policy ideas.

• Organization of the legislative branch.

Think tanks and other research organizations.– Brookings, Cato, Urban Institute, Rand, American

Enterprise Institute.• Ideological, scholarly, and methodological distinctions.

Unofficial Actors and Their Roles in Public Policy Communications media.

– The news media are important actors in the policy process.

• Newspapers – National versus regional versus local.

• TV is the central news medium. Older population, networks; younger population, cable news.

– Entertainment programming can be equally important.

• Movies, t.v., videogames.

– Media’s primary function in policy process is agenda-setting. Media coverage correlates with institutional attention.

Unofficial Actors and Their Roles in Public Policy Communications media.

– News media are not just passive actors.• Interest try to arouse media focus.

• Time and space constraints require discretion.

• Profit-driven businesses.

• Competitive biases of news gathering: dramatic and narrative qualities of the story.

Unofficial Actors and Their Roles in Public Policy Subgovernments, issue networks, and domains.

– Policy domain is the substantive area of policy over which participants in policy-making compete and compromise.

– The political culture and legal environment influence the domains.

– Policy community inside the domain consists of the actors actively involved in policy making in that domain.

• Iron triangles one way of organizing the policy community.

• Issue networks may be more accurate description.

Unofficial Actors and Their Roles in Public Policy Subgovernments, issue networks, and domains.

– Prying open policy networks (major corporate interests usually dominate).

– But, policy change is possible by prying open a domain.

• Focusing events.

• Social movements and mobilization.

• Exploiting the decentralization of American government.

• Going public.


Recommending Policy Actions.

Recommendation in Policy Analysis The policy-analytic procedure of

recommendation enables analysts to produce information about the likelihood that future courses of action will result in consequences that are valuable to some individual, group, or society as a whole.

Recommendation in Policy Analysis The procedure of recommendation involves

the transformation of information about policy futures into information about policy actions that will result in valued outcomes.

Policy recommendations are normative (advocative) rather than empirical (descriptive) or evaluative.

Recommendation in Policy Analysis Characteristics of advocative claims.

– Actionable: Advocative claims focus on actions that make be taken to resolve a policy problem.

– Prospective: Advocative claims occur prior to the time that actions are taken.

– Value laden: Advocative claims require both that actions have the predicted consequences, but also that those consequences have value for society.

– Ethically complex: Advocative claims can be intrinsic (valued as ends in themselves) or extrinsic (valued because they will produce some other value).

Recommendation in Policy Analysis Simple model of choice.

– Advocative claims are only possible when the analyst is confronted by a situation of choice between two or more alternatives.

– Simple model.• The definition of a problem requiring action.• The comparison of consequences of two or more

alternatives to resolve the problem.• The recommendation of the alternative that will

result in a preferred outcome.

Recommendation in Policy Analysis Simple model of choice.

Conditions for choice.– Single decision maker.

– Certainty.

– Immediacy of consequences.

A1 O1

A2 O2

O1 > O2

A1

Recommendation in Policy Analysis Complex model of choice.

– Conditions.• Multiple stakeholders.

• Uncertainty about outcomes.

• The passage of time between actions and consequences.

– Results.• Intransitivity of choice.

Recommendation in Policy Analysis Forms of rationality.

– Given the conditions of complex choice, there are multiple forms of rationality.

• Technical rationality.

• Economic rationality.

• Legal rationality.

• Social rationality.

• Substantive rationality.

Recommendation in Policy Analysis Rational-comprehensive theory.

– An individual or collective decision-maker must identify a policy problem on which there is a consensus among all relevant stakeholders.

– An individual or collective decision-maker must define and consistently rank all goals and objectives whose attainment would represent a resolution of the problem.

– An individual or collective decision-maker must identify all policy alternatives that may contribute to the attainment of each goal and objective.

Recommendation in Policy Analysis Rational-comprehensive theory.

– An individual or collective decision-maker must forecast all consequences that will result from the selection of each alternative.

– An individual or collective decision-maker must compare each alternative in terms of its consequences for the attainment of each goal and objective.

– An individual or collective decision-maker must choose that alternative that maximizes the attainment of objectives.

Recommendation in Policy Analysis Disjointed incremental theory.

– Consider only those objectives that differ incrementally from the status quo.

– Limit the number of consequences forecast for each alternative.

– Make mutual adjustments in goals and objectives, on the one hand, and alternatives on the other.

– Continuously reformulate problems – and hence goals, objectives, and alternatives – in the course of acquiring new information.


Monitoring Policy Outcomes

Introduction

The consequences of policy actions are never fully known in advance and, for this reason, it is essential to monitor policy actions after they have occurred.

In fact, policy recommendations may be viewed as hypotheses about the relationship between policy actions and policy outcomes: if action A is taken at time t1, outcome O will result at time t2.

Monitoring in Policy Analysis

Monitoring is the policy-analytic procedure used to produce information about the causes and consequences of public policies.

Monitoring, since it permits analysts to describe relationships between policy-program operations and their outcomes, is the primary source of knowledge about policy implementation.

Monitoring is primarily concerned about with establishing factual premises about public policy.


Monitoring performs at least four major functions:– Compliance: monitoring helps determine whether

the actions of program administrators, staff, and other stakeholders are in compliance with standards and procedures imposed by legislatures, regulatory agencies, and professional bodies.

– Auditing: monitoring helps determine whether resources and services intended for certain target groups and beneficiaries have actually reached them.


Monitoring performs at least four major functions:– Accounting: monitoring produces information

that is helpful in accounting for social and economic changes that follow the implementation of broad sets of public policies and programs over time.

– Explanation: monitoring also yields information that helps to explain why the outcomes of public polices and programs differ.


Sources of information.– Information must be:

• Relevant.– Macronegative versus micropositive.

• Reliable.– Observations precise and dependable.

• Valid.– Information about policy outcomes should measure what

we think it is measuring.


Sources of information.– Information on policy outcomes.

Historical Statistics of the United States Current Opinion

Statistical Abstract of the United States United States Census of Population by States

County and City Data Book Congressional District Data Book

Census Use Study National Opinion Research Center General Social Survey

Social and Economic Characteristics of Students Survey Research Center National Election Studies

Educational Attainment in the United States National Clearinghouse for Mental Health Information

Current Population Reports National Clearinghouse for Drug Abuse Information

The Social and Economic Status of the Black Population in the United States

National Criminal Justice Reference Service

Female Family Heads Child Abuse and Neglect Clearinghouse Project

Monthly Labor Review National Clearinghouse on Revenue Sharing

Handbook of Labor Statistics Social Indicators

Congressional Quarterly State Economic and Social Indicators

Law Digest


Sources of information.– When information is not available from

existing sources, monitoring may be carried out by some combination of questionnaires, interviewing, field observation, and the use of agency records.


Types of policy outcomes.– Consequences:

• Policy outputs – the goods, services, or resources received by target groups and beneficiaries.

• Policy impacts – actual changes in behavior and attitudes that result from policy outputs.

– Populations:• Target groups – individuals, communities, or organizations on

whom policies and programs are designed to have an effect.

• Beneficiaries – groups for whom the effects of policies are beneficial or valuable.


Types of policy actions. Policy actions have two major purposes.– Regulation – actions designed to ensure

compliance with certain standards or procedures.

– Allocation – actions that require inputs of money, time, personnel, and equipment.

– Regulative and allocative actions may have consequences that are distributive or redistributive.


Types of policy actions.– Policy inputs – the resources – time, money,

personnel, equipment, and supplies – used to produce outputs and impacts.

– Policy processes – the administrative, organizational, and political activities and attitudes that shape the transformation of policy inputs into policy outputs and impacts.


ISSUE AREA

POLICY ACTIONS POLICY OUTCOMES

Inputs Processes Outputs Impacts

Criminal Justice. Dollar expenditures for salaries, equipment, maintenance.

Illegal arrests as percentages of total arrests.

Criminals arrested per 100,000 known crimes.

Criminals convicted per 100,000 known crimes.

Municipal Services.

Dollar expenditures for sanitation workers and equipment.

Morale among workers.

Total residences served.

Cleanliness of streets.

Social welfare. Number of social workers.

Rapport with welfare recipients.

Welfare cases per social worker.

Standard of living of dependent children.


Definitions and indicators.– Variables versus constants.

– Definitions.• Constitutive definitions – gives meaning to words used to

describe variables by using synonymous words.

– Provide no concrete rules or guidelines for actually monitoring changes.

• Operational definitions – gives meaning to a variable by specifying the operations necessary to experience and measure it.

• Indicators – directly observable characteristics.

Approaches to Monitoring

APPROACHTYPES OF CONTROL

TYPE OF INFORMATION

REQUIRED

Social systems accounting QuantitativeAvailable and/or new information

Social experimentation Direct manipulations and quantitative

New information

Social auditingQuantitative and/or qualitative

New information

Research and practice synthesis

Quantitative and/or qualitative

Available information

Approaches to Monitoring

POLICY POLICY POLICY POLICYINPUTS PROCESSES OUTPUTS IMPACTS

PRECONDITIONS UNFORESEEN SIDE-EFFECTSEVENTS AND

SPILLOVERS

CONTROLLED OUTCOMESMANIPULABLE ACTIONS

UNMANIPULABLE CAUSES UNCONTROLLED EFFECTS

In1

In2

.

.

.

Ini

P1

P2

.

.

.

Pj

O1

O2

.

.

.

Om

Im1

Im2

.

.

.

Imn

PC1

PC2

.

.

.

PCp

E1

E2

.

.

.

Eq

SES1

SES2

.

.

.

SESr

Social Systems Accounting

An approach and set of methods that permit analysts to monitor changes in objective and subjective social conditions over time.


The major analytic element of social systems accounting is the social indicator.– Statistics that measure social conditions and

changes therein over time for various segments of a population. By social conditions, we mean both the external (social and physical) and the internal (subjective and perceptional) contexts of human existence in a given society.

Social systems accounting

AREA INDICATORHealth and illness Persons in state mental hospitals

Public safety Persons afraid to walk alone at night

Education High school graduates aged 25 and older

Employment Labor force participation by women

Income Percent of population below the poverty line

Housing Households living in substandard units

Leisure and recreation Average annual paid vacation in manufacturing

Population Actual and projected population

Government and politics Quality of public administration

Social values and attitudes Overall life satisfaction and alienation

Social mobility Change from father’s occupation

Physical environment Air pollution index

Science and technology Scientific discoveries

Representative social indicators.

Social Systems Accounting Must assume that the reasons for indicator change

are related to policy actions. This is not always the case.

Advantages.– Identify areas of insufficient information.– When indicators provide reliable information, possible

to modify policies.

Social Systems Accounting Limitations.

– Choice of indicators influenced by values of analysts.– Social indicators frequently do not cover manipulable

policy actions.– Most social indicators are based available data about

objective (rather than subjective) social conditions.– Social indicators provide little information about how

inputs are transformed into outcomes.

Coefficientsa

13.516 2.561 5.277 .000

.340 .243 .322 1.400 .179

71.419 11.060 6.458 .000

-1.551 .403 -.705 -3.849 .002

-6.745 22.133 -.305 .764

1.218 .494 .525 2.465 .025

(Constant)

Number of years

(Constant)

Number of years

(Constant)

Number of years

Model1

1

1

Breaks indisaster patterns1950 - 1971

1972 - 1988

1989 - 2006

B Std. Error

UnstandardizedCoefficients

Beta

StandardizedCoefficients

t Sig.

Dependent Variable: Total Disaster Declarationsa.

Model Summarya

Breaks in disaster patterns R R Square Sig.1950 - 1971 0.322 0.1031972 - 1988 0.705 0.497 ***1989 - 2006 0.525 0.275 *aPredictors: (Constant), Number of years*p<.05; **p<.01; ***p<.005


Social Experimentation

Use of social indicators often leads to random innovation rather than systematic change.

Social experimentation is the process of systematically manipulating policy actions in way that permits more or less precise answers to questions about the sources of change in policy outcomes.


Characteristics and procedures.– Direct control of experimental treatment.– Comparison (control) groups.– Random assignment.

Goal: internal validity – the capacity of experiments and quasi-experiments to make valid causal inferences about the effects of actions on outcomes.


Threats to internal validity.– History.– Maturation.– Instability.– Instrumentation and testing.– Mortality.– Selection.– Regression artifacts.


Social experimentation is weakest in the area of external validity or generalizability.

Neglects policy processes.

Seattle-Denver Income Maintenance Experiment A social experiment is a field test of one or

more social programs — or, to use the phraseology of the natural sciences, a test of one or more "treatments."

A social experiment is a field test in the sense that families or individuals are actually enrolled in a pilot social program offering some type of special benefit or service.

Seattle-Denver Income Maintenance Experiment It is "experimental" in the sense that

families or individuals are enrolled in each of the tested programs on the basis of a random assignment process, for example, the flip of a coin.

Seattle-Denver Income Maintenance Experiment To draw conclusions about the effects of the

treatment, it is necessary to collect information about the people who are enrolled in each experimental program and about those who receive no special treatment (called the control group), and then to compare them on the basis of the collected information.

Seattle-Denver Income Maintenance Experiment The primary focus of SIME/DIME — the

effect of income maintenance on work effort or labor supply — was a good subject for experimentation .

An increase in unearned income or a decrease in the effective wage rate were hypothesized to lead to decreases in labor supply.

Seattle-Denver Income Maintenance Experiment In the case of SIME/DIME, the experiment

was in fact designed to test the effect of two different kinds of social programs on participant work effort.

The two policies were a variety of cash transfer or negative income tax programs and several combinations of job counseling and education or training subsidy programs.

Seattle-Denver Income Maintenance Experiment Any observed effects of the experiment was

interpreted as the differential effects of the experimental treatment compared to existing government programs.

Thus any observed experimental-control differences in outcomes must be interpreted as estimates of the effect of replacing the early 1970s status quo with the experimental programs.

Seattle-Denver Income Maintenance Experiment

Table 4.

Labor Supply Response of Husbands:

A. Overall NIT response

(Percentage difference in annual hours worked)

Year

Sample Group 1 2 3 4 5

3-year sample -1.6 -7.3 -7.3 -0.5 -0.2

5-year sample -5.9 -12.2 -13.2 -13.6 -12.3

Total Sample -3.1 -9 -9.3 — —

Seattle-Denver Income Maintenance Experiment Table 5.

Labor Supply Response of Wives:



Year


3-year sample -4 -16.5 -15.2 -2 +13.4

5-year sample -15.1 -26.5 -21.6 -27.1 -24

Total Sample -8.1 -20.1 -17.4 — —

Seattle-Denver Income Maintenance Experiment Table 6.

Labor Supply Response of Female Heads:



Year


3-year sample -5.5 -14.1 -21.6 -8.9 -7.7

5-year sample -7.9 -15 -21.2 -28.3 -31.8

Total Sample -6.3 -14.3 -21.4 — —

Social Auditing

One of the limitations of social systems accounting and social experimentation is that both approaches neglect or oversimplify policy processes.

Social Auditing

Social auditing explicitly monitors relations among inputs, processes, outputs, and impacts in an effort to trace policy inputs from disbursement to final recipient.

Social auditing helps to determine whether ineffective outcomes are the result of inadequate inputs or processes that divert resources or services from intended target groups or beneficiaries.

Social Auditing

Processes monitored of two main types:– Resource diversion (from target or beneficiary

groups).– Resource transformation (changes in meaning

of policy actions from administrator to recipient).

Social Audit Steps

The three phases of a social audit Phase 1: Design and data collection

– Clarify the strategic focus– Design instruments, pilot test– Collect information from households and key

informants in a panel of representative communities

Social Audit Steps

Phase 2: Evidence-based dialogue and analysis– Link household data with information from

public services– Analyze findings in a way that points to action– Take findings back to the communities for their

views about how to improve the situation– Bring community members into discussion of

evidence with service providers/planners.

Social Audit Steps

Phase 3: Socialization of evidence for public accountability– Work-shopping– Communication strategy– Evidence-based training of planners and service

providers– Media training– Partnerships with civil society

Research and practice synthesis.

An approach to monitoring that involves the systematic compilation, comparison, and assessment of the results of past efforts to implement public policies.

Two primary sources of available information.– Case studies of policy formulation and implementation.

– Research reports that address relations among policy actions and outcomes.

Research and practice synthesis.

Two methods.– Case survey method – a set of procedures used

to identify and analyze factors that account for variations in the adoption and implementation of policies.

• Requires case coding scheme, a set of categories that capture key aspects of policy inputs, processes, outputs, and impacts.


Two methods.– Research survey method – a set of procedures

used to compare and appraise results of past research on policy actions and outcomes.

• Yields empirical generalizations about sources of variation in policy outcomes, summary assessments of the confidence researchers have in these generalizations, and policy alternatives or action guidelines that are implied in these generalizations.

• Requires the construction of a format for extracting information about research reports.


Advantages.– Comparatively efficient way to compile and

appraise an increasingly large body of cases and research reports on policy implementation.

– The case survey method is one among several ways to uncover different dimensions of policy processes that affect policy outcomes.

– The case survey method is a good way to examine both objective and subjective conditions.


Limitations.– All related to reliability and validity of

information.

Research and Practice Synthesis: Example David H. Greenberg; Charles

Michalopoulos; Philip K. Robins. Do Experimental and Nonexperimental

Evaluations Give Different Answers about the Effectiveness of Government-Funded Training Programs?

Research and Practice Synthesis: Example Abstract This paper uses meta-analysis to investigate

whether random assignment (or experimental) evaluations of voluntary government-funded training programs for the disadvantaged have produced different conclusions than nonexperimental evaluations.

Information includes several hundred estimates from 31 evaluations of 15 programs that operated between 1964 and 1998.

Research and Practice Synthesis: Example The results suggest that experimental and

nonexperimental evaluations yield similar conclusions about the effectiveness of training programs, but that estimates of average effects for youth and possibly men might have been larger in experimental studies.

POSC 431 – Program Evaluation

Source

Basic Guide to Program Evaluation– Written by

Carter McNamara, MBA, PhD, Authenticity Consulting, LLC. Copyright 1997-2006.

– Adapted from the Field Guide to Nonprofit Program Design, Marketing and Evaluation.

A Brief Introduction ... Note that the concept of program evaluation can

include a wide variety of methods to evaluate many aspects of programs in nonprofit or for-profit organizations.

However, personnel do not have to be experts in these topics to carry out a useful program evaluation. The "20-80" rule applies here, that 20% of effort generates 80% of the needed results. It's better to do what might turn out to be an average effort at evaluation than to do no evaluation at all.

Some Myths about Program Evaluation Many people believe evaluation is a useless

activity that generates lots of boring data with useless conclusions.

More recently (especially as a result of Michael Patton's development of utilization-focused evaluation), evaluation has focused on utility, relevance and practicality at least as much as scientific validity.

Some Myths about Program Evaluation Many people believe that evaluation is about

proving the success or failure of a program. This myth assumes that success is implementing

the perfect program and never having to hear from employees, customers or clients again -- the program will now run itself perfectly.

This doesn't happen in real life. Success is remaining open to continuing feedback

and adjusting the program accordingly. Evaluation gives you this continuing feedback.

Some Myths about Program Evaluation Many believe that evaluation is a highly unique and

complex process that occurs at a certain time in a certain way, and almost always includes the use of outside experts.

Many people believe they must completely understand terms such as validity and reliability.

They don't have to. They do have to consider what information they need to

make current decisions about program issues or needs. And they have to be willing to commit to understanding

what is really going on.

So What is Program Evaluation?

First, we'll consider "what is a program?" Typically, organizations work from their mission to

identify several overall goals which must be reached to accomplish their mission.

In public agencies and nonprofits, each of these goals often becomes a program.

Public and nonprofit programs are organized methods to provide certain related services to constituents, e.g., clients, customers, patients, etc.

Programs must be evaluated to decide if the programs are indeed useful to constituents.

So What is Program Evaluation?

Program evaluation is carefully collecting information about a program or some aspect of a program in order to make necessary decisions about the program.

Don't worry about what type of evaluation you need or are doing -- worry about what you need to know to make the program decisions you need to make, and worry about how you can accurately collect and understand that information.

Where Program Evaluation is Helpful Understand, verify or increase the impact of

products or services on customers or clients. Improve delivery mechanisms to be more

efficient and less costly. Verify that you're doing what you think

you're doing.

Other Reasons to Conduct Evaluations Facilitate management's really thinking

about what their program is all about, including its goals, how it meets it goals and how it will know if it has met its goals or not.

Produce data or verify results that can be used for public relations and promoting services in the community.

Other Reasons to Conduct Evaluations Produce valid comparisons between

programs to decide which should be retained, e.g., in the face of pending budget cuts.

Fully examine and describe effective programs for duplication elsewhere.

Basic Ingredients: Organization and Program(s) You Need An Organization:

– This may seem too obvious to discuss, but before an organization embarks on evaluating a program, it should have well established means to conduct itself as an organization.

You Need Program(s):– To effectively conduct program evaluation, you should

first have programs. That is, you need a strong impression of what your customers or clients actually need.

– Next, you need some effective methods to meet each of those goals. These methods are usually in the form of programs.

Programs

Inputs are the various resources needed to run the program, e.g., money, facilities, customers, clients, program staff, etc.

The process is how the program is carried out.

Programs

The outputs are the units of service, e.g., number of customers serviced, number of clients counseled, children cared for, artistic pieces produced, or members in the association.

Outcomes are the impacts on the customers or on clients receiving services, e.g., increased mental health, safe and secure development, richer artistic appreciation and perspectives in life, increased effectiveness among members, etc.

Planning Your Program Evaluation Depends on what information you need to

make your decisions and on your resources.– Your program evaluation plans depend on what

information you need to collect in order to make major decisions.

– But the more focused you are about what you want to examine by the evaluation, the more efficient you can be in your evaluation, the shorter the time it will take you and ultimately the less it will cost you.

Key Considerations

For what purposes is the evaluation being done, i.e., what do you want to be able to decide as a result of the evaluation?

Who are the audiences for the information from the evaluation, e.g., customers, bankers, funders, board, management, staff, customers, clients, citizens, etc.

Key Considerations

What kinds of information are needed to make the decision you need to make and/or enlighten your intended audiences, e.g., information to really understand the process of the product or program (its inputs, activities and outputs), the customers or clients who experience the product or program, strengths and weaknesses of the product or program, benefits to customers or clients (outcomes), how the product or program failed and why, etc.

Key Considerations

From what sources should the information be collected, e.g., employees, customers, clients, groups of customers or clients and employees together, program documentation, etc.

How can that information be collected in a reasonable fashion, e.g., questionnaires, interviews, examining documentation, observing customers or employees, conducting focus groups among customers or employees, etc.

Key Considerations

When is the information needed (so, by when must it be collected)?

What resources are available to collect the information?

Goals-Based Evaluation

1. How were the program goals (and objectives, is applicable) established? Was the process effective?

2. What is the status of the program's progress toward achieving the goals?

3. Will the goals be achieved according to the timelines specified in the program implementation or operations plan? If not, then why?

4. Do personnel have adequate resources (money, equipment, facilities, training, etc.) to achieve the goals?

Goals-Based Evaluation

5. How should priorities be changed to put more focus on achieving the goals? (Depending on the context, this question might be viewed as a program management decision, more than an evaluation question.)

6. How should timelines be changed (be careful about making these changes - know why efforts are behind schedule before timelines are changed)?

7. How should goals be changed (be careful about making these changes - know why efforts are not achieving the goals before changing the goals)? Should any goals be added or removed? Why?

8. How should goals be established in the future?

Process-Based Evaluations

1. On what basis do employees and/or the customers decide that products or services are needed?

2. What is required of employees in order to deliver the product or services?


3. How are employees trained about how to deliver the product or services?

4. How do customers or clients come into the program?

5. What is required of customers or client? 6. How do employees select which products

or services will be provided to the customer or client?


7. What is the general process that customers or clients go through with the product or program?

8. What do customers or clients consider to be strengths of the program?

9. What do staff consider to be strengths of the product or program?

10. What typical complaints are heard from employees and/or customers?


11. What do employees and/or customers recommend to improve the product or program?

12. On what basis do employees and/or the customer decide that the product or services are no longer needed?

Outcome-Based Evaluations Identify the major outcomes that you want to

examine or verify for the program under evaluation.

Choose the outcomes that you want to examine, prioritize the outcomes and, if your time and resources are limited, pick the top two to four most important outcomes to examine for now.

For each outcome, specify what observable measures, or indicators, will suggest that you're achieving that key outcome with your clients.

Outcome-Based Evaluations

Specify a "target" goal of clients, i.e., with what number or percent of clients you commit to achieving specific outcomes?

Identify what information is needed to show these indicators.

Decide how can that information be efficiently and realistically gathered.

Analyze and report the findings.

Methods to Collection Information

Questionnaires, surveys, checklists. Interviews. Documentation review. Observation. Focus groups. Case studies.

Methods to Collection Information

Ethics: Informed Consent from Program Participants Note that if you plan to include in your evaluation,

the focus and reporting on personal information about customers or clients participating in the evaluation, then you should first gain their consent to do so.

They should understand what you're doing with them in the evaluation and how any information associated with them will be reported.

Ethics: Informed Consent from Program Participants You should clearly convey terms of

confidentiality regarding access to evaluation results.

They should have the right to participate or not.

Have participants review and sign an informed consent form.

Selecting Which Methods to Use

The overall goal in selecting evaluation method(s) is to get the most useful information to key decision makers in the most cost-effective and realistic fashion.

Note that, ideally, the evaluator uses a combination of methods, for example, a questionnaire to quickly collect a great deal of information from a lot of people, and then interviews to get more in-depth information from certain respondents to the questionnaires.

Perhaps case studies could then be used for more in-depth analysis of unique and notable cases, e.g., those who benefited or not from the program, those who quit the program, etc.


1. What information is needed to make current decisions about a product or program?

2. Of this information, how much can be collected and analyzed in a low-cost and practical manner, e.g., using questionnaires, surveys and checklists?

3. How accurate will the information be (reference the above table for disadvantages of methods)?

4. Will the methods get all of the needed information?

5. What additional methods should and could be used if additional information is needed?


6. Will the information appear as credible to decision makers, e.g., to funders or top management?

7. Will the nature of the audience conform to the methods, e.g., will they fill out questionnaires carefully, engage in interviews or focus groups, let you examine their documentations, etc.?

8. Who can administer the methods now or is training required?

9. How can the information be analyzed?

Four Levels of Evaluation: There are four levels of evaluation information

that can be gathered from clients, including getting their:1. reactions and feelings (feelings are often poor

indicators that your service made lasting impact)2. learning (enhanced attitudes, perceptions or

knowledge)3. changes in skills (applied the learning to enhance

behaviors)4. effectiveness (improved performance because of

enhanced behaviors)

Four Levels of Evaluation:

Usually, the farther your evaluation information gets down the list, the more useful is your evaluation. Unfortunately, it is quite difficult to reliably get information about effectiveness. Still, information about learning and skills is quite useful.

Analyzing and Interpreting Information Always start with your evaluation goals:

– When analyzing data (whether from questionnaires, interviews, focus groups, or whatever), always start from review of your evaluation goals, i.e., the reason you undertook the evaluation in the first place. This will help you organize your data and focus your analysis.

Analyzing and Interpreting Information Basic analysis of "quantitative" information (for

information other than commentary, e.g., ratings, rankings, yes's, no's, etc.):1. Make copies of your data and store the master copy

away. Use the copy for making edits, cutting and pasting, etc.

2. Tabulate the information, i.e., add up the number of ratings, rankings, yes's, no's for each question.

3. For ratings and rankings, consider computing a mean, or average, for each question.

4. Consider conveying the range of answers, e.g., 20 people ranked "1", 30 ranked "2", and 20 people ranked "3".

Analyzing and Interpreting Information Basic analysis of "qualitative" information

(respondents' verbal answers in interviews, focus groups, or written commentary on questionnaires):1. Read through all the data.2. Organize comments into similar categories.3. Label the categories or themes, e.g., concerns,

suggestions, etc.4. Attempt to identify patterns, or associations and causal

relationships in the themes.5. Keep all commentary for several years after

completion in case needed for future reference.

Interpreting Information:1. Attempt to put the information in perspective.

a) Compare results to what you expected; b) Management or program staffc) Common standards for your servicesd) Original program goals (program evaluation)e) Indications of accomplishing outcomes (outcomes evaluation)f) Description of the program's experiences, strengths, weaknesses,

etc. (process evaluation).

2. Consider recommendations to help program staff improve the program, conclusions about program operations or meeting goals, etc.

3. Record conclusions and recommendations in a report document, and associate interpretations to justify your conclusions or recommendations.

Reporting Evaluation Results

1. The level and scope of content depends on to whom the report is intended,

2. Be sure employees have a chance to carefully review and discuss the report. Translate recommendations to action plans, including who is going to do what about the program and by when.

Reporting Evaluation Results

3. Bankers or funders will likely require a report that includes an executive summary (this is a summary of conclusions and recommendations, not a listing of what sections of information are in the report -- that's a table of contents); description of the organization and the program under evaluation; explanation of the evaluation goals, methods, and analysis procedures; listing of conclusions and recommendations; and any relevant attachments,

4. Be sure to record the evaluation plans and activities in an evaluation plan which can be referenced when a similar program evaluation is needed in the future.

Pitfalls to Avoid

1. Don't balk at evaluation because it seems far too "scientific." It's not. Usually the first 20% of effort will generate the first 80% of the plan, and this is far better than nothing.

2. There is no "perfect" evaluation design. Don't worry about the plan being perfect. It's far more important to do something, than to wait until every last detail has been tested.

Pitfalls to Avoid

3. Work hard to include some interviews in your evaluation methods. Questionnaires don't capture "the story," and the story is usually the most powerful depiction of the benefits of your services.

4. Don't interview just the successes. You'll learn a great deal about the program by understanding its failures, dropouts, etc.

5. Don't throw away evaluation results once a report has been generated. Results don't take up much room, and they can provide precious information later when trying to understand changes in the program.

Research and Practice Synthesis: Example The results also suggest that variation

among nonexprimental estimates of program effects is similar to variation among experimental estimates for men and youth, but not for women (for whom it seems to be larger), although small sample sizes make the estimated differences somewhat imprecise for all three groups.

Research and Practice Synthesis: Example The policy implications of the findings are

discussed. © 2006 by the Association for Public Policy

Analysis and Management

Techniques for Monitoring

APPROACHGRAPHIC DISPLAYS

TABULAR DISPLAYS

INDEX NUMBERS

INTER-RUPTED TIME-SERIES ANALYSIS

CONTROL-SERIES ANALYSIS

REGRES-SION DIS-CONTI-NUITY ANALYSIS

Social systems accounting X X X X X O

Social auditing x x x x x o

Social experimen-tation x x x x x x

Research and practice synthesis x x o o o o

Recommendation in Policy Analysis Disjointed incremental theory.

– Analyze and evaluate alternatives in a sequence of steps, such that choices are continuously amended over time, rather than made at a single point in time.

– Continuously remedy existing social problems, rather than solve problems completely at one point in time.

– Share responsibilities for analysis and evaluation with many groups in society, so that the process of making choices is fragmented or disjointed.

Recommendation in Policy Analysis Arrow’s impossibility theorem.

– It is impossible for democratic decision makers in a democratic society to meet conditions of the rational comprehensive model.

• Individual choices cannot be aggregated through majority voting procedures to create a collective decision that will produce a single best solution for all parties.

The Voter’s ParadoxCOMMITTEE MEMBER PREFERENCE

Brown (criterion: risk) A (solar) preferred to B (coal)

B (coal) preferred to C (nuclear)

A (solar preferred to C (nuclear)

Jones (criterion: feasibility) B preferred to C

C preferred to A

B preferred to A

Smith (criterion: efficiency) C preferred to A

A preferred to B

C preferred to B

Majority (intransitive and cyclic) A preferred to B

B preferred to C

C preferred to A

Recommendation in Policy Analysis Arrow’s impossibility theorem.

– Reasonable conditions for democratic decision procedures.

• Nonrestriction of choices.

• Nonperversity of collective choice.

• Independence of irrelevant alternatives.

• Citizen’s sovereignty.

• Nondictatorship.

Recommendation in Policy Analysis Bounded rationality.

– Decision makers engage in satisficing behavior (identify courses of action that are “good enough.”).

• Consider the most evident alternatives that produce a reasonable increase in benefits.

Rationality as constrained maximization.– Rational choice within the boundaries of

constraint.

Recommendation in Policy Analysis Criteria for policy recommendation.

– Effectiveness – does a given alternative result in the achievement of a valued outcome (technical rationality).

– Efficiency – the amount of effort needed to produce a given level of effectiveness (economic rationality).

– Adequacy – the extent to which any given level of effectiveness satisfies the needs, values, or opportunities that gave rise to the problem.

• Fixed costs and variable effectiveness (type I, maximize effectiveness).

• Fixed effectiveness and variable costs (type II, minimize costs).• Variable costs and variable effectiveness (type III, efficiency).• Fixed costs and fixed effectiveness (type IV, do nothing).

Recommendation in Policy Analysis Criteria for policy recommendation.

– Equity – the distribution of effects and effort among different groups in society (legal and social rationality).

• Maximize individual welfare.• Protect minimum welfare.• Maximize net welfare.• Maximize redistributive welfare.

– Responsiveness – satisfies the needs, preferences, or values of particular groups.

– Appropriateness – the value or worth of a program’s objectives and the tenability of assumptions underlying these objectives.

Approaches to Recommendation

Public versus private choice.– Nature of public policy processes.

• Numerous stakeholders with conflicting values.

– Collective nature of public policy goals.• Multiple conflicting criteria for choice.

– Nature of public goods.• Specific, collective, and quasi-collective goods.

Supply and demand (market mechanisms).– Discuss.


Public choice.– Problems with supply – demand models of

public policy.• Multiple legitimate stakeholders.

• Collective and quasi-collective goods.

• Limited comparability of income measures.

• Public responsibility for social costs and benefits.


Benefit-cost analysis.– Characteristics.

• Measure all costs and benefits to society of a program including intangibles.

• Traditional benefit-cost analysis emphasizes economic rationality: net benefits are greater than zero and higher than alternative uses.

• Traditional benefit-cost analysis uses the private marketplace as the point of departure in recommending programs.

• Social benefit-cost analysis also measures redistributional benefits.


Types of costs and benefits.– Inside versus outside costs and benefits.– Tangible versus intangible costs and benefits.– Direct versus indirect costs and benefits.– Net efficiency versus redistributional benefits.

Approaches to Recommendation Tasks in benefit-cost analysis.

– Problem structuring.– Specification of objectives.– Identification of alternative solutions.– Information search, analysis, and interpretation.– Identification of target groups and beneficiaries.– Estimation of costs and benefits.– Discounting of costs and benefits.– Estimation of risk and uncertainty.– Choice of decision criterion.– Recommendation.

Benefit-cost and Cost-effectiveness Analysis Benefit-cost analysis is an applied branch of

economics that attempts to assess service programs by determining whether the total societal welfare has increased (in aggregate more people have been made better off) because of the project or program.

Steps.– Determine the benefits of a proposed or existing

program and place a dollar value on those benefits.– Calculate the total costs of the program.– Compare the benefits and costs.

Benefit-cost and Cost-effectiveness Analysis Simple steps pose real challenge, especially

estimating intangible benefits. Procedure still useful in uncovering

assumptions and estimating value of intangibles.

Benefit-cost and Cost-effectiveness Analysis Cost-effectiveness analysis.

– The major costing alternative to benefit-cost analysis.• Relates the cost of a given alternative to specific measures of

program objectives.– For example, dollar per life saved on various highway

safety programs.

– Often the first step in a benefit-cost analysis.– Especially useful if analyst cannot quantify benefits, but

has fairly specific program objectives.– Key problem: situation where there are multiple benefits.

Results often very subjective.– 2nd key problem: does not produce a bottom line number.

Benefit-cost and Cost-effectiveness Analysis A private sector analogy.

– Benefit-cost analysis similar to financial analysis in private sector.

• Should the firm have done the project at all, i.E., Is the project producing a satisfactory rate of return?

• Public version: is the program a success, i.E., Has it improved social welfare?

• What other options are there for the use of the firm’s resources?

• Public version: should the program be continued when weighed against alternative uses for the government’s funds.

Benefit-cost and Cost-effectiveness Analysis A private sector analogy.

– Benefits and costs do not occur simultaneously in either private or public sector.

• R&D costs, marketing, capital investment, training.

• Government not priced, thus benefits are more broadly defined.

– Alternative uses of resources (opportunity costs).

Benefit-cost and Cost-effectiveness Analysis Benefit-cost illustration.

Costs

Year R & D Capital O & M Total Benefits1 $1500 $2000 $0 $3500 $0

2 500 2000 2000 4500 3500

3 2000 2500 4500 5500

4 2000 3000 5000 6500

5 2000 3500 5500 8500

Totals $2000 $10000 $11000 $23000 $24000

Year Benefit-Costs Present Value, Benefits-Costs (10%)1 ($3500) ($3500)

2 (1000) (909)

3 1000 826

4 1500 1127

5 3000 2049

Totals $1000 ($407)

Net Present Value @ 10% ($407)

Net Present Value @ 5% 219

Benefit-cost and Cost-effectiveness Analysis Formula for net present value.

Formula sensitive to choice of rate of return. Could also use return on investment.

– Discount rate that would make the present value zero.

12

332211

)1(...

)1(1

x

yxyxyyyyyy

r

CB

r

CB

r

CBCBNPV

Continuing or not continuing the project.

Benefit-cost and Cost-effectiveness Analysis

Costs

Year R & D Capital O & M Total Benefits6 $500 $400 $4500 $5400 $9000

7 400 5000 5400 8500

8 400 6000 6400 8500

9 400 7500 7900 8000

10 400 8000 8400 7500

Totals $500 $2000 $31000 $33500 $41500

Year Benefit-Costs Present Value, Benefits-Costs (10%)6 $3600 3600

7 3100 2818

8 2100 1736

9 100 75

10 (900) (615)

Totals $8000 $7614

Net Present Value @ 10% $7614

Net Present Value @ 5% $7803

Benefit-cost and Cost-effectiveness Analysis Total versus marginal benefits and costs.

– When considering the overall profitability of a project, an agency will consider the total costs involved in getting the project started through its operation’s cycle.

– But at any point in time, when an agency is continuing or discontinuing a project or program, it only considers the marginal costs or benefits.

– Definitions.• Marginal cost: incremental cost of producing one more unit of

output.• Marginal benefit: incremental benefit of that one unit of output.

– Public sector usually does not do it on the basis of a single unit, but what are the benefits being generated now versus the costs now?

Benefit-cost and Cost-effectiveness Analysis Public-private sector differences and

similarities.– Similarities – alternative uses for funds, one-

time costs, recurring costs, land, labor, capital.– Differences.

• Distributional considerations.

• Spillovers.

Framework for AnalysisBenefit Indicator Measure Dollar value Assumptions

Real

Direct

Tangible

Intangible

Indirect

Tangible

Intangible

Cost Indicator Measure Dollar value Assumptions

Real

Direct

Tangible

Intangible

Indirect

Tangible

Intangible

Transfers

Framework for Analysis

Real benefits and costs versus transfers.– Real: net gains and losses to society.– Transfers: merely alter the distribution of

resources in society.


Direct and indirect benefits and costs.– Direct costs and benefits are closely related to

the primary objectives of the project.• Direct costs – personnel, facilities, equipment,

material, project administration.

– Indirect costs and benefits are byproducts, multipliers, spillovers, or investment effects.

• Indirect costs are unintended costs that result from government action.

• Indirect benefits might include benefits of space exploration.


Tangible and intangible benefits and costs.– Tangible benefits and costs can be converted

readily into dollar figures.– Intangible benefits and costs are those things

that cannot be directly assigned an explicit price.

Determining the geographic scope of analysis.– Spillover effects may determine true

geographic jurisdiction.

Framework for Analysis (Agricultural Dam Example)Real Benefits Nature of Benefit/Cost

Direct

Tangible Increased farm output

New supply of water

Intangible Maintaining family farms

Indirect

Tangible Reduced soil erosion

Intangible Preservation of rural society

Real Costs

Direct

Tangible Construction material, labor, operations and maintenance, direct program supervision by agency

Intangible Loss of recreational value of land or river

Indirect

Tangible Administrative overhead of government

Diversion of water and its effects

Increased salinity

Intangible Loss of wilderness area

Transfers Relative improvement of profit for farm implement industry

General taxpayers may be subsidizing farmers.

Measuring Benefits

Evaluation problem difficult for government because of multiple benefits and intangibles.

Measuring Benefits

Sources of data.– Existing records and statistics kept by agency.– Feedback from clients.– Ratings by trained observers.– Experience of other governments or private or

nonprofit corporations.– Special data gathering.

Whenever possible analyst should use market value or willingness to pay.

Measuring Benefits

Valuing benefits.– Cost savings.– Time saved.– Lives saved.– Increased productivity or wages.– Recreational benefits.– Land values.– Alternatives to market prices.

See handout.

Measuring Costs

Cost categories.– One-time, fixed, or up-front costs.– Ongoing investment costs.– Recurring costs.– Compliance costs.– Mitigation measures.

Measuring Costs

Valuing indirect costs.– Flat overhead figure.

• Does the project actually costs increased administrative burden?

– Costs to the private sector.

– Valuing the use of capital.

– Valuing the damage effects of government programs.

– Other cost issues.• Sunk costs.

• Interest costs.

Analysis of Benefits and Costs

Framework.– Retrospective.– Snapshot.– Prospective.


Importance of using present value.– Choice of an appropriate discount rate.

• Private sector rate.

• Low social discount rate.

• Long-term treasury bill rates.

– Adjustment for inflation.


Presenting the results.– Net present value (B-C).– Benefit cost ratio (B/C).– Return on investment (discount rate to reduce

present value to zero). Appropriate perspective.

– Costs and benefits vary across stakeholders.– May conduct analyses from several

perspectives.


Sensitivity analysis.– Use spreadsheet analysis to vary the assumptions.

Intangibles.– Relate intangibles to dollar results: if there are net

costs, do the intangible benefits overcome the deficit?

Particular problems.– Equity concerns (weighting of values).

– Multiple causation and co-production problems.


Lecture 8d – How to Deliver Oral Testimony Based on a Written Statement

Introduction

Goal:– To speak authoritatively and to answer

questions responsively in public deliberation.

Objective:– Skill of writing speakable text, skill of speaking

easily from written text, and readiness to answer anticipated and unanticipated questions.

Introduction

Scope:– Pinpointed topic pertinent to hearing’s purpose and the

witness’s role.

Product:– Two expected communication products:

• Short oral summary.

• Full written statement.

Strategy:– Confident and useful public testimony resulting from

advance preparation.


Lecture 8c – How to Inform Policy Makers in a Briefing Memo or Opinion Statement

Introduction Goal:

– Recognition of meaningful information in a mass of details and representation highlighting the significance of information for a user.

Objective:– Skills of distilling, listening, recording, observing,

evaluating sources, relating details to context, interpreting details accurately in context, and selecting details according to relevance; capability of stating informed opinion that is aware of and responsive to other opinions.

Introduction

Product:– One- to two-page written memo, or one- to-two

paragraph written statement, possibly with attachments.

Scope:– Only essential topics in an identified context to

target a specific information need.

Introduction

Strategy:– Why is this communication necessary?– What is the subject?– What is the purpose?– What will this communication do? What can

happen as a result?– What is the context?– What is the situation of reception?– Why are you providing the information?

Task #1. Develop the Information

From meetings.– Attendance, notes, handouts, statements.

From varied sources.– Publications and materials– Consultation.

From informed reflection and analysis.– Update, summarize, consult.

Task #2, Write the Memo or Statement.

Chapter 2 method. Quick comprehension and ready use. Choose the right presentation. Stationery, header, overview, etc.

Developing a Testimony

Present concisely and responding credibly to questions.

Know the context. Know your message. Know your role. Know the communication situation. Rehearse your delivery.

Task #1. Write the Testimony

Use the method in Chapter 2 for both the oral and written forms.

Write out your oral summary, even if only talking points.

Everything in an oral testimony is public record.

Task #1. Write the Testimony Template.

– Title page or header to identify organization and witness, the agency holding the hearing, the topic, the date, and the location of the hearing.

– Greeting to thank the organizers for the opportunity to testify and to state why the topic is important to the witness.

– Message to state the main information the testimony provides.– Support (evidence, grounds) for the message.– Relevance of the message to the hearing’s purpose.– Optional: discussion or background to add perspective on the

message (only if relevant).– Closing to conclude the testimony and invite questions.

Task #2. Write the Full Statement

Might use the same organization as the oral summary.

May be longer, more details, and include appendices.

Must be well organized to ensure use.

Task #3. Present the testimony.

Summarize. State the message clearly and emphatically. Stay within time limits. Listen. Answer credibly.

Task #3. Present the testimony.

Questions-and-answer.– Listen to the questions of other witnesses.– Make sure you hear each question correctly.– Answer the question that is asked.– Stop when you have answered the question.– Do not lie or invent information.– Handle the following situations carefully:

• You are asked your personal opinion.• You don’t know the answer.• Your credentials are challenged or your credibility is

attacked.


Lecture 2c – APA Editorial Style

Punctuation Period.

– Use a period to end a complete sentence (also abbreviations, quotations, numbers, and references).

Comma.Use a comma– Between elements (including before and and or) in a series of

three or more items.• The height, width, or depth.

– To set off a nonessential or nonrestrictive clause, that is, a clause that embellishes a sentence but if removed would leave the grammatical structure and meaning of the sentence intact.

• Switch A, which was on a panel, controlled the recording device.

Punctuation

Comma (contd.)– To separate two independent clauses joined by

a conjunction.• Cedar shavings covered the floor, and paper was

available for shredding and nest building.

– To set of the year in exact dates.• April 18, 1992, was the correct date.

• But, April 1992 was the correct date.

– To separate groups of three digits in most numbers of 1,000 or more.

Punctuation Comma (contd.).Do not use a comma

– Before an essential or restrictive clause, that is, a clause that limits or defines the material it modifies. Removal of the clause would alter the meaning.

• The switch that stops the recording device also controls the light.

– Between the two parts of a compound predicate.• The results contradicted Smith’s hypothesis and indicated

that the effect was nonsignificant.

– To separate parts of measurement.• 8 years 2 months.

Punctuation

Semicolon.

Use a semicolon– To separate two independent clauses that are

not joined by a conjunction.• The participants in the first study were paid; those in

the second were unpaid.

– To separate elements in a series that already contains commas.

• The color order was red, yellow, blue; blue, yellow, red; or yellow, red, blue.

Punctuation Colon.Use a colon Between a grammatically complete introductory clause

(one that could stand as a sentence) and a final phrase or clause that illustrates, extends, or amplifies the preceding thought. If the clause following the colon is a complete sentence, it begins with a capital letter.– For example, Freud (1930/1961) wrote of two urges: an urge

toward union with others and an egoistic urge toward happiness.– They have agreed on the outcome: Informed participants perform

better than do uninformed participants.

Punctuation

Colon (contd.).Do not use a colon– After an introduction that is not a complete

sentence.• The policy alternatives included

The status quo, which reflected the current policy choices,

Alternative A, which required direct intervention, and

Alternative B, which required indirect intervention.

Punctuation Dash

– Use a dash to indicate only a sudden interruption in the continuity of a sentence. Do not overuse.

• These two alternatives—reducing benefits and disqualifying recipients—significantly reduced the size of the program.

Quotation marks– Use double quotation marks– To introduce a word or phrase used as an ironic comment,

as slang, or as an invented or coined expression. Use only the first time cited.

• Considered “normal” behavior.• The “good-outcome” variable . . . The good-outcome

variable.

Punctuation

Quotation marks (contd.)– To reproduce material from a test item or

verbatim instructions to participants.• The first question was “what is your gender?”

Use italics and not double quotation marks– Identify the anchors of a scale.– To cite a letter, word, phrase, or sentence as a

linguistic example.– To introduce a technical or key term.

Punctuation Parentheses

Use parentheses– To set off structurally independent elements

• The patterns were significant (see Figure 5).

– To set off reference citations in text.• Kingdon (2003) suggests

– To introduce an abbreviation• The Federal Emergency Management Agency (FEMA)

– To set off letters that identify items in a series within a sentence or paragraph.

• The policies include (a) welfare policy, (b) energy policy, and (c) defense policy.

Punctuation

Parentheses (contd.)Do not use parentheses– To enclose material within other parentheses

(use brackets to enclose material within parentheses).

• (the Department of Housing and Urban Development [DHUD]).

– Back to back.• (e.g., policy learning; May 1990).

Punctuation Brackets

Use brackets– to enclose parenthetical material that is already within

parentheses.• (The results for the control group [n=8] appear in Figure 2.)• Exception: do not use brackets if the meaning is clear

using commas.– Not (as Imai [1990] later concluded)– But (as Imai, 1990, later concluded)

– to enclose material inserted in a quotation by someone other than the author.

• “when [the author’s] words are quoted” (Dummy, 1995, p. 151).

Punctuation

SlashDo not use a slash– When a phrase would be clearer.

• Not: Smith acted as a supervisor/mentor.

• But: Smith acted as a supervisor or mentor.

– For simple comparisons. Use a hyphen or short das (en dash) instead.

• Test-retest reliability

• Not: test/retest reliability.

Spelling Merriam-Webster’s Collegiate Dictionary is the standard

spelling reference for APA journals and books. The more comprehensive version is the Webster’s Third

New International Dictionary. Plural forms of Latin or Greek originSingular PluralAppendix appendicesCannula cannulasDatum dataMatrix matricesPhenomenon phenomenaSchema schemas

Spelling

Hyphenation– Use the dictionary to determine the use of

hyphens in compound words.• Follow-up is a noun or adjective, but follow up is a

verb.

• If a compound is in a dictionary, it is considered a permanent compound (e.g., high school, caregiver, and self-esteem).

• Spelling can also change (life-style became lifestyle; data base became database).

Spelling General principles of hyphenation

– Do not use a hyphen unless it serves a purpose. If a compound adjective cannot be misread, do not use a hyphen.

• Grade point average.• Health care reform.

– In a temporary compound that is used as an adjective before a noun, use a hyphen if the term can be misread or if the term expresses a single thought (all words modify the noun).

• Different-word lists (lists of different words).• Different word lists (different lists of words).

– Most compound adjective rules are applicable only when the compound adjective precedes the term it modifies. If it follows the term, do not use a hyphen.

• Client-centered advice.• But: the advice was client centered.

Spelling

General principles of hyphenation.– Write most words with prefixes as one word;

however, there are exceptions.– When two or more compound modifiers have a

common base, this base is sometimes omitted in all except the last modifier, but the hyphen is retained.

Capitalization Words beginning a sentence

– The first word of a complete sentence.– The first word after a colon that begins a complete

sentence. Major words in titles and headings

– Not conjunctions, articles, or short prepositions, but all words four letters or longer. Capitalize all verbs, nouns, adjectives, adverbs, and pronouns. When a capitalized word is hyphenated, capitalize both words. Capitalize the first word after a colon or dash in the title.

– Major words in article headings and subheadings.– Major words in table titles and figure legends.– References to titles of sections within the same article.

Capitalization

Proper nouns and trade names.– Proper nouns and adjectives and words used as

proper nouns.– Names of departments if they refer to a specific

department.– Trade and brand names of drugs, equipment,

food, programs, etc.– Do not capitalize names of laws, theories,

models, or hypotheses (except retain uppercase in proper names).

Capitalization

Nouns followed by numerals or letters.– On Day 2 of Experiment 4.– Do not capitalize nouns that denote common

parts of books or tables followed by numerals or letters.

Titles of tests– Capitalize complete, exact titles of published

and unpublished tests.– Do not capitalize shortened, inexact, or general

titles of tests

Italics

Use italics for– Titles of books, periodicals, and microfilm

publications.– Genera, species, and varieties.– Introduction to a new, technical, or key term or

label (do not italicize after the first use).– Letter, word, or phrase used as a linguistic

example.– Words that could be misread.– Periodical volume numbers in reference lists.

Italics

Do not use italics– Foreign phrases and abbreviations (ad lib, et al.,

per se, vis-à-vis.– Greek letters.– Mere emphasis.

Abbreviations Use abbreviations sparingly.

– Do not overuse because it creates confusion.– Do not underuse. If you introduce an abbreviation, and only use it

two or three times subsequently, you are better spelling it out in all cases.

Explain the abbreviation the first time, and use the abbreviation subsequently.

Some abbreviations are in dictionaries. They can be used without explanation.– IQ, REM, ESP, AIDS, HIV, NADP, ACTH.

Use the standard Latin abbreviations only inside parentheses. Spell out the English equivalent in the main text (e.g., use and so forth for etc.).

posc 431 – public policy analysis introduction to public policy analysis

Documents

introduction eightfold

introduction policy

public policy analysis

policy problems

introduction iteration

diagnostic problem definitions

results of policy analysis

analytic concepts