PRODUCTIVITY IN PUBLIC ADMINISTRATION AND PUBLIC POLICYCharles R. Wise

Indiana University

The United States is now going through a stage in whichgovernment in general and public administration in particularis placing a high priority on accountability. We may lookback at some point and view the current period as the "ageof accountability." One aspect of this concern that hasreceived increased attention in public administration hascome to be referred to as the productivity movement(Newland, 1976).

The purpose here is to delineate the meaning of produc-tivity for public policy processes and to raise some of thepolicy questions that are implied by this thrust in publicadministration. In order to do this, there will first bepresented a definition of the concepts involved. Second,there will be a discussion of the development of efforts tothis point. Third, there will be a discussion of expectedroles for productivity analysis in policy formation. Fi-nally, questions posed by this movement for prospectivepublic policies and for the functioning of policy processeswill be raised.

T1.HE CONCEPTS. As is apparent from the literature and fieldreports (for example see ICMA's Interjurisdictional Guide)"Productivity" has come to mean a number of ideas and activi-ties. However, there is a thread of concensus runningthrough the movement about the essentials concepts and there-fore the concerns implied (Wise and McGregor, 1976). Pro-ductivity as a management improvement movement is associatedwith the twin concepts of efficiency and effectiveness. Asefficiency it is concerned with relating the outputs ofpublic programs and organizations to the resources utilizedto produce these outputs. As effectiveness it is concernedwith the quality of the output or service being delivered orthe impact on society of the output. Public administratorsare warned against a singular emphasis on either efficiencyor on effectiveness (Hatry, 1976; Thomas, 1976). Produc-tivity improvement and productivity measurement are not thesame; however a tenet of the movement is that various typesof measures can be useful in providing the basis forproductivity improvement decisions.

The measurement component of the productivity movementincludes attention to both efficiency and effectiveness.There are several types of efficiency measures including:productivity measurement--measures of end product or serviceintended for use outside the organization in relationship toinputs; work measurement—measures of intermediate productsor services related to labor hours; work standards—measureof time a task should take; degree of utilization of facili-ties, equipment, or employees, and of manhours involved inproducing end products.

The measurement of effectiveness has also developed inseveral ways meaningful to government. One parallel movementto that of productivity concerns itself with effectiveness interms of whole societies, communities, or regions. This ofCourse is the social indicators movement (Bauer, 1966; DHEW,1969; Plessas and Fein, 1972).

97

Another level of effectiveness may be conceived of bythe impact of particular agencies or service efforts. Crimerates or crime clearance rates, or the degree of streetcleanliness would be examples. Program evaluation is re-lated to effectiveness at another level. Performance indica-tors do not indicate the extent to which a particulargovernment program was responsible for the observed changesas distinguished from nonprogram factors. Program evalua-tion seeks to separate out program effects from externaleffects to provide this insight. A final thrust of themeasurement of effectiveness is involved with the determina-tion of citizen perception and satisfaction. The concernhere is that a significant dimension of government programeffectiveness goes beyond those impacts intended by adminis-trators to the quality dimensions perceived by citizens.

Productivity unlike some other management improvementefforts in government has not envisioned any particulardecision-making process but assumes that information gainedcan serve as an input into various types of decision proc-esses. This can be a failing in that management analystshave long learned that the best of information can bedefeated by organizational structures, political processes,and behavioral factors. Knowing this many of the profes-sionals working in the field emphasize the necessity ofdealing with organizational, political, and human factors asa prerequisite for success in productivity improvement ef-forts (Balk, 1975; Gaciala, 1975; Huneke, 1975; Shallman,1975; Weiss, 1970; Porter, 1975).

CURRENT ACTIVITIES. A relevant question at this point is howfar along are we in developing measures that give insightinto effectiveness and efficiency? A definitive attempt atthis question is beyond the scope of this effort, but someindications may provide an inkling of the extent of develop-ment. On the federal level, the Joint Financial ManagementImprovement Program—a joint effort by the General AccountingOffice, Office of Management and Budget, and Civil ServiceCommission--has surveyed federal agencies and has reportedproductivity measurement covering 1.7 million federal civil-ian employees or about 60 percent of the total civilian workforce. For the years 1967-70 they reported a productivityincrease for the federal government as a whole of 11.7 percent or an average annual increase of 1.8 percent for anestimated savings of two billion dollars (JFMIP, 1974).These efforts constitute the aggregation of productivitydata over many agencies without making any claims of com-pletely solving the measurement and comparability problemsinvolved. However, given the volume of activity the trendanalysis would seem to provide some insight on a macro basisthat exceeds the measurement error. The effort has also ledto efforts to encourage the development of better measure-ment procedures under the auspices of the JFMIP.

Effectiveness information has also been gatherecj througha variety of program evaluation and other data collectionactivities. However, one study of federal evaluation ofsocial programs reported that their most impressive findingabout evaluation in the federal government is that substan-tial work in the field has been almost non-existent. Asecond finding was that few significant studies had beenundertaken and that there is nothing akin to a comprehensive

98

federal evaluation policy (Wholey, et al., 1970). Anotherstudy of three hundred and eighty-two federally sponsoredevaluation studies found that impact measurement was includedin only about three-quarters of evaluation studies and thatabout one-half of the studies that do measure impact aredeficient either in design, sampling, or validity of measures(Bernstein, 1975, p. 97).

Reasons for this include: (1) lack of methodologicalknow-how, (2) difficulty in securing cooperation from pro-gram personnel, (3) problems in developing specificationsand guideiines for evaluation research, (4) lack of fundsdedicated to evaluation, (5) problems in the communication ofresults from researchers to program personnel, (6) no sys-tematic evaluation policy, (7) failure to spell out objectives(Wholey, et al., 1970; Guba, 1969; Bernstein, 1975).

While there is no executive branch evaluation policythere has been some activity on the legislative side. TheLegislative Reorganization Act of 1970 (PL91-510, 91stCongress) directed the General Accounting Office to reviewand analyze the results of government programs which re-sulted in standards for program results auditing promulgatedby GAO (Comptroller General, 1972). Secondly, Title VII ofthe Congressional Budget and Impoundment Control Act of 1974authorizes Congressional Committees to run their own evalua-tion activities or to require agencies to conduct evaluationsand report results to them. At the same time it expandedGAO's role (Public Law 93-344, 93rd Congress).

In summary, on the federal level, there has been atleast some success in gathering efficiency data in terms ofmacro productivity trends and more detailed efficiency datawithin a few federal departments. With respect to effec-tiveness the record is even more spotty with several observ-ers calling for a much more systematic approach and afederal evaluation policy that includes central agency(probably OMB) leadership and technical assistance to stateand local governments (Wholey, et al., 1970; Bernstein, 1975;Jones, 1974).

At the same time as state governments have been rapidlyassuming new responsibilities because of the decentralizationof the new federalism, they also have been experiencingdeclining revenues. This has forced retrenchment and re-evaluation of programs, and at least some state executiveslook to productivity improvement as an essential survivalstrategy (Lucey, 1975). With respect to efficiency measure-ment one study that examined state budget documents andsurveyed state budget agencies found that productivityindices (data that reflect the percent change in the rela-tionship between inputs and outputs for agencies over time)were nonexistent. The researchers did find that an increas-ingly used type of efficiency measure for internal managementpurposes was performance efficiency in terms of work stan-dards. The same study found little effectiveness measurementin terms of degree of post-program client improvement, clientsatisfaction, percent of total relevant need fulfilled, acces-sibility of services or impact on major groups or subdivisionsof the state. Those areas where at least some informationWas available included vocational rehabilitation, human re-sources training, and elementary and secondary education(Urban Institute, 1975).

99

Local governments have also been experiencing signifi-cant fiscal retrenchment and some have looked to productivityimprovement in order to maintain services without increasingtaxes. Local governments engage more in direct delivery ofservices than do state governments and would seem to providefertile ground for productivity measurement and improvement.Hatry of The Urban Institute concluded that the currentstate of productivity measurement for local government ispoor--both at the individual and local government level andat the national level. Many local governments do not sys-tematically keep workload data on many of their servicesand fewer report productivity indices. Effectiveness mea-surement is also quite rare (Hatry, 1973). Nonetheless,several municipalities in recent years have launched exten-sive productivity measurement programs and a variety ofmeasures are in use (Urban Institute, 1974). And researchis proceeding to develop not only multiple measures oflocal services but also multiple modes of measurement. (Seefor example Ostrom, 1975.)

There is not room here to go into all the difficultiesof measurement incurred in these efforts. While considerabledevelopment is underway it is fair to say that the currentstate of the art is in the initial stages. Measurement dif-ficulties facing governments include problems in determiningwhat is to be measured, especially quality aspects; guardingagainst the perverse effects of measurements; trade-offsbetween the need for simplicity of measurement (understand-ability) and complexity (perspective); the need for multiplemeasures of services; the need for multiple modes and assess-ment of their costs; and the need to more adequately defineinputs and link them to specific outputs (Hatry and Fisk,1972; Wise and McGregor, 1976; Ostrom, 1976).

PRODUCTIVITY AND POLICY FORMATION. At this point it isnecessary to ask what role we can expect productivity, inthe sense defined here, to play in policy formation in theUnited States. Its effect on agencies at all levels ofgovernment and their relationships with elected and appointedpolitical executives, legislators, and diverse publics is yetto be determined. Should we expect it will influence govern-ment at all?

It is clear government faces renewed calls for demonstra-tion of its accountability. In democratic societies ends andmeans are considered grist for the public mill and thereforeit is necessary to consider accountability in two senses—(1) process accountability and (2) outcome accountability.Of course the two are intertwined.

The drive for process accountability has concerneditself with legislation and administrative regulations con-cerning conflicts of interest, legislation regulatingcampaign financing, the development of codes of ethics, pro-visions for public disclosure of agency information andaccess to official policy deliberations--sunshine laws; andnumerous mechanisms and requirements for citizen participa-tion at various levels of policy decision processes. Threeassumptions underlie such efforts: (1) in themselves suchefforts are democratic and therefore the right or just wayto conduct government; (2) such efforts will serve as acheck on any tendencies of public officials engaging inillegal or immoral behavior; and (3) such efforts are likely

100

to result in public services that are more effective, i.e.fulfill critical public needs.

The drive for outcome accountability concerns itselfwith the need for empirical demonstration of the actions ofpublic agencies and individual public servants. It assumes(1) measurement of agency activities will reveal validimpacts of agency outputs and the efficiency with which out-puts were produced that resulted in those impacts; (2) suchmeasurement will lead to an increase in the knowledge ofagency operations and their consequences by participants inthe governmental process including political executives,legislators, managers, and citizen groups; (3) an increasein knowledge of the consequences of agency operations willresult in increase in the levels of agency outputs alongwith improvements in the quality of these outputs in ful-filling public needs. Both process and outcome accountabil-ity assume that increases in the effectiveness of agencyoutputs will result in an increase in citizen support forgovernment. It should be pointed out however that there isno assurance that they are compatible in a given context.Steps taken to make a decision process more participativeand thus more accountable may well result in less efficientprovision of services.

It is clear that productivity may be measured at dif-ferent levels in government that speak to different kinds ofdecisions. Examples of such levels include:

A. Public resources expended for all services at alllevels of government—i.e., total government sectorproductivity.

B. Public resources expended for all services by aparticular government body—e.g.. Federal Governmentproductivity.

C. Public resources expended for a single service bya particular governmental body, e.g., city inspectionservices productivity.

D. Public resources expended by a single agency, e.g.,highway department productivity.

E. Public resources expended for a single service bysome sub-unit of an agency, e.g., solid wastecollection productivity.

F. Public resources expended by a work group, e.g.,collection crew productivity.

G. Productivity of individuals.

It might even be desirable to determine productivity at sev-eral of these levels including the expenditure of privateresources. Statements of the value of different approacheshave appeared.

There are also different types of decisions at eachlevel that could be aided by such information, assuming forthe moment the information could be obtained. For exampleat level D—a single agency--decision areas include thedetermination of the success of management initiatives and

101

management capability, the desirable distribution of rolesamong jurisdictional departments, and the determination ofpriorities among departments.

It is thus essential that the type of measurementprogram be developed on the basis of the types of decisionscontemplated and the relevant policy process surroundingthose types of decisions. The problem is that data developedfor one type of decision at one level may not be easilyaggregated or disaggregated for use in another type ofdecision at another level. For example evaluation of localprojects m.ay not be useful for program or multiproject deci-sions (Williams, 1971, p. 87). One of the implications ofthis is that participants at various levels in publicorganizations with different interests are relied upon tocollect data which has little payoff for decisions at theirlevel.

It is also clear that the information obtained in pro-ductivity analysis will constitute but one type of input intoa particular policy process. Proponents of such effortshypothesize that information on the efficiency and effective-ness of public programs elevate policy deliberations, but weare made all too aware of other components of policy proc-esses that condition the impact of such informational inputs—political feasibility for example is one such consideration(Meltsner, 1972; Weiss, 1970; Adams, 1975).

Not only do the nature of existing policy processescondition the interpretation of such information but theyalso determine the ability of governments to obtain it inthe first place. This brings us to a key point. Produc-tivity measurement and improvement including both considera-tions of effectiveness and efficiency potentially can providea useful focus for policy formation. That focus is toorganize and operate public programs in such a way that theiractivities and effects can be revealed for purposes of policyreformulation.

In part this interest in measurement springs from areaction to large scale public programming that has gener-ally been thought to have had marginal effects and in whichthere has been little empirical evidence collected in pro-gram implementation to suggest the reasons why. Researchersin the field have questioned the assumptions of action pro-grams that are developed and organized as if they werecertain to succeed with no feedback mechanism to indicateneeded adaptions to improve efficiency or effectiveness whenthey did not succeed (Suchman, 1970; Campbell, 1969; Rossi,1966).

It is argued that programs could have been designed toproduce information on their effectiveness, but they werenot (Rivlin, 1971). The solution offered by these practi-tioners is a systematic experimental approach to publicpolicy information in which new programs or methods aredesigned for specific impacts and in such a way that welearn by empirical methods the degree to which they weresuccessful and the reasons why in order to provide a basisfor retention, imitation, modification, or elimination(Campbell, 1969; Rivlin, 1971; Suchman, 1970).

A way to implement an experimental approach would be toanticipate the policies and programs that will be considered

102

in the future. Many policies enacted have considerablehistories. The idea would be to continually launch aseries of experiments with the understanding that only somewill be successful (Bernstein, 1975, p. 138). Given thatsome programs will have built up such political supportthat full scale implementation is likely to proceed anyway,another approach may be to build into program designs, atleast for a sample of the operative cases, the mechanismsnecessary to produce productivity information. This ofcourse is dependent on how serious policy bodies and over-sight agencies insist on meaningful program design initia-tives in this regard and not just accept tacit adherence topaper requirements. With the decentralization of many domes-tic programs to state and local governments, federal agencieshave formulated extensive evaluation and planning require-ments. For some of these it is clear that the planningrequirements have not been taken seriously. It will beinteresting to see how the cabinet agencies and the GeneralAccounting Office treat the evaluation requirements.

A potential danger in this focus is that the emphasison evidence may be used as an excuse for disparaging all newideas and retrenching on any commitments to the disadvantaged,i.e., lets not do anything until the information on produc-tivity tells us the program is perfected (Williams, 1971;p. 129; Moorehouse, 1972). Decision-making must foreveroccur under conditions of imperfect knowledge. The offeringof this focus is to reduce the degree of uncertainty and noteliminate it. The decision to pursue policy in this mannerfor a particular policy area involves a tradeoff between thepotential degree of uncertainty reduction and the opportunityCosts involved. In each policy area we can expect to hearproponents for both sides of the issue. Some programs willprobably receive priority on equity grounds alone regardlessof efficiency or effectiveness at least for a time. Thequestion remains how much equity can be realized withoutefficiency and effectiveness?

The potential implications of this productivity policyfocus are significant. It places the responsibility onprogram designers and program managers not only to allocatepriorities to given problems and groups but to design theirefforts in such a way that they can empirically demonstratethe results of their activities, not only for their own usebut also for others in the policy process. It adds a dimen-sion which purely distributive politics lacks. The questionis not who gets what but how efficiently and effectively dothey get it. It also implies a renewed emphasis and capabil-ity for government accepting a responsibility for contributingto the productivity of the economy of the nation (Peterson,1972). The degree to which the government contributes to ordetracts from national productivity offers a focus for policydebate and a new responsibility for elected officials.

Public administration has witnessed other movements thathave not really had much impact on policy formation. It hasbeen charged one of the reasons for this is that these ini-tiatives have proceeded from false premises of the policyprocess. One factor that quickly enters the question of whatrole productivity can be expected to play in policy formationis the degree of rationalism implied and involved. There isno intention here to replay the incrementalism--comprehensiverationalism debate. However, there is clearly an element of

103

rationalism involved here. The question arises, how muchand to what effect?

The fate of Planning-Programming-Budgeting (PPB) in thefederal government for example, has been widely debated withfailure (Schick, 1973) and success (Worthley, 1974) bothclaimed. One reason given for failure was that PPB attemptedto superimpose an all encompassing rationality onto aresource allocation process essentially bargaining at basein which expression and representation is given to diversepolitical interests which might be neglected if budget choicewere centralized in the hands of analysts (Wildavsky, 1966).

In viewing productivity analysis and program evaluationmight not one ask "are we going down the PPB road again?"Clearly some think we are. The idea is presented that in thepluralistic political system bargaining does indeed charac-terize the decision process and program evaluation is anothertool of scientific decision-making which shares an emphasison efficiency. As such it is proposed it will have anuneasy time in a world that gives high priority to valuesother than efficiency and effectiveness (Poland, 1974).Another objection is that the design of organizations isconstraining. The proposition is offered that organiza-tional structure implies stability and generates commitmentto programs and clients while evaluation suggests change andspeaks to the relationship between action and objectives(Wildavsky, 1972).

It is true that productivity analysis and programevaluation are tools of scientific decision-making and theymust be viewed not as a panacea but as providing one set ofinputs into pluralistic bargaining processes. In addition,success in utilization does depend on the realities of organ-izational dynamics. However, these dynamics are not the samefor all public agencies and do not necessarily act to thedetriment of productivity analysis and program evaluationefforts. Not all agencies are characterized by ambiguoustask structures, irregular information flows (Quinn, 1975),ambiguous objectives, or high degrees of conflict.

In addition, unlike the across-the-board implementationof PPB in the federal government there is no single methodor procedure implied that seeks to revolutionize decision-making arrangements. There is rationalism here but it isnot all encompassing. Secondly while organizational dynamicsdo condition the implementation and utilization of such anal-yses, it does not necessarily follow that these dynamics willdefeat them. If it did, we would be admitting that any sci-entific aid to decision-making would be doomed at inception.The fact is that organizations and programs do change formany reasons (Kaufman, 1971). Productivity analysis does notseek to supplant political decision processes but tosupplement them.

IMPLICATIONS FOR POLICY AND POLICY PROCESSES. In approachingthis task it is necessary for the implementers to address therealities of those processes. These involve several implica-tions for public policies as well as implications for policyprocesses. The answers to these questions will greatlyaffect the role productivity analysis will play in policyformation.

104

Questions of policy suggested by the productivitymovement include:

(a) should efficiency and/or effectiveness measurementbe applied to all public programs.

(b) should evaluation be conducted by staff withinagencies or by outsiders.

(c) should the federal government take the lead andestablish a national policy on evaluation.

(d) should evaluation be confined to the use of harddata or should softer techniques with largerjudgement factors also be used.

(e) what should be done about administrative practicesthat limit productivity improvement but addressother values.

(f) what are the implications of productivity measure-ment for training, promotion, job mobility, employeedislocation and other human resource policies ofgovernment agencies.

In addition there are a number of questions regardingthe relationship between productivity and the nature ofpolicy processes in the United States:

(a) Can we develop ways in which to conduct informedpolicy debates utilizing evaluative data in publicdecision forums.

(b) Should the emphasis of the evaluative informationbe on distinguishing the successful from the unsuc-cessful or should the emphasis be on guiding develop-mental change within programs. If the latter canparticipants in policy processes in the United Statesrefrain from making success-failure judgementsprematurely.

(c) Can the timing of evaluative research be reconciledwith decision cycles in policy processes.

(d) To what extent can sufficient agreement be reachedon objectives of public programs and can these beclearly operationalized for measurement purposes.

(e) What incentives can be provided for productivityimprovement and can disincentives be altered.

(f) What are the implications of productivity measure-ment for the capacity of clients and citizens toparticipate in policy decisions.

There are obviously a number of contending forces inthe policy processes operating in the United States whichwill determine the answer to these questions. Whatever theparticular outcome it is clear that an added informationaldimension will be added to policy debates in many forums.That dimension is the contribution of public policy to pro-ductivity as well as the contribution of productivity toimprovement in public policy.

REFERENCES

Adams, Harold J., "Solutions As Problems: Tbe Case of Productivity,"Public Productivity Review, Voi. 1 No. 1, (1976) pp 36-43.

Balk, Walter, Improving Government Productivity: Some Policy PerspectivesBeverly Hills, Calif.: Sage (1975).

Bauer, Raymond A., Social Indicators, Cambridge, Mass.: M.I.T. Press(1966).

105

Bernstein, Ilene and Howard Freeman, Academic and Entrepreneuriai Research:The Consequences of Diversity in Federal Evaluation Studies. New York:Russel Sage (1975).

Campbell, Donald, "Reforms as Experiments," American Psychologist, Voi. 24No. 4, (1969) pp 409-420.

Comptroller General of the U.S., Standards for Audit of GovernmentalOrganizations, Programs, Activities, and Functions. Washington, D.C:U.S. Government Printing Office (1972).

Gaciaia, Jonathan S., "Gommentary on Productivity in State Government" inC Wise and E. B. McGregor eds. Productivity and Program Evaiuation.Bloomington, Indiana, Midwest Intergovernmental Training Committee(1975).

Guba, Egon, "The Faiiure of Educationai Evaiuation," Educational Tech-nology. Vol. 5, No. 5, (1969) pp 29-38.

Huneke, Richard, "Productivity in the State of Michigan," in C Wise andE. B. McGregor eds. Productivity and Program Evaluation. Bloomington,Indiana, Midwest Intergovernmental Training Committee (1975).

Hatry, Harry, "Approaches to Productivity Measurement and Program Evalua-tion," Pubiic Productivity Review Vol. 1, No. 3, (1976) pp 21-28.

Hatry, Harry P., "Issues in Productivity Measurement for Local Govern-ments," Pubiic Administration Review, Vol. 32, No. 6, (1973) pp llb-lSh.

Hatry, Harry P. and Donald Fisk, The Ghallenge of Productivity DiversityWashington, D.C: Nat. Technicai Information Service (1972).

International Gity Management Association, Jurisdictional Guide to Produc-tivity Improvement Projects, Washington, D.C (1975).

JFMIP, Joint Financial Management Improvement Program, Report on FederalProductivity: Productivity Trends by 1967-1973, Vol. 1 (1974).

Jones, Harry H., "Evaluation Bases and Management," Public AdministrationReview. Vol. 35, Speciai Issue, (i975) pp 737-742.

Kaufman, Herbert, The Limits of Organizational Change. University,Alabama: Univ. of Ala. Press (1971).

Lucey, Patrick J., "The Need for Productivity in Government" in C Wiseand E. B. McGregor, eds. Productivity and Program Evaluation.Bloomington, Indiana, Midwest Intergovernmental Training Committee(1975).

Meitsner, Arnold J., "Political Feasibility and Policy Analysis," PublicAdministration Review Vol. 32, No. 6, (1972) pp 859-867.

Moorehouse, Thomas A., "Program Evaluation: Social Research Versus PublicPoiicy," Pubiic Administration Review, Vol. 32, (1972) pp 868-874.

Newland, Chester A., "Policy Program Objectivies and Federai Management:The Search for Governmental Effectiveness," Public AdministrationReview, Vol. 36, No. 1, (1976) pp 20-27.

Ostrom, Elinor, Measuring Urban Services: A Multi-Mode Approach.Bloomington, Indiana. Workshop in Political Theory and Poiicy Analysis,Indiana University (1975).

Ostrom, Elinor, "Multi-Mode Measures: From Potholes to Police," PublicProductivity Review Vol. 1, No. 3, (1976) pp 59-66.

Peterson, Peter G., "Productivity in Government and the American Economy,"Pubiic Administration Review, Vol. 32, No. 6 (1972).

Plessas, P. S. and Ricca Fein, "An Evaluation of Sociai Indicators,"American Institute of Planners Journal, Vol. 38, (1972) pp 43-51.

Poland, Orville F., "Program Evaluation and Administrative Theory," PublicAdministration Review, Vol. 34 (1974) pp 333-338.

106

Porter, Elsa A., "Commentary on Implications for Trainers and PersonnelManagers" in C Wise and E. B. McGregor, eds. Productivity and ProgramEvaluation. Bloomington, Indiana, Midwest Intergovernmental TrainingCommittee (1975).

Quinn, Robert E., "Management Information Systems, Task Structure, andOrganizational Effectiveness" in Walter Balk ed. Administering StateGovernment Productivity Improvement Programs. Albany, N. Y.: S.U.N.Y.(1975).

Rivlin, Alice M. , Systematic Thinking For Social Action, Washington D.C,Brookings (1971).

Rossi, Peter, "Boobytraps and Pitfalls in the Evaluation of Social ActionPrograms," in Carol Weiss ed. Evaluating Action Programs. Boston:Allyn and Bacon, (1972) pp 224-235.

Schick, Allen, "A Death in the Bureaucracy: The Demise of Federal PPB,"Public Administration Review, Voi. 33, pp 146-156.

Shallman, William, "Commentary on Implications for Trainers and PersonnelManagers" in C Wise and E. B. McGregor, eds. Productivity and ProgramEvaluation. Bloomington, Indiana, Midwest Intergovernmental TrainingCommittee (1975).

Suchman, Edward A., "Action For What? A Critique of Evaluative Research"in The Organization, Management, and Tactics of Social Research,Richard O'Toole, ed. Cambridge, Mass,: Schenkman (i97O).

Thomas, John, "Government Accountability—For What?" Public ProductivityReview. Voi. 1, No. 2, (1976) pp 2-7.

The Urban Institute, The Status of Productivity Measurement in StateGovernment: An Initial Examination, Washington, D.C. (i975).

Tbe Urban Institute and I.CM.A., Measuring the Effectiveness of BasicMunicipal Services. Washington, D.C, ICMA (1974).

U.S. Department of Health, Education, and Weifare, Toward a Social Report,Washington, D.C: U.S. Gov't. Printing Office (1964).

Weiss, Carol H., "The Politicization of Evaluation Research," Journal ofSocial Issues. Vol. 26, No. 4 (1970) pp 57-68.

Wboley, Joseph S. et al̂ . , Federal Evaluation Policy: Analyzing the Effectsof Public Programs. Washington, D.C: The Urban Institute (1970).

Wildavsky, Aaron, "The Political Economy of Efficiency: Cost-BenefitAnalysis, Systems Analysis, and Program Budgeting," Public AdministrationReview, Vol. 34 (1966) pp 333-338.

Wildavsky, Aaron, "Tbe Self-Evaluating Organization," Pubiic AdministrationReview. Voi. 32 (1972) pp 509-520.

Williams, Walter, Social Policy Research and Analysis, New York: Elsevier(1971).

Wise, Charies, and Eugene B. McGregor, "Government Productivity andProgram Evaluation Issues," Public Productivity Review, Vol. 1, No. 3,(i976) pp 5-19.

Worthley, John A., "PPB: Dead or Alive," Public Administration Review,Vol. 34, (1974) pp 392-394.

107