accountability agenda in higher education

8/6/2019 Accountability Agenda in Higher Education

1/24

Accountability in Higher Education: Bridge over Troubled Water?Author(s): Jeroen Huisman and Jan CurrieSource: Higher Education, Vol. 48, No. 4 (Dec., 2004), pp. 529-551Published by: SpringerStable URL: http://www.jstor.org/stable/4151570 .

Accessed: 07/04/2011 11:01

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you

may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=springer. .

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed

page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

Springeris collaborating with JSTOR to digitize, preserve and extend access toHigher Education.
http://www.jstor.org/action/showPublisher?publisherCode=springerhttp://www.jstor.org/stable/4151570?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/action/showPublisher?publisherCode=springerhttp://www.jstor.org/action/showPublisher?publisherCode=springerhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/4151570?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=springer


2/24

HigherEducation 48: 529-551, 2004.? 2004 KluwerAcademicPublishers. Printed n the Netherlands. 529

Accountability in higher education: Bridge over troubled water?JEROENHUISMAN1& JAN CURRIE21Centeror HigherEducationPolicyStudies,Universityof Twente,P.O.Box 217, 7500AEEnschede,TheNetherlands E-mail: [email protected]); Schoolof Education,MurdochUniversity,SouthStreet,Murdoch,WesternAustralia6150, Australia(E-mail:[email protected])Abstract. This article discusses the impactof accountabilityon highereducationpoliciesin Europeand the United States. We describe how the accountabilitymovementrelates tootherpolicy trends n highereducation,providingempiricaldata on how accountabilitywasimplementedand how academics andmanagers n four universitiesperceivedthese policies.We close the article with a reflectionon the observed shift from professional to politicalaccountability hatuses 'soft' mechanisms that seem to offer little change in the qualityofeducation n thesecountries.Keywords: academics,accountability, ighereducationpolicy,universitymanagement

IntroductionAccountability s on the higher educationpolicy agendain many systems.In a numberof countriesaccountability s institutionalizedand commonlyaccepted,in othersit is a recentphenomenon,andin othersit is a contestedissue on the higher education agenda. Some analysts think that govern-ments and other stakeholdersdo not have the right to make academicsformallyaccountable or theirperformance.To supporttheirview most ofthese analystsrefer to the concepts of academicfreedom and professionalautonomy.Othersbelieve that the increasingattention o public,measurableaccountability s the logical consequence of governmentsretreatingfromclosely monitoringhighereducationandallowingan increase n institutionalautonomy.Moreover,others are preoccupiedwith the intendedand unin-tendedconsequencesof the growingattention o accountability.Given theseconcernsmanyinterestingquestionsariseregardingaccountability.This articlehas five aims. First,we presenta conceptualexplorationofthe concept of accountability.Second, we investigatethe rise of accounta-bility: why andhow has accountability nteredhighereducation?Third,wereview what has happenedat the nationalpolicy level regardingaccounta-bility in France, the Netherlands,the United States and Norway. Fourth,we reporton how accountabilitys actuallyimplementedandperceivedby


3/24

530 JEROENHUISMANAND JANCURRIEstaff members and managersin four universitiesin France (UniversityofAvignon), the Netherlands(Universityof Twente),Norway (UniversityofOslo) andthe United States (BostonCollege).Fifth,we discuss the paradoxemerging romtheempiricaldata:despitegrowingattention o accountabilityat the national evel, at the shop-floor evel staffmembersare to some extentcynical about the ability of currentaccountabilitymechanismsto improvequality.Althoughsome of our respondentsstatedthat accountabilitycouldlead to greater mprovements,heyfelt thatthe currentmechanismswere notvery beneficial.At the extremeend therewere those who disliked externalforms of accountabilitypreferring nsteadto rely on internalmotivationtoimprove heirteachingandwho countedon theprofessional ntegrityof theircolleagues for quality improvement.There was a noticeable gap betweenpolicy rhetoricdemandingharsher,managerial orms of accountabilityandthe lack of its implementationn ourfourcase studies.This is explored n theconclusion.

Conceptual xplorationAnalysts of accountabilitygenerallyagree that it is the "answerability orperformance"Romzek2000, p. 22) or "theobligationto report o others,toexplain,tojustify, to answerquestionsabouthow resourceshave been used,and to what effect" (Trow 1996, p. 310). Both Romzek and Trowsupple-ment these definitionswith the question:who is to be held accountable, orwhat, to whom, and throughwhat means?Trow (1996) also questionstheconsequences(see Wagner 1989) for a similarapproach,andKogan(1986)fora slightlydifferentmethod).

Romzek(2000) offers the most comprehensive ramework or analyzingtypes of accountabilityrelationships.She identifies four basic types: hier-archical, egal, professionalandpolitical.The last two are the types thataremore often found in highereducationcurrently. n some countries therehasbeen a movement from professionalto political accountabilityas nationalgovernmentsbegin to 'steer froma distance'(Kickert1991;Marceau1993),allowinginstitutionsgreaterautonomyat the same time as making hemmoreaccountable.Professionalandpoliticalaccountability ystemsreflectsituations"wherethe individualor agencyhas substantiallymore discretionto pursuerelevanttasksthanunder egal or hierarchical ypes.And thereviewstandards,whenthey are invoked,are much broader" Romzek2000, p. 25). Romzek notesthat the difference between professional and political accountability is thesource of the standard for performance. "Professional accountability systemsare reflected in work arrangements that afford high degrees of autonomy to


4/24

ACCOUNTABILITYN HIGHEREDUCATION 531individualswho base theirdecision-makingon internalizednormsof appro-priatepractice"(2000, p. 26). Political accountabilityrelationshipsaffordmanagersthe discretionor choice to be responsiveto the concerns of keyinterestgroups, such as elected officials, clientele groups,and the generalpublic.Trow (1996) adds to Romzek's frameworkby more explicitly pointingto the functions of accountabilityand more specifically focusing on thehigher educationcontext. Regardingthe functions, he first maintainsthataccountabilitys a constrainton arbitrary ower,therebydiscouraging raudand manipulation,and strengthening he legitimacyof institutionsthat areobligatedto report o appropriate roups.Second, accountabilitys claimedto sustainor raise the qualityof performanceby forcing those involvedtoexaminetheiroperations riticallyand to subjectthemto criticalreviewfromoutside.Third,accountability anbe used as a regulatorydevice through hekindof reportsandtheexplicitandimplicitcriteria o be metby thereportinginstitutions.Accountability in higher education: Where does it come from?Why has therebeen an increasedemphasison accountability n the 1980sand 1990s in both Europeand the United States? Here we explore someglobalizingpracticeswhich have led to this increase and the shift towardsmore public, political accountability.A number of writers (Henry et al.2001; Rhoades and Sporn2002; Vidovich 2002) have arguedthata globalmodelof qualitypolicyin highereducationhasemerged hroughprofessionalmechanisms,such as annualconferences and the internationalcirculationof professionals,as well as through he influenceof internationalorganiza-tions, such as the Organisationor EconomicCooperationandDevelopment(OECD) and the InternationalNetwork of QualityAssuranceAgencies inHigherEducation(INQAAHE).Rhoades and Sporn convincingly demon-strate he diffusionof qualityassurancemodels betweentwo 'core' regions,the USA and Europe.Qualityassurance n Europeas in most partsof theworldemergedin the 1990s whereas it existed in a varietyof forms in theUSA since the late 1800s. However,the global model thatis emergingandinfluencingmost countriesdiffersto some extent from the accreditationofinstitutionscommon in the USA. It resemblesmorethe qualityissues intro-ducedat the statelevel by stateboardsandlegislaturesemphasizing"qualityreviewprocesses ... in the context of strategicmanagement ffortsto refocusinstitutions"(Rhoades and Sporn 2002, p. 361). These forms of qualitymanagementhave been takenfrom business and the federalgovernment ntheUSA and are related o the efficientandeffectiveuse of publicresources.


5/24

532 JEROENHUISMANAND JANCURRIESimilar rends whichhadalreadyexisted in the USA for at leasttwo decadespriorto the 1990s beganimpactingon highereducation n Europe.However,as much as it is clear that "the cross-Atlanticand intra-European atternsof influenceare evident in the professionaldiscourse of highereducation"(RhoadesandSporn2002, p. 369), therearealso local differencesandresist-ance on thepartof some Europeanpolicy makers o followingthe Americanmodel. Furthermore,here are countriesthat are takingthe lead in Europe,such as Englandand the Netherlands,and developing their unique brandsof qualityassurancesystems. In coining the term 'glonacal' MarginsonandRhoades state that "at every level - global, nationaland local - elementsand influences of other levels arepresent" 2002, p. 289). Thereare policynetworksat every level that are influencingthe creationof quality agendasthat are similarto each other,yet have unique attributes hat significantlyrelateto theirgeographicand historicalcontexts. At the sametime, thereareoverridingnfluences hatdeterminemajorshifts in highereducationpolicies.The ideological shift towardsthe New Right led to greaterprivatizationofhighereducationand was a majorinfluencebringingmarket orces to bearon universities. In addition,the following global trendsinfluencinghighereducationsystems from the 1980s to the presenthave affected the type ofqualityassuranceprograms andthusaccountabilitymechanisms)establishedin differentnationalsystems(see also e.g., De Boer et al. 2002; WorldBank2002).

* Changing relationshipsbetweengovernmentsand universities: n mostsystems there was a relativelystrong bond between governmentandhigher education institutionsthrough funding, legislation, and plan-ning mechanisms.However,governmentshave retreatedand openedthe arena for greater autonomyand market mechanisms(Gornitzkaetal. 1999). In this context, Neave's (1988, 1998) analysis of develop-ments in WesternEurope s revealing.He points to the strikingchangefrom ex ante governmental ontrolby legislationandprocedures o expost justificationby qualityassuranceandaccountabilitymeasures.Thisdevelopmentwasparticularly isible in WesternEurope n the 1980sandin Centraland EasternEurope n the 1990s.The changeoccurred arlierin theUnitedStates wherepublicpolicies combinedwithmarketmech-anismsparticularlyn the 1970sand 1980swhenpoliticalaccountabilityovertook professionaljudgment in universitiesas the quality mech-anism.Accordingto Trow(1996), accountabilitybeganto replacetrustin professional ntegrity n the USA during hisperiod.

* Efficiencyand value or money:A related,yet autonomousdevelopmentis the growingtrendof governments o documentvalue formoney.Thisis partlydue to the massificationof highereducation around he world


6/24


7/24

534 JEROENHUISMAN AND JAN CURRIEwill differdependingon the historicalcontext andthe way nationalgovern-ments decide to implementaccountabilitymechanisms and how they areapproaching lobalizationas a neo-liberaleconomicideology.

Accountability in national contextsOnthebasis of ananalysisof developmentsn thefourcountries see Currie tal. 2003 for moredetails),we observedthataccountabilityandglobalizationwere particularlyvisible in policies that stressed the importanceof highereducation n its competitiverole, thatis, supporting he nationin the globaleconomy.Thischallengesnationalgovernments o keepa close watch on theeffectiveness and efficiency of highereducation nstitutionsandmakethemmoreaccountable.Ourconclusionsfor universitiesarerathersimilar to those of Leithwoodet al. regardingaccountabilityatthe school level:"The currentpreoccupationwith educationalaccountabilityappearsto have begun in most developedcountries n the 1960s, acquiringsignificantnew energy duringthe mid-to-late 1980s. Thereasonsforthese calls forgreateraccountability,urthermore,areto be found in the widereconomic,political,and social context of whichschoolsareapart.Thesecontexts arenot uniformacross all countries" 1999,p. 11).There re omenoteworthyifferencesmonghecountrieseexamined.In Norway, ewpublicmanagementdeasandthecountry'sositionntheEuropeanoliticalandscape,hat s partof Europe, ut not of theEuropeanommunity,ere riggersor ntroducingreater ccountability.The government ntended to introduceaccountabilitymechanisms,such asactivityplanningand qualityassurance;however,concurrently he govern-mentdecentralizednumber f activities,hus nitiating reaternstitutionalautonomy. saresult,heaccountabilityolicieswerestrippedf their hornsduringhe mplementationrocessand npractice ecameess effectualhanthegovernmentplannedDutchuniversities lso gainedgreater utonomyn a number f areas.This generaltrendrequirestwo qualifications.First, the governmentmain-tained ts powerover a number f aspectsof highereducation. econd, nsomeareas t was not so mucha questionof moreor less autonomy utof shifting responsibilitiesand accountabilitymechanisms,in other words,'steeringfroma distance'.The most importantdevelopment n practicewasthe introductionof a nationalqualityassurancesystem. The originalinten-tions of the governmentseemed far-reaching.But, similarto the Norwegiancase and due to the Dutch corporatistmodel of policy making, the actualimplementationf accountabilityurned uttobe moremodest.


8/24

ACCOUNTABILITYN HIGHEREDUCATION 535Theface of accountabilityn theUnited Stateschangedover time.Broadlyspeaking,therewas a shift froman internally-oriented ystem of accounta-bilityaimedatimprovementowardsanexplicit, externally-orientedne. Thereasons for the change lie significantly n the fact that the costs of highereducationgrewenormouslywithconsequencesfor nationaland statebudgetsand the generalpublic's view thathighereducationwas not deliveringvalueformoney.Thesechangesclearlysawa movement romprofessiornalo polit-ical accountabilityas universitiessoughtto gain greaterpublicapproval ortheirquality.The French higher education system is still largely controlledby theMinistry,despite some developmentstowardsderegulationand decentrali-zation. The efforts of the government o implementformal accountabilitymechanisms were not accepted wholeheartedlyand turned out to focusmainlyon monitoringdevelopments n highereducation.The qualityassur-ance through he ComitwNational d'Evaluation(CNE) was introducedas a

voluntary cheme and therehavebeen few consequencesfor the institutionsinvolved,otherthanincreasing heirprestigeandreputation.

Accountability in institutional contextsWenow take a further tepin our nvestigationof accountabilityby lookingathow it was implementedwithinfouruniversities.Wedescribe he impactthatnationaldebatesandpolicies actuallyhadon dailypracticesandon theviewsof academics and administrators,ummarizing he main findingsaccordingto three central themes/questionswe discussed with the interviewees:theaccountabilitymeasuresatthegovernment nduniversity evels,the measuresat the individual evel, and theeffectivenessof these measures.SampleWegatheredqualitativedatathroughn-depth nterviewswithacademicsandmanagers romfourdiverseuniversitiesn France,theNetherlands,Norway,and the United States. It is important o emphasize that this researchwasnot a strictlycomparativestudy because we did not control the sample ofinstitutionsor the interviewrespondents o enable a statisticalorexplanatorycomparisonof ourfindings.However,we chose a similarset of participantsneachof theuniversitiesandtheywereasked a similarset of twentyquestionsaboutgovernance,accountability, ompetitionandgenerating unds,and newtechnologies,ending with a few questionsregarding he role of tenureandthe futureof the university.Thus,this allowedus to observethe similaritiesand differencesin the trendsthatexisted during 1998 and 1999 across the


9/24

536 JEROENHUISMAN AND JANCURRIEfouruniversities.In this article,we focus on the threequestionsaskedaboutaccountability.Therewere 131 interviewswith a small numberof seniormanagersandan approximately qualnumberof academic staff fromprofessionalschools(education, applied languages, and/or law), sciences, and social sciences(arts).The interviews were conductedface-to-face, almost entirelyby oneof theauthors, nsuringconsistencyin questioninganddepthof probing.Thesample ncluded37 individuals romBoston,32 fromAvignon,31 fromOslo,and 31 from Twente. The academics interviewedranged from professorsto assistantprofessors, consisting of more men thanwomen, particularlyatAvignonandTwente,with a moreequalrepresentationt BostonCollegeandOslo. The seniormanagers nterviewed ncludedpresidents,vice rectors/vicepresidents,provosts,anduniversity ecretaries/registrars.We chose particularuniversitiesthat were not representativeof univer-sities in their countries but were chosen to representdifferent types ofuniversities(large/small; capital-based/provincial; ublic/private;research-oriented/teaching-oriented;managerial/collegial). Boston College is amedium-sized (12,500 students), private, Jesuit university located in theUnitedStates,which becamehighlymanageddue to a brushwithbankruptcyin the early 1970s. The Universityof Avignonis a small (7,100 students),provincial, iberal artsuniversity ocatedin southernFranceand is mainlyanundergraduatenstitution.TheUniversityof Oslo is a large(34,400 students)university ocated in the capitalof Norway and is mainlyresearch ocused.TheUniversityof Twente s a small(5,500 students),entrepreneurialniver-sity located in the easternpartof the Netherlandsandcombinestechnologyandsocial sciences. Oslo andTwentedetermine heirfutures,having greaterautonomythanAvignon, yet not as much as Boston College with its needandcapacityto garnerprivatefunds.However,Twente s less dependentongovernmentfunding, becoming an 'entrepreneurial' niversity by buildingup money from researchand consultingcontracts.Avignon and Oslo staffmembersare beginning to embrace and serve local communityeconomicinterests, yet their traditionalnature and reliance on governmentfundingremains ntactmore thanBostonCollege orTwente.Studentsat Oslo did notpaytuitionfees; atAvignon,they paida few hundreddollars n administrativecosts;atTwente hey paidabout$1,500;and at Bostontuition ees were about$21,500 (US dollarequivalents n 1999).Accountabilitymeasures nforce at universityanddepartmentevelHere respondentswere asked to mention the accountabilitymeasures,forexample research indices, quality reviews, and teaching evaluations thatwereintroducedby the government o monitoruniversitiesanddepartments.


10/24

ACCOUNTABILITYN HIGHEREDUCATION 537The Universityof Avignon seems the least preoccupiedwith accountabilityrequirements y the government.A smallnumber 16 percent)of responsesindicated hat thereis no controlor monitoring n place anda little less thana third(29 percent)of the responsesstated thatthere is no change or onlydebates akingplace on the issue of accountability.n sum,almosthalf of theresponsesstated hat there was a lack of accountabilitymechanismsor a lackof changeother thanthe monitoringof studentnumberswhichwas seen as atraditionalorm of accountability.

I would say that the structuresof evaluationhaven't really changedmuch.Asfor your examples,I amnot reallyconvinced hattheMinistertakesmuchnotice of thepass rates;howeverthepercentageof studentsenrolled,yes, evidently.But the only thingthat this is usedfor is so thattheMinistercan establisha budgetarynotation to allocate creditsto theuniversity, utafterthis has beendone,we are theones whodecidewhatwe aregoing to do with the unds and whetheror notwe aregoing to cutcertaincourses or keep them. So the Ministerin a way rids himselfofthis responsibility.OtherwiseI wouldsay that in this area, nothinghasreally changed.Therehas been a changein theMinister'sdiscourse,achange of methods,but not a change in the proceduresof evaluation.(Avignon,Junior,Male,Academic,ProfessionalSchool)About half (47 percent)of the responsesindicatedthataccountabilityexists'out there' but no immediateeffect was noticeable.Most of the elementsofaccountability,or instancemonitoringof studentchoice, pass rates,requiredqualificationsof academics,andperformance elatedfundingfor smallpartsof the university budget seemed relativelyharmless.Whereasthe generaltendency in the responses was that accountabilitymechanisms were notnecessary,a few responses(8 percent)mentionedthatthereshouldbe someexternalscrutinyfrom the government.The following quote identifiesthattheevaluation hould be on the contentof thecourse,not on the performanceof the professor,which seems to be an importantdistinctionthatthe Frenchwould want to maintain.

Weare in the middle of an important ransformationat the moment,which is based on the Englishand Americansystems.Up untilnow,wehad no studentcourseevaluationsat all, andIfeel thatthiswas missingfromoursystem.Thestudentsshouldbe allowed to evaluatethecontentsof theircourses, (as opposedto evaluatingor makingcriticismsof theactualprofessors)and to makean evaluationof theoverallwayinwhichtheir choiceof subjecthas beenpresented o them theyshouldbegivenan opportunity o say which areas of the course could be improved,or what needs to be added, etc. I think that this is a very good idea,


11/24

538 JEROENHUISMAN AND JANCURRIEhoweversome of mymoretraditionalist olleagues have a very negativeattitudetowardsthis kindof thing, saying that it wouldundermine heirauthorityand prestige, which may I add, I thinkis quitefar from thetruth. Avignon,Senior,Male, Academic,Social Sciences)

Boston College could be positionedat the other end of the spectrum.Itmustbe stressed thatalthough he university,being private, s not monitoredby the statelegislature,74 percentof the responsesindicatedthataccounta-bility is all aroundthe place, mainly by externalreviews. Nevertheless,anumberof responses ndicated hatexternalreviews were a fairlyrecentmoni-toringdevice, introduced o improvethe performanceof some departmentsandused to rewardothers.Theuniversity uts togetheran evaluationor thedepartment s a whole.Thereare sticks and carrotswith respectto monitoring.Theuniversityhasjust come out with an award or teaching or faculty witha littlecashprizeof $4000, notmuch.It'smorerecognition hana monetaryreward.Ourperformance s measured ndividuallybutalso departmentalwise.In the university'sopinion,are we allocating too manyresources to thegraduateprogramversusundergraduate?Do we have enoughelectivesonthebook?(BostonCollege,Senior,Male, Academic,SocialSciences)

In additionto the externalreview mechanisms,15 percentof the responsesnamedexplicitly the internalscrutinyof class sizes and other elements ofthe educationalprocess.The attentionpaidto accountabilitydoes not alwaysmeanthatrespondentsareseriously'bothered'by accountability.Theacademic vicepresidentand deans do not seem to monitordepart-ments all that directly.Thereis an annual report that'sput in, but Idon'tknowexactlywhathappensto these.It goes up to thedean, and Ihave neverhadany eedback.(BostonCollege, Senior,Male,Academic,ProfessionalSchool)

Most of the responses in this category,however,acceptedexternalreviewsand statedthat these were helpful for improvingtheirprogramsand proce-dures.Also a numberof responses ndicated hat there weredepartmentshatwere hardlyevaluatedor held accountable o managementat a higherlevel.In addition to the accountabilitymeasuresdescribedabove, some responses(6 percent)related o nationalrankingsof theiruniversity.This should notbeliterallytaken as a directaccountabilitymeasure,for externalorganizationsuse universitydatato rank nstitutions,ndirectly mplyingthattheuniversitycould be asked to explainor justify its performancevis-&-vis ts regulatorybodies (in this case, its Board of Trustees).Others (5 percent)mentioned


12/24

ACCOUNTABILITYN HIGHEREDUCATION 539directactionby thedepartmenthair(exhortationso do better)ordismissinga departmenthair f reportswere very negative.The Universityof Twente and the Universityof Oslo took middleposi-tions between Boston College and the Universityof Avignon. At Twentealmost all respondents 78 percent)referred o externalreviews of differentaspects, teachingand researchseparately,by nationalvisiting committees.The compulsoryparticipationn these quality assuranceprocesses impliesthat each study programfor education and each department/facultyorresearchwould write a self-evaluationand a peer review committeewouldvisit the program o assess the reportsand makes recommendations n areasthatmayneedimprovement.

Theuniversity sfairly well set up in termsof qualityassuranceproce-dures and accountabilityprocedures.The aculties are reviewedtwiceyearly,andall thatis done in termsofperformance ndicators,reviews,student evaluations- everythingyou can imagine. (Twente, Senior,Male, Academic,Social Sciences)A number of responses (15 percent) mentionedthe funding mechanismsthatpartlytook into accountthe performanceof the universitybasedon thenumberof graduatesat the Masterslevel, time to complete a degree, andnumberof PhDsgranted.A small number 7 percent)of theresponsesrelatedto the internalmonitoringpractices of the university,partly as a prepara-tion for the nationalquality assurancesystem, and partlyas a preparationfor obtainingaccreditation.At the time of the survey,this was a voluntaryactivity,soon to becomepartof the obligatoryaccreditationmechanisms.At the University of Oslo, most responses (57 percent) reportedtherole of completionratesin accountabilityproceduresand about a third(31percent)mentionedthe annualproductivity orms. A minorityof responses(7 percent)mentioneddiscipline reviews. It was clear from responsesthatcompletionrates and annualproductivityreportingpracticesdid not lead tosevere consequencesin terms of the budget, as these indicatorsdetermineonly a smallpartof the budget.

Completionratesof students were introduced everalyears ago by thegovernment, o part of thefunding is related to creditpoints. It is stillnot a majorelement,but it is there.Research indices are still ratherprimitive.It is essentially a question of publications and the numberof doctoralcandidates we produce.It's not sophisticated,and so far itis not specificallylinked to the budgetsor any rewardsor punishment.Quality reviews are done not on an extensive, regularbasis, but thegovernmentasks the Research Council to conductperiodic reviewsonthe state of a discipline.It is done on a national basis. And teaching


13/24

540 JEROENHUISMAN AND JANCURRIEevaluations,again there has been a consistentpush I would sayfromthe Ministry,even supportedby prizes, for those who do this the waytheformer Minister would like it to be done. So yes, again there arereally no heavy sanctionsfor those who don't use these evaluations.Thepressureso far has been essentiallyto use course evaluationsona regularand a fairly systematicbasis. The resultsof the evaluationshave ew consequences or those involved.Asyet, there s no mechanismfor translating he resultsof these reviews nto decisionsaboutbudgets.(Oslo, Senior,Male, Academic,Social Sciences)

There is a push to pay attention o quality monitoringand qualityreviews,but there are no mechanismsfor translating he results of the reviews intodecisions about budgets. Some parts of the university appear to take theaccountabilitymechanismsseriously.There s a lot of evaluation,and that's new. Wedidn't have that at allmanyyears ago, and ofcourse,I think hatit is quiteuseful.Wehavehadinternationalcommitteesevaluatingour research.And as to teaching,this institutehas been doing thatfor a longer time than other insti-tutes. Thestudentorganizationpicks out a couple of different eachingunits every semester to evaluate. Togetherwith the teacher in chargetheychoose the questions,and one of the students s given some moneyto do the statistics. I thinkit worksquite well. (Oslo, Senior,Female,Academic,Sciences)

Given the fact that many interviewed at the University of Oslo were notawareof all the mechanisms n placeto monitor heiractivities,the accounta-bilitypracticesseem ratherceremonialwithoutanyreal sanctions thatmightthreaten he survivalof departments.Accountabilitymeasures nforce at the individual evelAlthoughthereis some overlap n the responseson accountabilitymeasuresat the organizationaland the individual evels, we decided to discuss themseparately.At the University of Oslo, teaching evaluations (43 percent of theresponses)and annualreports 40 percentof the responses)were mentionedthe most as accountabilitymechanismsat the individual level. For manyrespondents, hereappears o be some frustrationxpressedabout heamountof time thevariousreports ake and the amountof overlap n demands orthesameinformation.Wehave, of course, now an increased amount of reporting:that yourecordyour plansfor teaching or the term,then after each term,then


14/24

ACCOUNTABILITYN HIGHEREDUCATION 541after each year, you report what you have been doing. How muchteachingyou have done. Whatkindof researchyou have carriedout.Also all sorts of publications and things like that. This is new to theuniversity. t is not somethingthathappened n that ormat earlier on.And the head will have a conversationwith each memberof the staffonce a year (Oslo,Senior,Male, Academic,ProfessionalSchool)

Eleven percentof the responsesrelated to the internationalpeer reviews oftheirresearch, uchas grantapplicationsand ournalarticles.A smallnumberof responses (7 percent)regardeduniversity prizes and dialogue meetingsbetween centraladministration nd the faculties as othermonitoringmech-anisms. The lattercan easily be interpretedas 'soft' monitoring,that is,discussingproblemsandraising possible solutions;the former s morediffi-cult to directlyconnectto accountability.Oneshould nterprethementionofprizes,however,as a mechanism hroughwhich individualsare not so muchheld accountable,as rewardedon the basis of a comparisonof their meritsor achievements.In sum, the following quote suggeststhatqualityis not somuchthe game as quantityat theUniversityof Oslo.

Maybewe shouldcall it countability,because it is alwaysa questionofquantitynot quality.Theyare countingteachinghours,articleswritten,conferencesattendedandprojects planned. We use quite a lot of timeto report about what we are doing. (Oslo, Senior, Male, Academic,ProfessionalSchool)At Boston College, 62 percentof the responses referred o the annualreviews,built on the evaluationof each individual'steaching,research,and

communityservice.Everybodyhas to submitan annualreport.In collaboration, hedepart-mentchair;deanandassociate deanscrutinize hat annualreport.So welook at thepublications,the teachingevaluations,and the service.Andthe assumption s that everybodywill be given a small raise, and thenon top of thatsomewherebetween one and maybeup to twopercentinmeritpay can be added to thatdependingon ourjudgment.Sofacultymembersare aware thattheyare expectedto be productive.I thinkthecriteria or productivityarefairly well known. We endto relyon somequantitativemeasuresacross the board and the chaircan add the voicethat looks at qualityas well. (BostonCollege, Senior, Male, Academic,ProfessionalSchool)A minority of responses (13 percent) mentioned mentoring programs andpeerteaching evaluations and another 13 percent mentioned tenure reviews. With


15/24

542 JEROENHUISMANAND JAN CURRIErespect to the latter:those without tenurego throughthe tenuringprocess,as has always happened raditionally, ndthose who aregoing forpromotionto full professorare scrutinized n greatdetail. There is now also discussionof post-tenurereviewsat Boston College, which could involve sanctionsforthosewhoseproductivitys belowthe normor whose teaching s evaluatedasbelow average.A small proportion 10 percent) identified no formal eval-uation mechanisms,suggesting some discrepancybetween departmentsatBoston College. Others arenot particularly atisfied with the feedbacktheyget as individuals.

I don't thinktheyare very effectiveat all. The sanctionor rewardof asalary increase is I thinkeffective.It will tellyou prettydirectlywhattheinstitution hinksof you, but the lackof anykindofformativeevaluationI thinkshows and the institutionwould be betteroff if it did a better obof that.(Boston,Senior,Male, Academic,ProfessionalSchool)At the University of Twente, four direct methods of monitoring theperformanceof academicswere mentioned.Overa third(35 percent)of theresponsesreported hat annual individual reviews were used without sanc-tions, 28 percentof the responses mentionedteaching surveys, 20 percentmentionedannualreportsgiven to the department hair,and 13 percentofthe responses identifiedannual reviews with bonuses, task reassignments,or performanceassessment plans. Only four percent mentioned indirectmeasures,such as counting he numberof PhD studentsandformula unding.

Wehave some system of personal interviewsevery year linked to theannual report.Theremay be some sort of taskreassignment,and somedepartmentsdon't seem to haveany system.This is a majorpolicy issuethat the board of the universitywants to deal with, and it is infavorof a systematicapproachof academic managementby the dean andthe department hairs.This would enabledepartmental hairs or groupleaders to implementa system of interviewseveryhalf a year and linktheseto an assignmentof tasks and resultsof the tasks, ookingbackandlooking orward. (Twente,Senior,Male,Manager)

The personalinterviews were criticized for their lack of usefulness, thatis,theydidnotresult n salary ncrementsor evenpromotionunless anacademicasked for specific advice in this regard.Thus the annual review meetingswere not usually used to discuss career development. Regarding the teachingsurveys, respondents ndicated that these were mostly appliedto first yearcourses and that the focus was to a largerextent on the courses and theprograms ather hanon the individualacademicparticipatingn aprogram.A


16/24

ACCOUNTABILITYN HIGHEREDUCATION 543good exampleof a rather ffectivemethod of evaluatinga courseis describedby a female academic n the sciences.

Ourfavourite type of evaluation is the oral method, evaluating thecourse witha smallgroupofstudents.Wehaverepresentatives f eachofthe studentprojectgroups,say about ive or six students,andone of mycolleaguesandI meet with themabout once everymonthor so. Becausethis way whensomething s going wrong you can immediatelychangethings. Wepreferthatto having the writtenevaluationafter the coursebecause then the changes can only be made the next year. (Twente,Senior,Female,Academic,Sciences)

At the Universityof Avignon,thepatternwas similarto thatof accounta-bility at the organizationalevel. Altogether68 percent said there was noevaluation;althoughout of this percentage,20 percentreportedthat therewere discussionson this issue andevaluationsmay occurin the future.Thislatterresponse s expressed n the followingquote:For the moment,thereare none at all, but I knowthat there is talk ofestablishing accountabilitymeasuresduring the new reformand thatthese measures will include student evaluation of courses. This hasbeen plannedfor sometimethis year; but up until now, nothing hasmaterialized. Avignon,Senior,Male,Academic,Sciences)

Some felt there was a need for indirect and/or informal evaluations(15 percent), and clearly some actually felt that some type of individualaccountabilitymeasures 17 percent)would be of benefitto the university.

I wouldverymuch like to show the inspectorsa piece of paper withalist of all my accomplishments, ll thethingsthatI amproudof, becausenobody takes into account what actually happened during the year.Quality goes unnoticed n this system/qualitys sacrificed or quantity.Figures are used as an indicationof quality.(Avignon, Senior,Male,Manager,ProfessionalSchool)

How effectiveare these measures?Here the respondents were asked to react to the question of whetherthe accountabilitymechanisms in place were effective in either moni-toring qualityor improvingthe qualityof teachingand researchwithin theuniversity.Do accountabilitymeasuresimprovethe qualityof teachingandresearch?


17/24


18/24

ACCOUNTABILITYN HIGHEREDUCATION 545to research or mentorshipbut thereare some good teachingavenues.(BostonCollege, Senior,Female,Academic,Sciences)

No one felt thatthesalary ncrementswereenoughto really changebehavior,butreceivinga negativeincrementor evenjust an average ncrementwouldsignal disapprovalof performanceand may effect a change. Teachingandresearchawardswerealso seen as beneficial.Onlyoneresponse ndicated hattryingtojudge the effectivenessof these measureswasreallytoo difficult.At theUniversityof Twente here is a morepositive pictureof the impactof accountabilitymeasures.Slightlyover half (54 percent)of the responsesindicated hatthe measureswereeffective.Yes,I thinktheydo havepositive influencesin the long run. I've beenworkingat the university or twentyyears, and in the beginningtherewerefew monitoringactivities in research or in teaching.And now Iwould say that these monitoringactivities have improvedthe qualityof teaching and research. (Twente, Junior, Male, Academic, SocialSciences)

A little over a third(38 percent)of the responses showed that there weredoubts about the effectiveness of accountabilitymeasures.The commentsmadewereabout he lackof effectivenessof thecurrentmechanisms,becausethey did not thinkthatthese mechanismsreally changedthe motivationtoresearch,andthey believed thattheremightbe othermechanisms hatcouldhavegreater mpact.It is a difficultquestionto answer,because the qualityof education hasfor sure improved n the last decade and researchthe same. But I'mnot sure if themajor impactof this improvements due to the measureswe havebeendiscussingorpressure rom outside,suchas internationalcompetition or funding, national competition or studentsand so on.(Twente,Senior,Male,Manager)

A few responses n thiscategoryalso doubtedwhether hequalityof researchor theeffectivenessof teachingcould be assessed.A small number 8 percent)of the responsesexplicitlyindicated hat the mechanisms n force neededtobe changed.They suggestedintroducingmorecollaborative,non-individualmeasures,peerreviews,andmoreformativeevaluations.Giventhelackof accountabilitymechanismsat theUniversityof Avignon,people responded to a general question of whether they thought thesemeasures could potentially be effective in improving the quality of teachingand research in their university. About a fifth (22 percent) of the responsesimplied a positive response. They argued that the university needed to be


19/24

546 JEROENHUISMAN AND JANCURRIEmoreefficientandopen-minded o the scrutinyof externaldemands.Overahalf (56 percent)of theresponsesweredoubtful,but in a positivesense. Theyqualified heirresponsesby sayingthat nternalmotivationwasalsonecessaryandthatthere was a riskif assessmentwouldonly take into accountquantita-tive indicators..n this categoryof responses,therewere also academicsandadministratorsndicatingthatit would be difficultto develop accountabilitymechanisms.

I believe that academics should have the responsibilityor evaluatingtheir own work;it is part of their ob. But it is difficult o knowexactlywhich mechanisms should be put into place to make sure that thisprocedure is put to use effectively,and that it actually does becomepart andparcel of their role in the university. Avignon,Junior,Female,Academic,ProfessionalSchool)Somearguments elated o the choice of criteriaor whatspecificallyshouldbeevaluated.A minorityof responses(14 percent) ndicated hataccountabilitywould not improveacademicperformance.A further8 percentfelt thattheydidnotknowenoughtocomment.Insum,somemightsuggestthat here s ananarchistic endencyat theUniversityof Avignon.If this is morewidespreadthanjust this university, hen it will make it difficult for the government oimpose 'hard'accountabilitymechanismson academics.Thesolutionmaybeto implement softer' mechanisms hataremoreformative,usingpeerreviewsandvoluntary eachingevaluations.

DiscussionWhen we look at the type of mechanisms n use, both at the organizationalandthe individual evel, it is striking hatmost mechanismswould becategor-ized as 'soft' measures: monitoringand explanation,and a few of themwould be described as strongmeasures: ustification.Regarding he effectsof accountabilitymechanisms,manyacademicsin this studywere skepticalabout heeffectivenessof currentmeasures.Therewas some opposition o thebureaucratic rocedures, he amountof work involved and the focus on quan-tifiableindicators.Manyrespondentsdoubtedwhether heprocedureswouldindeedhave the presumed mpact.Across the fourcountries,many respond-ents arguedfor less formal, more individualisticprocedures,and pleadedfor a culture in which informal procedureswere accepted as part of theworkingenvironment.On the otherhand,therewere also arguments romaminorityof respondentswho wanted o introduceaccountabilitymechanismsthatrewardedgood practicesandpunished ow qualityperformance.The responses clearly were influenced by the contexts in which theaccountabilitydebatetook place and the experiencesgainedwith accounta-


20/24

ACCOUNTABILITYN HIGHEREDUCATION 547bility.The four universitiesapparentlywere in differentstages of develop-ment. Boston College and the Universityof Twente seemed to be the twoinstitutions n which accountabilitymeasureswere to an extentinstitutional-ized, althoughthe Boston case showed considerablevarietyby department.Anotherdifferencewas thefactthat n theDutchcase, theuniversity ollowedthe nationalquality assurancerequirements,whereas in the United Statescase, therewas much more varietyin proceduresused by the departments.Aside fromregionalaccreditationagencies in the United States, there is nonationalbody to monitoraccountabilitymechanismsand most universitiessetuptheirown accountabilityprocedures.TheUniversityof Oslo seemedtobe representative f universities n whichaccountabilitywas to some extentimplemented,but at the sametimepracticeswere mostly ceremonial.At theUniversityof Avignonaccountabilitywas least visible, debateswere takingplace,butin practicenot manymechanismswereimplemented. nparticular,thepracticesat the Europeannstitutionswerereflectionsof developmentsatthe national evels.Connectingthe findings at the case study level to those at the nationallevels and the theoreticalframework, here are some interestingrelation-ships. Romzek's (2000) types of accountabilitymost found in professionalorganizationsweretrueof these universitieswith a slightmovementtowardsgreaterpoliticalaccountability.The use of performance ndicatorsby statelevel authorities n the United States - to show 'value for money' illus-trates his shifttowardspoliticalaccountability.Anothertypeof dynamism sshownin most nationalcases that llustrate egitimateauthorities'shiftfrominvokingone type of accountability,proactive by legislative requirements,towards other output-oriented orms, such as accreditation,performance-based funding and reporting performance ndicators.Most accountabilitymechanismsbelonged to the categoryof 'soft' mechanisms.They were notset upto sanction ndividualsor theiractivities.The national and institutional case studies seem to indicate that thechanging relationshipbetween governmentand universities is the mostimportant actor affecting the rise of accountability.The shift in types ofsteering relationshipstowards more institutionalautonomy and to someextent increasingmarketmechanisms invoked accountabilitymechanismsor new types of accountabilitymechanisms. Neither administratorsnoracademicswholeheartedlywelcomedmanyof these.

Conclusion: Are accountability policies failing universities?One of the striking findings was that 'soft' accountabilitymeasureswerefavouredover'hard'measures hatwould involve rewardsandsanctions.Thisimpliesa deviation from the contents of most of the policy proposals(either


21/24


22/24

ACCOUNTABILITYN HIGHEREDUCATION 549controlover the qualityassurancesystemandto 'convince' the governmentnot to collect performance ndicators o gain insight into the qualityof theuniversities Huisman2003).PolicyrhetoricOne interpretationf the subversionof accountabilitymechanismscould bethat hegovernmentpolicies simplyfailed to implementmoreseverepolicies.This wouldnot be surprising, or there aremanyexamplesof policy failuresorpartlysuccessfulpolicies in highereducation CerychandSabatier1986).On the otherhand, it would be surprising,given the varietyof contexts inwhichthepolicies wereformulatedand the varietyof instruments uggested,thatall attemptsacross the countries and institutions o implementharsherforms of accountability.Another explanationcould be that most attention to accountabilityin(government) olicypapers s merelyrhetoric.That s, governmentspleadforaccountabilitymeasuresbut actuallyrefrainfrom enforcingspecific policyinstruments.t is actuallydifficultfor themto monitorwhetherhighereduca-tioninstitutions eallyaccountfor theirperformance.There s also usuallyanemphasison processesrather hanoutcomesandrarelyare sanctionsapplied.Valueof 'soft'mechanismsandprofessionalaccountabilityA third ine of reasoningpoints to themanagementof highereducation nsti-tutionsastheweakest ink intheaccountability hain.Governmentsmayhavebeen successful in putting orwardaccountability olicies, butif institutionalleaders do not 'translate' the policies into institutionalmechanisms,thennothingchanges.There are some indications in the empiricalmaterialthatindeed institutionalmanagementwas hesitantto implementsuch measures.An important xplanation at least in the Europeancontinentalcontext- isthatuniversitymanagements still in its infancy.Thatis, only recentlyhaveinstitutional eadersbeen grantedthe power to really manage their institu-tions. Before the mid-1980s (in some countriesearlier or later)institutionalleadership o a considerableextentimpliedceremonialbehaviorandroutineleadership.Since the 1990s, the roles andfunctions of these leadersbecamemuchstrongerandfromthe late 1990s on, we beganto see a new generationof leaders n power(again: hereareimportantdifferencesbetweenthecoun-tries).These new leadersstill haveto get used to their newroles, whichcouldimplythatstronger eadership s emerging,butonly gradually.Some leadershave grasped the opportunity to lead and implemented far-reaching changesin their institutions. Others have tended to stay in a more traditional role anddecided against managing in an overt and aggressive way. This could explain


23/24

550 JEROENHUISMANAND JAN CURRIEthe ratherweak implementation f accountabilitymeasures n the Europeancountries.Anotherexplanationof the hesitancyof university eaders is not so muchbasedon a lack of managerial kills, but based on a positive readingof theircurrent eadership:purposivelythe managersside-step strongeraccounta-bility measures for softer ones which they believe are more effective inleading na collegialand collaborativemanner.Thustheygaingreaterprofes-sional integrity romtheir staff. In the case of Boston College, which has astrongmanagerialculture and where a rangeof accountabilitymechanismsexist, theremaybe goodreasonswhy the case formorepunitivemeasureshasnotbeenmade.Themanagers heremayhaverecognizedthat'soft' mechan-isms aremore effective in a professionalenvironment.Based on Romzek'stypes,universitiesareplacesof highautonomywhere thekind of monitoringthat would suit them best is one based on professional expertise. This iswhy qualitativeudgmentsmadeby one'speersarepreferable o hierarchicalassessmentsbased on efficiency and hardquantitativendicators that ofteneschewqualitativeudgments.There is alreadyconsiderable udgmentexercised in universities that isdependentuponpeer expertise, orexample npeerreviews of grants,articles,promotionapplications,and teaching awards. There are limits to the typeof managerialaccountability mposed from above. It may be effective forsome type of workers,but for universityscholars 'soft' monitoringmaybe betterbecause they are more often motivatedby intrinsic ratherthanextrinsicrewards.Managers n these fourcase studiesmay be intuitivelyintouch with this idea and have avoided the worst excesses of hierarchicalormanagerialaccountabilityor 'hard'monitoring,which would use quantifi-able performance ndicators and sanctionsthat may not produce the mosteffective results in a universityenvironment.If this explanationholds, theuniversitymanagershavebeenable to builda bridgeover troubled accounta-bility) water,easing the mindsof theiracademics.If the managers'strategyseems effective,policy makersat national evels maywant to heed the adviceof universitymanagersand especially academic researcherswho may bebest placed to assess the type of accountabilitymechanismsmost suited touniversities.

ReferencesAltbach, .G.2001). Higherducation nd heWTO:Globalizationunamok', nternationalHigherEducation3,2-4.Bleiklie,I., Hgstaker,R. andVab0,A. (2000).PolicyandPractice n HigherEducation.ReformingorwegianUniversities.ondon:essicaKingsley.


24/24

ACCOUNTABILITYNHIGHERDUCATION 551Boer, H. de, Huisman, J., Klemperer,A., Meulen, B. van der, Neave, G., Theisens, H.and Wende,M. van der (2002). Academia in the 21st Century.An Analysis of Trendsand Perspectives n Higher Education and Research.The Hague:Advisory Council forScience andTechnologyPolicy.Cerych,L. and Sabatier,P. (1986). GreatExpectationsand Mixed Performance.Stoke-on-Trent:TrenthamBooks.Currie,J., Boer, H. de, DeAngelis, R., Huisman, J. and Lacotte, C. (2003). GlobalizingPracticesand UniversityResponses. Westport:GreenwoodPress.Gornitzka,A.,Huisman,J., Klemperer,A., Maassen, P.,Heffen,O. van,Maat,L. van de andVossensteyn,H. (1999). StateSteeringModels withRespectto WesternEuropeanHigherEducation.Enschede:CHEPS.Henry,M., Lingard,B., Rizvi, F. and Taylor,S. (2001). The OECD, Globalisation andEducationPolicy.London:Pergamon.Huisman,J. (2003). 'Institutional eform n highereducation:Foreverchanges?',in Denters,B., Heffen,0. van, Huisman,J. andKlok,P-J.(eds.), InteractiveGovernanceandMarketMechanisms.Dordrecht:Kluwer,pp. 113-128.Kickert,W.J.M.(1991). 'Steeringat a distance:A new paradigmof public governanceinDutch highereducation'. Presentedat the EuropeanConsortiumor Political Research,

Universityof Essex.Kogan,M. (1986). EducationAccountability.AnAnalyticOverview.London:Hutchinson.Leithwood, K., Edge, K. andJantzi,D. (1999). EducationalAccountability:TheStateof theArt.Gtitersloh:Bertelsmann.Marceau,J. (1993). Steering from a Distance: InternationalTrends n the Financing andGovernanceof HigherEducation.Canberra:AustralianGovernmentPublishingService.Marginson,S. andRhoades,G. (2002). 'Beyondnationalstates,marketsandsystemsof highereducation:A glonacal agencyheuristic',HigherEducation43(3), 281-309.Neave, G. (1988). 'On the cultivationof quality,efficiency and enterprise:An overview ofrecent trends in higher educationin WesternEurope',EuropeanJournal of Education23(1/2), 7-23.Neave, G. (1998). 'The evaluativestatereconsidered',EuropeanJournalof Education33(3),265-284.Rhoades,G. andSporn,B. (2002). 'Qualityassurancen EuropeandtheU.S.:Professionalandpoliticaleconomicframingof highereducationpolicy', HigherEducation43(3), 355-390.Romzek,B.S. (2000). 'Dynamicsof publicaccountabilityn an era of reform',InternationalReviewofAdministrativeSciences66(1), 21-44.Stensaker,B. (1997). 'Fromaccountability o opportunity:Therole of qualityassessments nNorway',Quality n HigherEducation3(3), 277-284.Trow,M. (1996). 'Trust,marketsand accountabilityin higher education:A comparativeperspective',HigherEducationPolicy9(4), 309-324.Vidovich, L. (2002). 'Qualityassurance n Australianhigher education:Globalisationand'steeringat a distance",HigherEducation43(3), 391-408.Wagner,R. (1989). Accountability n education:Aphilosophicalinquiry.New York/London:Routledge.WorldBank(2002). ConstructingKnowledgeSocieties: New Challenges or TertiaryEduca-tion.Washington:WorldBank.

accountability agenda in higher education

Documents