institutional evaluation in the mercosur: transformation or control?

International Journal of Educational Development 20 (2000) 277–286www.elsevier.com/locate/ijedudev

Institutional evaluation in the Mercosur: transformation orcontrol?q

Ana Canen*, Mabel Tarre´ de OliveiraFaculty of Education, Federal University of Rio de Janeiro, Av. Pasteur, 250-Fundos, 22290 — 240, Rio de Janeiro, RJ Brazil

Abstract

In a globalised setting, the centrality of institutional evaluation for quality education and production of knowledge hasbeen emphasised in educational reforms. This paper discusses theoretical concepts and practical implications involved ininstitutional evaluation as developed in contemporary higher education policies in two countries of the Mercosurblock — Argentina and Brazil. In a critical, intercultural framework, it argues that homogenised evaluation practices,which are blind to the sociocultural diversity of the Mercosur countries, do little to improve the institutions’ overallquality of education. From the evidence of the study, some potentialities and limits of institutional evaluation to promoteeducation for equity and transformation are presented. 2000 Elsevier Science Ltd. All rights reserved.

Keywords:Comparative education; Educational policies; Multicultural education; Evaluation; Institutional evaluation; Interculturalperspective

1. Introduction

The formation of trading blocks, among whichthe latest one is the Mercosur (the Common Marketof the South Cone, comprising Argentina, Brazil,Paraguay and Uruguay), is pointed out as one ofthe main trends brought about by the globalisationprocess. Likewise, the centrality of institutionalevaluation so as to guarantee quality teaching,learning and production of knowledge for anincreasing technologised and competitive marketseems to be particularly conspicuous in educationalreforms in a globalised setting, including those car-

q A preliminary version of this paper was presented at theXth World Congress of Comparative Education, Cape Town,South Africa, July 12–17 1998.

* Corresponding author. Tel.:+55-21-295-32-46; fax:+55-21-295-32-46.

0738-0593/00/$ - see front matter 2000 Elsevier Science Ltd. All rights reserved.PII: S0738 -0593(99 )00069-3

ried out in the Mercosur countries in the last fewyears.

The present paper seeks to discuss theoreticalconcepts and practical implications involved ininstitutional evaluation in higher education asdeveloped in the latest educational policies in twocountries in the Mercosur: Argentina and Brazil. Itfocuses on the extent to which the multiculturalnature of the countries of the Mercosur, as well asthe institutional actors’ perceptions and identitiesare taken into account in institutional evaluation ineducational policies in the area. It argues that, eventhough a consensus may be brought about the rel-evance of evaluating educational institutions toenhance their performance in the overall globalscenario, the kind of theoretical concepts and prac-tical implications involved in institutional evalu-ation as conceived in educational policies couldeither enhance or be detrimental to educators’ per-

278 A. Canen, M. Tarre´ de Oliveira / International Journal of Educational Development 20 (2000) 277–286

formance and, as a consequence, to the institution’soverall quality education.

In the context of multicultural countries such asthose of the European Union and the Mercosur,authors such as Grant (1997) and Canen (1995,1997a) point out to the need to take into accountsociocultural plurality and disparities in teachereducation and evaluation. In Argentina and Brazil,these remarks acquire particular significance.Being the most industrialised countries in the Mer-cosur block, Argentina and Brazil share some con-textual differences as well as similarities worthnoting. Extension of territory and demography arethe first to impinge: Brazil occupies nearly one-half of the entire South America, with a populationof around a hundred and fifty million people, beingconsidered the “giant” member of the Mercosur.Argentina is the second largest country in the con-tinent and its population is around thirty millionpeople.

Cultural diversity is prevalent in both countries.In Argentina, even though the predominance ofEuropean descendants occur, a great manyindigenous groups and languages survive, parti-cularly in the Andean region. In Brazil, there canbe found 206 indigenous ethnic groups, each withits own cultural patterns, alongside Africandescendants and a large number of immigrantsfrom several parts of the world. Socioculturalinequality and disparities are also noteworthy. Pol-itically, both countries have recently emerged frommilitary dictatorships and have been trying tostrengthen democratic institutions.

In the face of the above mentioned culturalspecificities just briefly commented on, the role ofhigher education institutions in promoting researchand teaching practices tuned to cultural diversityand imbued by transformative, equity-orientededucational perspectives are crucial. It is arguedthat homogenised evaluation practices linked toglobal, market-oriented views of higher educationwhich are blind to sociocultural diversity are detri-mental to quality enhancement (Canen, 1997b),since they impose a uniform approach to a multi-cultural reality. Also, authors such as Davini(1997) point out that teachers’ sense of identitycould be jeopardised by institutional evaluationwhich fails to involve them and take their percep-

tions and experiences into account. In those cir-cumstances, rather than a boost to the quality ofeducation, institutional evaluation would representa hindrance.

Building on the above framework, the first partof the paper tries to conceptualise institutionalevaluation in transformative, equity-orientedapproaches as opposed to homogenised, control-oriented ones. The second part analyses the speci-ficity of the institutional evaluation as conceivedin educational policies in Brazil and Argentina,exploring the extent to which they have been tak-ing into account sociocultural plurality and insti-tutional identity. The last part tries to extract limitsand potentials of institutional evaluation for edu-cation for equity and transformation.

2. Educational evaluation: a political issue

Evaluation has been traditionally associated withjudgement of values and decision making, whichmakes it more than a mere technical activity. How-ever, the assumption of the neutrality of evaluationprocedures has long hovered over literature in thearea. Reduced to quantitative measurement of pro-ducts in terms of educational achievements to thedetriment of the analysis of educational processes,evaluation has been reduced to a technical activity,making its culturally, value-laden dimension andthe political impact of its results upon sociocultur-ally and economically diverse societal groups anon-issue.

Sustained criticism of the so-called classificatoryevaluation based on the above mentioned assump-tions of neutrality and technicality of the evaluationprocess has produced a considerable impact on thethinking in the area. Authors such as Luckesi(1996), Broadfoot (1994), Perrenould (1993),Canen (1997b) and others have pointed out to thepolitical nature of evaluation and the need to clar-ify its purposes and possible effects on educationalexclusion in multicultural, sociocultural disparatecontexts. It is rightly claimed that the stronglybehavioural dimension of classificatory evaluation,which relies heavily on quantitative indicators ofperformance, would be working towards homogen-isation of educational practices, to the detriment of

279A. Canen, M. Tarre´ de Oliveira / International Journal of Educational Development 20 (2000) 277–286

cultural plurality of institutions and of those whocome to them. As argued by Canen (1995,1997a,b) and Grant (1997), such a perspectivewould silence sociocultural plurality and unequalpower relations which legitimise dominant culturalpatterns to the detriment of cultural diversity inmulticultural societies. Furthermore, based uponliberal (and, more recently, neo-liberal) premises,the product-oriented evaluation perspective takessociety as a group of individuals in which theability to successfully compete reflects in success-ful evaluation results. Such a view does not takeinto account sociocultural diversity and conflicts,and the fact that classificatory evaluation practiceslegitimise the exclusion of those whose culturalvalues do not correspond to the dominant onesshould come as no surprise.

As opposed to a view which limits evaluation toproduct measurement, the above mentionedauthors propose evaluation conceived as a forma-tive, diagnostic process aimed at supporting teach-ing and learning and improving institutional per-formance. That would be achieved in as much asdiagnostic, formative evaluation would not be lim-ited to a final judgement of results. Rather, it wouldgo hand in hand with the development of edu-cational practices, as a continual research processaimed at fostering a permanent dialogue betweenview worlds and cultural values partaken by cul-turally diverse sociocultural groups who come tothe educational system. The aim is to ensure effec-tive teaching and learning and an inclusive, ratherthan excluding educational system. In a critical,intercultural perspective (Canen, 1995, 1997a,b;Grant, 1997), it is argued that the multicultural nat-ure of societies should be taken into account ineducation and evaluation practices, so as to worktowards fostering tolerance and appreciation ofcultural diversity, taken as an asset rather than aconstraint in the construction of citizenship andsociety.

In this sense, underlying the formative and diag-nostic evaluation process would be a project of tak-ing into account the diverse voices which come toeducational institutions, so as to build upon themin transformational educational practices. Theemphasis would be on the qualitative dimension ofthe process, namely the extent to which evaluation

results can represent starting points for discussionsand negotiations aimed at improving educationalpractices, optimising the dialogue between socioc-ulturally diverse groups and enhancing institutionalperformance. In an ironic analogy between pro-duct-oriented evaluation and the process of selec-tion of potatoes, Gandin (1995) remarks that bothshare a “final judgement” perspective (big, middle-sized and little potatoes are sorted out with nochance of changing or improving their character-istics; likewise, students would be classified as suc-cessful or unsuccessful with no chance fortransformation). On the other hand, the formative,diagnostic evaluation process is compared by theauthor to the development of a tree, whereby thegardener will constantly check its development,making sure its needs are met. In this sense, whilstclassificatory evaluation relates to the past, forma-tive evaluation builds on the present, with viewsto a future emancipatory, inclusive educationalproject.

3. Institutional evaluation and transformation:theoretical concerns

Bringing the above considerations to the focusof institutional evaluation, authors working fromtransformative, diagnostic and formative orientedevaluation perspectives such as Sobrinho (1996)and Cunha (1997) conceptualise institutionalevaluation as relating both to the object beingevaluated (the institution) and the subjectsinvolved in the process (institutional actors). In asimilar vein, Lourau (1996) suggests that any insti-tutional analysis should take into account the twocomponents of an institution, namely: the “insti-tuted” (meaning its static, rational dimension com-prising rules and norms) and the “instituting”(designating its dynamic, transformative aspect, inthe sense of founding, creating, constructing). Thereferred author contends that the separation of bothdimensions in institutional analysis and evaluationwould represent a distorted view of the very con-cept of institution. Such a fragmented view wouldlegitimise assumptions of stability and consensualnorms (institution) to which individuals wouldhave to comply, to the detriment of institutional


actors’ wills and perceptions. In this sense, Lourau(1996: 142) suggests that any institutional analysisshould capture the social action in its dynamics,“ trying to find out where the institution really is,that is, the relationships between the establishedrationality — rules, social aspects and codes —and the everyday happenings and social move-ments dialectically related to them”.

In the context of multicultural societies, localinstitutional evaluation should also take intoaccount institutional cultural distinctive projects. Ina highly interesting article, Teasdale (1998: 502)talks about the need for higher education insti-tutions to ensure “the maintenance of local culturalidentity while simultaneously equipping studentswith the skills to cope with the realities of the mod-ern world”. In order to achieve that, the primaryinitiative in curriculum development and evalu-ation would have to come from within the insti-tutions themselves, even though it might be legit-imated and facilitated by outsiders. That woulddiffer from a classificatory, product-oriented insti-tutional evaluation, external to the academic com-munity and heavily based on quantitative indi-cators of homogenised, market-orientedcompetencies. By ignoring the culturally diverse“instituting” dimensions, classificatory, global,market-oriented institutional evaluation would dolittle to transform education and the institutionitself. It would be silencing the sociocultural pro-cesses and the institutional actors’ wills involvedin them. On the other hand, an institution as a cul-turally plural, collective, everyday constructionrather than an abstract, essencialised and fixedentity should underlie institutional evaluation con-ceived as a formative, diagnostic, process-ori-ented mechanism.

The next section will explore the extent to whicha transformative perspective is prevalent in thecontext of the Mercosur educational policiesrelated to higher education institutional evaluation.

4. Institutional evaluation and the Mercosur:the cases of Argentina and Brazil

Talking about the latest developments at the endof the century and their impact on international and

comparative education, authors such as Watson(1998) point out to the growing sense of uncer-tainty, the globalisation process, the new regionalgroupings and the break-up of old political struc-tures, with ensuing deep social transformations.

An overview of the context in which the glo-balisation processes have been filtering to thecountries comprising the Mercosur groupingshould help cast a light on the particular dis-cussions and contentions surrounding institutionalevaluation in the area, as well as to the theoreticalapproach and practical implications underlying itsconcept. As commented on by authors such asCunha (1997) and Catani (1996), the Mercosurblock was created following a trend of formationof economical blocks to face an increasingly com-petitive world market. The neo-liberal perspectiveof production of knowledge for competitive edgein a highly globalised market has been movingtreaties on the generation of knowledge among thecountries within most of those blocks.

As pointed out by the above-mentioned authors,the particular context of the Mercosur, which com-prises recently democratised countries highlyindebted to more developed ones, could partlyexplain the influence of transnational agencies suchas the World Bank in the filtering of globalisedperspectives in the areas, as well as in the runningof educational policies.

Being the most industrialised countries in theMercosur, apart from sharing similar historic con-ditions as recently democratised countries follow-ing a long period of military dictatorship,Argentina and Brazil offer interesting illustrationsas to the impact of the globalisation process oninstitutional evaluation as developed in educationalpolicies in the Mercosur. As pointed out by Sob-rinho (1996), transnational agencies’ prescriptionsfor educational policies in these countries viewinstitutional evaluation as an instrument of controlfor ensuring competitiveness and return of edu-cational investments, rather than as a diagnostic,transformational set of practices. In this sense, suchpolicies ignore the cultural plurality of both coun-tries, as discussed earlier in this paper, being blindto institutional cultural distinctive research projectsand curriculum approaches.

Summarising the World Bank perspective con-


cerning institutional evaluation, Sobrinho (1996:17) indicates its efficientist and classificatoryemphasis, privileging quantitative indicators whichreduce institutional evaluation to the most

“visible and easily describable dimensions,such as built-in area, qualification of staff,description of students and workforce, as wellas lists of services, products... [and] numericalexpressions supposedly representing educationquality, such as number of citations and publi-cations, with the further ranking of institutionswith serious effects upon distribution ofresources and social organisation of studentsand staff”.

The university situation and the specific legis-lation that has called for institutional evaluation inArgentina and Brazil attest to the influence of theabove World Bank prescriptions, as will be com-mented on next.

4.1. Institutional evaluation in Argentina

As argued by Davini (1997), similarly to mostLatin American countries, the growing influence oftransnational agencies in the running of edu-cational policies in Argentina has been felt parti-cularly from 1990 onwards. Until that time, parti-cularly during the 1980s, those countries had beenmostly involved in recovering democratic insti-tutions in the aftermath of the political dictatorialgovernments. Leite (1997) points out that insti-tutional evaluation came to the limelight inArgentina precisely in 1991, following a contro-versial educational project developed in partner-ship between the World Bank, the Ministry ofScience and Education and the National UniversityCouncil. Known as “Subproyeto 6 — Programa deFortalecimiento a la Gestio´n y Coordenacio´nUniversitaria” (Sub-Project 06 — Program forStrengthening University Administration andCo-ordination) (Mollis, 1998), it was aimed at there-organisation of universities so as to fosteroptimisation of educational investments and gearcurriculum practices towards the competenciesrequired for a global market. The governmentexpected to be provided with information that

would allow the ranking of the higher institutions,according to parameters of efficiency and pro-ductivity, linking its financial support to the classi-ficatory evaluation thus conceived.

As explained by Mollis (1998) and Krotsch(1997), even though technical problems preventedthe ranking taking place at the time, the impactof evaluation in a product-oriented approach andexternal to universities’ initiatives was perceivedas a threat to institutional autonomy. The hom-ogenisation of evaluation procedures was also per-ceived as menacing cultural diversity of univer-sities and imposing a uniform neo-liberal modellinked to market demands. At the same time, thefact that evaluation measures had started evenbefore a specific legislation had been carried outwas greatly resented.

A “critical resistance” (Mollis, 1998) started tak-ing place at the universities, and regional commit-tees were formed by rectors and academic sec-retaries which generated the “ConsejoInteruniversitario Nacional — CNI” (National Uni-versity Council) from 1992 onwards. This councilbegan meeting periodically for the elaboration ofalternative institutional evaluation proposalsgeared at respect for institutional cultural identities,education for citizenship (as against for globalmarket) and qualitative change.

Mollis (1998) illustrates those aspects that makea formative, local model of evaluation, as dis-cussed by the CNI, distinctive from a global,classificatory one, in her description of the experi-ence of institutional evaluation being carried out inthe Faculty of Economical Sciences of the Univer-sity of Buenos Aires since 1994. The first distinc-tive aspect is that institutional evaluation in theexperience described is thought of as a gradual andformative process, rather than a “final sentence”based on product measurements. A second distinc-tive aspect is that it has been carried out by theacademic community itself, as against evaluationby controlling bodies external to it. Also, ratherthan limited to measures of staff’s academic quali-fications, it has been focusing particularly on theirperformances, as perceived by themselves, theirpeers and students. The local, cultural distinctiveinstitutional dimension has been consistently takeninto account in that those performances are contex-


tualised in the local institutional environment, aswell as within the framework of its cultural ident-ity. For instance, in the first phase of the process,questions were presented to members of staff abouttheir perceived roles, the institutional context andits aims, as well as the kinds of enhancing or detri-mental factors in the university environment togood teaching and research practices. Students’views on curriculum development, assessment pro-cedures and institutional ethos have also beengauged. Another distinctive aspect is that such alocal, formative evaluation builds on qualitative aswell as quantitative data (as opposed to a mainlyquantitative approach espoused by classificatoryinstitutional evaluation). As argued by Mollis(1998), the analysis of data so far has beenallowing critical analysis of weaknesses as well asstrong points. By emphasising processes ratherthan products, as well as taking into account thespecificity of the institutional ethos and aims, it isperceived by the academic community as a boostto quality improvement and to respect for cul-tural distinctiveness.

Summarising legislation on institutional evalu-ation in Argentina, Mollis (1998) argues that dur-ing the last six years, that issue has been a fieldof battle among three main institutional actors: theMinistry of Culture and Education, the NationalUniversity Council and the World Bank. She con-tends that, in that battle, the National UniversityCouncil’s critical resistance to classificatory insti-tutional evaluation has been having limited impactas compared to the “know how (material andsymbolic)” of the other two in the implementationof educational policies.

In fact, on August 7th 1995, the central govern-ment enacted an educational law — the Law24.521/95 for Higher Education, perceived byKrotsch (1997) as a tentative compromise of cen-tral government between the academic communi-ties’ claims and government’s perceived need to fitinto the World Bank prescriptions for institutionalevaluation in a global market perspective. Such acompromise was translated in the two complimen-tary institutional evaluation procedures institutedwithin the legislation: one internal, to be carriedout by each university, following its own criteriaand theoretical concerns, and another external, by

the central government, through the National Com-mission for University Evaluation andAccreditation, to be carried out every six years. Asargued by Krotsch (1997), the core of the law isstill geared towards differentiating higher edu-cation institutions according to market demands,and the central role of institutional evaluation inachieving that purpose seems to be clear. Also, bybeing applied at the end of every six years for pur-poses of accreditation and distribution of resources,the emphasis of the evaluation is in “control”,rather than change and improvement. Even thoughemphasising the link between the process of insti-tutional evaluation and the particular educationalproject of each institution, Leite (1997) points outthat the classificatory dimension of institutionalevaluation thus conceived has been perceived as a“punitive” rather than a transformative set of insti-tutional evaluation practices in that country.

It is argued that such homogenised, product-ori-ented institutional evaluation not only fails to con-template the distinctive institutional projects andcultural identities, but also favours institutionsgeared towards the neo-liberal project, perpetratingeducational inequality. Krotsch (1996) calls theattention to the fact that the above mentioned lawrepresents the basis for access to a credit of 160millions of dollars, delivered by the World Bank.He argues that unless Argentina’s institutionalevaluation proposals and practices take the insti-tutional actors’ perceptions and the cultural ident-ities of the institutions on board, little will be achi-eved in terms of institutional improvement andtransformation.

4.2. Institutional evaluation in Brazil

In Brazil, Cunha (1997) and Sobrinho (1996)situate the discussion of institutional evaluationwithin the higher education reforms proposed bythe central government, particularly in the lastthree years. Similarly to what happens inArgentina, the impact of the World Bank proposalson such reforms can be felt by the academic com-munities. As pointed out by Sobrinho (1996), suchproposals “push” the universities into market com-petitiveness, establishing institutional evaluationbased on the same market criteria of profitability


and competitiveness underlining commercial andindustrial enterprises.

Cunha (1997) points out that the new Braziliangovernment, which started in 1995, had begun anintense educational reform activity even beforelegislation had been implemented. Projects of law,provisory measures and others, inspired by theWorld Bank prescriptions, were put into motion,among which institutional evaluation of highereducation was one of their main “stars”. Ignoringinitiatives of the academic community in local, for-mative paradigms of institutional evaluation suc-cessfully carried out (to be commented upon soon),“the focus of institutional evaluation was changedto individual evaluation” (Cunha, 1997: 38) by thecentral government, in 1995. That means thathigher education institutions started being evalu-ated according to the results of students’ perform-ances in National Examinations of Higher Edu-cation Courses, mandatory for all studentsconcluding higher education in Brazil.

The so-called “prova˜o” (Brazilian nickname forthe “big” National Examination of Higher Edu-cation Courses) consists of homogenised questionswhich assess market-oriented competencies in stu-dents concluding higher education in Brazil,intended to provide central government with infor-mation to establish a ranking between those insti-tutions for accreditation and resource distributionpurposes. This way, a product-oriented, classifi-catory perspective similar to the selection ofpotatoes alluded by Gandin (1995) starting imbu-ing institutional evaluation from 1995 onwards.Rather than representing a part of the evaluationprocess, the quantitative, easily measured indicatorof students’ performance was initially taken as ameasurement of institutional performance, asopposed to the intercultural, diagnostic and trans-formative approach (Canen, 1995; Sobrinho, 1996;Lourau, 1996). Even though recognising the influ-ence of a market-perspective, a product-orientedset of principles originated in international agen-cies’ prescriptions for education in the Mercosurarea, Cunha (1997: 48) contends that nothing simi-lar to the “provoes” (plural noun of “prova˜o”) hadever occurred to the consulting body of theWorld Bank.

Critical resistance from the academic com-

munity to institutional evaluation as conceived bythe central government built mainly on the follow-ing points: the reduction of institutional evaluationto students’ evaluation; its emphasis on homo-geneous, global, market-oriented competencies tothe detriment of local cultures and culturally dis-tinctive institutional projects; its external dimen-sion, ignoring the actors’ initiatives; the “impover-ished” concept of knowledge as something feasibleto be assessed in one final examination, rather thana process of socio-cultural construction; the ensu-ing “exam-oriented” direction to be prevalent incourses all over big and multicultural Brazil.

As commented on earlier, alternative insti-tutional evaluation models had been proposed andsuccessfully implemented in some Brazilian uni-versities since 1988. Saul’s (1988) emancipatoryinstitutional evaluation experience carried out in aBrazilian University was among them. Someaspects made such a local, formative institutionalevaluation different from a global, classificatoryone: it was conceived and implemented within theacademic community itself, it relied on qualitativeas well as quantitative data and it tried to contex-tualise those data in the specific aims and culturalprojects of the institution. The main result of theexperience described by Saul (1988) was a docu-ment in which potentials and limitations of thecourse were pointed out, and a project outliningobjectives and methodological strategies for futuredevelopments was presented. In this sense, a trans-formational, diagnostic perspective imbued a self-evaluative stage in the institutional evaluation car-ried out, and a concerted effort of the academiccommunity towards improvement and changeunderlined the approach to the task undertaken.

A localised alternative to the current form ofglobal institutional evaluation, in practical terms,has also been taking place since 1993. Asexplained by Cunha (1997), a National AssessmentCommittee was created in Brazil in 1993, and theacademic community representatives began meet-ing regularly to discuss institutional evaluation ina formative approach. A Program of InstitutionalEvaluation of Brazilian Universities (PAIUB) waselaborated, in which self-evaluation and externalevaluation were previewed. The PAIUB hadreceived economical support from the Ministry of


Education from 1993 to 1995, and partly sponsoredself-evaluation processes in seventy Brazilian uni-versities.

A document summarising the main principlesunderlying the local, transformational institutionevaluation perspective underlining the PAIUBinitiative was issued (PAIUB, 1996). As argued byRistoff (1996) those principles make the PAIUBinstitutional evaluation distinctive from the global,classificatory one. The first one is “a holisticapproach” — understood as institutional evaluationwhich is not reduced to one aspect of institutionallife (as against evaluating universities exclusivelythrough students’ performance, for example). Thisprinciple proposes continual assessment of relevantaspects such as research projects, teaching stra-tegies, curriculum development, staff’s qualifi-cations, power relations and others. The secondprinciple is “comparability”, which implies theavoidance of a “ranking” mentality, but stresses theneed for equivalent evaluation procedures in orderto compare institutional performances. The thirdprinciple highlights the need for “respect for insti-tutional identity”, since “lower mountains and shal-lower rivers also have important contributions”(Sobrinho, 1996: 49). That is the principle whichallows for institutional cultural distinctive projectsto be valued. It builds on the need for institutionalevaluation to respect and foster cultural diversity,particularly relevant in multicultural countries suchas Brazil, as against homogenised evaluation prac-tices tuned to globalised market-oriented com-petencies.

The last principles refer to the need for insti-tutional evaluation to be conceived within the aca-demic community (as against externally conceivedones), to be gradual and continuing (and notreduced to measurement of final “products”, as ithappens in global, classificatory perspectives) andto be dissociated from “punishing or rewarding”mechanisms (as against global, classificatory insti-tutional evaluation models which link institutionalperformance to resource distribution, for example).

The document also provides detailed sugges-tions for qualitative as well as quantitative indi-cators for institutional evaluation coherent with theabove-mentioned principles. As argued by Sob-rinho (1996) and Cunha (1997), even though quan-

titative measures and data should be a relevant partin the cognisance of institutional organisation andrationalisation (the instituted dimension, called byLourau, 1996), they should not exhaust the evalu-ation process lest a classificatory, product-orientedperspective prevails, which would limit insti-tutional evaluation to its “instituted” dimension,rather than taking into account the pluralistic nat-ure of its “instituting” component.

Based on those agreed principles and suggestedindicators and variables, Brazilian universitiesstarted their efforts and experiences in local, for-mative institutional evaluation. Among them, Biel-schowski (1996) describes the process being car-ried out in the Federal University of Rio de Janeirosince 1994. Its starting point was to detect worry-ing aspects of the university, as perceived by staffand students through in-depth interviews and ques-tionnaires. From the data thus obtained, assessmentquestions were formulated which aimed to focuson the cultural identity of the institution, its aims,weak and strong points and the theoretical lensthrough which data would be evaluated. Theresults of that and of other university experiencesbased on the PAIUB principles were generally per-ceived as of great impact in institutional life, giv-ing voice to institutional actors and contributing tothe strengthening of institutional cultural diversity.However, financial support to the program ceasedin 1995, when the new Brazilian governmentstarted to implement a market-oriented approach tohigher education and institutional evaluation. ThePAIUB experiences still go on in many univer-sities, although their impact is seriously reduced,in the face of the legislation enacted in 1996.

In fact, the Law 9.394/96, known as Lei dasDiretrizes e Bases da Educac¸ao Nacional — LDB(Law of Directives and Bases for NationalEducation), enacted in 1996, maintained the mainprescriptions of the World Bank: its chore is onthe need for universities to adapt to marketdemands. The government has, according to thisLaw, total control over institutional evaluation inhigher education. As argued by Cury (1997), thatlegislation has institutional evaluation as one of itsmain axes. The word “evaluation” appears in thir-teen articles of the LDB, which has only 92 articlesaltogether, and is repeated 23 times. Article 45


presents the explicit need for establishing a hier-archy of higher institutions, whilst article 46 assertsthe role of central government in the accreditationof higher education institutions, based on theresults of periodical institutional evaluation of theirperformance in relation to market demands.

Due to the pressures of the academic communi-ties and of the university students’ unions, stu-dents’ national examinations — the “provo˜es” —are not to be considered the sole means for insti-tutional evaluation, albeit being still the most vis-ible one (the results of the first exams which tookplace in November 1996 were extensively exhib-ited in the press, with the ensuing ranking of Braz-ilian higher education institutions). Leite (1998)points out that the Decree 2026, of October 1996,issued by the Ministry of Education one monthbefore the “provo˜es” took place, presented insti-tutional evaluation as comprising the followingdimensions: evaluation of post-graduate courses(Master and PhD) in the format successfully under-taken so far; evaluation of graduate courses,through national examinations of students (the“provoes”); self-evaluation of higher educationinstitutions; and analysis of global institutional per-formance and graduation courses, to be undertakenby central government specialists, based upon cri-teria and assessment indicators as yet unknown.

Institutional evaluation in Brazil is thereforebeing implemented according to the Law 9394/96and its complementary Decree 2026/96. Cury(1997) argues that institutional evaluation thusconceived is product and control-oriented,rewarding the “high-achievers” with moreresources and condemning universities with cul-tural distinctive projects which do no fit into theneo-liberal mould to poverty and exclusion. In fact,the homogenisation of institutional evaluation inthe name of competitiveness and of global marketforces as imposed by the above-mentioned policiesdoes not take into account the sociocultural diver-sity and variety inherent in the Mercosur countries,ignoring institutional cultural identities, their his-tories and their “instituting” dimension. In multi-cultural countries beset by sociocultural disparitiesand inequalities, the educational exclusion legit-imised by classificatory evaluation practices(Canen, 1997b) will arguably be likely to attain

institutions which fail in the ranking promoted bythe homogenised, product-oriented institutionalevaluation, which does not take into account theirspecific roles, processes and cultural identities.

As for the Decree 2026/96, it could be arguedthat the complicated scheme proposed by it is anattempt to reconcile a classificatory, product-ori-ented perspective in evaluation to an intercultural,diagnostic and transformative one. However, dueto the differing and even conflicting theoreticalassumptions underlying each of them, as discussedearlier in this paper, such an attempt of compro-mise could arguably be compared to a servant try-ing to serve two masters.

5. Some conclusions

Viewed by some as an instrument of control forguaranteeing institutional competitive edge in anincreasing globalised market, and by others as ameans for developing change and transformationthrough a concerted effort of the socioculturalagents involved, institutional evaluation is a polit-ical, value-laden field of contentions. By exploringthe theoretical concepts and practical implicationsof evaluation conceived in a classificatory, control-oriented perspective and a transformative, diagnos-tic, interculturally-oriented one, it has been arguedthat in multicultural, deeply disparate countriessuch as the Mercosur ones, institutional evaluationshould take into account institutional culturalidentities, history and sociocultural interactionsand processes, lest homogenised evaluation prac-tices fail to produce a significant effect for insti-tutional change and educational equity and trans-formation.

The analysis of institutional evaluation as con-ceived by the latest educational policies in highereducation in two Mercosur countries — Argentinaand Brazil, provided an illustration of the chal-lenges faced by recently democratised countries intrying to undertake institutional reforms in the faceof an increasing influence of transnational agenciesin a globalised setting. The attempts to reconcilesociocultural identities and transformational evalu-ation practices to a market-oriented perspectivehave not as yet produced convincing answers to


the academic community in neither of the countriesdescribed. However, the road is long, and a lot maystill come up from the hot debates surroundinginstitutional evaluation in the area.

Even though the focus of the present paper hasbeen on higher education, it should be noted thatclassificatory, homogenised and market-orientedprograms of institutional evaluation based exclus-ively on “products”, such as the results of students’final examinations, have been more and more fre-quent at school level in both countries. Possiblesimilarities and differences of the impact of glo-balisation at these different levels of education begto be further researched into.

As claimed by Freire (1996: 39), educationinvolves “taking risks, being willing to accept thenew and rejecting any form of discrimination”. Ina transformational institutional evaluation perspec-tive, that would involve being sensitive to culturaldifferences and institutional cultural identities, sothat a democratic, inclusive, and equity-orientedapproach to educational goals permeate insti-tutional evaluation in the Mercosur countries.

References

Bielschowski, C.E., 1996. Avaliaçao na Universidade Federaldo Rio de Janeiro: A Metodologia. Avaliaçao 1 (1), 29–32.

Broadfoot, P., 1994. The 1993 SERA lecture, exploring the for-gotten continent: a traveller’s tale. Scottish EducationalReview 26 (2), 88–96.

Canen, A., 1995. Teacher education in an intercultural perspec-tive: a parallel between Brazil and the UK. Compare 25 (3),227–238.

Canen, A., 1997a. Competeˆncia Pedago´gica e Pluralidade Cul-tural: Eixo na Formaçao de Professores? Cadernos de Pes-quisa 102, 89–107.

Canen, A., 1997b. Avaliaçao Diagnostica: Rumo a` EscolaDemocratica. In: Um Salto para o Futuro, Ensino Funda-mental, se´rie XII. TVE, Ministerio da Educaçao e doDesporto — MEC.

Catani, A.M., 1996. Apresentaçao. In: Catani, A.M. (Ed.),Universidade na Ame´rica Latina: tendeˆncias e perspectivas.Sao Paulo: Cortez, pp. 7–10.

Cunha, L.A., 1997. Nova Reforma do Ensino Superior: A Lo´g-ica Reconstruı´da. Cadernos de Pesquisa 101, 20–49.

Cury, C.R.J., 1997. Reforma Universita´ria na Nova Lei dasDiretrizes e Bases da Educaçao Nacional? Cadernos de Pes-quisa 101, 3–19.

Davini, M.C., 1997. Novas Tecnologias Sociais, Reforma Edu-cacional e Formaçao Docente. Cadernos de Pesquisa 101,141–151.

Freire, P., 1996. Pedagogia da Autonomia. Saõ Paulo: Paz eTerra.

Gandin, D., 1995. Algumas Ideías sobre Avaliaçao Escolar.Revista de Educaçao AEC 24 (97), 00–00.

Grant, N.D.C., 1997. Some problems of identity and education:a comparative examination of multicultural education. Com-parative Education 33 (1), 9–28.

Krotsch, P., 1996. O Novo Cena´rio da Universidade Argentina:O Peso do Passado e os Desafios do Futuro. In: Catani, A.M.(Ed.). Universidade na Ame´rica Latina. Saõ Paulo: Cortez.

Krotsch, P., 1997. El Peso de la Tradicioń y las Recientes Tend-encias de Privatizacioń en la Universidad Argentina: Haciauna Relacioń Publico-Privado. Avaliaçao 2 (4/6), 31–44.

Leite, D., 1997. Avaliaçao e Tenso˜es Estado. Universidade eSociedade na Ame´rica Latina, Avaliaçao 2 (1/3), 7–18.

Leite, M.C.L., 1998. Avaliaçao e Relaçoes de Poder: Paiub eExame Nacional de Cursos. Avaliaçao 3 (1/6), 59–68.

Lourau, R., 1996. A Ana´lise Institucional. Petro´polis: Vozes.Luckesi, C.C., 1996. Avaliaçao da Aprendizagem Escolar. Saõ

Paulo: Cortez.Mollis, M., 1998. El Campo de la Evaluacioń Universitaria

Argentina y los Organismos Internacionales: entre la auto-nomia y la heteronomia. Avaliaçao 3 (1/6), 11–22.

Perrenould, P., 1993. Formaçao em Avaliacao: Entre IdealismoIngenuo e Realismo Conservador. In: Pra´ticas Pedago´gicas,Profissaõ Docente e Formaçao. Lisboa: Dom Quixote, pp.155–170.

Programa de Avaliaçao Institucional das Universidades Brasi-leiras — PAIUB, 1996. Avaliaçao 1 (1), 43–66.

Ristoff, D.I., 1996. Princı´pios do Programa de Avaliaçao Instit-ucional. Avaliacao 1, 47–53.

Saul, A.M., 1988. Avaliaçao Emancipato´ria. Sao Paulo: Cortez.Sobrinho, J.D., 1996. Avaliaçao Institucional: Marco Teo´rico e

Campo Polı´tico. Avaliacao 1, 15–24.Teasdale, G.R., 1998. Local and global knowledge in higher

education: a search for complementarity in the Asia-Pacificregion. International Journal of Educational Development 18(6), 501–511.

Watson, K., 1998. Memories, models and mapping: the impactof geopolitical changes on comparative studies in education.Compare 28 (1), 5–32.

institutional evaluation in the mercosur: transformation or control?

Documents