actor brief: higher education institutes (heis) · the development of research activities with...

OECD Innovation Policy Platform www.oecd.org/innovation/policyplatform

ACTOR BRIEF: HIGHER EDUCATION INSTITUTES (HEIs)

What are Higher Education Institutes?

Higher Education Institutes (HEIs) encompass a wide range of organisations including universities, institutes of technology, colleges, academies, specialized or professional institutes, trade schools and other organisations awarding academic degrees or professional certification. The structure of higher education and the importance of some of these organisations in the overall picture can, however, differ strongly between different countries, and some institutes, such as the French Grandes Ecoles, German Fachhochschulen or Italian Politecnicos are country-specific. Universities have traditionally played a central role in higher education. Today, they are no longer the only actors and the notion of “tertiary education” has emerged to encompass new types of institutes and new types of functions that they fulfil (OECD 2008).

What roles do HEIs perform?

There are several ways to characterise the roles HEIs perform. The distinction between three different missions has arguably been the most influential in recent debates. This account stipulates a historical development from a situation where education and research dominated HEIs’ activities to another where increasing pressure is put on them to also focus on a “third mission” of knowledge transfer to the private sector and wider society. This third mission is first presented, followed by a brief account of the more traditional missions of education and research and their contributions to innovation.

Knowledge transfer

Engaging in direct relations with industry through the rapid transfer of knowledge to the marketplace is assumed to hold important economic and social benefits. In practice, this has resulted in enlargement of public-private cooperation and encouraged a move to the “entrepreneurial university”. This “capitalization of knowledge” takes different forms and is sustained by an ever-growing array of policies and organisational structures including technology transfer offices, “incubator” and “spin-off” firms, and science and technology parks.

However, third-mission activities cannot be limited to purely economic dimensions: for example, as environmental or sanitary problems become increasingly difficult to tackle without scientific advice, demands for involvement of HEIs are also articulated in political arenas and through not-for-profit organisations, social movements or foundations that are increasingly linked to universities. Policy-makers should therefore bear in mind that third-mission activities encompass a wide array of activities through which HEIs interact with the economy, political arenas and civil society.


HIGHER EDUCATION INSTITUTES – OECD 2011 2

Furthermore, the overall picture might be more complex than captured in the distinction between three missions. HEIs already interact in several ways with wider society through their educational activities and contribute in several ways to knowledge production and transfer.

Educational activities of HEIs

International statistical conventions distinguish three programme levels in tertiary education:1

• Provision of mass tertiary education. This is typically the case for organisations with high percentages of bachelor programmes. 5B and some 5A level diplomas dominate.

ISCED 5A level programmes are largely theoretically based and are intended to provide sufficient qualifications for gaining entry into advanced research programmes (history, philosophy, mathematics, etc.) and professions with high skills requirements (e.g. medicine, dentistry, architecture, etc.). ISCED 5B level programmes focus on practical or technical education and provide occupationally specific skills geared for entry into the labour market. They are typically shorter than 5A programmes. ISCED level 6 programmes aim at advanced research qualification (e.g. doctoral programmes). Using these categories, the different education-related roles HEIs perform can be characterised as follows:

• Provision of professional specialized higher education and research. The central outputs of this type of educational activity are professional masters or equivalent diploma. 5A level programmes with a focus on high-level professional formation and research dominate.

• Formation of future generations of scientists. HEIs with this profile have high percentages of PhD students and prepare for level 5A and especially level 6 qualifications.

Contributions of HEIs to knowledge production and diffusion

The contributions of HEIs to global knowledge resources are limited neither to economically exploitable findings, nor to specific discoveries. Instead four ways in which HEIs contribute to the use of knowledge in socio-economic contexts can be distinguished:

• Building knowledge bases. Besides “breakthrough” or “blue-sky science”, research efforts by HEIs also involve patient accumulation of knowledge through incremental research, data accumulation through monitoring, improvement measures, better instrumentation or new uses of research technologies. These knowledge bases are crucial for a wide array of actors.

1. International Standard Classification of Education (ISCED).



• Creation of capabilities. The formation of human capital through education is part of the traditional functions of higher education. Two dimensions can be distinguished in this process: the inculcation of specific forms of knowledge and skills (‘tacit knowledge’) and the development of problem-solving capacities. Both are achieved through science and engineering training, including training in social sciences and humanities.

• Diffusion of knowledge. HEIs are active in spreading knowledge they produce. This occurs through a) publications that can be followed and monitored by companies and other social actors b) direct interaction with such actors via collaborative research programmes, joint ventures, consultancies and informal channels and c) development and diffusion of ways of doing, heuristics, and fruitful ways of searching.

• Maintenance of knowledge. Storing and maintaining knowledge in libraries, databases, conferences and computing resources is important in innovation processes, as old knowledge can have new applications, or become relevant as the socio-economic context changes. Such knowledge has to be stored in an accessible manner, which implies substantial costs for HEIs.

From missions to functions?

With growing pressures for specialisation, distinct profiles of HEIs have emerged that combine the different roles in education and research in specific ways. Some scholarly observers of innovation processes therefore propose to move from a distinction of different “missions” to one of three functions of HEIs (Larédo 2007):

• A first function involves the provision of an educated work force for the local economy through mass tertiary education. While training in scientific methods can be part of curricula, research activities are not central.

• A second function is the training of national and international specialists and the development of research activities with close ties to non-academic actors (industry, politics, civil society). Institutions with this profile (e.g. Institutes of Technology and different forms of Engineering Schools) focus on the professional master as a central output and on problem-solving research.

• A third function concerns the conduct of academic research. Concentration on this function implies high percentages of PhD students and academic articles as main outputs. Usually, only some departments within a university develop an academic profile, and the success of such a specialisation depends on long-term strategic choices and historical developments.



The motivations and interests of HEIs

The structure of motivations and interests of HEIs depends on the political and wider socio-economic context in which they operate. The traditional organisation of higher education as a largely independent, autonomous domain is captured in the metaphor of the “Republic of Science” (Polanyi, 1962). In this model, higher education is insulated from governmental steering and its governance is first of all in the hands of academics, through self-assessment and quality assurance mechanisms like peer-review. Popular at the same time in the German idealist tradition built around the Humboldtian model and in the American functionalist sociology of professions, it found a powerful normative framework in the Mertonian sociology of science.

While elite formation has always been a fundamental activity of universities, the (indirect) contribution to welfare through basic research gained increasing importance in justifications of public funding for HEIs after WWII. This was reinforced by the fact that classical economics came to conceive education and basic research as public or semi-public goods. From this point of view, the State has to fund HEIs and protect produced knowledge from appropriation by market forces. Inversely, individual researchers and HEIs have strong incentives in such a context to place their findings in the public domain, because research careers and the reputation of institutions depend mainly on publication records and other indicators of scientific excellence (e.g. Nobel Prizes).

Several contextual developments challenge this model, forcing HEIs to redefine their interests and to reposition themselves in a changing environment. These developments include:

1. Massification. Formerly reserved for elite education, HEIs now accept over 50% of the eligible age group in many countries, a number reaching up to 60-80% in some countries. From 1991 to 2004, the number of students enrolled in tertiary education has risen globally from 68 million to 132 million. The fact that important parts of scarce state resources today are allocated to higher education and research explains partly increasing calls for accountability of HEIs to the State and wider society.

2. Globalization. The growth of economic activity across national boundaries has three important implications for HEIs. Firstly, new pressures result from international competition for students, researchers and grants. Secondly, growing homogenisation across national policy frameworks challenge long-standing national systems and force HEIs to adapt. Thirdly, globalisation offers opportunities as new possibilities for international networking and collaboration emerge and students and scientists become more mobile and can be recruited on a world-wide basis. Furthermore, education services have been included in the new services negotiations of the General Agreement on Trade in Services under the auspices of the World Trade Organization (WTO) in 2000. This evolution could have potentially important implications for higher education globally.



3. Calls for accountability. In increasingly knowledge-based societies, research and education are seen as closely linked to economic development. This has seen states play a more direct role in mediating the interests of society and orienting the development of higher education. In this context, policy concern of an “innovation paradox” has recently spurred much debate: although public research in some regions and countries is of high quality, it fails to translate into innovation. A range of policy initiatives have been developed to facilitate knowledge transfers, ranging from subsidies for the establishment of technology transfer offices (TTOs) to changes to IPR frameworks. Even though the empirical evidence suggests the “European Paradox” is in part an artefact resulting from the indicators used (e.g. Dosi et al. 2005), issues of more direct contribution of HEIs to wealth creation, innovation, and social progress powerfully reshape policy frameworks and the ways HEIs perceive their interests.

4. New public management (NPM). NPM has become a global paradigm for determining the roles and functions of various actors in public administrations. It introduces a market orientation to public administration and seeks to improve its effectiveness and efficiency through management. Linked to the development and implementation of adequate instruments, the three “E’s” (economy, efficiency, effectiveness) have become important measures and indicators for good governance and best management practice. Core elements of NPM include: the decentralization of decision-making processes, the steering by outcomes or outputs, the flattening of hierarchical structures, and the introduction of market-type mechanisms, private sector management instruments and best practice concepts. As a result, politics define goals and objectives for relatively independent entities with global budgets. Performance agreements, combined with targeted and competitive funding procedures replace direct state intervention.

5. Regional integration processes. There have been initiatives for regional collaboration in regions including the European Union, MERCOSUR and ASEAN. The European Bologna process is the most advanced example for collaboration in tertiary education. In order to create a European Higher Education Area 46 European governments (by 2007) agreed on ten action lines aimed at making higher education in Europe more compatible and comparable, and more competitive and attractive for students and researchers in Europe and worldwide.

6. Information and communication technologies. The development of information and communication technologies has increased capacity and lowered costs to store, transmit and access information. This has considerable effects on: a) teaching and learning, as it facilitates, for example, distance education; b) research and publishing activities, for example, by opening access to information where good libraries are lacking; and c) roles of libraries, whose activities now include the provision of access to databases, web sites and IT-based tools, while some resources (articles, books) become increasingly accessible by distance.



7. Demographic and social developments. In most OECD countries, the student population has become diversified. First, the average age of students has risen as more mature students enter HEIs. A consequence of this is that life-long learning has become a greater focus of education activities. Second, the tertiary student population is increasingly heterogeneous in terms of gender (female participation has risen significantly), socio-economic background, ethnicity and previous education.

As a result of these developments, HEIs have been forced to reorient themselves in a changing environment. Several trends can be identified in respect to structures of motivations and interests:

Diversification between and differentiation inside HEIs. As a result of the growing importance of tertiary education, new organisational types such as polytechnics, university colleges, and technological institutes have emerged. Additionally, educational offerings within institutes have multiplied, as universities adopt different strategic profiles in order to meet societal demands and to attract students, grants and researchers. Thus, we observe the transformation from a homogeneous landscape of institutes to a highly differentiated one and from monolithic institutes to institutes that are internally divided into competing divisions.

New forms of institutional governance. In a context of increasing national and international competition, university leadership has been remodelled in most countries from collegial governance to top-down management. In particular, as academic leaders are increasingly seen as managers, coalition-builders or entrepreneurs, department heads and university presidents are given direct power over nominations and strategic orientations in education and research.

Growing complexity of external relations and internal decision-making. The definition, implementation, interpretation and verification of policies affecting HEIs, formerly concentrated in a central ministry, are increasingly divided and assigned to different parties and different levels of decision making. As multiple stakeholders articulate their interests, HEIs’ internal decision making also involves more factors and becomes more complex.

Increased focus on third party funding. One of the consequences of the substitution of government steering by market steering is the increased focus on third party funding. As this increases uncertainty of resource flows, one organisational consequence is that HEIs need new specialists for the creation of contracts, public relations, and fund raising.

(Global) networking, consortia building and contract activities. HEIs are no longer pure public institutes, but hybrids that combine different norms and values (e.g. public and private ones). Additionally, cross-border funding of research activities and the internationalisation of certain areas of research provide strong incentives for HEIs to act as strategic actors in a global market. Various forms of networking and collaboration among institutes, scholars, students and other actors (industry, civil society) become a priority for HEIs.



From basic research towards applied contract research? While this trend is less clear than the others, it is apparent that project-based grant attribution, pressures from stakeholders and the necessity to attract third party funding have important impacts on research agendas. While this can contribute to pushing HEIs to pursue explicit societal goals and objectives, it is often regarded as a threat to academic freedom.

These developments pose new challenges and questions. Several observers raise doubts about their long-term effects because of the potential impacts on circulation of knowledge (increased transaction costs due to patenting, changing publication practices) and basic research (funding for research that is not directly ‘marketable’ might decrease). These questions are returned to throughout this briefing note.

What resources do HEIs draw upon to fulfil their roles?

In order to perform their different roles, HEIs draw upon different kinds of resources. In an innovation context, two of the most critical are human resources and finance.

Human resources

Both the people directly engaged in research and teaching activities and those often ‘invisible’ technicians, managers and administrators providing support services in all areas are an essential resource. As regards research in HEIs, there are sharp differences across countries in terms of the ratio of research personnel in higher education institutes, arguably reflecting the specificity of national research and industrial structures.

To expand recruitment and participation, attention has focused on enhancing the attractiveness of research careers, women’s participation and international mobility. Firstly, a rise in the number of doctoral holders, playing an increasingly important role in research, has not been matched by an expansion of permanent academic positions. This makes development of early stage researchers difficult and, therefore, research careers less attractive. Among the priority issues is doctoral candidates’ status as students or employees, as well as their working conditions (OECD 2008a). Secondly, women’s participation in research careers has increased but remains low in comparison to men. To tackle this issue, most OECD countries have introduced specific programmes to achieve a gender balance. Finally, there is increasing concern with attracting foreign and expatriate researchers as well as retaining domestic researchers in a more competitive and internationalised skilled-staff market. Initiatives to facilitate mobility and create attractive environments (e.g. quality of research infrastructure) have been taken in many countries as a supplement to domestic human capital creation.



Finance

Funding is a core aspect of HEIs governance directly linked to outcomes in quality, efficiency, equity, and system responsiveness. In spite of differences among OECD countries, some trends can be identified regarding a) expenditure on HEIs and b) the development of new funding arrangements (OECD 2008a).

a) Expenditure: HEIs are predominantly funded by the state. The proportion of public funding ranges from 20 % to more than 90 % of the total budget, with the large majority lying between 60 % and 90 % (OECD 2007). The treatment of higher education as public or private good varies considerably across countries, as well as the importance and the amount of tuition fees and the roles of other funding sources (research agencies, EU, industry, business, property revenues and services to students). Over a quarter of expenditure on educational institutions across the OECD is accounted for by HEIs. Over the last 15 years, public expenditure on HEIs and the relative proportion of expenditure by private sources increased. Higher education R&D expenditure also increased in absolute terms. However, such increases have not been often matched by larger investments in relative terms. For example, in about half of the OECD countries public expenditure per student on HEIs declined.

b) Funding mechanisms and allocation criteria: Over the last decades, major policy reforms were implemented with a view to tackling shortcomings of traditional input based funding (e.g. number of students) vis-à-vis a result-oriented public management (e.g. France, Germany, Italy). New funding mechanisms and criteria were introduced across OECD countries in an attempt to reflect public policy objectives rather than institutional needs, as well as to reward institutions for actual rather than promised performance. This pervasive trend has produced diverse national arrangements that combine in variable proportions traditional mechanisms and criteria and new ones characterised by greater targeting of resources, performance-based funding, and competitive procedures, as well as by a diversification of funding sources.



Table 1. Comparison between traditional and performance-based allocation mechanisms

Traditional Performance-based

Negotiated budgets: Allocations of state funds are negotiated between government agencies and institutions drawing on input criteria (e.g. historical trends, number of students, etc.)

Performance agreements: Governments enter into regulatory agreements with institutions to set mutual performance-based objectives.

Categorical funds: Categories of HEIs designated as eligible for funds for specific purposes including facilities, equipment and programmes.

Competitive funds: HEIs compete on the basis of peer-reviewed project proposals against a set of objectives (e.g. improving quality and relevance, promoting innovation, etc.)

Funding formulas based on size of staff or number of students enrolled.

Funding formulas based on priority (e.g. critical labour force needs) or performance measures including output (e.g. number of graduates per year) or outcome (e.g. academic ranking of the HEI).

Source: Adapted from OECD (2007a), taken from Salmi, J. and A. M. Hauptman (2006).

Project-based research funding is now widespread across OECD countries. In this model, allocation is made by intermediate funding agencies (e.g. Research Councils or Science Foundations) through competitive, peer-reviewed selection processes. In order to provide a stable funding stream, project-based funding can be combined with core research funding and research centre funding.

In many countries, like Australia, the Czech Republic, Finland, or the United Kingdom, a funding formula is used to determine allocations to research core funding. The performance measures attached to funding formulas include output numbers like post-graduate students, research degrees awarded, scientific publications, patents and licenses issued, spin-offs, research contracts with companies, and external research income.

Public funding is also increasingly concentrated in a limited number of centres performing measurable world standard research. Increasing importance is put on the establishment of ‘centres of excellence’ as a means to foster well-identified, sustainable structures characterised by a ‘critical mass’ of high-level scientists and/or technology developers and excellence in specific research areas, high levels of scientific and/or industrial connectivity, and a capacity to play dynamic roles in the surrounding innovation system. The meaning of the key notion of ‘critical mass’, linking funding and evaluation mechanisms, remains however unclear (OECD 2008).



Table 2. Main government mechanisms used for the allocation of funding research in HEIs

Research core funding

A fixed block grant that is provided periodically (e.g. annually);

Research centre funding

Funds are allocated to specific research centres (e.g. centres of excellence)

Project-based funding

Funds are granted to an individual researcher or group of researchers to carry out a specific research project on the basis of a project application.

Source: adapted from OECD 2008a.

Some countries also combine the allocation of core research funding and research centre funding with competitive criteria, thus linking the allocation of funds to assessments of research quality. This is the case in the UK where the Research Assessment Exercise (RAE) is used periodically to assess the quality of research and to inform the distribution of public funds for research. Similar strategies have been adopted in other countries to direct funds selectively to the most highly rated, raising the profile of research and stimulating the development of supporting infrastructure.

What interactions do HEIs rely upon in performing their roles?

Traditional interactions of HEIs were concentrated on the funding relationship with the state and were therefore relatively monopolistic. Multi-actor, multi-level governance frameworks have since emerged in many countries. This is essentially due to the emergence of new stakeholders and new forms of partnerships that increase the role of external actors and their influence with respect to the internal affairs of HEIs. The relationships with some of these actors are discussed below.

The State

The State interacts with HEIs on different levels (federal and state or central and local levels) and within central government at the level of different ministries. Its main functions are funding, defining a regulatory framework, and assessment of HEI performance.

• Government. Ministries and local governments are important sources of funding for tertiary education in most countries, and central government lays down the operating rules for the public service.



• Funding bodies. In the case of higher education independent funding bodies (e.g. NSF and NIH in the US) have become increasingly important, even in countries and regions where funding was traditionally allocated by the central public authority to the principal research organisation who in turn distributed it among its units (e.g. CNRS in France). Today, models based on the competitive allocation of funds by national agencies and dual systems prevail.

• National quality assessment bodies. For a variety of reasons, the question of how to assess the performance, or quality, of higher education has become a central issue worldwide in the last few decades. While the organisation of quality assessment differs greatly among countries and regions, some general trends can be identified. Firstly, quality assessment is increasingly carried out by meta-level managing agents that are relatively independent from government (voluntary associations in the US, Higher Education Quality Council in the UK, Comité National d'Évaluation in France, etc.). Secondly, quality assessment tends to be linked to, but not rigidly tied to the allocation of funds. This improves the acceptance of evaluation procedures by the scientific community. Thirdly, common methods of quality assessment are emerging (see section 6 below).

Academia

Interactions with academics from other organisations are important for HEIs because:

• A great number of funding decisions and evaluation procedures include peer review processes. With new developments in HEIs (competitive allocation of grants, ex-post programme evaluations), peer review becomes more and more used in new fields. At the same time, the nature of many assessment processes has changed, as extended peer-review procedures include merit criteria and experts external to the scientific community.

• Research careers are no longer bound to national borders. Internationalisation of research fields and the necessity to attract (foreign) researchers in a more competitive environment push HEIs to specialisation and extensive networking.

Students

With the massification of HEIs, students formerly reduced to ‘pupil’ status have become an important factor in the governance of HEIs. However, this new role is different from trends to participatory opening of university governance in the 1970s that defined students as ‘partners’ in the operation of the university. As education is increasingly conceived in market terms, newer developments tend to define students as ‘users’ and universities compete to attract students through specific profiles and attractive learning environments. Important differences between countries exist depending on the existence and importance of tuition fees.



Other HEIs

With current globalisation processes and incentive schemes, networking and collaboration between HEIs has become increasingly important. This can take various forms ranging from informal research contacts to formalized agreements to facilitate exchange programmes and the building of consortia in order to enhance the opportunities to acquire public or private funding.

Private sector

The interaction of HEIs with the private sector is not a new phenomenon, but calls for “value-for-money” and the increasing importance of innovation processes in knowledge societies have given this dimension of HEIs’ interactions unprecedented importance. Private enterprises are increasingly seen as stakeholders in the strategic management of higher education systems and as partners for the commercialisation of research findings. Different types of interactions with HEIs can be distinguished:

• Research collaborations. Collaborative research includes R&D joint ventures, consultancy and contract work and informal relationships. Benefits of collaboration are often mutual (staff mobility, bi-directional knowledge flows, funding issues, etc.). The share of higher education R&D expenditure financed by industry is an indicator of linkages and shows wide variation across countries ranging from 37% in China to 1% in the Slovak Republic. Numbers also show the overall share of these activities has been remarkably stable since 1990 (6-7%). Another indicator is the proportion of innovating firms collaborating with HEIs. Variations from 33% in Finland to 5% in Spain reflect differences in the structures of innovation systems across countries. Finally, it is noteworthy that large firms tend to collaborate considerably more with HEIs than small ones (OECD, 2008).

• IP related interactions. The commercialisation of IP generated in HEIs through national guidelines for licensing, data collection systems, strong incentive structures and specific policy instruments like the Bayh-Dole Act (United States) has been a key policy focus in many countries over recent years. At the level of HEIs, this has led to an increase in patenting activities, the establishment or reorientation of technology transfer offices (TTOs), the creation of spin-offs designed to bring specific discoveries to the market, and interaction with venture capital and business angels to successfully commercialise them. However, the record is rather mixed. Patenting-related activities of TTOs, for example, have not brought the hoped-for revenue flows and only very few universities worldwide have successfully been able to generate significant revenues from patents and by commercialising inventions.



• Employment-related interactions. The private sector is traditionally important for HEIs as private companies are future employers of graduates. The existence of networks between HEIs and the private sector offering attractive possibilities for internships and job offers after graduation can be an important argument to attract students. More generally, human mobility is considered to be another important dimension of knowledge transfer. People hold tacit knowledge that is not readily transferable, and mobility between public and private sectors can contribute to the circulation of know-how and skills. The placement of researchers and research students in and out of the private sector is also a way to enhance absorptive capacity of companies.

Associations and civil society

Scientific knowledge becomes ever more important in public debates and political decision-making as social problems (like health issues, environmental problems, technological risks, etc.) are often at the same time associated with high uncertainties and high urgency. Barriers between academia, political spheres and civil society become less rigid as NGOs increasingly develop their own expertise, often collaborating with individual researchers or whole departments. Similarly, expert committees are created in all domains and at all levels of political decision-making in order to inform public debate. Another evolution has been the call for “democratization” of research through public engagement and stakeholder participation. This has led to new forms of interactions between HEIs and wider society (”citizen juries”, “focus groups”, “consensus conferences”, etc.).

Regions

There has recently been important debate about HEIs’ contributions to regional development. Several activities can create positive returns for regions:

1. Firstly, mass tertiary education provides local enterprises with a qualified workforce and thus contributes to regional human capital formation. This can be furthered through internship programmes, a focus on lifelong learning, and through closer HEI-business interactions. For example, in some regions industry plays a consultative role in curriculum design.

2. Secondly, HEIs are among the most important sources of innovation, and governments are rethinking how benefits from higher education can be maximized not only at the national, but also at the regional level. However, evidence suggests that direct economical impact of HEIs on regions in terms of innovation is often limited. Initiatives that have been taken to overcome this problem include the creation of science shops designed to facilitate access to knowledge to regional actors and the creation of science parks and incubators to take research findings to the market and simultaneously favour the development of local knowledge-intensive jobs.



3. Thirdly, HEIs can contribute to local communities, culture and environment. Involvement with the local social and natural environment can take several forms ranging from community service activities to local case studies and pilot projects (e.g. energy efficiency programmes, coastal zone management, etc.).

Evaluating HEIs’ performance

In a context marked by a shift towards NPM in many OECD governments and by a growing awareness of the role of HEIs in an ever-more competitive knowledge-driven global economy, accountability and performance evaluation are at the forefront in HEIs’ governance. Evaluation concerns all core activities of HEIs and has raised important challenges in terms of approaches, methods, actors and indicators involved.

Higher education quality assurance systems

Over the last decades, OECD governments have progressively developed sophisticated quality assurance systems that, mixing different mechanisms shifted responsibility of the assessment process from the institutes themselves to an external national quality agency and funding bodies, often relying upon a sequence of self-evaluation and site visits followed by a peer-review and the preparation of an evaluation report. Along with such systems, performance-based funding, market mechanisms, participation of external stakeholders, and publication of performance measures are other means used to pursue accountability of HEIs.

The issue of quality assurance is not without controversy, however, reflecting differences in perceptions of quality by different stakeholders. There is also debate about distinguishing between accountability-driven (centralised structures and external auditors measuring quantitative indicators of success) and improvement-driven (facilitative structures using peer review to assess more qualitative indicators of success) quality assurance mechanisms (Sachs 1994). Whereas in the first case the locus of control is external, in the second it is internal. The issue is whether and how a balance can be struck between instruments and methods used to accomplish both accountability and improvement. In most OECD countries, accountability-driven approaches tend to be dominant, although often used in combination with quality improvement approaches. In this respect, peer-review has been highlighted as a good way of bringing more legitimacy to external evaluation mechanisms. The UK’s higher education quality assurance system is the only predominantly improvement-oriented (OECD 2008a).



Arrangements of quality assurance of programmes and institutes across OECD countries often involve a mix orientations towards accreditation, assessment and audit:

• Accreditation: Most OECD countries use some form of accreditation process for the establishment of the status, legitimacy or appropriateness of an institute, programme or module of study. Such mechanism assesses whether a threshold standard and/or qualification for a certain status are met. Missions, resources, and procedures are examined. The outcome of the process is a yes/no decision with, sometimes, possible graduations. It is accountability-driven.

• Assessment (or evaluation): In many countries, processes for assessing the quality and appropriateness of the teaching and learning process are used in association with accreditation mechanisms (e.g. China, France, Mexico, Norway, Poland, Portugal, Spain and Sweden). This results in graded judgments about quality of outputs. Emphasis is often geared at disciplines and/or programmes, more than at institutes. It is well suited to both accountability and improvement objectives.

• Audit (or review): Audit processes have been adopted in a number of countries. The process consists in checking that procedures in an institute or programme are in place to assure quality of provision and outcomes. It assesses the extent to which procedures are effective in attaining results. The audit produces a qualitative description. It is an improvement-driven mechanism.

Table 3: Typology of quality assurance approaches

Activity Question Emphasis Outcomes

Accreditation Are you good enough to be approved?

Comprehensive (mission, resources, processes)

Yes/No or Pass/Fail decision

Assessment (Evaluation)

How good are your outputs?

Outputs

Grade (including Pass/Fail)

Audit (Review)

Are you achieving your own objectives? Are your processes effective?

Processes Description, qualitative

Source: OECD (2008a), Based on Woodhouse (1999).



The processes used in quality assurance vary and can involve any of the following:

• Processes of internal self-review and quality monitoring: Independently implemented internal quality assurance system can be used as a means to enhance quality, particularly in countries where regimes are improvement-driven. Guidelines have become common tools to assist HEIs in the design of their internal quality assurance systems, but also in carrying out their self-evaluations and preparing self-evaluation reports for the purpose of external evaluations.

• Stakeholders’ involvement in quality assurance: Groups of concerned actors can include, on the supply side, governments, quality assurance agencies, HEIs and individual academics, and on the demand side, students, graduates, employers, and civil society at large. Policies and practices differ widely across countries. However, the involvement of demand side stakeholders in the design and implementation of quality assurance activities has acquired great importance in the last few years, becoming best practice. Professional and industry bodies may be involved in the quality assurance of programmes preparing for occupations in which some form of professional accreditation is required.

• Rankings: Institutional and media rankings that combine different quantitative variables into a single “score” have become increasingly salient proxies of quality outside formal national assurance frameworks, with a strong impact on student and academic mobility at the global and regional levels. In particular, the powerful Shanghai Jiao Tong University ranking has lead to intensified global competition for getting “HiCi” researchers and Nobel Prize winners. However, the usefulness of rankings as proxies of quality is questionable. For example, they may mislead perceptions of HEIs’ quality by giving too much emphasis to research excellence. Worse still, they can distort incentives and lead HEIs to focus efforts on research at the expense of teaching and knowledge transfer in its various forms. Since rankings seem to be here to stay, different policy responses are being elaborated (e.g. developing quality rankings within the national quality assurance framework or publishing quality-related information at institute level).

• Performance indicators and statistical data: Performance indicators as policy tools in higher education play a major role in accountability. The most commonly used indicators of performance relate to completion rates and time needed for degree completion. These serve to assess student progress, dropout and graduation rates as well as destinations and employment rates of graduates. The use of these indicators is, however, poorly systematized and varies significantly across countries.



Measuring research quality, transfer and impact

The increased emphasis on efficiency and effectiveness evaluation of HEIs outputs has raised a number of important conceptual and methodological challenges as regards measuring research impact and knowledge transfer from HEIs.

In order to measure the quality of research in HEIs, most countries carry our periodic evaluations with different frequency (e.g. every year in the Czech Republic; every 8 years in Estonia, ad hoc in Finland) and on different units (e.g. the whole R&D system, the institute level, the research field, and the individual researcher). Most countries use publication data as an indicator to assess research quality. Peer reviews, awards and prizes, academic staff data and research student data are also used. Indicators relating to knowledge transfer and impact on innovation include patents, patent citations, and the relevance of research to business. The alignment of research with national strategic priorities is a less common indicator.

A general trend in the last few decades has been an increasing focus on measuring commercialisation and potential use of research outcomes and HEIs intellectual property. Such focus tends, however, to neglect the need to assess less tangible but equally important impacts. In this sense four main problems have been pointed out (OCDE 2006a). First, the effects of research often emerge long after the research has been completed. Second, a given innovation may draw upon many research projects and a given research project may affect many innovations. Third, because the beneficiaries of research may not be the people or organisations that perform the research, it may not be obvious where to look for effects. And fourth, in a given project portfolio, the distribution of impacts is typically highly skewed, as a small number of “blockbuster” projects may account for most effects, while others only advance knowledge in a general way.

How sensitive are the activities of HEIs to policy intervention?

HEIs are under growing pressure as they are exposed to various demands from different stakeholders. Despite major differences across countries, the OECD has identified common challenges and suggested policy directions, each implying specific trade-offs across the many facets of higher education policy – governance, funding, quality assurance, equity, research and innovation, academic career, links to the labour market and internationalisation. The table below presents them in a synthetic manner.



Table 4: Main policy challenges and policy directions in higher education policy

Domain Main challenges Main suggested policy directions

Steering tertiary education

Articulating the nation’s expectations of the tertiary education system

Aligning priorities of institutions with economic and social goals Creating coherent systems of HE

Finding the proper balance between steering and autonomy

Developing institutional governance arrangements to respond to external expectations

Develop a coherent strategic vision for tertiary education Establish sound instruments for steering tertiary education Ensure the coherence of the tertiary education system with extensive diversification Build system linkages Strengthen the ability of institutions to align with the national tertiary education strategy Build consensus over tertiary education policy

Funding tertiary education

Ensuring the long-term financial sustainability of tertiary education. Devising a funding strategy consistent with the goals of the tertiary education system Using public funds efficiently

Develop a funding strategy that facilitates the contribution of the tertiary system to society and the economy Use cost-sharing between the State and students to shape the funding of tertiary education Publicly subsidise tertiary programmes in relation to the benefits they bring to society Make institutional funding for instruction formula-driven (input and output indicators) and including strategically targeted components Improve cost-effectiveness

Back the overall funding approach with a comprehensive student support system




Quality of tertiary education

Developing quality assurance mechanisms for accountability and improvement

Generating a culture of quality and transparency Adapting quality assurance to diversity of offerings

Design a quality assurance framework consistent with the goals of tertiary education Develop a strong quality culture in the system (e.g. internal quality assurance mechanisms) Commit external quality assurance to an advisory role as the system gains maturity but retain strong external components in certain contexts Align quality assurance processes to the particular profile of TEIs Avoid fragmentation of the quality assurance organizational structure

Equity in tertiary education

Ensuring equality of opportunities.

Devising cost-sharing arrangements which do not harm equity of access Improving the participation of the least represented groups

Assess extent and origin of equity issues Strengthen the integration of planning between secondary and tertiary education systems Consider positive discrimination policies for particular groups Provide incentives for TEIs to widen participation

The role of tertiary education in research and innovation

Fostering research excellence and its relevance Building links with other research organisations, the private sector and industry

Improving the ability of tertiary education to disseminate the knowledge it creates

Improve knowledge diffusion rather than strengthening commercialization via stronger IPRs Improve and widen channels of interaction and encourage inter-institutional collaboration Use the tertiary education sector to foster the internationalization of R&D

Broaden the criteria used in research assessments Ensure the shift towards project-based funding is monitored and provide a mix of funding mechanisms

The academic career

Ensuring an adequate supply of academics Increasing flexibility in the management of human resources Helping academics to cope with the new demands

Give institutions ample autonomy over the management of human resources Reconcile academic freedom with institutions’ contributions to society Improve the entrance conditions of young academics

Develop mechanisms to support the work of academics




Links with the labour market

Including labour market perspectives and actors in tertiary education policy Ensuring the responsiveness of institutions to graduate labour market outcomes

Providing study opportunities for flexible, work-oriented study

Coordinate labour market and education policies Improve data and analysis about graduate labour market outcomes

Strengthen career services of HEIs Enhance provision with a labour market orientation

Include labour market perspectives and actors in policy development and institutional governance

Internationalisation of tertiary education

Designing a comprehensive internationalization strategy in accordance with country’s needs Ensuring quality across borders

Enhancing the international comparability of tertiary education

Develop a national strategy and comprehensive policy framework Improve national policy coordination Encourage HEIs to become proactive actors of internationalization Promote the national tertiary education system

Develop on-campus internationalization

Source: based on OECD (2008a).

Developing a coherent vision on how to ensure HEIs’ contribution to economic and social benefit is the overarching challenge. This may be achieved through policy intervention in different areas to foster coordination between HEIs and stakeholders (including SMEs, new firms, NGOs, concerned groups and community actors). Along with this comes a need to improve HEIs abilities to transfer knowledge and technology and move beyond commercialisation of IP. Making regional engagement an explicit objective of HEIs entails particular challenges and calls for changes in higher education legislation and mission strategies, the development of suitable indicators to assess impacts of HEIs on regions, and in some cases participation of community stakeholders in funding and quality assessment.

A highly sensitive point in this context is the trade-off between accountability and academic autonomy. This is especially important as competitive project-based research funding may have negative impacts on the long-term development of the research and innovation system (OECD 2008), since it tends to reward mainstream research over riskier breakthrough or interdisciplinary research. There is also increasing concern about negative impacts on investment in research infrastructure in HEIs.



Funding strategies may help to reconcile autonomy with accountability. Formula-based funding can protect allocation decisions from different pressures and help make the funding strategy clear to all stakeholders. Carefully linking performance evaluation, funding, and systematic system monitoring by stakeholders can help to articulate accountability and organisational autonomy. Performance-based mechanisms may, however, produce undesired effects on research and training quality. Linking publication numbers to funding may induce researchers to publish more, but not necessarily better. Likewise, incentives to increase university patenting have resulted in a rapid increase in ‘low quality’ patents being granted to universities (Henderson et al., 1998). The combination of peer review and indicators is a way of reducing perverse outcomes of assessment exercises.

Fostering research excellence and its relevance is critical. However, increasing focus on academic ‘excellence’ as a result of rankings and evaluation mechanisms has also raised several concerns as it may impact negatively on the status of teaching activities, the treatment of applied and interdisciplinary research, as well as on the construction and maintenance of industry and community linkages. Additionally, the need for universities to buy-in leading researchers to enhance their profiles is viewed with concern as it leads to the emergence of a transfer market for academics.

On a final note, the process of policy design should be based upon informed diagnosis, supported by adequate research evidence, and designed so as to be consistent with other policies in place. In order to build consensus and foster the chances of successful policy implementation, government authorities need to improve communication on the long-term vision of what is to be accomplished for higher education as the rationale for proposed reform packages.

Further resources

Dosi, G., P. Llerena and M. Sylos Labini (2005), “Science-Technology-Industry Links and the ‘European Paradox’: Some Notes on the Dynamics of Scientific and Technological Research in Europe”, LEM Working Paper Series, 2005/02, Italy.

Henderson, R., A. Jaffe and M. Trajtenberg (1998), “Universities as a Source of Commercial Technology: A Detailed Analysis of University Patenting, 1965-1988”, Review of Economics and Statistics, 80, pp.119-127.

Larédo, P. (2007), “Revisiting the Third Mission of Universities: Toward a Renewed Categorization of University Activities?”, Higher Education Policy 20, 441–456.

OECD (2009), Education Today. The OECD Perspective. OECD Publishing, Paris.

OECD (2008a), Tertiary Education for the Knowledge Society: Volume 1/2, OECD Publishing, Paris.

OECD (2008b), Education at a Glance: OECD Indicators, OECD Publishing, Paris.



OECD (2007a), Funding Systems and their Effects on Higher Education Systems – International Report, Education Working Paper No. 6, OECD Publishing, Paris.

OECD (2007b), Higher Education and Regions: Globally Competitive, Locally Engaged.

OECD (2006), Education Policy Analysis, OECD Publishing, Paris.

OECD (2004), Career Guidance and Public Policy: Bridging the Gap, OECD Publishing, Paris.

Polanyi, M. (1962), “The Republic of Science: Its Political and Economic Theory”, Minerva, Volume 1, pp.54-74.

Sachs, J. (1994), “Strange Yet Compatible Bedfellows: Quality Assurance and Quality Improvement”, Australian Universities’ Review, Vol. 37, No.1.

Salmi, J. and A. M. Hauptman (2006), “Resource Allocation Mechanisms in Tertiary Education: A Typology and an Assessment”, in Global University Network for Innovation (ed.), Higher Education in the World 2006, The Financing of Universities, Palgrave MacMillan, Basingstoke, U.K.

Woodhouse, D. (1999), “Quality and Quality Assurance”, Quality and Internationalisation in Higher Education, OECD, Paris.

Last updated: June 2011

actor brief: higher education institutes (heis) · the development of research activities with...

Documents