www.che.de

What Can National Rankings Learn from the U-Multirank-Project ?

Gero Federkeil, CHE, Germany

IREG-Forum: National University Rankings on the RiseBratislava, 10-11 Oct. 2011

2

Part 1

- U-Multirank basics

IREG Forum | Gero Federkeil | Bratislava 2011-10-10

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 3

The project

• Commissioned by the European Commission• 2-year project, 2009 – June 2011• Report now available:

http://ec.europa.eu/education/higher-education/doc/multirank_en.pdf

• Ján Figel, the former European Commissioner for Education, Training, Culture and Youth:

“- to allow stakeholders to make informed choices; - to help institutions to position themselves and improve their performance”

• Two phases:o Design of new instrumento Testing the feasibility of new instrument

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 4

Specification of U-Multirank

• Five dimensions:o Teaching & learningo Researcho Knowledge transfero International orientationo Regional engagement

• Long list of indicators to be tested in pilot project

• development of data collection tools and processes (question-naires, definitions, FAQs, communication + feedback processes)

• methods for building ranking groups instead of league tables

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 5

Testing U-Multirank

• Two levels:• Institution (FIR)• Fields (FBR)

• Global sample of higher education and research institutions: • 159 (target: 150), 2/3 Europe, • 109 completed institutional questionnaires

• Two fields: • Business studies• Engineering (electrical and mechanical)

Bais logic: Mapping Diversity

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 6

Diversity of higher education institutions in Europe & the world

Identifying comparable institutions that can be

compared in one ranking

CLASSIFICATION

Description of horizontal diversity

Types/profiles

RANKINGS

Assessment of vertical diversity

Performance

Complementary instruments of transparency

+

Mapping and Ranking

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 7

Mapping: Selection of a comparable set of universities based on institutional profiles

Teaching and learning

Research involvement

Knowledge exchange

Regional engagement

International orientation

Student profile

Example:

• Comprehensive, teaching oriented institution• Mainly undergraduate education• Low research orientation• Low international orientation• Regionalyl embedded (e.g. recruiting)

Subset of comparable institutions to be compared in a ranking

Mapping and Ranking

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 8

Ranking: Multi-dimensional ranking for subset of institutions

No composite

indicator!

No number 1 !

9

Part 2

(What) can national rankings learn from U-Multirank?

IREG Forum | Gero Federkeil | Bratislava 2011-10-10

Mapping and Ranking

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 10

Most national HE systems are diversified HE systems: Different types/profiles of institutions exist

Need to identify comparable institutions for ranking

Mapping systems can increase the comparabiliy and improve the quality of rankings

U-Map defines indicators for mapping & is setting a standard for Europe

Multi-dimensional Approach

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 11

• Multi-Multirank identified a set of indicators for 5 dimensions

• U-Multirank introduced 2 „new“ dimensions:

• knowledge transfer• regional engagement

• Indicators have been discussed intensively with stakholders

Innovative indicators

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 12

Teaching and learning:

For rankings which want to inform (prospective) students indicators based on students‘ assessment of their teaching and learning experience are highly useful and are feasible (in most settings)

Knowledge transfer:

• Joint publications with industry• Research funds from industry

• But problems with regard to data (e.g. on spin offs/licenes)

Innovative indicators

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 13

International Orientation

Rating indicator on international orientation of programmes is more meaningful than linear ranking of number of int. students

Regional Engagement:

• Important for many HEIs yet most problematic dimension in U-Multirank

• Bibliometric indicator: Regional co-publications• Further development is necessary

User-driven Approach

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 14

• Intensive stakeholder consultation helped to increase acceptance

• Multi-dimensional, personalised rankings allow• individual users to produce ranking based on their own

preferences and • networks and aossciations of universities to start

benchmarking / create their own ranking

Data collection

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 15

• International rankings have to rely on self-reported data due to lack of international data bases (except bibliometric, patent data)

• Feedback loop with universities concerning self-reported data on institution, faculties & programmes helped to increase consistency & quality of data

• Parallel / conflicting national data collections (e.g. student surveys)

raises issue of coordination national – international rankings in general

16

Outlook

IREG Forum | Gero Federkeil | Bratislava 2011-10-10

Outlook: National rankings and U-Multirank

IREG Forum | Gero Federkeil | Bratislava 2011-10-10 17

• There will be a continuing demand for national rankings !

• Definition of a core set of indicators for national rankings and U-Multirank?

• Network of national rankings, e.g. Germany – Austria – Switzerland - Netherlands – Spain …. that share data which can be used for U-Multirank

18

Thank you very much!

More information:

Gero.federkeil@che-ranking.de

www.che-ranking.de

www.u-multirank.eu

www.che.de

What Can National Rankings Learn from the U-Multirank-Project ?

Gero Federkeil, CHE, Germany

IREG-Forum: National University Rankings on the RiseBratislava, 10-11 Oct. 2011