u multirank pptx ankara11 14
TRANSCRIPT
1 @UMultirank
U-Multirank: ideas and experiences
Prof. Dr. Frank Ziegele
Ankara, November 2014
Presented by:
www.umultirank.org
Part One U-Multirank, why and how Part Two U-Multirank in practice Part Three Launching U-Multirank Part Four What’s to come
2
Agenda
@UMultirank
• Rankings are here to stay
• They receive high public attention
• They have impacts on HEIs (obsession about “world class universities”)
• They have political impacts (e.g. on excellence initiatives, mergers, immigration policies; see EUA report)
• But until now there were no rankings that fulfilled certain functions
• The current rankings do not help to support decision making
• They don’t take into account diversity of stakeholders’ preferences and HEIs’ missions
@UMultirank #UMRinAthens 4
Why another ranking?
U-Multirank is a transparency tool for many stakeholders
• Helps students to identify study-programmes
• Assists university leaders in strategic management
• Serves policy-makers in developing higher education, research and innovation policies
• Assists business & industry in finding partnerships
• Serves society with accurate open access information
@UMultirank #UMRinAthens 5
6
Teaching and learning Research
Knowledge transfer International orientation
Regional engagement
“University X is the fifth best in the world”
Multi-dimensional ranking: ranks on 30 individual indicators (performance measures) in
five dimensions of performance
@UMultirank #UMRinAthens
A new instrument to compare university performances
7
“Indicator 1 counts for 20%, indicator 2 for 30%, etc., altogether this leads to a score of X”
No composite overall scores, transparency lost if scores are weighted / added
User-driven: user decides on areas of performance to compare (and on the kind of
university to be compared) – YOUR WAY!
@UMultirank #UMRinAthens
A new instrument to compare university performances
8
ranking performances of universities as a whole
ranking performances in specific fields or disciplines
@UMultirank #UMRinAthens
A new instrument to compare university performances
9
The “global system” of higher education is characterised by an enormous diversity in university profiles and missions
Diverse preferences and needs of students, university leaders, academics, business and industry, governments etc.
Different groups of stakeholders are
interested in different types of
performances by universities
But diversity is not yet
transparent or understood
@UMultirank #UMRinAthens
Higher education needs this new kind of comparison
@UMultirank 10
Quotes representing the basic ideas
Dr. José Joaquín Brunner, Director Cátedra UNESCO de Políticas Comparadas de Educación Superior, Chile,
“U-Multirank is a great instrument for students, faculty, the public and policy makers.”
Dr. Simon Marginson, Professor of International Higher Education at the Institute of Education, The University of London,
“U-Multirank gives students and other higher education stakeholders much more information to work with in
making considered decisions.”
target groups
informed decisions
@UMultirank 11
Quotes representing the basic ideas
Prof. David D. Dill, Professor Emeritus of Public Policy University of North Carolina, Chapel Hill, USA,
“U-Multirank permits students and academic staff to compare institutions of similar missions and types…”
institutional diversity, like with
like
Fernando M. Galán, Vice-Chairperson of the European Students Union (ESU)
“U-Multirank will be an extremely useful tool for students all over the world, being the first global
ranking that includes in a serious manner the teaching and learning dimension.”
more than research
@UMultirank 12
Quotes representing the basic ideas
Andreas Schleicher, Director Education and Skills and Special Advisor on Education Policy at the OECD,
“With a first-of-its-kind multi-dimensional approach to comparing institutional performance, U-Multirank is now putting students and
the public into the driver’s seat of determining which universities are doing well on what and why.”
Professor Sir Howard Newby, Vice-Chancellor, University of Liverpool, UK,
“U-Multirank provides an elegant solution to a problem which undermines conventional rankings. It does not seek to weight
different activities of a university but allows users to devise their own ranking according to what they believe is important."
multi-dimensional, user-driven
U-Multirank indicators show diversity of performance profiles
@UMultirank
Teaching and Learning Institutional
ranking Field-based
ranking
• Student-staff-ratio
• Graduation rate (BA and - separately - MA)
• Percentage of academic staff with PhD
• Percentage of students graduating within
normative period (BA and –separately - MA)
• Rate of graduate employment
• Inclusion of work experience in degree programme
13
@UMultirank
Teaching and Learning – Student Satisfaction Indicators Institutional
ranking Field-based
ranking
Overall learning experience
Quality of courses & teaching
Organisation of the programme
Contact to teachers
Social climate
Facilities (libraries, laboratories, rooms, IT)
Research orientation of teaching /programme
Inclusion of work experience /practical elements
14
U-Multirank indicators show diversity of performance profiles
Research Institutional
ranking Field-based
ranking
• External research income (per fte academic staff)
• Doctorate productivity
• Total research publication output (per fte
academic staff)*
• Art related output
• Field-normalised citation rate*
• Highly cited research publications *
• Interdisciplinary research publications*
• Research orientation of teaching (student survey)
• Number of post-doc positions
@UMultirank 15
U-Multirank indicators show diversity of performance profiles
@UMultirank
Knowledge Transfer Institutional
ranking Field-based
ranking
• Income from private sources (service contracts,
consultancies, licenses, royalties, trials, etc.)
• Joint research publications with industry*
• Patents (per fte academic staff)
• Co-patents with industry (per fte academic staff)
• Number of spin-offs (average over three year
period)
• Patent citations to research publications*
• Revenues from Continuous Professional
Development
U-Multirank indicators show diversity of performance profiles
16
@UMultirank
International Orientation Institutional
ranking Field-based
ranking
• Educational programmes (BA/MA) in foreign
language
• International orientation of degree programmes
• Opportunities to study abroad (student survey)
• Student mobility (composite of incoming,
outgoing, joint degree students)
• Percentage of international academic staff
• Percentage of PhDs awarded to foreign students
• International joint research publications*
• International research grants
17
U-Multirank indicators show diversity of performance profiles
@UMultirank
Regional Engagement Institutional
ranking Field-based
ranking
• Percentage of graduates working in the region
• Student internships in regional enterprises
• Degree theses in cooperation with regional
industry
• Regional joint research publications*
Income from regional sources
18
U-Multirank indicators show diversity of performance profiles
@UMultirank 19
and how does it really work?
Part 2: U-Multirank in practice
live presentation of web tool
21 21
universities are included with publicly available data 850
of the 879 universities have provided comprehensive data 500
faculties/departments are included in the four field based rankings (physics, electrical and mechanical engineering, business studies)
1,000
study programmes within these faculties are included 5,000
students completed the student satisfaction survey 60,000
70 countries have universities in U-Multirank
@UMultirank
U-Multirank 2014 is the most comprehensive international data comparison in higher education
Alex Usher, President of Higher Education Strategy Associates (HESA), Canada “U-Multirank is the most global, most diverse and least reductive ranking ever.”
22
Worldwide coverage
In 2014, 62% of U-Multirank universities are from Europe, 17% from North America, 14% from Asia and 7% from Africa, Latin America and Oceania
PhD-awarding institutions
+ universities of
applied sciences
small +
large universities specialised institutions
+ comprehensive
universities old (pre 1870)
+ young institutions
(after 1980)
@UMultirank
U-Multirank offers a global view of a large variety of university profiles
23
Turkish universities in U-Multirank – Full participation
Name of the University Participation 2014
Bilkent U Institutional, Business, Electrical Engineering
Hacettepe U Institutional, Business, Electrical & Mechanical Engineering
Istanbul Aydin U Business
Istanbul U Institutional, Business, Electrical & Mechanical Engineering, Physics
Koç U Institutional, Business, Electrical Engineering, Physics
Kocaeli U Institutional, Business, Electrical & Mechanical Engineering, Physics
Mugla Sitki Kocman U Institutional, Business, Physics
Sabanci U Institutional, Business, Electrical & Mechanical Engineering
@UMultirank
+ 4 universities only with public data
+ 16 new registrations in 2015
24
Results Institutional Ranking 2014 – Full participant
Name of the
University
Nr of A’s
Teaching &
Learning
Nr of A’s
Research
Nr of A’s
Knowledge
Transfer
Nr of A’s
International
Orientation
Nr of A’s
Regional
Engagement
Bilkent U 1 / 4 2 / 8 2 / 8 3 / 6 3 / 5
Hacettepe U 3 / 4 4 / 8 1 / 8 1 / 6 3 / 5
Istanbul U 0 / 4 1 / 8 2 / 8 1 / 6 4 / 5
Koç U 1 / 4 2 / 8 1 / 8 4 / 6 3 / 5
Kocaeli U 1 / 4 1 / 8 1 / 8 1 / 6 1 / 5
Mugla Sitki Kocman U
1 / 4 0 / 8 0 / 8 0 / 6 2 / 5
Sabanci U 3 / 4 4 / 8 3 / 8 2 / 6 1 / 5
Results: Webtool
@UMultirank
25
Many univs. have specific strengths
No univ. has “across the board”
“A” scores
Around 100 univs. have a wide range (>10) of “A” scores
Very few univs. with no “A” scores
On individual indicators performance differences between univs. are clearly
visible
300 univs. never before seen in global rankings. Of these, 30 have
more than 10 “A” scores
@UMultirank
U-Multirank shows a wide distribution of “A” scores
“A” scores per univ. % of univs.
0 2 %
1 to 5 43 %
6 to 10 43 %
11 to 15 10 %
16 to 22 2 %
Tom Boland, Chief Executive of the Higher Education Authority, Ireland, “U-Multirank demonstrates very well that different universities have different kinds of strengths.”
26 @UMultirank
The difference in university profiles becomes visible
You can’t say that one university is better than the other! You can‘t find the world‘s top 100!
And many special
analyses…
27
Comparisons per
university profile
Comparisons per
dimension
Comparisons per country
Comparisons per indicator
Comparisons per
field/discipline
@UMultirank
U-Multirank offers a wealth of information and the potential for many more analyses
Thomas D. Parker, Senior Associate at the Institute for Higher Education Policy, Washington, USA “U-Multirank is the ‘thinking man’s ranking’. Until now, rankings have been somewhere between bad
journalism and bad science, and we’re beginning to see much more serious work.”
U-Multirank also provides specific benefits to participating institutions
Basic benefits • Visibility of institutions with different profiles, marketing use • Visibility of specific profiles: tool to look for “similar” institutions • Comparison of performance with like • Allowing differentiated SW-analysis, evidence-based discussions both
on institutional and on field level
Additional analysis of own data
• Compared with the total sample • Institutional & field-based data • Detailed analysis of student survey
Support for benchmarking processes
• Either for networks or organisations • Multi-level: institutional and fields @UMultirank
28
• cost of data collection differs largely (according to implemented data systems)
• many indicators without problems, high degrees of completeness
• difficult indicators: graduate unemployment, internships in the region, degree theses with local companies (also definition of “region”)
• some challenges with response rates to student surveys (parallel surveys, e.g. effort to use British NSS data); if response is too small we do not show data (statistical checks)
• intensive communication loops
• challenges of institutional structures (mergers, internal) 29 @UMultirank
Experiences with data collection
@UMultirank 31
U-Multirank will be extended in 2015
+ 150 institutions (at the moment
>280 new regsitrations)
+ 3 new fields (psychology,
computer science and medicine)
Current sample of 879 institutions
U-Multirank 2015 release in March 2015
next chance to
join: register for 2016
@UMultirank 32
• For more information on U-Multirank visit us, at www.umultirank.org.
• Contact:
• +49 (0) 5241 9761 58
• Like/follow us:
• /U-Multirank
• @UMultirank
• @Umultirank
More information/Contact