©2013 michael fung rankings and benchmarking workshop – vietnam rankings workshop strategies for...

©2013 MICHAEL FUNGRANKINGS AND BENCHMARKING WORKSHOP – VIETNAM 1

RANKINGS WORKSHOPSTRATEGIES FOR MARKETING, BRANDING, AND

IMPROVING INTERNATIONAL RANKINGS

MICHAEL FUNGDIRECTOR OF PLANNING & INSTITUTIONAL RESEARCH

THE HONG KONG UNIVERSITY OF SCIENCE AND TECHINOLOGY

26 MARCH 2013VIETNAM NATIONAL UNIVERSITY

HO CHI MIN CITY, VIETNAM


INTRODUCTIONS


INTRODUCTIONS

• Acknowledgements– VNU– INFEQA– QS


ABOUT TRAINER

• Hong Kong University of Science & Technology (HKUST)

• Singapore Management University (SMU)• Singapore Government• Carnegie Mellon University (CMU)


ABOUT HKUST

• Established in 1991• 4 Schools – Science, Engineering, Business,

Humanities & Social Science• Total Enrolment: 12,603 (UG 8,699; PG 3,904)• Faculty: 537 (core), >1,200 (extended)• 60-hectare campus

*Figures as at Sep/Oct 2012


HKUST – INTERNATIONALIZATION

• Highly internationalized mix of students (16%, 56 countries) and staff (80% non-local, 80% US/UK PhDs)

• Operations in Nansha-Guangdong, Shenzhen, Hangzhou-Zhejiang, Foshan, Xian

• >200 worldwide partners, 600 exchange students on campus every semester

• 36% of UG students have one or more semester of international experience, mid-term target of 50%

• Language of instruction – English


HKUST – BRAND

• Focused, elite, research university• International university with strong global and

Mainland ties• Interdisciplinary, entrepreneurial spirit, innovative

thinking• Institute for Advanced Study – incubator for top

scientific leaders and research breakthroughs• Spectacular physical setting and environment


HKUST – RANKINGS

99

QS World University Rankings – overall 33

QS World University Rankings by Subject – Chemistry 11

– Computer Science 13

– Statistics & Op. Research 14

QS Asian University Rankings – overall 1

– Engineering 7

QS Top 50 Under 50 2

Times Higher Education World University Rankings – overall 65

– Engineering 23

– Social Sciences 50

THE Top 100 Under 50 3

SJTU Academic Ranking of World Universities – overall 276

– Computer Science 26

– Economics & Business 48

Financial Times MBA 10

Financial Times EMBA 1

UTD Top 100 Business Schools 17

Top in the World Top in Asia Top in HK


PARTICIPANTS INTRODUCTION

• Introduce yourself – name, organization, role• What you hope to get out of this workshop• Keep within 20 seconds per person


SESSIONS OUTLINE

1) The global higher education and rankings landscape

2) Rankings and methodologies3) Strategies to improve rankings performance4) Driving change – the human factor5) Planning for action


LEARNING OBJECTIVES

• Overview of global trends in higher education and international rankings of universities

• Understanding of various rankings and associated methodologies

• Insights and strategies to improve rankings performance

• Knowledge on bringing about organizational change to respond positively to rankings


GROUND RULES

• Be present, be engaged• Sharing, learn from one another• Open to ideas, suspend judgment• Listen intently, speak constructively• Reflect, think about how to take action


GLOBAL HIGHER EDUCATION LANDSCAPE


THE GLOBAL ENVIRONMENT

• What are the observed changes in recent 5-10 years?

• What are the driving forces leading to these changes?

• Which factors are most relevant for Vietnam?



• Globalization– Economies– Internet– Information & knowledge– Language – English

• Internationalization– Branch campuses– Offshore programs, franchises



• Massification– The promise of higher education– Aspirations of students, parents– Growing range of degree-granting institutions– Private providers – national interest, quality assurance– No longer elite?

• Mobility– International students– Competition for staff– Migration of talents and strengths



• Alternative access, learning modes– Private providers– Distance programs– Online / MOOCs

• Funding, resources– Traditional sources– New sources


THE RANKINGS LANDSCAPE

• What has happened in the recent 5 years around university rankings?

• What do you think are the driving forces behind these developments?



• In the Beginning….– US News & World Report - College & University

Rankings– The Guardian University Guide– The Times Good University Guide



• Today….– Academic Ranking of World Universities, G-factor,

HEEACT—Ranking of Scientific Papers, High Impact Universities: Research Performance Index, Newsweek, Professional Ranking of World Universities, QS World / Asian University Rankings, SCImago Institutions Rankings, Times Higher Education World University/Reputation Rankings, Webometrics, …. …and the list goes on….



• Trends – Localized Globalized – Limited phenomenon Worldwide proliferation



• Impact– Students, parents, donors: Perception = Reality?– Funding and quality assurance agencies:

Expectations and pressure to act and react– Institutions: Branding and marketing, respond to

public commentary– Stakeholders: Interest and responses keeps the

business going



• Drivers for the proliferation of rankings– Relative ease of data collection, publication – Simplification of a complex construct –

quantitative and qualitative comparisons – Inclination towards “mind-less” approximation of

the real world – Salience, perception, media coverage – Internationalization of higher education


RANKINGS & METHODOLOGIES


MAJOR RANKINGS

• Quacquarelli Symonds (QS)– World University Rankings (WUR)– Asian University Rankings (AUR)– Subject rankings– Top 50 under 50


MAJOR RANKINGS

• Times Higher Education (THE)– World University Rankings (WUR)– World Reputation Rankings (WRR)– Subject rankings– Top 100 under 50


MAJOR RANKINGS

• Shanghai Jiaotong University (SJTU)– Academic Rankings of World Universities (ARWU)

• Financial Times (FT)– Global MBA Rankings– Global EMBA Rankings


OTHER RANKINGS

• National Taiwan University Ranking (NTUR)

• SCImago Institutions Rankings (SIR)

• Leiden Ranking


OTHER RANKINGS

• University of Texas Dallas (UTD)– Top 100 Business School Research Rankings

• Tilburg University (TU)– Top 100 Worldwide Economics Schools Research

Ranking

• Nature Publishing Index


January

Financial Times MBA Ranking

February

March

THE World Reputation Rankings

UTD Top 100 Business School

Rankings

Nature Publishing Index

April

May

QS Asia University Rankings

QS Top 50 Under 50

Tilburg U Top 100 Worldwide Econ.

Sch. Research

June

QS World University

Rankings by Subject

THE Top 100 Under 50

July

SJTU Academic Ranking of World

Universities

August September

QS World University Rankings

SCImago Institutions

Rankings

October

THE World University Rankings

Financial Times EMBA Ranking

November

National Taiwan University Ranking

December

Leiden Ranking

RANKINGS RELEASE TIMELINES


• Established in 2004

• Published by annually around September

• Ranks over 700 universities around the world

• Types of rankings:

– World University Rankings

– Reputational Rankings by Faculty Area

– World University Rankings by Subject

– Top 50 Under 50

QS WORLD UNIVERSITY RANKINGS


Overall Ranking

Performance Indicators

Descriptions

Research Quality (60%)

Academic Reputation

40% Entirely based on Academic Peer Review surveyAll previous respondents are invited to provide

viewpoint on the quality of universities in their broad field.

Citations per paper

20% Citation and paper counts in a 5-year periodScopus is the major source for calculation

Teaching Quality

Student-Faculty Ratio

20% No. of UG and PG students in FTE divided by FTE faculty

Graduates Employability

Global Employer Review

10% Entirely based on online survey of graduates’ employers incl. domestic & international from various sectors.

International-ization(10%)

International Faculty

5% Normalize against the total faculty and student population.

International Students

5%




QS World University Rankings (2012)

1 2 3 5

33 47 6395 97

137 156 159155

Note: Bar height represents scores; numbers represent rank positions


Reputational Rankings by Faculty Area

Entirely driven by the academic reputation survey.

Five faculty area:– Arts & Humanities– Engineering & Technology– Life Sciences & Medicine– Natural Sciences– Social Sciences &

Management

WUR By Subject (29 subjects)



QS Top 50 Under 50 University Rankings– Ranks the first Top 50 young Universities (under 50 yrs-old)– Same methodology and weightages as QS WUR


Under 50 rank

QS 2011 rank

Institution CountryFounding

year


Survey samples cover global regions, with Asia being relatively under-represented

2012 Academic responses by country and region



2012 Academic responses by faculty and subject area

21%

28%

18%

23%

30%



• Characteristics– Strong bias on reputation/ perception (carrying the weight of 50%)– Favour older and well known institutions, esp. at the subject level


0102030405060708090

100

QS Academic and Employer Reputation Scores (2012)

Academic Reputation Employer Reputation


• Characteristics– Strong bias on reputation/ perception (carrying the weight of 50%)– Favour older and well known institutions, esp. at the subject level

0102030405060708090

100

QS University Subject Rankings Scores (2012)

Engineering Social Science & Mgt Life Science



QS ASIAN UNIVERSITY RANKINGS


• Released annually around May since 2009• Ranks 400 universities in Asia

Overall Ranking Performance Indicators

Descriptions

Research Quality (60%)

Academic Peer Review 30% Survey distributed to previous respondents & subscribers of 2 key databases: 1) The World Scientific & 2) the Int’l Book Info. Service (IBIS)

Papers per Faculty 15% Citation and paper counts in a 5-year periodExtracted from Scopus database

Citations per paper 15%

Teaching Quality Student-Faculty Ratio 20% No. of UG and PG students in FTE divided by FTE faculty

Graduates Employability

Asian Employer Review

10% Entirely bases on online survey of graduates’ employers incl. domestic & international from various sectors.

Internationalization(10%)

International Faculty & Students

2.5% 2.5%

Each Indicator is factored against the overall number of faculty or student to normalize the size of institutions.

Exchange Students – in & out

2.5% 2.5%




QS Asian University Rankings (2012)

1 2 10 12 16 2029 31

417068

95





QS Asian University Rankings (2012)

3843

91

475956

65

3445

36

8678 89


Reputational Rankings by Faculty Area– Entirely driven by the academic reputation survey– Five areas:

• Arts & Humanities• Engineering & IT• Life Sciences & Medicine• Natural Sciences• Social Sciences & Management

• Characteristics– Strong bias on reputation/perception (carrying the weight of 40%)– Favour well known institutions– Survey samples from Asian regions that would be more representative

for Asian institutions



• Published by Times Higher Education since 2010 (split with QS in 2009)

• Ranks 400 top universities around the world

• Types of rankings: – THE World University Ranking– THE World University Ranking by Subject– THE World Reputation Rankings– THE Top 100 Under 50

TIMES HIGHER EDUCATION RANKINGS


Performance Indicators Remarks

Teaching (30%) Reputation Survey 15% Similar to QS WUR’s Academic Peer Review survey

PhD awards per academic 6% Normalized to reflect the variation btw disciplines.

UG per academic 4.5%

Income per academic 2.25%

PhD awards / UG awards 2.25%

Research (30%) Reputation survey 18%

Research Income (Scaled) 6% Normalized to reflect the variation btw disciplines.

Papers per academic + research staff 6%

Citations Citation impact (normalized average citations per paper)

30% The total no. of papers of each university is scaled against its size.

Citation counts in a 6-year periodIndustry income

Industry research income per academic

2.5%

International Outlook (7.5%)

Int’l domestic staff ratioInt’l domestic student ratioInt’l co-authored papers ratio

2.5%2.5%2.5% New indicator introduced in 2011





0

10

20

30

40

50

60

70

80

90

100

Times Higher Education World University Rankings (2012)

5 7

391

65124

68

182

1

29

4

242239 254


Survey Respondents’ Demographics



World University Rankings by Subject – Only ranks the top 50 universities of each of the 6 subjects

– Arts & Humanities– Engineering & Technology– Life Sciences– Physical Sciences– Clinical, Pre-Clinical & Health– Social Sciences

THE World Reputation Rankings• Introduced in March 2011• Teaching reputation and research reputation • Entirely based on the Reputation survey, i.e. the no. of times an institution is

cited by respondents as being "the best of (max.) 15 institutions " in their field of expertise.


©2013 MICHAEL FUNGRANKINGS AND BENCHMARKING WORKSHOP – VIETNAM53

Emphasis on both teaching and research reputation (constitute 33% for the overall ranking)



Top100 Under 50 ranking– Ranks the top 100 young Universities (under 50 yrs-old)– The same 13 indicators but different weightages, e.g. teaching

reputation and research reputation had been reduced from 15% to 10% and 18% to 12% respectively



• Established in 2003• Published by Shanghai Jiaotong University annually in

August• Ranks 500 universities around the world • Types of Rankings

– World University Rankings– Rankings by Field– Rankings by Subject

SHANGHAI JIAOTONG RANKINGS


Overall by Field

by Subject

Descriptions

Alumni 10% 10% - Alumni winning Nobel Prizes and Fields Medals

Award 20% 15% - Staff winning Nobel Prizes and Fields Medals

HiCi 20% 25% 25% Highly cited researchers in 21 broad subject categories

N&S 20% - - Papers published in Nature and Science

PUB 20% 25% 25% Papers indexed in Science Citation Index-expanded and Social Science Citation Index

PCP 10% - - Per capita academic performance of an institution

TOP - 25% 25% % of papers published in top 20% journals

Fund - - 25% Research expenditures





SJTU Academic Ranking of World Universities (2012)

1

3 5 610

2051 70 142 228 276 299 321 431


• Characteristics– Heavy Emphasis on awards of Nobel Prizes, Fields Medals and

science research– Favour research intensive institutions with long history of notable

faculty and alumni

ARWU by Field (#5) ARWU by Subject (#5) Clinical Medicine and Pharmacy Engineering/Technology and Computer Sciences Life and Agriculture Sciences Natural Sciences and Mathematics Social Sciences

Chemistry Computer Science Economics / Business Mathematics Physics




0102030405060708090

100

ARWU rank

ARWU - Alumni & Award Scores (2012)

Alumni winning awards Staff winning awards


• Release annually in January; published since 2001• Ranks 100 MBA programmes based on 20 indicators• Criteria of the participating the FT MBA Ranking:

– Full-time programme – In operation for at least 4 years– with first batch of graduates graduated at least 3 years ago– with a class size of at least 30 students

• Characteristics– Emphasis on the alumni’s current salaries and its’ increase

btw pre- and post-MBA (carrying the weight of 40%)– Heavily relies on the graduate survey input (55%)

FINANCIAL TIMES MBA


• Release annually in October; published since 2001• Based on the online survey done by the school and

alumni graduated 3 years ago• Ranks 100 EMBA programmes based on 16 indicators• Criterion of the participating the FT EMBA Ranking:

– At least 20% response rate from alumni

• Characteristics– Emphasis on the alumni’s current salaries and its’ increase

btw pre- and post-EMBA (carrying the weight of 40%)– Heavily relies on the graduate survey input (55%)

FINANCIAL TIMES EMBA


FINANCIAL TIMES MBA


Category Performance Indicators

MBA EMBA Descriptions

Alumni Salaries & Career Develop-ment (55%)

Weighted Salary/Salary today (US$)

20% 20% Alumni salary today with adjustment for salary variations between industry sectors. / Average salary 3 years after gradation

Salary percentage increase

20% 20% % increase in average alumni salary between pre and post of the programme

Value for money 3% - Salary earned by alumni today, course length, fees and opportunity cost of not working for the course

Career progress 3% 5% Changes in seniority level and company size in which alumni are working now versus before they studied MBA

Work Experience - 5% Prior work exp., e.g. seniority of positions, no. of year of same post, company size, and int’l work exp.

Aims achieved 3% 5% Extent to which alumni fulfill their goals for studying an MBA

Placement success

2% - Alumni evaluation on the effectiveness of the school career services

Employed at three months

2% - % of the alumni found employment within three months of graduation

Alumni recommend

2% - Alumni to name 3 business schools they would recruit MBA graduates

FINANCIAL TIMES MBA / EMBA


Category Performance Indicators MBA EMBA Descriptions

Diversity & Inter-national Research of the school and its MBA (25%)

Women faculty 2% 3% % of female faculty

Women students 2% 3% % of female students

Women board 1% 1% % of female members on the advisory board

International faculty 4% 5% % of faculty from other countries

International students 4% 5% % of students from other countries

International board 2% 1% % of the board from other countries

International mobility 6% - Employment movements of alumni between pre- and post-MBA.

International experience 2% 5% Weighted average of four criteria that measure international exposure of the program

Languages 2% 1% No. of extra languages required on completion of the MBA

Research Cap-abilities (20%)

Faculty with doctorates 5% 5% % of faculty with a doctoral degree.

FT doctoral rank 5% 5% No. of doctoral graduates in the past 3 years. Additional points will be given if graduates took up faculty position in one of the top 50 FT MBA schools.

FT research rank 10% 10% No. of faculty publication in 45 academic and practitioner journals; the total is weighted for faculty size.



FINANCIAL TIMES MBA / EMBAFinancial Times MBA Rankings (2013)

16 8 15

27 3136

52

897895



Financial Times EMBA Rankings (2012)

1 4 6 717

2635

54

7970

86


• Published by National Taiwan University annually around November• Mainly evaluates and ranks the scientific paper performance of the top

500 universities worldwide. • Consists of 8 indicators to come up with the overall ranking, 6 rankings by

field, and 14 rankings by subject category.

Performance Indicators Remarks

Research Productivity (25%)

Number of articles of the last 11 years 10% Data Source: ESI, for the Science Citation Index (SCI) and Social Sciences Citation Index (SSCI).

Number of articles of the current year 15%

Research impact (35%) Number of citations of the last 11 years 15%

Number of citations of the last 2 years 10%

Average number of citations of the last 11 years 10%

Research excellence (30%)

h-index of the last 2 years 10% h-index : measure both the productivity and impact of the publicationNumber of Highly Cited Papers 15%

Number of articles of the current year in high-impact journals

15%

NATIONAL TAIWAN UNIVERSITY RANKING


• Characteristics– Consider the scientific paper performance in both short-term and

long-term period (recent years and past 11 years) – Favour science intensive institutions with longer history– Institutions without subjects of agricultural, plant & animal,

pharmacology & toxicology and etc., would be disadvantaged.

NTUR by Field (#6)

NTUR by Subject (#14)

Agriculture Clinical Medicine Engineering Life Sciences Natural Sciences Social Sciences

Agricultural Sciences Chemical Engineering Chemistry Computer Science Civil Engineering Electrical Engineering Environment/Ecology

Geosciences Physics Plant & Animal Science Materials Science Mechanical Engineering Mathematics Pharmacology/ Toxicology

NATIONAL TAIWAN UNIVERSITY RANKING


• Published annually around September; published since 2009• Analyzes and evaluates the research performance of institutions

Methodology • Based on no. of research output tracked by Scopus• Three indicators were published (not contribute to the rankings):

Output The no. of scientific papers published in scholarly journals reveals the ability of an institution to produce scientific knowledge. Output values are affected by institution sizes and research profiles, among others factors.

IC International Collaboration – ratio of institution's output that collaborates with foreign institutions

Q1 Ratio of publications that are published in the most influential scholarly journals of the world – top quartile (25%) of journals by discipline, according to journal prestige (SJR)

NI Normalized Impact – ratio between the average scientific impact of an institution and the world average impact of publications of the same time period and subject area

SCIMAGO INSTITUTIONS RANKING


• Characteristics– Emphasis on the quantity of research output and

international collaborations, regardless of the number of citation and the impact of individual publication

– Favour research intensive institutions with more research faculty

SCIMAGO INSTITUTIONS RANKING


• Based on the publication in Thomson Reuters’ Web of Science database in a moving 5-year window.

• Consider only the document types as article, letter, and review.

• Mainly evaluates the publication performance in Science and Social Sciences of 500 major universities worldwide.

• Arts and Humanities are excluded.

LEIDEN RANKING


Impact Indicators

Mean citation score (MCS) The average no. of citations of the publications

Mean normalized citation score (MNCS)

The average no. of citations of the publications, normalized for field differences, publication year, and document type

Proportion top 10% publications (PPtop 10%).

The proportion of the publications, compared with other similar publications, belong to the top 10% most frequently cited.

Collaboration Indicators

Proportion collaborative publications (PPcollab).

The proportion of the publications that have been co-authored with one or more other organizations.

Proportion international collaborative publications (PPint

collab).

The proportion of the publications that have been co-authored by two or more countries.

Mean geographical collaboration distance (MGCD).

The average geographical collaboration distance

Proportion long distance collaborative publications (PP>1000

km).

The proportion of the publications that have a geographical collaboration distance of more than 1000 km.

LEIDEN RANKING


• Published by the University of Texas at Dallas (UTD) to track publications in 24 journals; announced in March

• Ranks top 100 business school rankings• Methodology

– Based on the no. of papers in 24 leading business journals in a moving 5-year window

• Characteristic– US Business schools dominate the ranking result (In 2012,

90% of the top 20 list and 72% of the top 100 list)

UTD TOP100 BUSINESS SCHOOLS RANKING


• Published by the Tilburg University annually around May; announced since 2004

• Based on the no. of publications published by the universities in 68 leading economics journals over a 5-year period.

Characteristic• Only counted on the sum of publication published in

those 68 economics journals.

TILBURG UNIVERSITY TOP 100 WORLDWIDE ECONOMICS SCHOOLS RESEARCH RANKING


• Ranks institutions according to the no.of primary research articles published in Nature journals. The publication is based on data for the period between 1 January and 31 December annually.

• Ranks by different regions to constitute: Nature Publishing Index – Global Top 100 (release around March annually), Asia-Pacific, Japan, China, Australia, Korea

• Articles consist of Research Articles, Letters and Brief Communications published in Nature journals.

Characteristic• Specified for Nature research publications • Based on the corrected count (cc) of publications

NATURE PUBLISHING INDEX


RANKINGS ANALYTICS SYSTEM

• Let’s have a look at several examples from the major ranking systems, to better appreciate what the metrics mean.– Time-series analysis– Benchmarking– Cross-country analysis


RANKINGS & METHODOLOGIES

• Many different rankings, with different methodologies

• Overall, the main areas being assessed are:– Reputation– Teaching performance– Research productivity and impact– Internationalization– Funding and sources of incomes– Faculty and staff strength– Student numbers

©2013 michael fung rankings and benchmarking workshop – vietnam rankings workshop strategies for...

Documents

michael fung rankings

rankings landscape

rankings performance

schools science

ug students

global higher education

exchange students

business schools17