measuring engagement and impact 15 and 17 february 2016 professor aidan byrne, chief executive...

Measuring Engagement and Impact

15 and 17 February 2016

Professor Aidan Byrne, Chief Executive OfficerAustralian Research Council

Web: arc.gov.au I Email: [email protected]

Outline• Overview of the Australian Research Landscape• ERA and ‘Engagement and Impact’• Trial measures

2015–16 Federal Budget—$429 billion

2015–16 Federal Budget—R&D $9.7 billion


Commonwealth Investment in R&D 2015–16 (%)

NB tax measures reflect money foregone, not direct funding

ARC NCGP % funding by institution 2008–2015

IRUATNGo8Unaligned unis & others

National Competitive Grants Programme

Area of box represents $$ funded over the period 2011–2015. N.B. - Centres of Excellence, SRIs, not awarded in every year. - ITRP and DECRA only funding since 2012

ARC NCGP funding by Fields of Research 2002–2014

ERA 2015—OutcomesERA 2015 results are available in: State of Australian University Research 2015-2016 , Volume 1, ERA National Report.*

ERA 2015 outcomes show Australia’s universities continue to produce high-quality research across a wide range of disciplines. Overall research quality is improving and reflects ERA’s role in focussing universities on research quality.

Of the UoEs assessed in both ERA 2015 and the previous round, ERA 2012, 56 per cent of these maintained the same rating, while 29 per cent improved their rating by one.

The ERA results show an extraordinary breadth of outstanding research performance in Australia.

*Available at: www.arc.gov.au.

ERA 2012 to ERA 2015 GrowthThere was steady growth in Australia’s university research activity between the ERA 2012 and ERA 2015 rounds, including increases in:

the total number of assessed UoEs, up 6% to 2460 the number of research outputs submitted, up 5% to 432,747 the number of research staff submitted, up 11% to 67,579 total number of patents, up 20% to 936.

Distribution of ERA 2015 ratings across all Australian universities

UoE ratings (aggregated four-digit results, grouped by two-digit FoR code)


High Performing two-digit Units of Evaluation

Two-digit Fields of Research codes:

01 Mathematical Sciences 02 Physical Sciences03 Chemical Sciences05 Environmental Sciences07 Agricultural and Veterinary Sciences

09 Engineering11 Medical and Health Sciences17 Psychology and Cognitive Sciences18 Law and Legal Studies21 History and Archaeology

Percentage of all research outputs in ERA 2010–2015

Percentage of open access research outputs in ERA 2015 Submissions


Considering Research Impact• Research Impact means the demonstrable

contribution that research makes to the economy, society, culture, national security, public policy or services, health, the environment, or quality of life, beyond contributions to academia.

• It is included in objectives and selection criteria

• The ARC is seeking better communication about research outcomes


Measuring Impact

• In 2016 the ARC will work with the higher education research sector, industry and other end-users of research to develop quantitative and qualitative measures of impact and engagement

• A pilot assessment will take place in 2017

• The ARC will conduct a full assessment in 2018 as a companion to ERA


Engagement and Impact Assessment• National Innovation and Science Agenda—7 December 2015—

$1.1 billion over four years

• A range of new initiatives to support research, incentivise innovation and entrepreneurship, reward risk taking, and promote science, maths and computing in schools

• Includes $11.2 million over five years ($2.2 million on average) to conduct a national assessment measuring engagement and impact in university research

• Government invests $3.5 billion a year in university research

• ARC and Department of Education and Training to develop impact methodology in 2016, including extensive sector consultation


Engagement and Impact Assessment

1. Allocation UK REF



2. Advocacy + Impact case studies




2. Advocacy + Impact case studies

3. Alignment

4. Assessment

5. Analysis

6. Advancement /Improvement

Some common difficulties• Attribution• Time-lag• Reference Periods• Data collection vs reporting burden• Finding fast feedback drivers

Universities are multi-dimensional We need a better and more complete description of activity.

IMPACTERA

Enga

gem

ent

Linkage fraction vs Discovery fraction

Mapping Engagement: Linkage Projects vs. Discovery Projects

Engagement by cohortStrength in Linkage and Discovery schemes

Engagement by universityStrength in Linkage and Discovery schemes

Thank you

measuring engagement and impact 15 and 17 february 2016 professor aidan byrne, chief executive...

Documents