Excellence in Research for Australia 2015 UpdateARC/NHMRC Research Administrators Seminar

16 November 2015

Presented bySarah Howard

Director, Research Evaluation

Web: arc.gov.au | Email: era@arc.gov.au

• ERA 2015—where are we up to? Currently finalising evaluation Preparing State of Australian University

Research: ERA National Report 2015–16• Submission review

- Highlights- Challenges- Review process

Introduction

Web: arc.gov.au | Email: era@arc.gov.au

Not yet….

Evaluation

Web: arc.gov.au | Email: era@arc.gov.au

WHAT?

• State of Australian University Research: Volume 1 ERA National Report 2015–16

• Includes additional analysis of ten (10) year longitudinal dataset (2003–2013)

• ERA 2010, ERA 2012, ERA 2015 comparisons

• PLUS—additional volumes to come…applied research, gender, open access

WHEN?

• Early December…

ERA National Report

Web: arc.gov.au | Email: era@arc.gov.au

ERA 2015 Submission Review

Web: arc.gov.au | Email: era@arc.gov.au

2015 Submission Upload Stats (excluding Stage 0)

Institution Upload AttemptsHighest 2112nd Highest 1663rd Highest 1444th Highest 1405th Highest 1345th Lowest 174th Lowest 163rd Lowest 152nd Lowest 13Lowest 13Grand Total 2396

Web: arc.gov.au | Email: era@arc.gov.au

• ARC-RO collaboration

• RO staff worked very hard—thank you!

• ARC ERA case managers—productive relationships between ERA staff and ROs

• Great feedback from ROs—thank you!

• Stage 0—continues to be successful

• Your feedback—thank you!

Submission Highlights

Web: arc.gov.au | Email: era@arc.gov.au

• Submission completed on time

• Quality of submissions very high

• We saw improvements in:- data quality- repository readiness- explanatory statements- NTRO research statements

Submission Highlights

Web: arc.gov.au | Email: era@arc.gov.au

• Purpose is to allow universities to upload trial submissions to test XML data structure

• Opened 19 January 2015 to 19 February 2015

• All business rules were available - except BR024 and BR117 relating to EID tagging

• New report—30% ERA peer review sample

• Units of evaluation were not created

• Submissions uploaded did not carry forward to Stage 1

• 40 of the 41 universities used Stage 0

• There were a total of 918 submission uploads in Stage 0

Stage 0

Web: arc.gov.au | Email: era@arc.gov.au

• Some unis considered the Stage 1 deadline a ‘soft deadline’ and did not provide a complete submission that we could check in Stage 2

• This caused delays and increased pressure on everyone (ARC and unis) re data checking in Stage 2

• Incentives for unis to move through Stages 1 and 2 before the deadlines?

• Allowing staggered upload of different indicators?

• Easter! If only we could have moved that to July

• ARC gives 9 months notice. Extending timelines compromises evaluation timing

Submission Stages 1–2

Web: arc.gov.au | Email: era@arc.gov.au

We are looking to improve the process, including…

• EID tagging—better guidance on allowable article types, citation provider interactions (data corrections and timing)

• Repositories—citation/alt metrics on landing pages, are metadata links and landing pages required? possibly a direct link to the output (many unis did this for ERA 2015)?

• —better guidance on what additional information can be provided (e.g. not a theatre programme)

Submission Challenges

Web: arc.gov.au | Email: era@arc.gov.au

There is always room for further improvement…

• data quality—in general

• repository readiness—REC members reported some issues

• explanatory statements—no alt metrics etc.

• NTRO research statements- variable quality- REC members reported some were not at all

helpful- key issue is identifying the research component

Submission Challenges

Web: arc.gov.au | Email: era@arc.gov.au

• We have collected and collated your feedback through submission—thank you!

• We will be conducting a range of other review and consultation processes through 2016—we will let you know…

Submission Review Process 2015–2016

REC Member and Peer Reviewer RecruitmentResearch Evaluation Committee (REC) Membership

• 8 REC Chairs announced in November—to ensure ERA evaluations are of the highest standard

• Full list of RECs members can be found on the ARC website

Peer Reviewers

• The call for Peer Reviewer nominations closed on 26 November 2014

• For ERA 2015 we had ~ 1300 peer reviewers—even better than 2012!

• We assigned material to ~ 1100 peer reviewers.

• ~ 1000 peer reviewers submitted reports.

ERA-HERDC Alignment Update

• Consultation paper on options for alignment (8 December–13 February)

• Sector working group (3 July–21 September)

• Draft specifications completed

• Next steps—sector consultation on draft specifications, when…?

Questions?

Thank You

Web: arc.gov.au | Email: era@arc.gov.au