Building and Measuring Research Impact

Luke GeorghiouVice-President for Research & Innovation

University of Manchester

Why are researchers asked to demonstrate impact?

• Principal motivations for assessing impact– Accountability for public expenditure– Justification against competing demands for resources– Learning and improving

• ‘The explicit assessment of impact in the REF for the first time demonstrates the importance placed on research outside the research community. It will enable universities to rigorously demonstrate the success of their research and the significant contribution it makes to the economy and society.'– David Sweeney, Director (Research, Innovation, and Skills) HEFCE

RCUK Mission for Societal and Economic Impact

• To advance knowledge, understanding and technology (including the promotion and support of the exploitation of research outcomes), and provide trained researchers

• To build partnerships to enhance take-up and impact, thereby contributing to the:– • economic competitiveness of the United Kingdom– • effectiveness of public services and policy– • enhancement of the quality of life and creative output of the nation.

• Mission and Statement of Expectation on Economic and Societal Impact– Research Council UK’s commitment to excellent research that extends

the boundaries of human knowledge remains as strong as ever. These documents signal a … recognises that publicly funded research should benefit us culturally, socially and financially.

Negative reactions

Petition decries 'impact' agenda in research11 June 2009

Scientists stage mock funeral outside parliament in funding protest

Science for the Future claims funding policies risk plunging British science and industry 'back into the Dark Ages'

Why resistance?

• Principal concern is that application of impact criteria will affect ex ante the direction of research– Funding decisions on grants– REF funding

• and that will result in work of lower quality, integrity and even impact– Argument that ‘blue-skies research’ ultimately creates

prosperity and well-being and that focus on immediate impact may be at expense of greater long-term impact

– Blue-skies is Anglicism translating as ‘investigator-driven, curiosity-driven or basic research’

Impact and Social Science and Humanities

• Worry in SSH communities that STEM subjects could be seen as more ‘impactful’ and secure larger share of resource

• Concerns grounded in the supposed predominance of a particular sequential model of impact– Discovery Development Commercialisation

– As opposed to interactive/co-production model more typical of SSH

– Also whether impact defined predominantly in economic terms

• In fact most social science disciplines founded in the expectation of research driving change in society

• Less clear in some parts of humanities but nonetheless strong commitment to culture

Response: British Academy Report

“Punching our weight: the humanities and social sciences in public policy making”

Taxonomies of Impact

• Two main ways to classify– Type of beneficiary– Nature of the impact

• Often mixed• Ongoing struggle with recognising public

engagement

REF Panel C Taxonomy• Impacts on creativity, culture and society

– Influencing knowledge, behaviours, practices, rights or duties • Economic, commercial, organisational impacts

– Businesses or other wealth creators• Impacts on the environment

– Natural, historic or built• Health and welfare

– Quality of life enhanced or harm mitigated• Impacts on practitioners and professional services

– Development and delivery of services and/or ethics• Impacts on public policy, law and services

– government, public sector and charity organisations and societies through implementation or non-implementation

Impact types by faculty at UoM

Expert

advic

e

Clinica

l Prac

tice

Consulting

Industry/N

on-Public Funds

Licen

sing

Spin-O

ut

Media covera

ge0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Engineering & Physical Sciences HumanitiesLife Sciences Medical & Human Sciences

Based on Research Performance Exercise Self-reporting

Impact in UoM vision and strategy

• Our work must have an impact beyond academia and yield economic, social and cultural benefits whenever the opportunity arises– Key relationships– Manchester heritage– Creating and demonstrating

impact– Commercialisation– Societal challenges and social

responsibility

Quality Impact

People

Focus Resources

Integrity Alignment

STRATEGY

Individual incentives at UoM

• Parity of esteem with curiosity-driven research for impact-generating, translation and knowledge transfer activities in promotion and PDR– applied research and development includes transfer of

intellectual property into the wider economy; translation of research findings into clinical solutions; development of innovation; research and consulting relationships with companies, government departments and other public bodies; and the enrichment of the wider culture ...

• Training & development – from doctoral training, through new academics

programme and beyond

Individual incentives continued

• Outside work policy – often seeding institutional links– register of interests to manage potential conflicts of

interests likely to arise• Radical IP policy– generous share to originators : 85% to originators up to

first £1m plus any re-invested into research, then 50% to originators

• Investment– Proof of principle and venture funds (eg UMIP Premier

Fund, social enterprise).

Rolling Research Profiling Exercise

Data base of all research-active staff (E Scholar repository) supporting annual reporting and

assessment of:• Research outputs (assessed by externally moderated

peer review NOT journal rankings or impact factors)• Research expenditure (benchmarked)• Research student supervision• Impact, engagement and esteem – Reported text boxes

Ability to deliver impact also rests in organisational competences

• Working with business and other non-academic partners requires:– Mutual trust and mutual benefit– Professional interface– Recognition and management of differences

• Works best in context of long-term strategic or ‘broadband’ relationship– Reduction of transaction costs

• Hidden but vital competence– Ability to configure multiple disciplines in seamless interdisciplinary

configurations to solve business and societal challenges– Key rationale for critical mass and economies of scope

Infrastructure is important

• Enterprise courses, conferences, staff and student business competitions and networking

• Developing regional innovation system and Corridor including cultural/creative zone

• Vectors – Policy@Manchester– Academic Health Sciences Networks

• Cultural institutiions

Living with REF Impact Assessment

• UoM submitted 186 REF Impact Cases • Some key challenges

– Identification • Especially of impacts where key players departed • No systematic organisational memory

– Comprehension• Getting academics to understand the detailed criteria

– Verification• Assembling credible supporting evidence

– Uncertainty• No track record on how criteria will be interpreted or how “reach

and significance” translate in to a scale across hugely different cases• Unclear boundaries – eg is it enough to demonstrate impact on a

policy or should the policy also be evaluated?

• A new stretch for modified peer review– But do the underlying assumptions of peer review hold?

Building Impact into your CV

• Consider engagement with user and stakeholder communities not as an add-on to research but as a potential source of ideas and insights even if you do not rely upon them for data

• Present your work in non-academic fora to interested parties eg at practitioner forums, via social and traditional media

• Profess the value of your field as well as your individual work• Ensure that interactions outside academic world are recorded

and note references in public documents• When engaging with non-academics consider whether it is

possible to collect evaluation evidence such as surveys or letters

• Monitor your presence in social media

Closing observations

• Not all research can be expected to have impact during the time it is being conducted and not all impact is realised by those performing the research

• Nonetheless where potential exists we have a duty to society to facilitate and realise impact from research we have already done and in some cases from research as it takes place

• The ability to generate and to record impact is a necessary organisational capability and a desirable individual attribute

• REF Impact assessment is a test and a hurdle but should not dominate what we do