real-world impacts from research: evidence & lessons
DESCRIPTION
Real-world impacts from research: Evidence & lessons. David Pannell Centre for Environmental Economics and Policy School of Agricultural and Resource Economics. For this PPT see www.davidpannell.net under “Talks”. Growing interest. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/1.jpg)
Real-world impacts from research: Evidence & lessonsDavid PannellCentre for Environmental Economics and PolicySchool of Agricultural and Resource Economics
For this PPT see www.davidpannell.net under “Talks”
![Page 2: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/2.jpg)
Growing interest· Perception: we need to do better at convincing
government about benefits of research
· ARC discussing how to include real-world impact in ERA
· UK’s Research Excellence Framework: 20% of funding based on “impact” from 2014.
![Page 3: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/3.jpg)
Trial by universities, 2012· Group of Eight (Go8) and
Aust Technology Network of Universities (ATN)
· Each university submitted cherry-picked case studies (165 submissions)
· Evaluated by people from industry & government
· 24 ‘best’ selected
![Page 4: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/4.jpg)
Plan· An example research project
Was selected in the GO8/ATN · Some evidence about impact
· Measuring impact
· Strategies for having impact
![Page 5: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/5.jpg)
Example
![Page 6: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/6.jpg)
2000: Salinity was a hot topic
![Page 7: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/7.jpg)
$1.4 billion of public funding
![Page 8: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/8.jpg)
I was shocked· Poor design of the program
· Program developers seemed to have been unaware of crucial areas of salinity research and their implications
· No chance of any significant benefits
![Page 9: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/9.jpg)
My response· Media· Discussion papers · Presentations· Submissions
![Page 10: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/10.jpg)
Tried to help them· Developed INFFER (Investment Framework for
Environmental Resources)A tool for integrating the science with other infoDevelop logical, evidence-based environmental
projectsAssess value for moneyPrioritise projects
![Page 11: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/11.jpg)
INFFER strategy· Extensive input by users· Make tools as simple as possible· Provide training and help desk for users· Readable documentation· Public critiques of existing approaches· Attempt to influence gov’t agencies to change the
signals
![Page 12: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/12.jpg)
Regional NRM application
![Page 13: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/13.jpg)
![Page 14: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/14.jpg)
Policy impacts· Senate inquiry (2006)
Recommended use of INFFER· NRM Ministerial Council (2007)
Endorsed new set of principles for investment in salinity
· Victorian Government, Biodiversity White Paper “INFFER will be utilised for the next five years”.
· Caring for our Country Influenced design of project template
![Page 15: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/15.jpg)
Lessons: Use of science· If you want people to use good science, the
people issues are crucialRelationshipsCommunication
· Most prospective users were happy with current (very poor) approach
· Didn’t perceive that government would reward them for doing it better
![Page 16: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/16.jpg)
Lessons: User capacity· Lack of capacity to formally integrate disparate
technical and socio-economic information for decision making
· Lack of expertise in economics and social science
· Lack of time to read things· People misinterpret things easily
![Page 17: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/17.jpg)
Research versus? Impact· Has taken considerable effort beyond traditional
researchTime commitmentNew skills and knowledgeNew networks
· Satisfying but very challenging to make a difference
· Worth it?
versus?and?
![Page 18: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/18.jpg)
Research versus? Impact· Various benefits for my research
· Interesting problems and issues arise
· Innovation - outside what’s currently in journals
· Better understanding of research relevance
· Journal papers generatedDirectly part of the INFFER work: 17Related/stimulated by: 16
· Reputation for useful research easier to get funding (unsolicited approaches offering $)
versus?and?
![Page 19: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/19.jpg)
Evidence about impact
![Page 20: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/20.jpg)
Evidence of high returns· Estimated rates of return to R&D are typically
very highCan indicate 30%, 50%, 100% annual rate of return
· Credible?$1 invested at 50% over 100 years = $4E17 (a
million times Australia’s annual GDP)· Sound analyses still show good returns
For both applied and basic research
![Page 21: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/21.jpg)
Heterogeneity· The distribution of benefits is highly skewed
· Most research has low impact
· A small number of projects have huge impactMore than enough to pay for the rest
![Page 22: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/22.jpg)
Example: CRC program· Benefits for 1991 to 2017
· The CRC program generated a net economic benefit of $7.5 billion over the study period
· Annual contribution of $278 million
· BCR = 3.1
![Page 23: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/23.jpg)
Impact is often slow· Lags to impact usually measured in decades
· e.g. US agriculture
· From first investment to peak impact = 24 years
· Still generating benefits after 50 years
· Several lagsResearch lagCommercialisation lagAdoption lag Impact lag
![Page 24: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/24.jpg)
Longer lags = lower net benefits· Discounting allowing for interest costs on the
up-front investment
· 30-year lag, 7% discount rate, benefits reduced by 87%
· The high measured rates of return occur despite the long time lags
![Page 25: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/25.jpg)
Supply push vs demand pull· Science push (Bush, 1945)
· Implicit in the “linear model”Basic R Applied R Technology Benefits
· Demand pull (Schmookler, 1966)Market demand Applied R Technology Benefits
· Big debate in the 1960s
· Resolved in the 1970s – innovation is an iterative process – both push and pull matter
![Page 26: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/26.jpg)
Measuring impact
![Page 27: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/27.jpg)
Determinants of benefits· Scale of relevance
· Adoptability of the research
· Benefits per unit
· Probability of research success
· Share of the credit attributable to particular research
· Time lags
![Page 28: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/28.jpg)
With vs without
0 5 10 15 20 250
10
20
30
40
50
60
70
80
90
100
With project
Without project
Year
Wel
fare
/util
ity/in
com
e
![Page 29: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/29.jpg)
Applicability?· The theory is relatively straightforward
· It has been applied successfully in many case studiesEspecially agriculture
![Page 30: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/30.jpg)
But …· It takes resources and skills
· Easier … for physical products than for knowledge if the benefits arise in markets if the benefits occur quickly for applied than for basic research
· Much university research is not in the categories that are relatively easy to evaluateKnowledge, public goods, long time lags, basic
![Page 31: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/31.jpg)
What will ERA do?· Perhaps copy the UK Research Excellence
Framework
· Two componentsCase studies of impactThe submitting unit's approach to enabling impact
from its research· They won’t expect an economic evaluation
![Page 32: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/32.jpg)
If it’s case studies, you’ll need to· Make the case/tell the story
· Link elements in chain from research to impact
· Provide evidence
· Note: in Go8/ATN trial, many nominations did this poorlyThe chain was incompleteThe evidence was weak/unconvincing
· If you can do it well, you’ll stand out
![Page 33: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/33.jpg)
Having an impact
![Page 34: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/34.jpg)
How to have an impact?· There is little research about this
· There are papers, but largely anecdotal
· Some resources at end of PPT
![Page 35: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/35.jpg)
Chain from research to impact· The chain varies widely from case to case
· Can have many links
· Understanding the chain for your research helps you tochoose, design and deliver research for greater
impactcommunicate impact provide evidence
![Page 36: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/36.jpg)
A chain from research to impact: Technology· Research and development
· Sell the IP
· Feasibility studies
· Design
· Manufacturing capacity
· Finance
· Marketing
· Sales
![Page 37: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/37.jpg)
A chain from research to impact: Information for policy· Research
· Something useful is learned (or isn’t)
· New information influences policy (or doesn’t)
· Policy change is implemented (or isn’t)
· If policy aims to change behaviour, people respond as intended (or don’t)
· Changes (relative to no research) result – social, environmental or economic benefits (or not)
![Page 38: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/38.jpg)
Risk of low benefits from research to influence policy· Nobody is listening
· You lack credibility with the decision maker
· The decision maker doesn’t understand
· The new results are not different enough from what we already know
· The decision depends more on other factors
· The decision options have similar payoffs
![Page 39: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/39.jpg)
Lessons: having impact· Need some demand pull
· Understand and respect potential users
· Be prepared for opposition
· Need perseverance, continual marketing
· Need repetition – government has short memory
· Seek a product champion
![Page 40: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/40.jpg)
Lessons: having impact· Need “absorptive capacity” in the organisation
· The political circumstances need to be right. You can’t change ideological positions of govt.
· Timing. Grasp opportunities.
· Good communicationSimplicity, brevity, clarityAvoid jargon, maths, complex graphs
· Think about impact which choosing what to research
![Page 41: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/41.jpg)
Conclusion· We are going to be asked to demonstrate real-
world impact
· It’s not just about communicating what we do better – we may need to change what we do to have genuine impact
· Pursuing impact is exciting and worthwhile but challenging – spinoff benefits for research
· The earlier in the research process you start thinking about impact, the better
![Page 42: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/42.jpg)
Resources· Pannell, D.J. and Roberts, A.M. (2009).
Conducting and delivering integrated research to influence land-use policy: salinity policy in Australia, Environmental Science and Policy 12(8), 1088-1099.http://dpannell.fnas.uwa.edu.au/dp0803.htm
· Pannell, D.J. (2004). Effectively communicating economics to policy makers. Australian Journal of Agricultural and Resource Economics 48(3), 535-555. http://dpannell.fnas.uwa.edu.au/j78ajare.pdf
![Page 43: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/43.jpg)
Resources· Weible et al. (2012). “Understanding and
influencing the policy process”, Policy Science 45, 1-12. http://link.springer.com/article/10.1007%2Fs11077-
011-9143-5
![Page 44: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/44.jpg)
Pannell Discussions (Blog posts)· 150 – Why don’t environmental managers use
decision theory?http://www.pannelldiscussions.net/2009/04/150-wh
y-dont-environmental-managers-use-decision-theory/
· 136 – Engaging with policy: tips for researchershttp://www.pannelldiscussions.net/2008/09/136-eng
aging-with-policy-tips-for-researchers/
![Page 45: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/45.jpg)
Resources· A relevant blog post by ecologist Brian McGill on
“What it takes to do policy-relevant science” http://dynamicecology.wordpress.com/2013/05/14/
what-it-takes-to-do-policy-relevant-science/
· Video: Ben Martin (U Sussex) “Science Policy Research - Can Research Influence Policy? How? And Does It Make for Better Policy?”http://upload.sms.csx.cam.ac.uk/media/747324
![Page 46: Real-world impacts from research: Evidence & lessons](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56815bac550346895dc9ae67/html5/thumbnails/46.jpg)
For this PPT see www.davidpannell.net under “Talks”