Connecting evidence, policy and practice in an era of austerity, complexity and

decentralised decision making

Maps, routes and shoes

Sandra Nutleysmn@st-andrews.ac.uk

The inspiration

• ‘Arguably the role of social research becomes more important to guide practice in an era of austerity than one of affluence’ (SRA, UK, 2010)

• ‘There seems to be broad [bipartisan] agreement: We need an evidence-based system to guide future budget decisions that assesses the relative performance and impact of all government programs’ (Center for American Progress, USA, July 2011)

Underpinning rationale:

Evidence-based policies and practices ‘more likely to be better informed, more effective and less expensive’

“There is nothing a politician likes so little as to be well-informed.

It makes decision making so complex and difficult.” (John Maynard Keynes)

“For me, politics shouldn’t be some mind-bending exercise. It’s about what you feel in your gut”

(David Cameron, April 2011)

Familiar concerns

Possibly not so familiar concerns

• ‘There is actually very little evidence that can adequately inform what KTE strategies work in what contexts’ (p 756)

• ‘KTE, at least as conceptualised to date, simply does not fit with the underlying politics of health policymaking’ (p 757)Mitton et al 2007

1. Policy and practice context 2. Knowledge-to-action (KTA) thinking, models

and initiatives3. Implications for connecting evidence, policy

and practice

1. Policy and practice context

Financial austerity

• Concurrent huge increase in demand for public services due to demographic and social pressures

• Conclusion: Radical reform required. Continuing to deliver services using current models won’t work

Outlook for Scottish public expenditure

Source: Christie Commission 2011

Austerity: more a threat than opportunity?• Job cuts for researchers in

government• Research and evaluation

budgets slashed• Researchers & evaluators

having to do more with less

‘One person's riot is another’s

research grant’

But• Research impact demands have

raised status of applied/policy-related research in universities

• Politicians still reach for research as a tactic

Complexity: ‘wicked’ nature of many social problems

• Social problems often interdependent with complex causal relationships

• Knowledge base for understanding problems and possible solutions often uncertain, fragmented and contested

• Stakeholders may hold divergent and extreme views, and be unfamiliar with and resist other perspectives

Head, 2008

Complexity: public service systems• Various combinations of hierarchy, market,

community and network approaches to service delivery

• Greater decentralisation of decision making, service commissioning and procurement in many countries

• E.g. Growing complexity of education systems due to:– Diversity of stakeholder preferences and expectations– More decentralised and flexible governance structures– Additional layers of governance at international and

transnational levels– Rapidly changing and spreading ICTs

OECD/CERI, 2007

2. KTA thinking, models and initiatives

Forms of research use

Direct (instrumental)– Knowledge-driven– Problem-solving (or engineering)

Indirect (conceptual)– Social interaction– Enlightenment (or percolation)

Symbolic– Political– Tactical

Source: Weiss 1979

Importance of

informal carers…Restorative justice

Choose life suicide policy

Harm reduction in substance misuse…

Importance of early years

Enhancing self-care…

The happiness and well-

being agenda…

Impact often occurs via indirect & subtle routes, which may involve symbolic use along the way

Three generations of knowledge to action (KTA) thinking

• Knowledge transfer

• Knowledge exchange

• Knowledge integration

• Knowledge a product –degree of use a function of effective packaging

• Knowledge the result of social & political processes – degree of use a function of effective relationships and interaction

• Knowledge embedded in systems and cultures – degree of use a function of effective integration with organisations and systems

Source: Best et al 2009

Good at…

• The importance of context, networks & systems;• Social and collective learning, and unlearning;• Interaction with other types of knowledge (tacit;

experiential; political awareness);• ‘Use’ as an adaptive process - not an event;• Non-individualised embedded uses of research;• Inherent non-linearity of systems.

…moving us away from ideas of research as “answers”, which need to be packaged

… problematising “knowledge transfer”

Begins to emphasise…

• Know-about problems: the nature and formation of social problems.

• Know-what works: what policies, strategies or interventions will bring about desired outcomes

• Know-how (to put into practice): e.g. knowledge about effective programme implementation.

• Know-who (to involve): e.g. getting stakeholder buy-in and building alliances for action.

• Know-why (requirements of action): relationship between values & policy/practice.

• Know-whether having any impact: monitoring, evaluation and accountability

Knowledge required for effective policies and services

Knowledge-to-action (KTA) models• A bewildering array of models, frameworks and theories (63

models identified by Ward et al 2009)• Drawn from wide range of disciplines - very varied levels of

evidential support• Most models reflect first or second generation thinking• Models focused on a spectrum of concerns (Davies et al 2011):

Increasing the use of explicit (what

works) knowledge

Creating environment that encourages engagement with

wide variety of knowledge

Most KTA models clustered towards the left-hand side of this spectrum

Implementation frameworks Interactional frameworks

KTA process often viewed as an iterative cycle, especially when concerned with instrumental use

(Jones et al 2009)

Knowledge-to-action initiativesMany and varied:

International/national/local focusPolicy/practice focus

Individuals/groups/organisations/systems focusTop down

Evidence to practiceBottom up

Practice to evidence

No child left behind (NCLB), USA

Project Oracle, London

Center for Court Innovation, NY, USA

Centre for Effective Services, Ireland

NB: Researcher/university driven initiatives still often based on knowledge push models (Qi & Leven, 2011)

3. Implications for connecting evidence, policy and practice

Those wishing to negotiate the research use/impact terrain…

• Need good maps of the policy/practice context at different levels of detail

Context matters: factors explaining use/non-use

• Political culture factors – e.g. shared or conflicting values, adversarial or consensual political frameworks, extent of deference to technical expertise

• Policy domain factors – e.g. settled or contentious problems, actors, structures and issue histories

• Coordination and integration capacities – e.g. extent of central control, system capacities for policy development and implementation

• Organisational capacities and processes – e.g. supply of analytical skills, receptivity to outside information, routine requirements and procedures

• Cross-sectoral relationships – institutionalisation of cross-sectoral processes such as bridging organisations and networks

Head (2011)

Those wishing to negotiate the research use/impact terrain…

• Will find many generic route guides, not all of which are based on fieldwork!

• Often not clear what routes to pursue in which context

• And to complicate matters, researchers need to decide which shoes they want to wear…

• Need good maps of the policy/practice context at different levels of detail

Researcher rightly take different stances in relation to policy/practice

• The consensual approach – working with the grain of current policy/practice.

• The critical and contentious approach – “keeping the system honest”.

• Paradigm challenging – subverting current thinking and perhaps proposing new principles for action.

There are some schemas of the relationship between certain routes and aspects of context that are worthy of further consideration and development

Contandriopoulos et al 2010Models of use in relation to cost-sharing and polarisation

dimensions of the context

Fazekas & Burns 2011

In the meantime, some general guidance based on RURU’s work

Research Unit for Research Utilisation (RURU) – developing cross-sector knowledge on research use

Generic features of effective practices to increase research use

Research must be translated - adaptation of findings to specific policy and practice contexts

Ownership is key – though there are exceptions where implementation is received or perceived as coercive

Need for enthusiasts - champions - personal contact is most effective

Contextual analysis - understanding and targeting specific barriers to, and enablers of, change

Credibility - strong evidence from trusted source, inc. endorsement from opinion leaders

Leadership - within research impact settings Support - ongoing financial, technical & emotional support Integration - of new activities with existing systems and

activitiesSource: Nutley et al 2007

smn@st-andrews.ac.ukResearch Unit for Research

Utilisation (www.ruru.ac.uk – new website coming soon!)

Thank You!

Key references• Best A, Terpstra JL, Moor G, Riley B, Norman CD, Glasgow RE, (2009)

"Building knowledge integration systems for evidence-informed decisions", Journal of Health Organization and Management, 23(6): 627 - 641

• Contandriopoulos, D., Lemire, M., Denis, J-L. & Tremblay, E. (2010) “Knowledge Exchange Processes in Organizations and Policy Arenas: A Narrative Systematic Review of the Literature” The Milbank Quarterly 88(4): 444-83.

• Davies HTO, Powell A, Ward V and Smith S (2011) Supporting NHS Scotland in developing a new Knowledge-to-Action Model, Unpublished report available from University of St Andrews

• Fazekas M & Burns T (2011) Exploring the complex interaction between governance and knowledge, A draft EDU/CERI paper, OECD

• Head B (2008) “Assessing network-based collaborations” Public Management Review, 10(6): 733-749

• Head B (2011) “Comparative Analysis of Research/Policy Relationships – challenges and agendas”, presentation to panel on the comparative study of EBPP, IRSPM conference, Dublin, 12 April

References (cont.)

• Jones et al (2009) Knowledge, policy and power: six dimensions of the knowledge-development policy interface, London: ODI

• Mitton et al (2007) “Knowledge transfer and exchange: Review and synthesis of the literature”, The Milbank Quarterly 85(4):729-768

• Nutley, S.M., Walter, I., & Davies, H.T.O. (2007) Using evidence: how research can inform public services. Policy Press. Bristol.

• Puttick R ed (2011) Using evidence to improve social policy and practice: perspectives on how research and evidence can influence decision-making, London: Alliance for Useful Evidence/NESTA

• Ward, V., House, A., & Hamer, S. (2009) “Developing a framework for transferring knowledge into action: a thematic analysis of the literature” Journal of Health Services Research and Policy 14: 156-164.

• Weiss CH (1999) “The interface between evaluation and public policy”, Evaluation 5(4): 468-486