the context of knowledge translation: a knowledge value framework for analysis and implementation...

The Context of Knowledge Translation:

A Knowledge Value Framework for Analysis and Implementation

Juan D. Rogers

School of Public Policy

Georgia Institute of Technology© Juan D. Rogers, 2006

Knowledge Translation Task Force 2November 14, 2006

Presentation Outline

Concepts and theories Knowledge Value Mapping What is so special about the health field? Concluding remarks


Yet another attempt at clarification…

Knowledge translation: What’s in the definitions:“Iterative, timely and effective process of integrating best evidence into the

routine practices of patients, practitioners, health care teams and systems in order to effect to optimal healthcare outcomes and to optimize health care and health care systems.” (CIHR 2004)

“The collaborative and systematic review, assessment, identification, aggregation, and practical application of high-quality disability and rehabilitation research by key stakeholders (i.e., consumers, researchers, practitioners, and policymakers) for the purpose of improving lives of individuals with disabilities.” (NCDDR 2005)


Clarification… Bottom line:

Movement of knowledge from creators to users. Additional considerations:

How exactly does it move? Collaborative, iterative, timely

Is it the best knowledge? Best evidence, systematic, assessment, identification,

aggregation, high quality research. Do we have the right goals?

Optimal healthcare outcomes, improving lives Do we have the right target?

Patients, practitioners, medical teams, health systems, people with disabilities

Does it make a difference? Integrated into routine practice, practical application


Clarification…

Definitions are not theoretical constructs Policy definitions: normative statements

Creation, movement, use of knowledge embedded in complex phenomenology Own causal mechanisms

Creativity patterns, reception and validation dynamics, field boundaries, disciplinary identities, incentive systems, productivity variation, scientific change, institutionalization, information management, and more.

Multi-level socio-technical effects Psychology of creativity and learning, organization of expertise, changes in

routines, economic impacts, collaborations, changing field boundaries, and more.

Complex set of relevant objects Individuals and individual behavior, texts, cognitive spaces, groups,

organizations, institutions.


Clarification…

KT agenda: Well defined knowledge object

Creators are known Conditions of creation are known Its meaning and implications are known Stable criteria for assessing quality

Well defined target of knowledge movement Clear vision of desired outcome Means will be found within the existing institutional

context


Nature of Scientific Knowledge

Positivism as the reference of all epistemological debatesPositive agenda: define how good knowledge

is madeNegative agenda: rule out without ambiguity

mere opinion and pseudo-scienceBuilding blocks of knowledge: observation

and logic


Knowledge…

The problem of observation: Individual, passive, unbiased observers do not exist

(Theory-ladenness of observation) Various theory based intervention mechanisms to

produce useful observations Observations themselves are fallible

The problem of induction: Facts do not speak for themselves through inductive

mechanisms The role of logic is defensive rather than constructive

(Falsifiability)


Knowledge…

Testing is open-ended (Duhem-Quine) Theory is underdetermined by data

Many relevant theories at play in one setting Difficult to separate substantive from controlling assumptions Difficult to replicate exactly the same experiment Significance of test outcomes takes time to emerge

Knowledge is a heterogeneous combination of: language (statements), arrangements of objects (machines, materials), practices (controls, display interpretation, tacit knowledge).


Knowledge…

Significant consequences: Objectivity is achieved through inter-subjective communication A measure of conventionalism is always present No clear-cut, completely final demarcation criterion (i.e. quality

related to assessment of practice rather then truth value) Facts and values cannot be completely disentangled (i.e. values

are embedded in conventions) Status of knowledge is not purely formal (i.e. changes in any of

the components of the knowledge production system affects its validity)

Validity needs constant work


Lessons on Knowledge Flows

Basic Research

Applied Research

Development Product or Process

Economic or social importance

TechnologyScience

University/Government Business/Market

The “linear” model of knowledge production and transfer


Knowledge flows…

Knowledge doesn’t travel in straight lines Not enough to add feedback loops

Logical relations of content are not in a bijective function with knowledge dynamics phenomena Language is a subset of knowledge creation and acquisition

experience In new contexts it gives rise to new meanings

Boundaries of model categories are arbitrary Suggests a hierarchy of knowledge (and knowers) The “other” meaning of linear

Factors don’t simply aggregate Changes in factors have disproportionate effects


Knowledge flows… Embarrassments and Omissions:

Inconclusive assessments of basic research payoff Strong involvement of industry in basic research Almost all research is supported with some argument about

potential payoff From our work: self-identifying basic researchers produce most

patents, licenses, etc. Most successful instances of technology transfer involve moving

people Knowledge management trend emerged from failure of equating

knowledge with information Appropriate technology perspectives arose from ignoring how

context changes the way technology works The list goes on…


Translation and Boundaries

Agenda: Bring together the two realms:

Policy concerns defined in KT Phenomenology of knowledge flows

Translation metaphor has benefits Acknowledges boundaries and different cultures Translation suggests something deeper than

transfer


Translation…

Metaphor and definitions have limitations: Too committed to the stability of meaning (focuses

almost exclusively on content) Uncritical about the nature of the boundaries Naïve about the ability to effect change It may be too hierarchical Assumes there is a translator competent in all the

relevant “languages” Should probably be considered a pilot rather

than a full blown solution


Translation…

There is a mismatch between the speed with which new knowledge is created and the speed at which social systems changeRoutines are institutionalized behaviorChain of insufficient conditions:

Education of individuals need not change an organization

Change in one organization need not change the institutional field it belongs in


Translation…

The field of knowledge flow is very diverse Physicians, clinicians Therapists, nurses, para-medical Suppliers, industry, structured markets Insurers, other financial institutions Regulators, service agencies Patients, people with disabilities Downstream research teams

Several levels of value Incentives, stakes Professional, cultural identity, discretion Individual, institutional


Knowledge Value Mapping: Can it Help?

Takes the complex field of knowledge dynamics as its point of departure Heterogeneity of knowledge phenomena Interaction of factual and value components Does not assume “frictionless” movement of knowledge Does not assume “changeless” movement of knowledge Expands the field of observation to capture knowledge

phenomena Answer: Probably, but no recipes. Makes systematic what some field leaders or oversight

agents already do


KVM and meaning

Consequences of heterogeneity of knowledge: Meaning is underdetermined by linguistic content Scientific knowledge claims do not contain all their

consequences within themselves Appropriation of knowledge implies attaching it to a new set of

circumstances It also implies interrogating it (new level of testing) for a new set

of problems (relaxing the ceteris paribus clauses) Similar to technology design: not all uses are anticipated in the

design The users will mobilize the new knowledge according to their

values

Knowledge use always involves some transformation


KVM and meaning…

Consequences: Complicated interaction of knowledge content and context There are no purely formal principles of knowledge flow There is no privileged knowledge stance from which

knowledge flow can be predicted and controlled The exact relationship of factual claims and normative

stances must be determined empirically Every boundary will involve some sort of implicit or explicit

negotiation of the implications of the knowledge that flows through


KVM Constructs

Knowledge Value Collectives Connected set of social actors Specific content area (embedded criteria) Heterogeneous membership (authors, managers,

advisors, etc.) Intended to be exhaustive Boundary can be ambiguous (such is life) Likely to contain specific field effects

Broad context of knowledge translation


KVM constructs…

Knowledge Value Alliances: Subsets of a KVC Explicit agreement to pursue common knowledge related

goals Allow articulation of some factors of knowledge flow

(collaboration, competition, lobbying, etc.) Constitute basic fact/value arrangements (negotiated

agendas) Combination of KVC and KVA allows general formulation

of evaluation questions Allows to see assessment of foreseen/desired outcomes in the

context of other unforeseen outcomes (realization of other values)


KVM and translation…

Procedure: Map the field (i.e. find the KVC)

Identify creators and users Explicit references (publications, citations, etc.) Documented relationships (advisory roles, collaborations, consortia) Nominations

Find the patterned dynamics (KVAs and other organized behavior)

Find the communication patterns Find evidence for and infer the normative stances of all

involved Articulate relevant patterns of knowledge flow


KVM and translation…

Retrospective KVM: Features of past or present knowledge flow Examples:

Division of knowledge labor in interdisciplinary teams or inter-sector teams

Emergence of new specialized roles in interdisciplinary research Institutional innovation in technology transfer Community valuation of hidden knowledge outcomes

Prospective KVM: Given KVM and proposed intervention, predict likely effects

Determination of facilitating and hindering factors for knowledge flows (relevant to evaluation and other policy concerns)


Some specifics on Health

Markets relevant to health have special features End user has incomplete information Value of health to the patient incommensurate with market value

attribution Technological products marketed to professionals, not end users Heavily regulated and affected by public good considerations

Knowledge field has special features Long-standing institutionalization of knowledge creation and

application (clinical counterpart to health research) Has specific division of cognitive labor: unique position of clinical

research (special rules for allowable observation and experimentation)


Health…

No serious questioning of relevance of health research (compared to other fields)

Institutionalization of knowledge flows has a long history Differentiation of sub-specialties Adaptation of professional roles

Why is there an application gap? Speed of knowledge creation is one (partial?) answer Suggests problem is at the institutional level Deserves a KVM sort of analysis (beyond scope today)


Health…

Synthesis, reviews and meta-analysis Constitutes knowledge creation in its own right Has its own rules of evidence Guided by theoretical claims that are not equivalent to the individual

contributions The very idea of Evidence Based Medicine is suggestive of a crisis

Quality control of knowledge (fact/value) In the knowledge creation context it is epistemology (validation criteria) In the knowledge application context it is pragmatic: does it work? In health: transfer of epistemology to the application domain

Obvious question of relevance or completeness of the fact/value issue This is a different certification issue than original epistemology

What is happening in the institutions of health knowledge?


Health…

Health knowledge institutions had their own knowledge translation mechanisms

Their re-invention raises important questions about health knowledge flow

Current suggestions seem to be short term partial solutions

Long term solutions not likely to be in the form of recipes or guidelines


Concluding Remarks

Heterogeneous nature of knowledge requires a broad empirical perspective on knowledge flows

Assessment and decision-making require articulation of specific fact/value arrangements

Outcomes at the level of changed professional routines are long term

Speed of knowledge change seems mismatched with current institutions

Need multi-level strategy Adaptation of existing KT procedures is good idea as short term

approach Requires assessment of fact/value topology of own field Need to include probable emergence of new professional roles Implies long term institutional changes