sharing guidelines knowledge: can the dream come true? medinfo panel cape town, september 15, 2010

Sharing guidelines knowledge: can the dream come true?

Medinfo panel

Cape Town, September 15, 2010

Motivation

The vision of sharing executable clinical knowledge can be achieved only if we: Standardize platforms for deploying scalable knowledge

based services Ensure services are mutually compatible and

interoperable and free of institution-specific details Develop reusable content and service components Support automated cross-verification for quality and safety Establish communities of practice who share, maintain,

update, and improve content

2

Objectives

Raise awareness of the practical challenges involved in maintaining repositories of sharable executable clinical knowledgeChallenges with maintaining a repositoryDefining what knowledge can be shared and howChallenges in piecing together knowledge into a

care plan and integrating it with EHR data If the knowledge if free, what’s the business model

and incentives for contributing knowledge?

3

Panel participants

John Fox, Department of Engineering Science, University of Oxford, UK

Robert Greenes, Ira A. Fulton Chair of the Department of Biomedical Informatics, Arizona State Univercity, Phoenix

Sheizaf Rafaeli, Head of the Graduate School of Management and Sagy Internet Research Center, University of Haifa, Israel

Mor Peleg, Head of the Department of Information Systems at the University of Haifa

4

What shall we discuss?

Life-cycle approach for sharable knowledge-based patient-care services

Methodology for distilling sharable knowledge from business/ implementation considerations

Methods for weaving medical knowledge services into an application and for mapping clinical abstractions into EHRs

Incentives and business models for a knowledge-sharing community

5

John Fox

Bob Greenes

Mor Peleg

Sheizaf Rafaeli

John Fox

6

Options for addressing open-source publishing of medical knowledge, drawing on lessons learned in the OpenClinical project

20th Anniversary Gold Medal Award

OpenClinical: Open Source?

John Fox

University of Oxford (Engineering Science)UCL (Oncology, Royal Free Hospital) www.cossac.org

www.OpenClinical.org

• Goal: To promote awareness and use of decision support, clinical workflow and other knowledge management technologies for improving quality and safety of patient care and clinical research.

• A resource and portal for technologists, clinicians, healthcare providers and suppliers

• Currently about 200,000 visitors a year (80% growth in 2010)

www.OpenClinical.net

• Experimental project to explore how to develop content for high quality clinical decision support and workflow services at the point of care

• Goal is to build a community of users, researchers and content providers who are willing to contribute to the development of a repository of open content, including applications and application components

OpenClinical.net test sitepro tem: modx.openclinical.net

Content development lifecycle

• Prototype development model for open source content repository on www.OpenClinical.net

• Currently limited to PROforma decision and process modelling language

• Intended to eventually multiple representations (e.g. GLIF, ASBRU, GELLO, OWL ...)

Load from, save to repository

Download tools www.cossac.org/tallis

Web publishing (“publets”)

Integrate and Deploy

Key questions for open content

• Quality and Safety– Quality lifecycles, safety culture, who is liable?

• Reusability and interoperability – Open technical standards, who is developing them?

• Functioning community (Sheizaf Rafaelli)– What will sustain the open source ethic?

• Facilitating infrastructure (Bob Greenes)– Three organisations; too little? too much?

• Sustainable business models– How do the proprietary/open source worlds coexist?

Sustainable business models (1)

• Traditional standalone apps? • Issues of integration and localisation• Likes fragmentation; hates

interoperability

• Pay per patient (analogous to pay per view)

• Who would/should actually pay? • No-one pays for Adjuvant! Online

Sustainable business models (2)

• Standard medical publishing model• Commercially viable on a publishing model?

(Clinical Evidence)• Discussion on www.berkerynoyes.com/

pages/innovations_in_evidence_based_medicine.aspx

• Open Source with value-adding services? (c.f. Linux model)• Attractive model but how can we achieve

critical mass of a content development community?

Towards an open content lifecycle?

Ioannis ChronakisVivek PatkarRichard ThomsonMatt SouthAli Rahmanzadeh

Thank you

Robert Greenes

22

Morris Collen Award

Morningside Initiative

Sharing medical knowledge involves separation between the medical content and the business/applications considerations

MUMPS

Toward sharing of clinical decision support knowledge

Robert A. Greenes, MD, PhDArizona State University

Phoenix, AZ, USA

A focus on rules

Purpose of this talk

• Identify key challenges to CDS adoption with focus on rules– Expressed in terms of 3 hypotheses:

1.Sharing is key to widespread adoption of CDS

2.Sharing of rules is difficult

3.Sharing can be facilitated by a formal approach to rule refinement

Hypothesis 1: Sharing is key to widespread adoption of CDS

• We know how to do CDS!– Over 40 years of study and experiments

• Many evaluations showing effectiveness

Rules as a central focus• Importance of rules

– Can serve as alerts, reminders, recommendations– Can be run in background as well as interactively– Can fire at point of need– Same logic can be used in multiple contexts

• e.g., drug-lab interaction rule can fire in CPOE, as lab alert, or as part of ADE monitoring

– Can invoke actions such as orders, scheduling, routing of information, as well as notifications

• Relation to guidelines– Function as executable components when GLs are

integrated with clinical systems• Poised for huge expansion

– Knowledge explosion – genomics, new technologies, new tests, new treatments

– Emphasis on quality measurement and reporting

Yet beyond basics, there is very little use of CDS

• Positive experience not replicated and disseminated widely– Largely in academic centers– <30% penetration– Much less in small offices– Pace of adoption barely changing

• Only scratching surface of potential uses– drug dose & interaction checks – simple alerts and reminders– personalized order sets– Narrative infobuttons, guidelines

Adoption challenges

• Possible reasons1. Users don’t want it2. Bad implementations

• Time-consuming, inappropriate• Disruptive

3. Adoption is difficult• Finding knowledge sources• Adapting to platform• Adapting to workflow and setting• Managing and updating knowledge

• But new incentives and initiatives rewarding quality over volume can address #1– Health care reform, efforts to reduce cost while preserving

and enhancing safety and quality• And #2 AND #3 can be addressed by sharing of

best practices knowledge– Including workflow adaptation experience

Hypothesis 2: Sharing of rules is difficult

• Rules knowledge seems deceptively simple:– ON lab result serum K+– IF K+ > 5.0 mEq/L– THEN Notify physician

• Even complex logic has similar Event-Condition-Action (ECA) form– ON Medication Order Entry Captopril– IF Existing Med = Dyazide

AND proposed Med = CaptoprilAND serum K+ > 5.0

– THEN page MD

Why is sharing not done?

• Perception of proprietary value– Users, vendors don’t want to share– Non-uptake even with:

• Standards like Arden Syntax for 15 years, GELLO for 5 years• Knowledge sources such as open rules library from Columbia since 1995,

and guidelines.gov, Cochrane, EPCs, etc., - most not in computable form• Failure of initiatives such as IMKI in 2001

• Lack of robust knowledge management– To track variations, updates, interactions, multiple uses

• Same basic rule logic in different contexts• Beyond capabilities of smaller organizations and practices to undertake

• Embeddedness– In non-portable, non-standard formats & platforms– in clinical setting– in application– in workflow– in business processes

Example of difficulty in sharing

• Consider simple medical rules, e.g., – If Diabetic, then check HbA1c every 6 months– If HbA1c > 6.5% then Notify

• Multiple translations– Based on how triggered, how/when interact,

what thresholds set, how notify– Actual form incorporates site-specific

thresholds, modes of interaction, and workflow

• Multiple rules have similar intent• Differences relate to how triggered, how

delivered, thresholds, process/workflow integration

• Challenge is to identify core medical knowledge and to develop a taxonomy to capture types of implementation differences

Setting-specific factors (“SSFs”)

• Triggering/identification modes– Registry, encounter, periodic panel search, patient list for day, …– Inclusions, exclusions

• Interaction modes, users, settings• Data mappings & definitions, e.g.,

– What is diabetes - code sets, value sets, constraint logic?– What is serum HbA1c procedure?

• Data availability/entry requirements– Thresholds, constraints

• Logic/operations approaches– Advance, late, due now, …

• Exceptions– Refusal, lost to follow up, …

• Actions/notifications– Message, pop-up, to do list, order, schedule, notation in chart, requirement

for acknowledgment, escalation, alternate. …

Hypothesis 3: Sharing can be facilitated by a formal

approach to rule refinement

• Develop an Implementers’ Workbench

• Start with EBM statement• Progress through codification and

incorporation of SSFs• Output in a form that is consumable

“directly” by the implementer site or vendor

Life Cycle of Rule Refinement

Start with EBM statementStage 1. Identify key elements and logic – who, when, what to be done

– Structured headers, unstructured content– Medically specific

2. Formalize definitions and logic conditions– Structured headers, structured content (terms, code sets, etc.)– Medically specific

3. Specify adaptations for execution– Taxonomy of possible workflow scenarios and operational considerations– Selected particular workflow- and setting- specific attributes for particular

sites

4. Convert to target representation, platform, for particular implementation

– Host language (Drools, Java, Arden Syntax, …)– Host architecture: rules engine, SOA, other– Ready for execution

Four current projects addressing this challenge

EBM statement

1. Identify key elements and logic – who,

when, what to be done

2. Formalize definitions and logic conditions

3. Identify possible workflow scenarios –

model rules, defining classes of operation

4. Convert to target representation,

platform, for particular implementation

Idealized life cycle / Morningside / KMR / AHRQ SCRCDS/ SHARP 2B

What we hope to accomplish

• Implementers’ Workbench (IW)• Taxonomy of SSFs• Knowledge base of rules• Approach

– Vendor, implementer, other project input, buy-in, collaboration

– Taxonomy as amalgam of NQF expert panel, Morningside/SHARP/Advancing-CDS workflow studies, SCRCDS implementation considerations

– Diabetes, USPS Task Force prevention and screening A&B recommendations, and Meaningful Use eMeasures converted to eRecommendations as initial foci

– Prototyping, testing, and iterative refinement of IW

What we expect to share

• Experience/know-how• Knowledge content• Methods/tools• Standards/models

Standards/models

• Representation• Data model/code sets• Definitions• Templates• Taxonomies• Transformation processes

Where CDS should go from here?

• Need for coordination– Multiple efforts underway– Need to coalesce and align these

• Need sustainable process– Multi-stakeholder buy-in, participation, support,

commitment to use

• Need to demonstrate success– Small-scale trials– Larger-scale deployment built on success

• Expansion to other kinds of CDS

Comments? Questions?

Mor Peleg

42

GLIF3New Investigator Award

ProcessMining

Biomedical Ontologies

KDOM

Data

K

Weaving medical knowledge services into applications.Using a mapping ontology to map medical knowledge into institutional data

Implementing decision-support systems by piecing sharable knowledge components

Mor Peleg, University of Haifa

Medinfo panel, Cape Town, September 15, 2010

Motivation Computerized guidelines have shown

positive impacts on clinicians but they take time to develop

Solution: Share executable GL components, stored in Medical Knowledge Repository

Assemble computerized GLs from components

Map the GL’s medical terms into institutional EHR fields

Examples of medical resources that could be shared and assembled

Medical calculators Risk-assessment tools Drug databases Controlled terminologies (e.g. SNOMED) Authoring, validation, and execution

tools for computer-interpretable GLs

Component interface

Peleg, Fox,

et al. (2005)

LNCS 3581

pp.156-160.

The interface can be used for

Sharing components Indexing and searching for components

Using the attributes: clinical sub-domain, relevant authoring stages, and goals

Assembling components into a GLSpecifying the guideline's skeleton

language (e.g., GLIF, PROforma) into which components can be integrated

Example: providing advice on regimens for treating breast cancer

Get patientData

Is patient eligible for evaluating

therapy choices?

Adjuvant's life- expectancy calculator

Filter out non-beneficial and

contraindicated therapies

Present choices to

userChoose optionCalculate

regimen

Prescribe regimen

Using Standards

Skeleton can be any GL formalism Eligibility criteria expressed in GELLO standard Referring to the HL7-RIM patient data model

Integrating assembled guideline with EHR data

50

Encode once but link to different EMRs Global-as-View Mapping Ontology + SQL Query Generator

RIM

Peleg et al., JBI 2008 41(1):180-201

Knowledge-Data Ontology Mapper (KDOM)

51

Query result: RIM view

KDOM mapping instances

SQL query

generator

Guideline Expression (need not use EMR’s terms)

GELLO interpreter

Evaluated expression

KDOM mapping classes:

Direct, Hierarchical,Logical, Temporal

“Breast Mass = true”

Patient has Palpable Breast Mass or Hard_Breast_Mass

Palpable Breast Mass is-a Breast Mass.Palpabale Breast Mass is stored in the Problems table

Observation of Breast Mass

true

SQL query

Summary

A repository of tested executable medical knowledge components that would be published on the Web

Framework for specifying the interface of components so that they could be searched for and integrated within a Computerized GL specification

KDOM used to integrate the medical knowledge with institutional EMRs

52

Thanks!

[email protected]

53

Hope to see you at AIME 2011, July 2-6, 2011, Bled, Slovenia

Sheizaf Rafaeli

54

survey and contrast social, technical, hierarchical and market-based models for motivating and maintaining the sharing of information and processing tools

Sharing Guidelines Knowledge: can the dream come true ?

Sheizaf [email protected]

http://rafaeli.net

MedInfo 2010

[email protected], http://rafaeli.net 56

Bits Replacing Atoms

Moore, Gilder, Metcalfe, Reed


utility

users

Information OverloadEconomics of Scarcity vs. Economics of Abundance?


What’s really new?• Access has become widespread

• Information as a commodity; IT as a commodity“Does IT matter “? Transmission has been solved

• Information is an experience good

• The impossible ease of copying

• Disintermediation

• Free information has become commonplace, normative, expected. Both free and for-fee information occupy the same net


• New Rules for the New Economy : 10 Radical Strategies for a Connected World by Kevin Kelly

• “Information Rules : A Strategic Guide to the Network Economy by Carl Shapiro, Hal R. Varian


“Free” as in free speech, or as in free beer?

How is UGC motivated?


The Value of Information

• Public source• Commodity• Overload• History• Technology• Psychology?

• Private source• Uniqueness• Timing• Presentation• Tailoring• Technology• Network effects


Emphasis on distinction between Private and Public Suggesting the Subjective Value of Info

כלים מוזרים לתיגמול

Wiki “barnstars”

Wikipedia: a system that shouldn’t work, but does. Participation Power Laws and Long Tail


Web 2.0

UGC

and

Co-production

Further personal stakes in info value

• Information markets http://answers.google.com • Online Scientific Journals

http://jcmc.indiana.edu • Citizens’ Advice Bureaus

http://shil.info • Wikis http://misbook.yeda.info • Online Higher Ed systems

http://qsia.org • Games and Serious Games


SHIL (שי"ל)שרות יעוץ לאזרח Citizen Advice Bureaux (CABs) Established

1957 55 “Brick and Mortar” offices Telephone hot line & Internet web site,

operated at the Univ. of Haifa Sagy Center Operated by Volunteers, coordinated and

funded by the Israeli Ministry of Social Affairs and Social Services in collaboration with municipalities.

Ownership… • Legal Perspective

Vs. Open Source, Peer-to-Peer,UGC, Web 2.0, etc.

• Apply 19th century property law to 21st century reality?

• Legality: "fair use" "first sale" "prior art" doctrines

• Open Innovation


Discussion

• Still LOTS to study and learn…

• Interactivity and Social Motivations seem to be king

• A high (too high?) overall subjective value for information.

• As predicted by the Endowment Effect theory, WTA for information was significantly larger than WTP for information

• This predicts undertrading. Implications for system design


Discussion (2)

• Information is a commodity. Nevertheless, information is still easier to duplicate, easy to share, and ownership of it proves more difficult to enforce

• Society has not yet adjusted its information consumption patterns to the present situation of information abundance

• Scoring and Governance Rules!


Thank you

[email protected] http://rafaeli.net

Provocative statements

77

Statement 1

A national or international effort can be put together to create a repository of implementable knowledge.

78

Statement 2

Guideline sharing could be achieved within 10 years

79

Statement 3

Guideline sharing at the implementation level requires separation into component steps that can be individually implemented, because of differences in process/work flow that prevent the guideline from being adopted in its entirety

80

Statement 4

True sharing of executable medical knowledge could never be achieved because knowledge could not be separated from institutional adaptations

81

Statement 5

Guideline formalization activities do not typically address implementation settings and requirements

82

Statement 6

The benefits of formalizing and sharing clinical knowledge are beyond dispute: the challenge now is to establish principles of safe deployment and use in clinical service design

83

Statement 7

As in so many other fields of engineering, one of the keys to effective and safe deployment will be open technical standards (covering medical concepts, clinical vocabulary, task models for example)

84

Statement 8

Adoption of standards will be necessary but will not be sufficient for success: another vital challenge is to persuade the commercial world of medical IT, publishing, etc. to develop business models that accept and build on open standards

85

Statement 9

If information “wants to be free” why discuss incentives for sharing anyway?

86

Statement 10

The only types of incentives for sharing are material, social, or ego-oriented.

87

Statement 11

Which of these incentives is more available (material, social, or ego-oriented)

Which is more likely to generate results (material, social, or ego-oriented)

Which has more leverage for potential participating scientists? (material, social, or ego-oriented)

88

Statement 12

Ever since Fred Brook’s “Mythical Man-Month” vs. Eric Raymond’s “The Cathedral and the Bazaar”, we’ve seen a conflict between orderly design and sharing. Following Brook’s recent “Design of Design”, should the notions of iterative design be applied to sharing; or is the Open Code approach the way to go?

89

Discussion

90

Thank you!

91

Rafaeli, S. and Raban, D. (2003) , The Subjective Value of Information: An experimental comparison of willingness to purchase or sell information, JAIS: The Journal of the Association for Information Systems (AIS). Vol. 4:5 pp. 119-139 Rafaeli, S. & Raban, D.R. (2003 ) The Subjective Value of Information : Trading expertise vs. content, copies vs. originals in E-Business, The Third International Conference on Electronic Business (ICEB 2003), pp. 451-455. Rafaeli, S. and Raban, D.R. (2005) Information Sharing Online: A Research Challenge, in the International Journal of Knowledge and Learning, (inaugural issue), Vol. 1, Issue 1-2, pp. 62-80. ,

Raban, D.R. and Rafaeli, S. (2006) , The Effect of Source Nature and Status on the Subjective Value of Information , Journal of the American Society for Information Science and Technology ( JASIST ), Volume 57, Issue 3 (p 321-329)

Rafaeli, S., Raban, R.D., & Ravid, G., (2005). Social and Economic Incentives in Google Answers. ACM Group 2005 conference, Sanibel Island, Florida, November 2005. http://jellis.net/research/group2005/papers/RafaeliRabanRavidGoogleAnswersGroup05.pdf

M. Harper, D. Raban, S. Rafaeli, J. Konstan, Predictors of Answer Quality in Online Q&A Sites. CHI 2008.

D. Raban, M. Harper, Motivations for Answering Questions Online. Book chapter in New Media and Innovative Technologies (Caspi, D., Azran, T. eds.), 2007.

Rafaeli, S., Raban, D.R. and Ravid, G. (2007) 'How social motivation enhances economic activity and incentives in the Google Answers knowledge sharing market', International Journal of Knowledge and Learning ( IJKL ), Vol. 3, No. 1, pp.1-11.