in Healthcare Analytics

Knowledge Management

Greg Nelson, MMCi, CPHIMSVice President, Analytics and StrategyVidant HealthGreenville, NC

Monica Horvath, PhDFormerly:Senior Reporting Solution ManagerUNC HealthcareChapel Hill, NC

NCHICA

Annual Conference

2019

AGENDA

NCHICA

IntroductionWhy knowledge management matters

KM DefinedWhat good looks like

ChallengesWhy don’t we all do it

Lessons LearnedWhat have we learned

Future DirectionsWhere are we in our journey

2

NCHICA

Introduction

• Why knowledge management matters• Why this matters• The cost of inaction

• The Learning Health Organization• Analytics’ role in the LHS• The value of KM

3

NCHICA 4

Analytics is a team sport… [and] requires a multidisciplinary approach to achieving value.. The Analytics Lifecycle Toolkit, 2018

NCHICA 5

NCHICA 6

n (n-1) /2

NCHICA 7

Business/ Operational View

7

NCHICA 8

Analytics View Reference data is not standard

No one can agree on the

measures

How do we make this

actionable?

Who should see what level of

data?

What is the O/E benchmark?

People aren’t using the

dashboard

Where does this data

come from?

Who should the data

steward be?

NCHICA 9

Value of Alignment

HORIZONTAL COORDINATIO

N

COORDINATED SYSTEMIC CHANGE

CONSISTENCY IN MEANING STANDARDIZED

BUSINESS RULES

People Collaboration

Content Context

NCHICA

Closed Loop System of Learning

* Source unknown

10

Learning Health System

NCHICA 11

LHS Applied to D&A

Data Engineer & Data Quality Data Scientist

BA

Prioritize Problem

Extract data

Prepare data

Explore features

Develop models

Evaluate models

Deploy models Monitor

DevOps

Integrated Knowledge Management, Collaboration, Source Control, Prioritization, Stakeholder Engagement/ Transparency, Team Processes, Peer Review, Quality Processes, Solution Exploration

Problem Definition

NCHICA * Source: The Analytics Lifecycle Toolkit, Wiley 2018 12

The Analytics Lifecycle

NCHICA

Value of KM

Tangible• Lower travel costs

• Increased productivity

• Reduced printing costs

• Improved closure time

• Shorter production times

• Reduced rework

• Improved reused

• Faster time to decision (customer satisfaction)

Intangible• Consistent use of data

• Increased metrics/ data accuracy

• Improving data sharing and usability

• Standard validation processes

• Data completeness and consistency

• Engaged team members

• Tighter teamwork

• Faster emergency communications

• Top of licensure teamwork

• Talent / career development

13

NCHICA

Knowledge Management Defined

• KM Defined• Definition• Components of KM

• KM Strategies• Bimodal analytics• Different types of KM

14

NCHICA 15

Knowledge management is a business process that formalizes the management and use of an enterprise's intellectual assets. Gartner, 2017

NCHICA 16

Components of KM

Our approach to support a data-driven culture is to ensure the alignment of people, processes, and technology that can be leveraged to accelerate our consistent and widespread use of knowledge.

PlatformContent People Process Knowledge Reuse

+ + x =

What? Where? Who? How? Why?+ + x =

NCHICA 17

Different Goals of Analytics

• Efficient/smooth DataOps• Reduce risk• Control costs• Information Security• Data privacy• Repeatability • Reliability

• Scalability• Performant• Reduce errors

• Creative• Innovative• Novel• Multiple perspectives• Design thinking/ empathy

• Fail fast (errors welcome)• Transparency• Agility

Mod

e 1

Mode 2

Gartner, 2017

NCHICA 18

Requires Different Approaches*

Top-downBest-practice drivenLeaders in chargeDrives alignment and governance

External sources of “excellence”

Bottom UpCollective, continuous learningParticipant-driveEvolving structures and themesProblem/ Solution Focus

Center of ExcellenceCo

mm

unity

of P

ract

ice

* Adapted from Gartner, 2017

Mode 1

Mode 2

NCHICA 19

Experience: Uses and value of data

Source: The Jurney–Warden data-value pyramid of (Agile Data Science 2.0)

Drive,value,

effect, alter, change, deliverCurate,

recommend, understand, infer,

learnStructure, link, metadata,

tag, explore, interact, share

Clean, aggregate, visualize, question

Collect, display, plumb individual records

Actions

Predictions

Reports

Charts

Records

… where we extract enough structure from our data to display its properties in aggregate and start to familiarize ourselves with those properties.

The data-value stack begins with the simple display of records

Next comes identifying relationships and exploring data through interactive reports.

This enables statistical inference to generate predictions.

Finally, we use these predictions to drive user behavior in order to create and capture value.

NCHICA 20

Illustrative Difference

Mode 1 Mode 2

BI Report• Report specification• Source to Target Mapping• Requirements traceability matrix• Refresh cycle• User Acceptance Criteria

Predictive Model• Purpose and description of the problem the model

tries to solve• Define the behavior to be predicted, and how that

behavior will be defined and measured• Define data sources available to be used as

predictors• Visual and statistical inspection/ observations• Feature extraction & selection• Define modeling sample (e.g.

training/validation/testing, holdout, cross-validation, etc.)

• Describe modeling techniques used to build model candidates

• Describe model validation techniques used to select final model

• Modeling results and discussion• Bias testing• Model implementation considerations• Model drift parameters

NCHICA 21

Example Supply Demand

Business definition Data stewardAssociations/ Groups

Data ConsumerReport WriterDashboard developerQA/ Validation

Source code Developer DeveloperBusiness AnalystQA/ Validation

Industry trend Anyone Anyone

Calculation Metric owner Developer

Aggregate and arrange contentOrganize content to satisfy their own preferencesProduced by resources inside and outside the organizationKnowledge creators generate and combine (mashup) content.As knowledge is consumed, it is refreshed.

Consumed in "chunks’" rather than in its entiretyConsumed at point of need.Tags, comments and ratings help define relevance and valueIntegration of multiple sources and types of information

Supply vs. Demand

NCHICA

Challenges and Opportunities

• Operational processes• Motivation• Systems and technologies• Speed/ velocity of change

22

NCHICA

No Easy Button?Instead of taking the comprehensive “boil the ocean” enterprise approach to design and implementation, you can take a “fundamentals” approach that focuses on the critical data-oriented improvements such as metadata management, data standards, data quality management and data governance.

23

Process – governance, report development, metrics, imperatives, operationalization, engagement

People – Culture, incentives, clarity, structural changes (rotations, etc.)

Technology – Collaborative, source control, platforms

NCHICA 1 Leadership for Change Programme Master Class 1: Systems Thinking With Myron Rogers," Leadership for Change.

Myron's Maxims 1

1. People own what they help create.

2. Real change happens in real work.

3. Those who do the work do the change.4. Start anywhere; follow it everywhere.

5. Connect the system to more of itself.

Creators Editors Viewers

Content Creation Content Consumption

24

NCHICA

Lessons Learned

• What can we learn from others• From our own “failures”• What it takes?

• Onboarding/ checklist/ peer reviews/ • Structural integrity to ensure this is sustainable

• Best practice repository

25

NCHICA 26

Community Models

The Three P's of Communities 2

§ Purpose: The shared domain that identifies the specific area with value to its members.

§ People: The individuals operating in the domain who collaborate in providing a social foundation to facilitate interaction and share knowledge.

§ Practice: The application of knowledge by practitioners to drive innovation, expertise and capability.

Practice

PeoplePurpose

1. Jean Lave and Etienne Wenger, 19912. Gartner, 2017

A. Activity Purpose + People + Practice =Community of interest/ Special interest group

B. Domain Purpose + People + Practice =Competency center/ Center of excellence

C. Learning Purpose + People + Practice =Professional learning community/ Technical Club

D. Outcome Purpose + People + Practice = Guild/ Community of Practice

"Communities of practice are groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly."1

A

B CD

NCHICA 27

Analytics CoP

Community of Practice Canvas

Target Group

Who are the target members for this community?

Which roles or activities does this community support?

How will the community be organized?

How will the community collaborate?

How will members benefit from joining this community?

What personal member needs are being addressed?

How will the community benefit the organization?

What business needs are being addressed?

Community

VisionWhy are you creating this community?

What is the overall purpose of the community?

Business Goals

Member Goals

• Report Writers• Data Scientists• Dashboard Developers• Data Engineers• Business Users• QA Leads

• Reduce rework• Accelerate innovation• Improve efficiency• Standardize processes• Clarify R&R’s• Increase collaboration

• Collaborative technology• Data Catalog• Business Glossary• Stewardship/ Curation

1. Gartner, 2017

NCHICA 28

Example Systems and Processes

Why? How? What?

• Business objectives

• Purpose / goals

• Business challenges

• ROI

• Business prioritization

• Alignment to value

• Linkage to strategy

• Approach

• Technology/

platform

• Architecture

• Execution plan

• Best practices

• Report

• Dashboard

• Metric

• Business Rule

• Glossary

• Definition

• Lineage

Implicit ExplicitKnowledge

Technical deliverables

Business definitionsProject CharterSpreadsheets

Project PlansSpreadsheetsDiagramsLessons Learned

NCHICA 29

Our Content and Collaboration Strategy

GitHub• Project-based

technical artifacts

• Design• Requirements• Code

SharePoint• Product Catalog• Stakeholder

communications

• Learning Home• Value Registry

Data Catalog• Definitions• Glossary• Data sources• Technical

Metadata• Data

Classification

Other• EverNote Library• ShareFile - Plans,

Documents• LucidChart -

Diagrams• Miro –

Brainstorming• Aha! – Product

Portfolio Mngmt• ServiceNow –

Incidents, Tasks,

NCHICA 30

Solution: Make It Easy to Understand the Metric

• Links to EADSpedia page

• Full metric details

• Other similar metrics are co-located on the page

• Where relevant, links to advanced analytics projects are

included

NCHICA 31

Solution: Document Everything Well and By Topic

Links to FY19 dashboard wiki page

Related metric topics are grouped

FY18 version

Metric details & governance

links

Table of Contents for Metrics, analytics

work, helpful links

Quality context

NCHICA 32

Data Catalog

NCHICA 33

Data Science Platform

NCHICA 34

Key Lessons

What we tried? What we learned?

Lots of technology! (Yammer, EverNote, OneNote, SharePoint, ShareFile, GitHub, Wikis)

• Prototyping is good; don’t try to operationalize too fast• Technical fluency matters• Repeated exposure (sell the change)• Accountability is key (anchoring the change)

Standardized vs. freeform content • Not everyone thinks like me (content organization)• Establish guardrails• Standardize (R&Rs, Procedures, Measurement)

Content Repositories • Automate anything that can/ should be automated• Don’t force “unnatural” behaviors• Social participation (people will go wherever it serves them)

“Collaboration and social within the company is 80% people, process and content and 20% IT.“

NCHICA

Future Direction

• Our respective journeys• Where we are and where we want to be• Measurement objectives• Technology aids• Value registry• Incentives

35

THANKYOU

NCHICA

Greg Nelson, MMCi, CPHIMSVidant Healthgreg.nelson@vidanthealth.com

Monica Horvath, PhDFormerly of UNC Healthcaremonicahorvathphd@gmail.com

36