5/18/17©UWMRF 2017 1

(OTT ID 1236)Inventors: Joseph Bockhorst and Scott Vanderbeck, Department of Computer

Science and Electrical Engineering , University of Wisconsin-Milwaukee; Samer Gawrieh, Department of Gastroenterology and Hepatology and

Director of Liver Transplantation, Medical College of Wisconsin

For further information please contact:

Automatic Pathology Software for Diagnosis of Non-Alcoholic Fatty Liver Disease

Jessica Silvaggi Licensing Manager

1440 East North Ave. Milwaukee, WI 53202

Tel: 414-906-4654jsilvaggi@uwmfdn.org

Shortfalls of current liver pathology assessment

Problems/Unmet Needs:

• Non-Alcoholic Fatty Liver Disease (NAFLD) is the most common liver disease in the U.S.

• Accurate distinction of mild vs. severe phenotypes is essential and liver biopsy is the only way to accurately diagnose and stage NAFLD

• Current NAFLD scoring method relies on manual pathologist assessment and grading of histological features

• Studies demonstrate substantial pathologist disagreement and inter-observer error (Gawrieh, S.,

Knoedler, 2011. 15: 19-24.)

• Poor reproducibility; intra-observer error

• Does not reflect the true continuous nature of “scoring” lesions

Technological Solution:

• The inventors have utilized a supervised machine learning technique for identification of white regions in liver biopsies with 92% accuracy (based on results from 47 patients)

• This is the first work to identify white regions using machine learning and automatically quantify lobular inflammation and hepatocyte ballooning

• This technology is automatic, faster, and more accurate than human pathologists

• Applications include diagnosis of NAFLD, assessment of candidate liver donors for transplants, and biopsy index database searching

5/18/17©UWMRF 2017 2

Road to Commercialization: Market, Intellectual Property, and Partnering

Market

• The global market for liver disease treatments was $400 million in 2009 and expected to increase to more than $700 million by 2014

• Computer aided detection and diagnosis (CAD), originally used in image analysis and the development of algorithms for image recognition, has evolved to offer full workflow management packages for a range of healthcare conditions

• Use of CAD can provide a more economically viable proposition for busy, cost-focused healthcare providers; CAD allows physicians to deal with enormous data sets more efficiently, making their job easier and in turn making physicians more accurate

Intellectual Property

• Copyrighted Software

Partnering

• We are looking for a partner to license the algorithm and software and to develop the technology into a final product for use by pathologists or other end users

5/18/17©UWMRF 2017 3

5/18/17©UWMRF 2017 4

Liver Anatomy-Lobule

A single lobuleMultiple lobules

• The bile duct, portal artery and vein, and central vein are some of the white regions that must be distinguished from the fatty white regions in a liver lobule

Liver Biopsies for Pathological Assessment

5/18/17©UWMRF 2017 5

H&E Stain Trichrome Stain

• Biopsy sections are cut into different planes for staining

• White regions are identified from the stained biopsy sections from a patient

Pathologists use the NAFLD Activity Score (NAS)

5/18/17©UWMRF 2017 6

• State-of-the-art scoring system for NAFLD (Kleiner et al., 2005)

• Based on 3 key histological features:

– Steatosis [0-3]

– Lobular Inflammation [0-3]

– Hepatocyte Ballooning [0-2]

• NAS = Steatosis + Lobular Inflammation + Hepatocyte Ballooning

Not

Steatohepatitits Borderline NASH

0 1 2 3 4 5 6 7 8

Examples of Histological Legions

5/18/17©UWMRF 2017 7

Steatosis Lobular

Inflammation

Hepatocyte

Ballooning

Normal

Approach for Machine Learning

5/18/17©UWMRF 2017 8

Pathologist

Annotations

Study Patients

Patient P

H&E Stained Images

White Region

Classifier

Lobular Inflammation Calculator

Lobular Inflammation Classifier

Ballooning Classifier

TC Stained Images

ImageTCImageHE

Portal Triad Identification

Heuristic

Ballooning Calculator

Steatosis Calculator

Fibrosis Calculator

FibrosisClassifier

SP FPBP LP

Classification of White Regions

5/18/17©UWMRF 2017 9

• Motivation

– Classifying all white regions provides a direct methods for quantifying steatosis

– Many anatomical land markers manifest as white or with a white center. This is useful for other tasks

Method for Classification of White Regions

5/18/17©UWMRF 2017 10

Hypothesis:

• Comparing previous methods that use hand crafted rules based purely on morphology, to a supervised machine learning approach that uses a richer feature vector representation

• Richer feature vector representations lead to more accurate classifications than ones based purely on morphology

– We compare 2 representations:

1. Morphology Only

2. Advanced feature set

Approach

• Represent each white region by a fixed length feature vector

• 10 fold cross validation

Results of White Region Classification

5/18/17©UWMRF 2017 11

Model Overall Accuracy P-Value

Morphology 74.8%

Advanced feature set 92.1% 0.0007

Supervised Machine Learning Results

Accuracy

The results show that the algorithm is predicting the white regions with very high accuracy.

Results of White Region Classification-Cont’d

5/18/17©UWMRF 2017 12

• Results again show that the algorithm is classifying white regions with very high accuracy. Of particular interest is how well it performs for macro and micro steatosis (fat)

Feature Precision Recall ROC Area

Bile Duct 0.900 0.849 0.981

Central Vein 0.732 0.788 0.938

Macro Steatosis 0.954 0.974 0.980

Micro Steatosis 0.930 0.927 0.967

Other 1.000 0.857 0.994

Portal Artery 0.786 0.767 0.973

Portal Vein 0.901 0.880 0.976

Sinusoid 0.924 0.899 0.982

OVERALL 0.921 0.921 0.974

Steatosis Grade Machine Learning

5/18/17©UWMRF 2017 13

• Percent Steatosis = total steatosis area / total tissue area

• Learn a different model for each patient using a leave one out approach

• The model is able to correlate best with the average grade

0

1

2

3

0%2%4%6%8%

10%12%14%16%18%20%22%24%26%

FLE0

38

FLE0

50

FLE0

13

FLE0

20

FLE0

41

FLE0

31

FLE0

32

FLE0

12

FLE0

46

FLE0

01

FLE0

45

FLE0

18

FLE0

37

FLE0

25

FLE0

16

FLE0

15

FLE0

47

FLE0

26

FLE0

34

FLE0

44

FLE0

22

FLE0

27

FLE0

36

FLE0

03

FLE0

04

FLE0

05

FLE0

51

FLE0

33

FLE0

35

FLE0

42

FLE0

28

FLE0

40

FLE0

09

FLE0

24

FLE0

17

FLE0

11

FLE0

48

FLE0

21

FLE0

07

FLE0

29

FLE0

49

FLE0

43

FLE0

23

FLE0

14

FLE0

08

FLE0

39

FLE0

30

Pat

ho

logi

st G

rad

e

Mo

de

l Pe

rce

nt

Ste

ato

sis

Correlation of Computed Percentage Steatosis with Pathologist Grade

Avg. Pathologist Grade

% Steatosis

y = 0.0252x2 + 0.0067x + 0.003R² = 0.9392

0%

5%

10%

15%

20%

25%

30%

0 0.5 1 1.5 2 2.5 3

Mo

del

Per

cen

t St

eato

sis

Pathologist Grade

Relationship of Computed Percentage Steatosis with Pathologist Grade

Patient Sample

Lobular Inflammation Classification

5/18/17©UWMRF 2017 14

Hypothesis/Approach:

• Lobular inflammation classification can be performed by supervised machine learning

• Provide a representative quantification for actual lobular inflammation regions

• Use a feature vector very similar to the one used for white regions

• 10 fold cross validation

Approach for Lobular Inflammation Classification: 95.6% accuracy (vs. 94.0% baseline)

5/18/17©UWMRF 2017 15

Feature Precision Recall ROC AreaLobular Inflammation 0.696 0.489 0.946

Not-Lobular Inflammation 0.968 0.986 0.946OVERALL 0.952 0.956 0.946

00.10.20.30.40.50.60.70.80.9

1

0 0.2 0.4 0.6 0.8 1

Pre

cis

ion

Recall

Precision vs. Recall Curve

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

Tru

e P

osit

ive R

ate

False Positive Rate

Receiver Operating Characteristics

Hepatocyte Ballooning: Classification Results

5/18/17©UWMRF 2017 16

• 98.9% accuracy (vs. 97.9% baseline)

Feature Precision Recall ROC Area

Hepatocyte Ballooning 0.912 0.542 0.983

Not-Hepatocyte Ballooning 0.990 0.999 0.983

OVERALL 0.989 0.989 0.983

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

Pre

cis

ion

Recall

Precision vs. Recall Curve

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

Tru

e P

os

itiv

e R

ate

False Positive Rate

Receiver Operating Characteristics

Overall Results for NAFLD Activity Score (NAS)

5/18/17©UWMRF 2017 17

• 73.8% Correlation with Pathologists

y = 0.7624x + 0.4561R² = 0.6532

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

0 1 2 3 4 5 6

Co

mp

ute

d M

od

el N

AS

Average Pathologist NAS

Comparison of Computed NAS with Average Pathologist NAS

Computed Model NAS

Linear (Computed Model NAS)

• Circled region accounts for diagnosis by software of a worse case

than that predicted by pathologist

Summary

5/18/17©UWMRF 2017 18

• Computer imaging techniques can effectively be used as a diagnostic aid for NAFLD / NASH

• Supervised learning techniques are superior to hand crafted computational rules

• Steatosis grading accurate enough for clinical use

• Next Steps

– Additional data to study lobular inflammation and ballooning

– Model refinement, tile size, etc.

– Features in other domains (FFT, Wavelets, etc)

– Impact of magnification of scan

Next Steps: Product Development

5/18/17©UWMRF 2017 19

• Company must convert Matlab algorithms into a production software environment for commercial use

• Estimated 500 hours of development to move this technology into production

31B Treats Ovarian Cancer in Mice

5/18/17©UWMRF 2017 20

Automatic Pathology Software for Diagnosis of Non-Alcoholic Fatty Liver Disease

(OTT ID 1236)

For more information contact:

Jessica SilvaggiLicensing Manager

1440 East North AvenueMilwaukee, WI 53202

Tel: 414-906-4654