1Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

Review

Glaucoma management in the era of artificial intelligenceSripad Krishna Devalla,1 Zhang Liang,1 Tan Hung Pham,1,2 Craig Boote,1,3,4 Nicholas G Strouthidis ,2,5,6 Alexandre H Thiery,7 Michael J A Girard 1,2

To cite: Devalla SK, Liang Z, Pham TH, et al. Br J Ophthalmol Epub ahead of print: [please include Day Month Year]. doi:10.1136/bjophthalmol-2019-315016

1Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore2Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore3School of Optometry & Vision Sciences, Cardiff University, Cardiff, UK4Newcastle Research & Innovation Institute, Singapore, Singapore5NIHR Biomedical Research Centre, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK6Discipline of Clinical Ophthalmology and Eye Health, University of Sydney, Sydney, New South Wales, Australia7Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore

Correspondence toDr Michael J A Girard, Department of Biomedical Engineering, National University of Singapore, Singapore 117583, Singapore; mgirard@ nus. edu. sg

Received 30 July 2019Revised 7 September 2019Accepted 5 October 2019

© Author(s) (or their employer(s)) 2019. No commercial re-use. See rights and permissions. Published by BMJ.

AbsTrACTGlaucoma is a result of irreversible damage to the retinal ganglion cells. While an early intervention could minimise the risk of vision loss in glaucoma, its asymptomatic nature makes it difficult to diagnose until a late stage. The diagnosis of glaucoma is a complicated and expensive effort that is heavily dependent on the experience and expertise of a clinician. The application of artificial intelligence (AI) algorithms in ophthalmology has improved our understanding of many retinal, macular, choroidal and corneal pathologies. With the advent of deep learning, a number of tools for the classification, segmentation and enhancement of ocular images have been developed. Over the years, several AI techniques have been proposed to help detect glaucoma by analysis of functional and/or structural evaluations of the eye. Moreover, the use of AI has also been explored to improve the reliability of ascribing disease prognosis. This review summarises the role of AI in the diagnosis and prognosis of glaucoma, discusses the advantages and challenges of using AI systems in clinics and predicts likely areas of future progress.

InTroduCTIonGlaucoma is a presently incurable condition that requires lifelong treatment and monitoring, and is the second largest cause of irreversible blind-ness worldwide.1 Primarily affecting the elderly, it is estimated that nearly 79.6 million people will have glaucoma by 2020.2 The central event in glau-coma is the irreversible damage of retinal ganglion cell (RGC) axons that carry visual information from the retina to the brain.3 When damaged, the RGCs undergo programmed cell death (apoptosis) resulting in vision loss.3 While an early diagnosis could minimise the risk of permanent vision loss,4 nearly half the patients affected by glaucoma remain undiagnosed until a relatively late stage5 due to the slow and asymptomatic nature of the disease in its earlier stages.6

Once diagnosed, predicting the progression of glaucoma is a complex endeavour that is time consuming, subjective and heavily dependent on the clinician’s experience and expertise, and requires multiple clinical tests.7 8 Often, these tests are repeated over several visits to rule out their inherent subjectivity and to account for patient variability.7 In fact, in 2016 the World Glaucoma Association acknowledged the absence of a single specific test that could be regarded as a perfect reference to predict the progression of glaucoma.9 This means that cases of overtreatment and undertreatment

are presently inevitable.9 Given that glaucomatous changes to the optic nerve head (ONH) tissues are irreversible, timely and reliable structural and functional evaluation of the eye could help in the early diagnosis of glaucoma, and to better predict its progression.10 11

In recent years, artificial intelligence (AI)-based systems have started to revolutionise the health-care industry.12–14 In the field of ophthalmology, a number of AI approaches have been explored for the diagnosis of retinal,15 16 macular,17 18 choroidal16 and corneal pathologies.19 20 Moreover, with the advent of deep learning (DL), a number of AI tools for the automated segmentation and enhancement of ocular images from optical coherence tomog-raphy (OCT)21–25 and fundus imaging modalities26 have also been proposed.

Modern AI algorithms are especially tailored to extract meaningful features from complex and high-dimensional data. Consequently, a number of AI studies have been proposed for the diagnosis and management of glaucoma based on the interpreta-tion of functional and/or structural information of the eye. This review summarises the role of AI in glaucoma, the clinical advantages and challenges, and projects potential future applications of AI in glaucoma.

undersTAndIng AI AlgorIThmsThe AI algorithms discussed in this review can be broadly classified into two categories based on the complexity of the data they handle.

The first category consists of machine learning classifiers (MLC) and artificial neural networks (ANN). MLCs such as random forest (RF), logistic regression (LR), support vector machine (SVM), Gaussian mixture model (GMM) and independent component analysis (ICA) are clustering algorithms that are extensions of classical statistical modelling. ANNs, on the other hand, are biologically inspired algorithms that pass the input data through a series of interconnected nodes (artificial neurons), and continuously modify the weights of each node to obtain the desired classification.

These algorithms learn to take input data (eg, clinical parameters) and automatically make a prediction (eg, presence of pathology, glaucoma severity) through a supervised or unsupervised learning process. In supervised learning, the algo-rithm (eg, SVM, RF, ANNs, LR) is trained with a fully labelled data set (eg, disease diagnosis as label). In unsupervised learning, the algorithm (eg, GMM, ICA) is trained with an unlabelled data set

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

2 Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

Figure 1 A comparison between the actual (second column) and the AI-predicted (third column) HFVs as reported by Wen et al.41 The input HFVs at different stages of glaucoma for the AI algorithm are shown in the first column. AI, artificial intelligence; HVF, Humphrey visual fields; MD, mean deviation; PMAE: point-wise mean absolute error.

(eg, only clinical parameters as inputs) in an attempt to identify new patterns/trends. Such algorithms are typically well suited for handling low-dimensionality numeric data (eg, simple numbers such as the vertical cup-to-disc ratio, intraocular pressure (IOP), age and sex).

The second category of AI algorithms are the sophisticated variants of the ANNs known as convolutional neural networks (CNN). They are well suited for exploiting high dimension-ality data (eg, fundus and OCT images) through multiple interconnected levels of data abstraction. In each level, convo-lution layers (layers of filters) attempt to organically extract the features (feature maps; eg, information on texture, edges, inten-sity, thickness) that best represent the task (eg, identifying the presence of a pathology, identifying a specific tissue) of the algo-rithm. Through an iterative learning process (training) that aims to minimise the error between the output of the network (eg, predicted diagnosis) and the ground truth (eg, clinical diagnosis), weights (parameters) of the extracted feature maps (to decide the influence of each feature towards the final decision) are contin-uously refined until the optimal weights (least error between the network output and the ground truth) are identified.

In the recent years, DL, an advanced and powerful incarnation of the CNNs, has become the go-to AI approach in the field of medical imaging for segmentation, enhancement and diagnostic applications.27 When exposed to diverse and large volumes of data (eg, a combination of clinical parameters and images), it is often critical to identify the important features present in those data that drive the accuracy of an AI model. This is a time-con-suming process known as feature engineering. In such scenarios, DL networks often outshine traditional methods owing to their

‘automated feature engineering’ approach (ie, automatically identifying the best set of features in the data that influence the performance of the algorithm). For instance, jointly under-standing the textural information (eg, hyper-reflectivity) and spatial arrangement (eg, between the retinal layers and choroid) might be crucial for identifying the retinal pigment epithelium layer while learning to segment the ONH tissues from OCT images. Nevertheless, a hybrid of traditional and advanced AI algorithms are also being explored to increase the robustness of these predictive models.

AI For evAluATIon oF FunCTIonAl dAmAgeIn the earliest study reported in 1994, Goldbaum et al28 29 proposed the use of ANNs to interpret visual fields (VF) data from standard automated perimetry evaluations. When trained on the absolute threshold sensitivity, age and VF values (read in sequence from nasal to temporal), the ANN was able to detect glaucomatous eyes almost as proficiently as a trained reader (ANN and expert agreement: 74%). Subsequently, other studies30–32 also concluded that ANNs and MLCs are able to match, or even outperform, human experts and more conven-tional algorithms.32 A study by Lietman et al33 concluded that ANNs designed to detect VF defects could also outperform accepted global indices such as the mean deviation and pattern SD at specificities greater than 90%. More recently, CNNs have been leveraged to detect abnormal VFs.34 35 Li et al34 developed a DL network that was trained on the probability map of the pattern deviation image to distinguish glaucoma from healthy VFs. The network achieved a diagnostic accuracy of 87.6%, higher than glaucoma experts (62.6%) and using traditional criteria such as Advanced Glaucoma Intervention Study (AGIS) (45.9%) and Glaucoma Staging System 2 (GSS2) (52.3%). In another study, Kucur et al35 reported that a DL system trained with images that were Voronoi representations of VFs could identify early glaucomatous VFs with an average precision of 87.4%.

AI For prognosIs oF FunCTIonAl dAmAgeThe progression of glaucoma is non-linear, dependent on the instrument used for assessment, and there presently exists no widely accepted clinical index to predict glaucomatous vision loss.36 Several research groups have explored AI approaches to increase the reliability of the clinical prognosis. In one of the initial studies, Brigatti et al37 developed an ANN using the VF data obtained from the Yale Glaucoma Section database. The network, when trained on the VF threshold points, mean defect, corrected loss, variance, false positive and false negative ratios and patient age, produced a good agreement (sensitivity: 73% and specificity: 88%) with the experienced observer. Sample et al38 reported that standard MLCs such as SVMs and mixture of Gaussians classifiers predicted the development of abnormal fields, on average, nearly 4 years earlier than more traditional methods (StatPac-like) in patients with ocular hypertension (OHT). Since OHT is a significant risk factor for glaucoma, the proposed VF prognosis approach could also help in the early diagnosis. Studies exploiting unsupervised techniques such as ICA also concurred that the VF loss predicted by ANNs was comparable, or even better than, existing clinical criteria.39 Simi-larly, Yousefi et al40 concluded that AI techniques could identify patterns of progression earlier than more conventional methods. Recently, Wen et al41 reported that DL networks could predict the future 24-2 Humphrey visual fields (HVF), up to 5.5 years (figure 1), from a single HVF as input.41

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

3Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

It is important to note that, while the above studies reported the success of AI in the assessment and prognosis of functional damage, their performance is variable due to the sheer subjec-tivity involved in obtaining VF data (eg, patient factors, measure-ment noise, fixation losses).42 A summary of the AI studies for the diagnosis and prognosis of glaucoma using functional evalu-ations can be found in table 1.

AI For evAluATIon oF sTruCTurAl dAmAgeEarly AI-based studies assessed glaucomatous structural damage using data obtained from confocal scanning laser ophthalmos-copy (CSLO), and scanning laser polarimetry (SLP) modalities. Although the clinical relevance of these modalities has decreased in recent years because of advances in ophthalmic imaging,42 43 their contribution in establishing the idea that glaucomatous structural damage precedes VF loss is noteworthy.44

Several studies reported that MLCs increased the discrimina-tory power of the optic disc parameters obtained from CSLO.45–50 Specifically, Bowd et al45 reported that MLCs, when trained on global and regional optic disc topographic parameters, offered a significantly higher area under the curve (AUC) when compared with more standard methods. The results also inferred that the peak height contour (temporal inferior), global cup shape and the disc area (nasal) were the most informative parameters in discriminating glaucomatous and healthy eyes. Moreover, studies also concurred that MLCs trained on the retinal nerve fibre layer (RNFL) measurements from an SLP device offered a diagnostic accuracy higher than the inbuilt software (GDx).51 52

With the advent of fundus colour photography, studies have explored the use of ANNs for the segmentation and classifica-tion of optic disc photographs.53–58 In the earliest study, Sinthan-ayothin et al58 reported that an ANN recognised the important regions (optic disc, blood vessels and fovea) of a fundus photo-graph with high sensitivity (80.4%–99.1%) and specificity (91.0%–99.1%). While these studies were not designed to directly help the diagnosis of glaucoma, they53–57 led the way for other groups to design methods to automatically extract relevant structural parameters (ie, cup-to-disc ratio, volume) for disease diagnosis and clinical management.

Subsequently, many DL approaches directly exploited the visual information and extracted glaucoma-related features from fundus colour photographs.59–61 In a landmark multiethnicity study, Ting et al61 developed a DL network trained on 500 000 fundus photographs (125 189 glaucoma images) that was capable of discriminating glaucomatous fundus photos with high confi-dence (AUC: 0.942; specificity: 87.2%; sensitivity: 96.4%). Li et al60 also developed a DL network (AUC: 0.982; specificity: 92.0%; sensitivity: 95.6%) able to detect referable glaucoma-tous optic neuropathy by training a CNN on 48 000 fundus photographs. In another study, Medeiros et al59 proposed a DL approach to predict the spectral-domain OCT average RNFL thickness measurements from fundus photos (mean error less than 10 µm). If successfully translated to clinics, the proposed DL-based system could cost-effectively diagnose and stage glau-coma merely from optic disc photographs.

Having demonstrated excellent reproducibility in measuring the RNFL thickness,62 OCT has emerged as the de facto stan-dard for objective quantification of glaucomatous structural damage of the ONH tissues.63 In the earliest study, Huang and Chen64 reported that ANNs successfully differentiated (AUC: 0.87) glaucomatous and healthy eyes using OCT measure-ments (RNFL thickness and ONH parameters). Furthermore, Burgansky-Eliash et al65 showed that MLCs offered excellent

discriminatory power (AUC: 0.98) in detecting glaucoma eyes using the OCT parameters obtained from macula, peripapillary and ONH regions. The study also concluded that MLCs were able to differentiate (AUC: 0.85) early from advanced glaucoma eyes. Other research groups that attempted to study the efficacy of other MLCs using OCT measurements also achieved similar results.66–69

Although the above-mentioned studies reported the general success of AI systems in identifying glaucoma eyes using OCT parametric data, their performance strongly depended on the accuracy of the automated measurements. For instance, the presence of blood vessel shadows can adversely affect the performance of these tools, yielding incorrect RNFL thickness measurements.70 Indeed, these issues are more pronounced in glaucoma subjects since they already exhibit a thinner RNFL,71 limiting the classification ability of AI systems. Besides structural parameters, there exists other visual information from OCT images (speckle pattern, tissue reflectance) that are associated with the progression of glaucoma.72 Thus, by better exploiting the information contained within OCT image data, it is feasible to increase the diagnostic power of the instrument in ophthal-mology clinics.

While the idea of AI-assisted OCT for glaucoma diagnosis is still quite nascent, a few studies have begun to explore its feasi-bility using deep CNNs. Muhammad et al73 proposed a hybrid approach that used a CNN to extract rich features from wide-field (9×12 mm) OCT scans that were later classified by an RF classifier to predict the existence of glaucomatous damage (AUC: 0.94). The extracted features included the RNFL and ganglion cell plus inner plexiform layer (RGC+) thickness measurements, thickness probability maps and the en face projection images. In a population-based study (2701 subjects; 135 glaucoma), Girard et al74 developed a novel ‘Co-Training’ DL network to simulta-neously segment (figure 2A) and diagnose glaucoma (figure 2B) from OCT images of the ONH. The DL network first isolated the individual neural and connective tissues, and subsequently combined the segmentation information with the OCT images to identify glaucoma from non-glaucoma subjects (AUC: 0.90). By leveraging 3D structural information, the DL network can discriminate glaucomatous eyes significantly better than methods exploiting the RNFL thickness values alone (figure 2B). Maetschke et al75 proposed a feature agnostic approach that used a 3D DL network to classify glaucomatous and healthy eyes directly from raw OCT volumes (AUC: 0.94). Further, they also concluded that the DL network focused on the neuroretinal rim, optic disc area, and the lamina cribrosa and its surrounding regions, while identifying a glaucoma scan (figure 3). These find-ings concurred with established clinical markers associated with glaucoma,76 thus adding ‘clinical explainability’ to these so-called ‘black-box’ tools. More recently, in a multidevice, multiethnicity study, Zhang et al77 reported that a DL network could offer a fast (less than 1 s) and simplified glaucoma diagnosis (AUC: 0.90) from just a single clinical test (OCT scan of the ONH). Finally, a number of studies have also used AI to detect glaucoma with varying success from anterior segment OCT images and measurements (AUC ranging from 0.85 to 0.96).78–81 A summary of the AI studies for the diagnosis of glaucoma using the struc-tural evaluations can be found in table 2.

hybrId AI ApproAChesGiven the inherent subjectivity of functional assessments and the variability in structural measurements, research groups have attempted to increase the discriminatory power of AI systems by combining both the structural and functional statuses of the eye.

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

4 Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

Tabl

e 1

Sum

mar

y of

the

AI s

tudi

es fo

r the

dia

gnos

is o

f gla

ucom

a us

ing

the

func

tiona

l eva

luat

ions

Aut

hor

Jour

nal

popu

lati

on (t

rain

ing/

test

ing)

Ass

essm

ent

Type

of A

Id

ata

type

AuC

sens

itiv

ity

spec

ifici

tyCo

mm

ents

Gol

dbau

m e

t al28

29

Inve

st O

phth

alm

ol V

is S

ci60

N a

nd 6

0 G

pat

ient

s (1

0-fo

ld c

ross

-va

lidat

ion)

Diag

nosi

s (F

)AN

NVF

s–

65%

71%

ANN

/exp

ert a

gree

men

t: 74

%

Gol

dbau

m e

t al30

Inve

st O

phth

alm

ol V

is S

ci18

9 N

and

156

G p

atie

nts

(10-

fold

cro

ss-v

alid

atio

n)Di

agno

sis

(F)

MLC

VFs

0.92

279

%90

%–

Chan

et a

l31IE

EE Tr

ans

Biom

ed E

ng18

9 N

and

156

G p

atie

nts

(25-

fold

cro

ss-v

alid

atio

n)Di

agno

sis

(F)

MLC

VFs

0.88

–0.9

258

.3%

–78.

2%90

%–

Bizi

os e

t al32

J Gla

ucom

a11

6 N

and

100

G p

atie

nts

Trai

ning

: 53.

7:46

.3%

(N:G

)Te

stin

g: 4

6.3:

53.7

% (N

:G)

Diag

nosi

s (F

)AN

NVF

s0.

984

93%

94%

–

Liet

man

et a

l33J G

lauc

oma

249

N a

nd 1

06 G

pat

ient

s (tr

aini

ng a

nd

test

ing:

hal

f-hal

f)Di

agno

sis

(F)

ANN

VFs

––

–O

utpe

rform

ed g

loba

l ind

ices

at h

igh

spec

ifici

ties

(90%

–95%

)

Li e

t al34

BMC

Med

Imag

ing

4012

PD

imag

es (t

rain

ing:

371

2; te

stin

g:

300)

Diag

nosi

s (F

)DL

VFs

(PD

imag

es)

–93

.20%

82.6

0%Ac

cura

cy: 8

7.60

%

Kucu

r et a

l35Pl

oS O

ne19

79 N

and

281

1 G

VF

imag

es (1

0-fo

ld

cros

s-va

lidat

ion)

Diag

nosi

s (F

)DL

VFs

(Vor

onoi

im

ages

)–

––

Aver

age

prec

isio

n: 8

7.40

%

Brig

atti

et a

l37Ar

ch O

phth

alm

ol18

1 G

pat

ient

s (th

reef

old

cros

s-va

lidat

ion)

Prog

nosi

s (F

)AN

NVF

s–

73%

88%

–

Sam

ple

et a

l38In

vest

Oph

thal

mol

Vis

Sci

Trai

ning

: 189

N a

nd 1

56 G

pat

ient

s (1

0-fo

ld

cros

s-va

lidat

ion)

Test

ed: 1

14 O

HT

Prog

nosi

s (F

)M

LCVF

s–

––

Pred

icte

d ab

norm

al fi

elds

3.9

2±0.

55 y

ears

ea

rlier

than

trad

ition

al m

etho

ds

Sam

ple

et a

l*39

Inve

st O

phth

alm

ol V

is S

ci66

OHT

,73

GS

and

52 G

pat

ient

s

Prog

nosi

s (F

)U

LVF

s–

––

ICA

quan

titat

ivel

y id

entifi

ed a

larg

er

perc

enta

ge o

f pro

gres

sion

in e

yes.

Yous

efi e

t al40

Am J

Oph

thal

mol

Cros

s-se

ctio

nal c

ohor

t: 11

46 N

and

677

pa

tient

sLo

ngitu

dina

l coh

ort:

139

N p

atie

nts

Test

–ret

est c

ohor

t: 71

G p

atie

nts

Prog

nosi

s (F

)U

LVF

s–

87%

96%

Mac

hine

lear

ning

hel

ps to

con

sist

ently

det

ect

the

prog

ress

ion

of g

lauc

oma

muc

h ea

rlier

th

an o

ther

con

vent

iona

l met

hods

.

Wen

et a

l41Pl

oS O

neTr

aini

ng: 3

972

patie

nts

Test

ing:

903

pat

ient

sPr

ogno

sis

(F)

DL24

-2 H

VFs

––

–Pr

edic

tion

of H

VFs

up to

5.5

yea

rs fr

om s

ingl

e HV

F

The

abbr

evia

tions

and

acr

onym

s us

ed in

the

tabl

e ca

n be

foun

d in

the

onlin

e su

pple

men

tary

app

endi

x.*S

tudy

con

side

red

glau

com

a su

spec

t/ear

ly-s

tage

gla

ucom

a su

bjec

ts.

AI, a

rtifi

cial

inte

llige

nce;

AN

N, a

rtifi

cial

neu

ral n

etw

ork;

AU

C, a

rea

unde

r the

cur

ve; D

L, d

eep

lear

ning

; F, e

valu

atio

n us

ing

func

tiona

l inf

orm

atio

n; G

, gla

ucom

a; G

S, g

lauc

oma

susp

ect;

HVF,

Hum

phre

y vi

sual

fiel

ds; I

CA, i

ndep

ende

nt c

ompo

nent

an

alys

is; M

LC, m

achi

ne le

arni

ng c

lass

ifier

; N, n

orm

al; O

HT, o

cula

r hyp

erte

nsio

n; P

D, p

atte

rn d

evia

tion;

UL,

uns

uper

vise

d le

arni

ng; V

F, vi

sual

fiel

d.

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

5Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

Figure 2 Recent deep learning (DL)-based networks for the segmentation (A), diagnosis (B) and enhancement (C) of the optical coherence tomography (OCT) images of the optic nerve head (ONH). A ‘Co-Training’ DL network proposed by Girard et al74 for the simultaneous segmentation of neural and connective tissues, and diagnosis of glaucoma (A and B). Devalla et al25 developed a DL algorithm to enhance the quality of ONH images (C). RNFL, retinal nerve fibre layer.

Figure 3 Heat maps (or class activation maps) are used to better understand the decision-making process of the 3D deep learning (DL) network.76 The first column represents the en face view for a glaucoma (A) and healthy (C) optical coherence tomography (OCT) scan; while the second column represents their respective side views (B and D). The red maps correspond to the regions that have influenced the DL network the most for the classification of healthy/glaucoma eyes. POAG, primary open-angle glaucoma.

The earliest such study by Brigatti et al82 assessed the perfor-mance of an ANN trained on the ONH parameters (cup-to-disc ratio, rim area, cup volume and RNFL thickness) and the VF indices (mean defect, correct loss variance and short-term fluctu-ation). The network offered a higher diagnostic accuracy (88%) when trained with both the structural and functional informa-tion, as opposed to only one of them (ONH parameters/VFs: 80%/84%). Subsequent studies that combined the structural measurements from OCT,67 83 CSLO84–86 and SLP parameters87 along with the perimetry evaluations also offered similar results.

In a slightly different hybrid approach, Oh et al88 reported that an ANN trained with a mix of ophthalmic (IOP, spherical equivalent refractive errors, vertical cup-to-disc ratio, presence RNFL defects) and systemic factors (sex, age, menopause, dura-tion of hypertension) could successfully differentiate (AUC:

0.89) between primary open-angle glaucoma (POAG) subjects and glaucoma suspects.

oTher AI ApproAChesWhile the AI algorithms mentioned above discriminated between the functional and/or structural status of the eye to identify glau-coma, a few studies have also explored the potential of applying AI to genetic data. In the largest genome-wide association study conducted on nearly 140 000 participants, Khawaja et al89 iden-tified 112 genomic loci (including 68 novel loci) associated with IOP and the development of POAG. Further, they developed a regression model that predicted POAG with an AUC of 0.76 based on these loci. Burdon et al90 used both regression and ANN models and concluded that a combination of disc parame-ters, IOP and POAG-associated loci could improve the accuracy of POAG risk prediction models, thus allowing scope for early treatment and prevention of blindness. A summary of the hybrid and genetic studies that use AI for the diagnosis of glaucoma can be found in table 3.

dIsCussIonIn this review, we discuss the role of AI in the diagnosis and prog-nosis of glaucoma using functional or/and structural evaluations of the eye. While early studies relied on simple MLCs or ANNs to detect glaucomatous eyes using parametric data, modern DL systems have successfully exploited high-dimensionality image data, thus increasing the diagnostic power of ocular imaging modalities such as fundus photography and OCT in clinics.

The use of AI algorithms in clinics may be viewed as a tool to assist clinicians, but not to replace them. AI methods can help speed up the triage process by collating data from multiple tests,

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

6 Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

Tabl

e 2

Sum

mar

y of

the

AI s

tudi

es fo

r the

dia

gnos

is o

f gla

ucom

a us

ing

the

stru

ctur

al e

valu

atio

ns

Aut

hor

Jour

nal

popu

lati

on (t

rain

ing/

test

ing)

Ass

essm

ent

Type

of A

Id

ata

type

AuC

sens

itiv

ity

spec

ifici

tyCo

mm

ents

Bow

d et

al45

Inve

st O

phth

alm

ol

Vis

Sci

189

N a

nd 1

08 G

pat

ient

s (1

0-fo

ld c

ross

-val

idat

ion)

Diag

nosi

s (S

)M

LCCS

LO p

aram

eter

s0.

945

83%

90%

–

Tow

nsen

d et

al46

Br J

Oph

thal

mol

60 N

and

140

G p

atie

nts

(eig

htfo

ld c

ross

-val

idat

ion)

Diag

nosi

s (S

)M

LCCS

LO p

aram

eter

s0.

904

85.7

%80

%–

Zang

will

et a

l47In

vest

Oph

thal

mol

Vi

s Sc

i13

5 N

and

95

G p

atie

nts

(10-

fold

cro

ss-v

alid

atio

n)Di

agno

sis

(S)

MLC

CSLO

par

amet

ers

0.96

485

%90

%–

Uch

ida

et a

l48In

vest

Oph

thal

mol

Vi

s Sc

i43

N a

nd 5

3 G

pat

ient

s (th

reef

old

cros

s-va

lidat

ion)

Diag

nosi

s (S

)M

LCCS

LO p

aram

eter

s0.

940

92%

91%

–

Adle

r et a

l49M

etho

ds In

f Med

98 N

and

98

G p

atie

nts

(thre

efol

d cr

oss-

valid

atio

n)Di

agno

sis

(S)

MLC

CSLO

par

amet

ers

–82

.79%

86.5

4%Er

ror=

15.4

%

Bow

d et

al*5

0In

vest

Oph

thal

mol

Vi

s Sc

i22

6 G

S pa

tient

sDi

agno

sis

(S)

MLC

CSLO

par

amet

ers

––

–HR

s (9

5% C

I): 1

.57

(1.0

3 to

2.5

9) fo

r SVM

forw

ard;

an

d 1.

70 (1

.18

to 2

.51)

fo

r SVM

bac

k

Wei

nreb

et a

l51Ar

ch O

phth

alm

ol84

N a

nd 8

3 G

pat

ient

s (1

2-fo

ld c

ross

-val

idat

ion)

Diag

nosi

s (S

)DA

SLP

para

met

ers

0.89

074

%92

%–

Bow

d et

al52

Inve

st O

phth

alm

ol

Vis

Sci

72 N

and

92

G p

atie

nts

(10-

fold

cro

ss-v

alid

atio

n)Di

agno

sis

(S)

MLC

SLP

para

met

ers

0.94

077

%90

%–

Li e

t al60

Oph

thal

mol

ogy

Trai

ning

: 23

433

N, 6

122

GTe

stin

g: 6

033

N, 1

537

GDi

agno

sis

(S)

DLFP

0.98

2595

.60%

92%

–

Ting

et a

l61JA

MA

Trai

ning

: 494

661

imag

es

(125

189

G)

Test

ing:

71

896

G

Diag

nosi

s (S

)DL

FP0.

942

96.4

0%87

.2%

–

Med

eiro

s et

al*

59O

phth

alm

olog

yTr

aini

ng: 4

76 N

, 674

GS

and

699

G p

atie

nts

(GS)

Test

ing:

128

N,1

64 G

S an

d 17

1 G

pat

ient

s

Diag

nosi

s (S

)DL

FP0.

944

(95%

CI 0

.902

to 0

.966

)90

%80

%Ac

cura

cy: 8

3.70

%

Huan

g an

d Ch

en64

Inve

st O

phth

alm

ol

Vis

Sci

100

N a

nd 8

9 G

pat

ient

s (1

0-fo

ld c

ross

-val

idat

ion)

Diag

nosi

s (S

)M

LCO

CT p

aram

eter

s0.

821–

0.99

172

%–1

00%

80%

–

Burg

ansk

y-El

iash

et a

l65In

vest

Oph

thal

mol

Vi

s Sc

i42

N a

nd 4

7 G

pat

ient

s (s

ixfo

ld c

ross

-val

idat

ion)

Diag

nosi

s (S

)M

LCO

CT p

aram

eter

s0.

981

92.5

0%95

%–

An e

t al66

J Gla

ucom

a10

5 G

pat

ient

s (1

0-fo

ld

cros

s va

lidat

ion)

Diag

nosi

s (S

)M

LCO

CT p

aram

eter

s–

––

Accu

racy

: 87.

8%

Kim

et a

l67PL

oS O

ne20

2 N

and

292

G (t

rain

ing

and

test

ing

split

: 80:

20%

)Di

agno

sis

(S)

MLC

OCT

par

amet

ers

and

VFs

0.97

998

.3 %

97.5

%Ac

cura

cy: 9

8%

Bare

lla e

t al68

J Oph

thal

mol

46 N

and

57

G p

atie

nts

(10-

fold

cro

ss-v

alid

atio

n)Di

agno

sis

(S)

MLC

OCT

par

amet

ers

0.87

764

.9%

80%

–

Chris

toph

er e

t al69

Inve

st O

phth

alm

ol

Vis

Sci

28 N

and

93

G p

atie

nts

(trai

ning

and

test

ing:

LO

O)

Diag

nosi

s (S

)U

LO

CT p

aram

eter

s0.

95–

––

Muh

amm

ad e

t al73

J Gla

ucom

a45

N a

nd 5

7 G

pat

ient

sDi

agno

sis

(S)

DLO

CT im

ages

––

–Ac

cura

cy: 9

3.1%

Gira

rd e

t al74

ARVO

Gen

eral

Mee

ting

2566

N a

nd 1

35 G

(tra

inin

g an

d te

stin

g sp

lit: 7

0:30

%)

Diag

nosi

s (S

)DL

OCT

Imag

es0.

90–

––

Cont

inue

d

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

7Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

Aut

hor

Jour

nal

popu

lati

on (t

rain

ing/

test

ing)

Ass

essm

ent

Type

of A

Id

ata

type

AuC

sens

itiv

ity

spec

ifici

tyCo

mm

ents

Mae

tsch

ke e

t al75

PloS

One

Trai

ning

: 216

N a

nd 6

72

G s

cans

Test

ing:

17

N a

nd 9

3 G

sc

ans

Diag

nosi

s (S

)DL

OCT

imag

es0.

94–

––

Zhan

g et

al77

ARVO

Gen

eral

Mee

ting

55 9

02 N

and

39

096

G im

ages

(tra

inin

g,

valid

atio

n an

d te

stin

g sp

lit:

70:1

5:15

%)

Diag

nosi

s (S

)DL

OCT

imag

es0.

90–

––

Xu78

IEEE

EM

BC20

48 G

imag

es (t

rain

ing

and

test

ing:

LO

O)

Diag

nosi

s (S

)DL

AS-O

CT im

ages

0.92

1–

85%

Fu e

t al79

Am J

Oph

thal

mol

2113

G (fi

vefo

ld c

ross

-va

lidat

ion)

Diag

nosi

s (S

)DL

AS-O

CT im

ages

0.96

090

%92

%–

Fu e

t al80

IEEE

Tran

s Cy

bern

Data

set

1: 2

113

G p

atie

nts

Data

set

2: 2

02 G

pat

ient

sDi

agno

sis

(S)

DLAS

-OCT

imag

esDa

ta s

et 1

: 0.9

62Da

ta s

et 2

: 0.9

52Da

ta s

et 1

: 93%

Data

set

2: 8

7.4%

Data

set

1: 9

0%Da

ta s

et 2

: 95%

bala

nced

acc

urac

yDa

ta s

et 1

: 91.

8%Da

ta s

et 2

: 91.

24%

F-m

easu

reDa

ta s

et 1

: 67.

7%Da

ta s

et 2

: 81.

8%

Niw

as e

t al81

Com

put M

etho

ds

Prog

ram

s Bi

omed

/74

G p

atie

nts

(trai

ning

and

te

stin

g: L

OO

and

10-

fold

cr

oss-

valid

atio

n)

Diag

nosi

s (S

)M

LCAS

-OCT

imag

es0.

95 (L

OO

met

hod)

88.9

0% (L

OO

met

hod)

93.3

3% (L

OO

met

hod)

Acc

urac

yLO

O m

etho

d: 8

9.20

%10

-fold

cro

ss-v

alid

atio

n:

85.1

2%

The

abbr

evia

tions

and

acr

onym

s us

ed in

the

tabl

e ca

n be

foun

d in

the

onlin

e su

pple

men

tary

app

endi

x.*S

tudy

con

side

red

glau

com

a su

spec

t/ear

ly-s

tage

gla

ucom

a su

bjec

ts.

AI, a

rtifi

cial

inte

llige

nce;

AS-

OCT

, ant

erio

r seg

men

t opt

ical

coh

eren

ce to

mog

raph

y; A

UC,

are

a un

der t

he c

urve

; CSL

O, c

onfo

cal s

cann

ing

lase

r oph

thal

mos

copy

; DA,

dis

crim

inan

t ana

lysi

s; DL

, dee

p le

arni

ng; F

P, fu

ndus

pho

togr

aph;

G, g

lauc

oma;

G

S, g

lauc

oma

susp

ect;

LOO,

leav

e on

e ou

t val

idat

ion;

MLC

, mac

hine

lear

ning

cla

ssifi

er; N

, nor

mal

; OCT

, opt

ical

coh

eren

ce to

mog

raph

y; S

, eva

luat

ion

usin

g st

ruct

ural

info

rmat

ion;

SLP

, sca

nnin

g la

ser p

olar

imet

ry; S

VM, s

uppo

rt v

ecto

r mac

hine

; UL,

un

supe

rvis

ed le

arni

ng; V

F, vi

sual

fiel

d.

Tabl

e 2

Cont

inue

d

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

8 Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

Table 3 Summary of the AI studies for the diagnosis of glaucoma using the hybrid (structural and functional) evaluations and genetic data

Author Journalpopulation (training/testing) Assessment

Type of AI data type AuC sensitivity specificity Comments

Kim et al67 PLoS One 202 N and 292 G patients (training and testing split: 80:20%)

Diagnosis (S and F)

MLC OCT parameters and VFs

0.979 98.3% 97.5% Accuracy: 98%

Brigatti et al82 Am J Ophthalmol 54 N and 185 G patients (training and testing split: 66.66:33.33%)

Diagnosis (S and F)

ANN Disc parameters and VFs

– 90% 84% Accuracy: 88%

Bowd et al83 Invest Ophthalmol Vis Sci

69 N and 156 G patients (10-fold cross-validation)

Diagnosis (S and F)

MLC OCT parameters and VFs

0.869 68% 90% –

Racette et al84 J Glaucoma 68 N and 144 G patients (10-fold cross-validation)

Diagnosis (S and F)

MLC CSLO and VFs 0.93 77.8% 90% –

Mardin et al85 J Glaucoma 88 N and 88 G patients (10-fold cross-validation)

Diagnosis (S and F)

MLC CSLO and VFs 0.977 95% 91% –

Bowd et al*86 Invest Ophthalmol Vis Sci/

264 GS eyes (47 progressed, 217 stable; 10-fold cross-validation)

Prognosis (S and F)

MLC CSLO and VFs 0.805 59.6% 85% –

Grewal et al*87 Eur J Ophthalmol Training: 30 N, 22 GS and 28 G eyesTesting: 5 N, 8 GS and 7 G eyes

Diagnosis (S and F)

ANN Clinical parameters, OCT parameters, SLP

0.773 71.4% 60% Classification rate: 65%

Oh et al*88 Invest Ophthalmol Vis Sci

Training: 257 patients (GS)Testing: 129 patients (GS)

Diagnosis (ophthalmic and systemic factors)

ANN Clinical and demographic

0.89 90% 84% Accuracy: 88%

Khawaja et al89 Nat Genet 2606 N and 2606 G (training and testing split: 80:20%)

Diagnosis (genetic)

Reg Genetic data 0.76 – – –

Burdon et al90 Am J Ophthalmol 1919 N and 67 G patients (training, validation, testing split: 70:15:15%)

Diagnosis (genetic)

ANN and Reg

Genetic data – – – Genetic variations related to open-angle glaucoma might be useful to detect glaucoma very early.

The abbreviations and acronyms used in the table can be found in the online supplementary appendix.*Study considered glaucoma suspect/early-stage glaucoma subjects.AI, artificial intelligence; ANN, artificial neural network; AUC, area under the curve; CSLO, confocal scanning laser ophthalmoscopy; F, evaluation using functional information; G, glaucoma; GS, glaucoma suspect; MLC, machine learning classifier; N, normal; OCT, optical coherence tomography; Reg, regression; S, evaluation using structural information; SLP, scanning laser polarimetry; VF, visual field.

detecting abnormalities and offering relevant referrals. Thus, AI systems can help create a clinically conducive environment that better uses specialised resources, reduce workload for clinicians, minimise diagnostic errors leading to incorrect treatment and improve the overall quality of ophthalmic care for patients with glaucoma.

With its ability to extract meaningful features from complex modalities, modern AI methods can help in the discovery of new biomarkers to improve our current understanding of glaucoma. This could be useful for the early detection of glaucoma and to promote research and development into new drugs and treat-ment. Besides, AI could also help us to identify new structural/functional signatures or traits that are extremely difficult to parameterise, perhaps leading to an enhancement of the accepted definition of glaucoma. Thus, a synergy between AI systems and clinicians could lead to mutual advancements in both glaucoma research and clinical practices.

The screening of large populations is logistically and econom-ically unfeasible.91 However, new telemedicine-based screening has offered the benefits of early detection, reduced travel times, increased specialist referral rates, thus saving costs for both the individual and the healthcare system.92 Incorporating AI systems in tandem with the ocular imaging modalities used in

telemedicine might be a long-term and cost-effective solution to increase screening efficacy, and to monitor patients in primary care and community settings where resources and access to specialists are limited.

The lack of a widely accepted clinical reference to predict the progression of glaucoma increases the likelihood of incorrect treatment and management strategies.9 Since structural changes to the eye almost always precede functional loss in glaucoma,44 AI tools for the segmentation21 93 and enhancement (figure 2C)25 of ONH tissues could help clinicians to better visualise and model the 3D structural information in OCT images (figure 2A) specifically for each patient, thus improving the reliability of the prognosis offered. Eventually this could open doors for a range of precision medicine tools for the clinical forecasting, person-alised pharmacological/surgical recommendations and moni-toring glaucoma therapeutic efficacy.

Lastly, by exploiting the data regarding the surgical outcomes of several patients, AI could also help clinicians to evaluate surgical options, preoperative and postoperative management strategies, thus improving surgical outcomes.

There exist several challenges in the clinical translation of AI tools that warrants further discussion. First, the performance of these tools primarily depends on the quality of training data

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

9Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

(quality of images and diagnosis, presence of artefacts). Further-more, a large (>100 000 images) and diverse training set with a good mix of ophthalmic (severity of glaucoma, presence of other conditions) and non-ophthalmic factors (age, ethnicity, imaging device) is generally recommended to ensure clinical robustness.12 Curating such a training set in practice is expensive and daunting.

Second, while all the above studies discussing the use of AI to assess glaucoma seem encouraging, it must be noted that the AUCs are extremely subjective and cannot be used to directly compare different studies due to the following reasons. First, the prevalence of glaucoma, its type and severity94 varies among different regions,95 ethnicities,96 age and gender.97 Second, even among people with glaucoma, due to its asymptomatic nature, only half of them are aware of their disease in the developed countries, and even lower in low and middle-income countries.95 This may change what constitute normal and glaucoma popula-tions across countries. Third, while the cohorts in retrospective clinic-based studies comprise patients who are diagnosed with glaucoma using well-defined criteria, the cohorts in prospective population-based studies can have patients with varying degrees of disease severity and prevalence depending on the region95 and demographics (ethnicity, gender, age).97 Thus, although the AUC is a single measure to summarise the overall diagnostic performance of a study, it must be interpreted with the context of the nature of study (clinic based/population based), popula-tion size and spectrum (type and severity) and demographics. Ideally, to truly assess diagnostic power, studies must be highly standardised and must use balanced data sets across multiple factors, including, but not limited to age, ethnicity, gender, type of glaucoma and severity. However, given the practical limita-tions in curating such data sets, the clinical acceptance for these algorithms thus lies in the repeated and reliable independent validations across multiple data sets. Extra caution must also be exerted when dealing with very high AUCs (>0.98) due to the discrepancies and sheer subjectivity in the diagnosis even among the experts.98 In addition, the diagnostic power calculated with AI on a given population should ideally be compared with that obtained with a gold standard parameter such as RNFL thick-ness on the exact same population. If performance is identical and high (AUC >0.90 for both cases), one should ultimately ask ‘Why is the AI algorithm’s diagnostic power not superior to that of a simple parameter?’ and ‘Why is the diagnostic power for RNFL thickness high for this given population?’ It is worth noting that it is relatively easy to get a very high AUC on a small population (~100 patients), but less so with data from tens of thousands or millions of patients. Finally, sensitivity (at 95% specificity) should also be reported to better understand performance.

Third, while longitudinal predictions from structural analysis of OCT images might help in the prognosis and even early diag-nosis of glaucoma, the development of such an AI system is a clinical challenge. This is because the recruitment and follow-up of a large cohort is very expensive and cumbersome. Besides, the subjective definitions for suspect/moderate phenotypes could compromise the integrity of training data, eventually affecting the performance of such tools.

Fourth, the ‘black-box’ nature of AI algorithms has long offered resistance to its clinical adoption. AI algorithms discriminate based on the correlations/patterns they infer from the training data. These correlations may or may not be in concurrence with the theoretical cause. When exposed to high-dimension-ality data (eg, a combination of ophthalmic and non-ophthalmic parameters), the algorithm might pick up intrinsic patterns in the data that might correlate to glaucoma, but might not be clinically correct. For instance, given the strong correlation in

the prevalence of glaucoma with demographics (ethnicity, age, gender),97 the AI algorithm might learn to identify pathology merely based on the demographics and neglecting the ophthalmic parameters. The chances of identifying such clinically irrelevant correlations increase when dealing with high-dimensionality data (eg, OCT images), if the algorithm is not designed and vali-dated carefully, thus leading to excess false positives and false negatives. Although a few studies16 75 have attempted to offer ‘clinical explainability’ of the results by visualising heat maps (or class activation maps) to understand the influence of different regions in the image towards the final diagnosis, these methods are still relatively brittle and under active development: the use of such visualisation tools is still in its infancy. The development of such tools is crucial for the widespread clinical adoption of AI algorithms.

Fifth, the ‘black-box’ nature also poses a host of conflicting medicolegal liability issues between clinical practice and industry.99 Given the performance subjectivity of these tools due to the variability in image quality and device, a clinical verifica-tion of the results is required before the final decision is made. However, clinical decisions based directly on the interpretation of AI systems lead to a higher liability on the manufacturer, thus affecting retail price.99 Clear medicolegal guidelines and a diag-nostic pipeline involving both the AI systems and clinicians to minimise errors are essential for an economically feasible adop-tion and increased acceptability of these tools.

Finally, the approval of regulatory bodies such as the Food and Drug Administration and the European Medicines Agency required for the clinical use of these diagnostic tools is likely to be difficult.99 This is a result of the fact that the required perfor-mance for clinical acceptance is not yet clearly defined. Thus, multiple clinical studies comparing AI and traditional diagnostic practices are needed to better understand these systems, and increase the patient/clinician confidence in them.

In conclusion, clinicians in future may expect a plethora of AI tools to assist them in the day-to-day diagnosis and management of glaucoma. While the persistence of new clinical and technical challenges is undeniable, one cannot dismiss the ways in which AI could positively impact ophthalmic research and clinical prac-tice in glaucoma.

Contributors SKD was the first author and drafted the manuscript. ZL coauthored and provided review on the usage of AI for functional analysis. THP coauthored and provided review on the usage of AI for structural analysis. CB, NGS, AHT and MJAG critically reviewed the manuscript.

Funding This work was supported by the Singapore Ministry of Education Academic Research Funds Tier 1 (R-397-000-294-114 (MJAG)); and the Singapore Ministry of Education Tier 2 (R-397-000-280-112, R-397-000-308-112 (MJAG)).

Competing interests MJAG and AHT are co-founders of Abyss Processing.

patient consent for publication Not required.

provenance and peer review Commissioned; externally peer reviewed.

orCId idsNicholas G Strouthidis http:// orcid. org/ 0000- 0003- 0705- 7678Michael J A Girard http:// orcid. org/ 0000- 0003- 4408- 5918

RefeRences 1 Resnikoff S, Pascolini D, Etya’ale D, et al. Global data on visual impairment in the year

2002. Bull World Health Organ 2004;82:844–51. 2 Quigley HA, Broman AT. The number of people with glaucoma worldwide in 2010 and

2020. Br J Ophthalmol 2006;90:262–7. 3 Osborne NN, Wood JP, Chidlow G, et al. Ganglion cell death in glaucoma: what do we

really know? Br J Ophthalmol 1999;83:980–6. 4 Tatham AJ, Weinreb RN, Medeiros FA. Strategies for improving early detection of

glaucoma: the combined structure-function index. Clin Ophthalmol 2014;8:611–21.

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

10 Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

5 Quigley HA, West SK, Rodriguez J, et al. The prevalence of glaucoma in a population-based study of Hispanic subjects: Proyecto VER. Arch Ophthalmol 2001;119:1819–26.

6 Weinreb RN, Aung T, Medeiros FA. The pathophysiology and treatment of glaucoma: a review. JAMA 2014;311:1901–11.

7 Schuman JS. Detection and diagnosis of glaucoma: ocular imaging. Invest Ophthalmol Vis Sci 2012;53:2488–90.

8 Stroux A, Martus P, Budde W, et al. Sequential classification in glaucoma diagnosis. Graefes Arch Clin Exp Ophthalmol 2003;241:277–83.

9 Weinreb DFG-H RN, Leung C, Medeiros FA, et al. 10Th consensus meeting: diagnosis of primary open angle glaucoma. Kugler Publications, 2016.

10 Butt NH, Ayub MH, Ali MH. Challenges in the management of glaucoma in developing countries. Taiwan J Ophthalmol 2016;6:119–22.

11 Leske MC, Heijl A, Hussein M, et al. Factors for glaucoma progression and the effect of treatment: the early manifest glaucoma trial. Arch Ophthalmol 2003;121:48–56.

12 Ting DSW, Pasquale LR, Peng L, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol 2019;103:167–75.

13 Golland P, Hata N, Barillot C, et al. Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis. In: Medical image computing and computer-assisted intervention – MICCAI 2014. Springer International Publishing, 2014.

14 Johnson KW, Torres Soto J, Glicksberg BS, et al. Artificial intelligence in cardiology. J Am Coll Cardiol 2018;71:2668–79.

15 Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus Photographs. JAMA 2016;316:2402–10.

16 De Fauw J, Ledsam JR, Romera-Paredes B, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 2018;24:1342–50.

17 Lee CS, Tyring AJ, Deruyter NP, et al. Deep-learning based, automated segmentation of macular edema in optical coherence tomography. Biomed Opt Express 2017;8:3440–8.

18 Burlina PM, Joshi N, Pekala M, et al. Automated grading of age-related macular degeneration from color fundus images using deep Convolutional neural networks. JAMA Ophthalmol 2017;135:1170–6.

19 Maeda N, Klyce SD, Smolek MK, et al. Automated keratoconus screening with corneal topography analysis. Invest Ophthalmol Vis Sci 1994;35:2749–57.

20 Maeda N, Klyce SD, Smolek MK. Neural network classification of corneal topography. Preliminary demonstration. Invest Ophthalmol Vis Sci 1995;36:1327–35.

21 Devalla SK, Renukanand PK, Sreedhar BK, et al. DRUNET: a dilated-residual U-Net deep learning network to segment optic nerve head tissues in optical coherence tomography images. Biomed Opt Express 2018;9:3244–65.

22 Fang L, Cunefare D, Wang C, et al. Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. Biomed Opt Express 2017;8:2732–44.

23 Roy AG, Conjeti S, Karri SPK, et al. ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomed Opt Express 2017;8:3627-3642.

24 Halupka KJ, Antony BJ, Lee MH, et al. Retinal optical coherence tomography image enhancement via deep learning. Biomed Opt Express 2018;9:6205–21.

25 Devalla SK, Pham TH, Wang X, et al. A deep learning approach to Denoise optical coherence tomography images of the optic nerve head 2018.

26 Retinal vessel segmentation via deep learning network and fully-connected conditional random fields2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI)20161316

27 Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60–88.

28 Goldbaum MH, Sample PA, White H, et al. Interpretation of automated perimetry for glaucoma by neural network. Invest Ophthalmol Vis Sci 1994;35:3362–73.

29 Goldbaum MH, Sample PA, White H, et al. Discrimination of normal and glaucomatous visual fields by neural network. Investigative ophthalmology & visual science 1990;31.

30 Goldbaum MH, Sample PA, Chan K, et al. Comparing machine learning classifiers for diagnosing glaucoma from standard automated perimetry. Invest Ophthalmol Vis Sci 2002;43:162–9.

31 Chan K, Lee T-W, Sample PA, et al. Comparison of machine learning and traditional classifiers in glaucoma diagnosis. IEEE Trans Biomed Eng 2002;49:963–74.

32 Bizios D, Heijl A, Bengtsson B. Trained artificial neural network for glaucoma diagnosis using visual field data: a comparison with conventional algorithms. J Glaucoma 2007;16:20–8.

33 Lietman T, Eng J, Katz J, et al. Neural networks for visual field analysis: how do they compare with other algorithms? J Glaucoma 1999;8:77–80.

34 Li F, Wang Z, Qu G, et al. Automatic differentiation of glaucoma visual field from non-glaucoma visual filed using deep convolutional neural network. BMC Med Imaging 2018;18:35.

35 Kucur Şerife Seda, Holló G, Sznitman R. A deep learning approach to automatic detection of early glaucoma from visual fields. PLoS One 2018;13:e0206081–e81.

36 De Moraes CG, Liebmann JM, Levin LA. Detection and measurement of clinically meaningful visual field progression in clinical trials for glaucoma. Prog Retin Eye Res 2017;56:107–47.

37 Brigatti L, Nouri-Mahdavi K, Weitzman M, et al. Automatic detection of glaucomatous visual field progression with neural networks. Arch Ophthalmol 1997;115:725–8.

38 Sample PA, Goldbaum MH, Chan K, et al. Using machine learning classifiers to identify glaucomatous change earlier in standard visual fields. Invest Ophthalmol Vis Sci 2002;43:2660–5.

39 Sample PA, Boden C, Zhang Z, et al. Unsupervised machine learning with independent component analysis to identify areas of progression in glaucomatous visual fields. Invest Ophthalmol Vis Sci 2005;46:3684–92.

40 Yousefi S, Kiwaki T, Zheng Y, et al. Detection of longitudinal visual field progression in glaucoma using machine learning. Am J Ophthalmol 2018;193:71–9.

41 Wen JC, Lee CS, Keane PA, et al. Forecasting future Humphrey visual fields using deep learning. PLoS One 2019;14:e0214875.

42 Sharma P, Sample PA, Zangwill LM, et al. Diagnostic tools for glaucoma detection and management. Surv Ophthalmol 2008;53 Suppl1:S17–32.

43 Zheng C, Johnson TV, Garg A, et al. Artificial intelligence in glaucoma. Curr Opin Ophthalmol 2019;30:97–103.

44 Gordon MO, Beiser JA, Brandt JD, et al. The ocular hypertension treatment study: baseline factors that predict the onset of primary open-angle glaucoma. Arch Ophthalmol 2002;120:714–20.

45 Bowd C, Chan K, Zangwill LM, et al. Comparing neural networks and linear discriminant functions for glaucoma detection using confocal scanning laser ophthalmoscopy of the optic disc. Invest Ophthalmol Vis Sci 2002;43:3444–54.

46 Townsend KA, Wollstein G, Danks D, et al. Heidelberg retina tomograph 3 machine learning classifiers for glaucoma detection. Br J Ophthalmol 2008;92:814–8.

47 Zangwill LM, Chan K, Bowd C, et al. Heidelberg retina tomograph measurements of the optic disc and parapapillary retina for detecting glaucoma analyzed by machine learning classifiers. Invest Ophthalmol Vis Sci 2004;45:3144–51.

48 Uchida H, Brigatti L, Caprioli J. Detection of structural damage from glaucoma with confocal laser image analysis. Invest Ophthalmol Vis Sci 1996;37:2393–401.

49 Adler W, Peters A, Lausen B. Comparison of classifiers applied to confocal scanning laser ophthalmoscopy data. Methods Inf Med 2008;47:38–46.

50 Bowd C, Zangwill LM, Medeiros FA, et al. Confocal scanning laser ophthalmoscopy classifiers and stereophotograph evaluation for prediction of visual field abnormalities in glaucoma-suspect eyes. Invest Ophthalmol Vis Sci 2004;45:2255–62.

51 Weinreb RN, Zangwill L, Berry CC, et al. Detection of glaucoma with scanning laser polarimetry. Arch Ophthalmol 1998;116:1583–9.

52 Bowd C, Medeiros FA, Zhang Z, et al. Relevance vector machine and support vector machine classifier analysis of scanning laser polarimetry retinal nerve fiber layer measurements. Invest Ophthalmol Vis Sci 2005;46:1322–9.

53 Al-Bander B, Williams B, Al-Nuaimy W, et al. Dense fully Convolutional segmentation of the optic disc and cup in colour fundus for glaucoma diagnosis. Symmetry 2018;10:87.

54 Fichtinger G, Martel A, Peters T. Sliding Window and Regression Based Cup Detection in Digital Fundus Images for Glaucoma Diagnosis. In: Medical image computing and computer-assisted intervention – MICCAI 2011. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011.

55 Muramatsu C, Hayashi Y, Sawada A, et al. Detection of retinal nerve fiber layer defects on retinal fundus images for early diagnosis of glaucoma. J Biomed Opt 2010;15:016021.

56 Muramatsu C, Nakagawa T, Sawada A, et al. Automated segmentation of optic disc region on retinal fundus Photographs: comparison of contour modeling and pixel classification methods. Comput Methods Programs Biomed 2011;101:23–32.

57 Acharya UR, Dua S, Du X, et al. Automated diagnosis of glaucoma using texture and higher order spectra features. IEEE Trans Inf Technol Biomed 2011;15:449–55.

58 Sinthanayothin C, Boyce JF, Cook HL, et al. Automated localisation of the optic disc, fovea, and retinal blood vessels from digital colour fundus images. Br J Ophthalmol 1999;83:902–10.

59 Medeiros FA, Jammal AA, Thompson AC. From machine to machine: an OCT-Trained deep learning algorithm for objective quantification of glaucomatous damage in fundus Photographs. Ophthalmology 2019;126:513–21.

60 Li Z, He Y, Keel S, et al. Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus Photographs. Ophthalmology 2018;125:1199–206.

61 Ting DSW, Cheung CY-L, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA 2017;318:2211–23.

62 Loh EH-T, Ong Y-T, Venketasubramanian N, et al. Repeatability and reproducibility of retinal neuronal and axonal measures on spectral-domain optical coherence tomography in patients with cognitive impairment. Front Neurol 2017;8:359–59.

63 Leung CK-S, Cheung CY-L, Weinreb RN, et al. Retinal nerve fiber layer imaging with spectral-domain optical coherence tomography: a variability and diagnostic performance study. Ophthalmology 2009;116:1257–63.

64 Huang M-L, Chen H-Y. Development and comparison of automated classifiers for glaucoma diagnosis using Stratus optical coherence tomography. Invest Ophthalmol Vis Sci 2005;46:4121–9.

65 Burgansky-Eliash Z, Wollstein G, Chu T, et al. Optical coherence tomography machine learning classifiers for glaucoma detection: a preliminary study. Invest Ophthalmol Vis Sci 2005;46:4147–52.

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from

11Devalla SK, et al. Br J Ophthalmol 2019;0:1–11. doi:10.1136/bjophthalmol-2019-315016

review

66 An G, Omodaka K, Tsuda S, et al. Comparison of Machine-Learning classification models for glaucoma management. J Healthc Eng 2018;2018:1–8.

67 Kim SJ, Cho KJ, Oh S. Development of machine learning models for diagnosis of glaucoma. PLoS One 2017;12:e0177726.

68 Barella KA, Costa VP, Gonçalves Vidotti V, et al. Glaucoma diagnostic accuracy of machine learning classifiers using retinal nerve fiber layer and optic nerve data from SD-OCT. J Ophthalmol 2013;2013:1–7.

69 Christopher M, Belghith A, Weinreb RN, et al. Retinal nerve fiber layer features identified by unsupervised machine learning on optical coherence tomography scans predict glaucoma progression. Invest Ophthalmol Vis Sci 2018;59:2748–56.

70 Ye C, Yu M, Leung CK-S. Impact of segmentation errors and retinal blood vessels on retinal nerve fibre layer measurements using spectral-domain optical coherence tomography. Acta Ophthalmol 2016;94:e211–9.

71 Mansberger SL, Menda SA, Fortune BA, et al. Automated segmentation errors when using optical coherence tomography to measure retinal nerve fiber layer thickness in glaucoma. Am J Ophthalmol 2017;174:1–8.

72 Huang X-R, Knighton RW, Zhou Y, et al. Reflectance speckle of retinal nerve fiber layer reveals axonal activity. Invest Ophthalmol Vis Sci 2013;54:2616–23.

73 Muhammad H, Fuchs TJ, De Cuir N, et al. Hybrid deep learning on single wide-field optical coherence tomography scans accurately classifies glaucoma suspects. J Glaucoma 2017;26:1086–94.

74 Girard MJA, Chin KS, Devalla SK. Deep Learning can Exploit 3D Structural Information of the Optic Nerve Head to Provide a Glaucoma Diagnostic Power Superior to that of Retinal Nerve Fibre Layer Thickness. In: Association for research in vision and ophthalmology (ARVO) annual meeting. Hawaii, USA, 2018.

75 Maetschke S, Antony B, Ishikawa H, et al. A feature agnostic approach for glaucoma detection in OCT volumes. PLoS One 2019;14:e0219126.

76 Gandhi M, Dubey S. Evaluation of the optic nerve head in glaucoma. J Curr Glaucoma Pract 2013;7:106–14.

77 Zhang L, Devalla SK, Cheng C-Y. A Multi-device, Multi-ethnicity deep learning algorithm to detect glaucoma from a single optical coherence tomography scan of the optic nerve head association for research in vision and ophthalmology (ARVO) annual meeting. Vancouver, Canada, 2019.

78 Automated anterior chamber angle localization and glaucoma type classification in OCT images. In2013 35th annual International Conference of the IEEE engineering in medicine and biology Society (EmbC)2013

79 Fu H, Baskaran M, Xu Y, et al. A deep learning system for automated angle-closure detection in anterior segment optical coherence tomography images. Am J Ophthalmol 2019;203:37–45.

80 Fu H, Xu Y, Lin S, et al. Angle-Closure detection in anterior segment OCT based on multilevel deep network. IEEE Trans Cybern 2019:1–9.

81 Niwas SI, Lin W, Bai X, et al. Automated anterior segment OCT image analysis for angle closure glaucoma mechanisms classification. Comput Methods Programs Biomed 2016;130:65–75.

82 Brigatti L, Hoffman D, Caprioli J. Neural networks to identify glaucoma with structural and functional measurements. Am J Ophthalmol 1996;121:511–21.

83 Bowd C, Hao J, Tavares IM, et al. Bayesian machine learning classifiers for combining structural and functional measurements to classify healthy and glaucomatous eyes. Invest Ophthalmol Vis Sci 2008;49:945–53.

84 Racette L, Chiou CY, Hao J, et al. Combining functional and structural tests improves the diagnostic accuracy of relevance vector machine classifiers. J Glaucoma 2010;19:167–75.

85 Mardin CY, Peters A, Horn F, et al. Improving glaucoma diagnosis by the combination of perimetry and HRT measurements. J Glaucoma 2006;15:299–305.

86 Bowd C, Lee I, Goldbaum MH, et al. Predicting glaucomatous progression in glaucoma suspect eyes using relevance vector machine classifiers for combined structural and functional measurements. Invest Ophthalmol Vis Sci 2012;53:2382–9.

87 Grewal DS, Jain R, Grewal SPS, et al. Artificial neural network-based glaucoma diagnosis using retinal nerve fiber layer analysis. Eur J Ophthalmol 2008;18:915–21.

88 Oh E, Yoo TK, Hong S. Artificial neural network approach for differentiating open-angle glaucoma from glaucoma suspect without a visual field test. Invest Ophthalmol Vis Sci 2015;56:3957–66.

89 Khawaja AP, Cooke Bailey JN, Wareham NJ, et al. Genome-Wide analyses identify 68 new loci associated with intraocular pressure and improve risk prediction for primary open-angle glaucoma. Nat Genet 2018;50:778–82.

90 Burdon KP, Mitchell P, Lee A, et al. Association of open-angle glaucoma loci with incident glaucoma in the blue Mountains eye study. Am J Ophthalmol 2015;159:31–6.

91 Bettin P, Di Matteo F. Glaucoma: present challenges and future trends. Ophthalmic Res 2013;50:197–208.

92 Thomas S-M, Jeyaraman MM, Jeyaraman M, Hodge WG, et al. The effectiveness of teleglaucoma versus in-patient examination for glaucoma screening: a systematic review and meta-analysis. PLoS One 2014;9:e113779.

93 Devalla SK, Chin KS, Mari J-M, et al. A deep learning approach to digitally stain optical coherence tomography images of the optic nerve head. Invest. Ophthalmol. Vis. Sci. 2018;59:63–74.

94 Heijl A, Bengtsson B, Oskarsdottir SE. Prevalence and severity of undetected manifest glaucoma: results from the early manifest glaucoma trial screening. Ophthalmology 2013;120:1541–5.

95 Quigley HA. Number of people with glaucoma worldwide. Br J Ophthalmol 1996;80:389–93.

96 Kosoko-Lasaki O, Gong G, Haynatzki G, et al. Race, ethnicity and prevalence of primary open-angle glaucoma. J Natl Med Assoc 2006;98:1626–9.

97 Rudnicka AR, Mt-Isa S, Owen CG, et al. Variations in primary open-angle glaucoma prevalence by age, gender, and race: a Bayesian meta-analysis. Invest Ophthalmol Vis Sci 2006;47:4254–61.

98 Bathija R, Gupta N, Zangwill L, et al. Changing definition of glaucoma. J Glaucoma 1998;7:165???169–9.

99 Schmetterer L. Challenges with deep learning in glaucoma. International Glaucoma Review: Glaucoma Opinion 2018;19:12–15.

52709191. Protected by copyright.

on October 29, 2019 at S

wets S

ubscription Service

http://bjo.bmj.com

/B

r J Ophthalm

ol: first published as 10.1136/bjophthalmol-2019-315016 on 22 O

ctober 2019. Dow

nloaded from