Jonathan M KaganDivision of Clinical Research

National Institute of Allergy & Infectious DiseasesBethesda, Maryland, USA

William MK TrochimDepartment of Policy Analysis & Management

Cornell UniversityIthaca, New York, USA

Integrating Evaluation with Business Process Modeling for Increased Efficiency and Faster Results in HIV/AIDS

Clinical Trials Research

The HIV/AIDS Problem

Total: 33 million (30 – 36 million)

Western & Central Europe

730 000730 000[580 000 – 1.0 million][580 000 – 1.0 million]

Middle East & North Africa380 000380 000

[280 000 – 510 000][280 000 – 510 000]

Sub-Saharan Africa22.0 million22.0 million[20.5 – 23.6 million][20.5 – 23.6 million]

Eastern Europe & Central Asia1.5 million 1.5 million [1.1 – 1.9 million][1.1 – 1.9 million]

South & South-East Asia4.2 million4.2 million[3.5 – 5.3 million][3.5 – 5.3 million]

Oceania74 00074 000

[66 000 – 93 000][66 000 – 93 000]

North America1.2 million

[760 000 – 2.0 million]

Latin America1.7 million1.7 million[1.5 – 2.1 million][1.5 – 2.1 million]

East Asia740 000740 000

[480 000 – 1.1 million][480 000 – 1.1 million]Caribbean230 000

[210 000 – 270 000]

Adults and children estimated to be living with HIV, 2007

Total: 33 million (30 – 36 million)

5

Evaluation Project Goals

• Support the success of the clinical trials programs• Provide empirically based evidence about process

and outcomes to guide decision making and program improvement

• Ensure the highest scientific priorities are addressed• Promote collaboration and shared learning• Increase efficiency and research integration• Develop a culture of ongoing evaluation

Stakeholder-Identified Critical Success Factor Concept Map

DAIDS Policies and Procedures Operations andManagement

Resource Utilization

Community Involvement

Scientific Agenda - setting

Biomedical objectivesRelevance to Participants

12

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77 78

79

80

81

82

83

84

8586

87

88

89

90

91

Operations, Policies and Resources

• Focus– Administrative polices, funder polices and procedures, process efficiency

and site capacity

• Lead Evaluation Question – How can the process of protocol development be improved to increase

efficiency and shorten the timeline?• How long does each phase of the protocol development process take?• What are the limiting factors in the protocol development process?• How can we shorten the timeline without compromising quality? • How can the process be improved?• What are reasonable targets for each phase?

Enterprise FoundationPerson and Institution Registry

ProtocolRegistration

ClinicalSite

Monitoring

DAIDS Adverse

Experience Reporting

System

INDManagement

ProtocolManagement

Desktops Wireless Web Services

DAIDS Enterprise Information System

InIn DevelopmentDevelopment

PendingPending Open to Open to AccrualAccrual EnrollingEnrolling Closed to Closed to

AccrualAccrualClosed toClosed toFollow UpFollow Up

WithdrawnWithdrawn

Participants Off StudyParticipants Off Study& Primary Analysis & Primary Analysis CompletedCompleted

ConcludedConcluded

ArchivedArchived

ProposedProposed

DAIDS Harmonized Protocol Statuses

SRC Review Total Elapsed (Days1) Maximum1 60

Minimum1 1

Median1 27

Target2 35

Difference (Median-Target) 2

Std. Deviation 10.41

# of Reviews 1061 Calendar days2 Business days

Note: The numbers shown above the bar represents the total number of days for SRC Review Process (A+B)A= Days from Protocol Receipt to SRC Review B= Days from SRC Review to Consensus Distribution

30

Days

60 90 120 180 210 240 270 300 330 360 390 420 450 480 510 540 570 600 660 720630 690150 750

133 days

100 daysReceipt to CSRC Review (Multiple)

SRC Review Completion to RAB Sign Off

Pending to Open to Accrual

Receipt to Comments Distribution

(single)

125 days

27 days

15 days

23 days

160 days

Pending to v1.0 Site Registration (US Sites)

517 days

Pending to v1.0 Site Registration (Non-US Sites)

Open to Accrual to Enrolling

Protocol Timeline Summary

Receipt to Review

(single) 233 days

358 days

381 days

780

DAIDS Regulatory Review Process

What Good Are These Time-Based Measures?

• Time measures provide a coarse-grain look at processes – kind of a composite ‘smell test’Sometimes it’s the process itselfOther times it’s the resources/capacity that’s rate-limiting

• Time measures can help develop focus on where improvements could be sought

• Excessive time is a disincentive to clinical research• We should evaluate quality/value but we may lack the

ability or know-how – time studies can get us started

What Have We Learned and How Can We Put it to Use?

Things that appear worth considering from these time-based protocol analyses:

•Shorten/simplify processes that consume time disproportionate to their ‘value’•Develop means of assuring performance accountability•Set a ‘drop dead’ date for protocol initiation

Acknowledgements

• Evaluation planning and concept mapping– Mary Kane, Concept Systems, Inc.– Kathleen Quinlan, Concept Systems, Inc.– Scott Rosas, Concept Systems, Inc.

• Protocol event analyses– Suresh Varghese and Alex Varghese, Digital Infuzion, Inc.

• NIAID HIV/AIDS Networks– Jeffrey Schouten, Network Coordinating Center, FHCRC– Network Leadership, Operations Centers, Data Centers