Clinical Research in the Emergency Department

Jim Quinn MD MSAssociate Professor of Surgery/Emergency Medicine

Research Director Emergency Medicine

Overview• Goals for research in academic emergency

medicine

• Problems/solutions for researchers in emergency medicine

• Problems/solutions for research implementation

• The new clinical research unit in the ED at Stanford

Academic Emergency Medicine

• Outstanding residency and clinical operations• Research lags behind educational and clinical

performance• Excellence in education, clinical operation

and research will lead to departmental status at the university

• Departmental status will lead to more academic and fiscal freedom

Improving ResearchHow Do We Get There

• Obstacles

- Too busy

- No training in research

- Too few mentors/role models

- No interest

Improving ResearchHow Do We Get There

• Solutions

1) Recruit researchers

2) Develop researchers

- Personal and academic investment

- Expose EM residents to advantages of academic career

- Funding through grants and career development awards

Research During Residency

• Train residents to appreciate research efforts, critically evaluate a study

- Study design, methodology, statistics• Exposure to opportunities for an academic

career• Some will decide to do research fellowships

and pursue academic medicine

Research During Residency

• Start early (1st year)• Develop own idea• Develop that idea with a faculty mentor- avoid doing research for them- find mentors with common interest - Research curriculum and support to facilitate

project development• Research Director- support and direction

Research CurriculumStructure and Support to Develop Your Idea

• Curriculum (tried and tested)- 12 hours – 6 sessions with core reading and homework

designed to develop your project

• Textbook

- “Designing Clinical Research” - Hulley and Cummings

- small paperback readable

• Help identify mentors and sources of data

The Five Page ProtocolGoal for the Research Curriculum

• Concise protocol- More concise than an NIH submission, but

often sufficient for small intramural grants- Discipline approach to planning the study- Provide the materials and answers for IRB

submission- Completed by the end of first year- Implement in years 2 and 3

Organization The Five Page Protocol

• Page One- Title, Specific objectives, significance• Pages 2-5- Overview of design (RCT, observational cohort/ cross

sectional, case/control)- Study subjects: selection criteria, exclusions, accessible

populations, plans for sampling and recruitment- Measurement – predictor and outcome variables- Statistical issues – sample size, proposed analysis- Quality control and data management- Timetable- Ethical considerations

Research Development

• Residents- EMF: Resident Research Grants - $5,000- EMF, SAEM: Research Fellowships -

$75,000- T and F awards from NIHFaculty- Career development awardsSAEM, EMF, K awards from NIH

What is a Career Development Award ?

Funding to protect your time so that you can develop your research skills

Research may be: Clinical Basic Science

Senior Fellowships (F33)

IndependentInvestigatorInternship/Residency SpecialtyMedical

School

Postdoctoral Fellowships (F32)

NIH Awards for Individuals with a Health-Professional Doctorate

InstitutionalTraining Grants (T32)

Midcareer Investigatorin Patient-Oriented

Research (K24)

Mentored Patient-Oriented Research CDA (K23)

Scientist Development Program (K12)

Mentored Clinical Scientist Development Award (K08)

Short-TermTraining Grant(T35)

Career Enhancement Award Stem Cells (K18)

Benefits of a Career Development Award ?

• Protected time

• Extra Training

• Step towards independence

• New relationships

Myth: NIH Grants/Study Sections

• Emergency medicine proposals, especially clinical research, will not be evaluated fairly, nor will they be funded consistently, until the NIH has a study section devoted to emergency medicine.

• NIH grants are rare and hard to get

What is Important to Study Section Members?

• Study section members don’t care what department the investigator is in.

• Study section members care about:

– The match between the proposed work and the goals of the program

– The quality of the proposal

– Investigator’s track record and preliminary data

– The institutional research environment

Selected NIH Panel Recommendations

• “The NIH must ensure fair and effective reviews of extramural grant applications for support of clinical research: panels that review clinical research– (a) must include experienced clinical

investigators and– (b) at least 30-50% of the applications

reviewed by these panels must be for clinical research.”

• “The NIH should improve the quality of training for clinical researchers by requiring grantee organizations to provide formal training experiences in clinical research and careful mentoring by experienced clinical investigators.”

Selected NIH Panel Recommendations

Does NIH Fund EM Research?

• A search of currently funded federal grants using the CRISP database and key words “Emergency” yielded 204 new grants in the years 2000-2002.

• Accurate numbers of grants submitted by specialty are difficult to find and interpret. No separate statistics are maintained for Emergency Medicine.

Research Options

• Basic Science

• Translational Research

• Clinical ResearchClinical Research

- Large database

- Retrospective reviews

- Clinical trials/Prospective cohorts

Myth: Large Databases

• Large administrative databases contain large amounts of clinically useful information.

Large Databases

• In general, large databases are collected:

– For non-research purposes (e.g., claims and billing databases)

– With no specific research question in mind (e.g., trauma center databases)

Large Databases

• Large databases often lack the specific outcome and risk stratification variables needed for a particular study, requiring assumptions and approximations to be made.

• Large databases often have a substantial proportion of missing or incorrect data which may reflect recording bias or other sources of bias.

Large Databases

• Even small biases, together with a large sample size, may yield results with impressively small p values that are, nonetheless, artifacts.

• Without independent methods for checking the accuracy and completeness of the data, these biases may be difficult to detect.

Clinical Trials

• Prospective trials

• Designed to answer specific questions

• More likely to answer the question correctly

Problems Implementing Clinical Research

• Where did all the patients go?

“ The best way to eliminate disease

is to study it”

• Nobody cares

• IRB/HIPAA issues

• Department too busy, too many protocols

“Tragedy of the Commons”

• A “Metaphor” to describe the sub-optimal use of a collectively shared resource

“best strategies for individuals conflict with the common good”

Clinical Research Unit

Goal – “conduct efficient and effect research in the chaotic environment of the ED for the common good”

- Comprehensive database of all ED patients- Real time data infrastructure- Real time notification and enrollment- Research director, research coordinator, volunteers- Research committee to oversee all projects to

ensure adequate resources

Clinical Research UnitReal Time Data Infrastructure

- HIPPA complaint ED Research database

- Hosted by SOM: secure, redundancy

- Allows for instant notification directly from database

- Web based enrollment

- Eventually paperless

- https://emerg-med.stanford.edu/

Clinical Research UnitResearch Coordinator and Volunteers

• Volunteers - Undergrads and med students- Help screen and enroll patients- Deal with paper flow• Coordinator - oversees volunteers: schedules- patient follow-up- resource for data and chart acquisition for ED

studies.

Clinical Research UnitResearch Committee

• Meets monthly – 30-60 minute meetings after faculty meeting 2nd Wednesday

• Open meetings

• Oversees and approves all protocols in ED

• Consists of research director, resident representation, at least 2 volunteer faculty members

Clinical Research UnitFunding and Resources

• Coordinator – 50% time primarily from grant funding

• New funding and studies could increase to 100%

• Non-EM researchers/industry will have to pay to use our data infrastructure/research unit

Clinical Research UnitRegistering Protocols

• Send e-mail with protocol to Dr.Quinn• Protocol will be reviewed at research committee for:- IRB approval- Funding Source- Resource Utilization- Benefit to EM- All external protocol will need to have an EM

faculty as an investigator/supporter on the protocol

Next Step

• Identify current projects utilizing ED patients/resources

• Hire coordinator – Completed Dec/Jan

• Volunteer recruitment - Ongoing

• First committee meeting in December

Clinical Research UnitThe First Studies

• Dog Bite Study

- Requires prospective enrollment of patients and consent

• NET-2

- Surveillance study to be part of large NINDS study, no consent

Are Prophylactic Antibiotics Beneficial in Dog Bites?

• Controversial

1) Meta analysis- Ann Emerg Med – 1994

- Recommend treating

2) Cochrane Review 2004

- Recommend Not Treating

3) Current recommendation is to treat high risk wounds

Are Prophylactic Antibiotics Beneficial in Dog Bites?

• Is it worth doing the study?

- Over 1,000 patients needed in a multi-center trial at great cost to determine a 5% difference (less power on sub group analysis)

- Is 5% an important difference?

Are Prophylactic Antibiotics Beneficial in Dog Bites?Value of Cost- Benefit Models

The models done ahead of a trial can;

1) Clearly define important outcomes to measure

2) Help determine MCID for sample size

3) Sometimes provide the answer

Dog Bite: Cost – Benefit ModelSE

longseWTP1 - COST1

NoSE#

WTP2 - COST2

Uncomplicatedinptunc

SElongse

WTP3 - COST3

NoSE#

WTP4 - COST4

Disabilitycompdis

NoSElongnose

WTP5 - COST5

SE#

WTP6 - COST6

Nodis#

Recovercomprec

Death#

WTP7 - COST7

Complicated#

InptRxfail

SElongse

WTP8 - COST8

NoSE#

WTP9 - COST9

Outptrecover#

OutptRXOutptRx

SElongse

WTP10 - COST10

NoSE#

WTP11 - COST11

Uncomplicatedinptunc

SElongse

WTP12 - COST12

NoSE#

WTP13 - C13

Disabilitycompdis

NoSElongnose

WTP14 - C14

SE#

WTP15 - C15

Nodis#

Recovercomprec

Death#

WTP16 - C16

Complicated#

InptRx#

FaceRxinfFaceRxinf

shortSEshortse

WTP17 - C17

noshortSE#

WTP18 - C18

FaceRxnoinf#

Rx

NoRx [+]

Bite

Clinical DataVariable Estimate Evidence Range Analyzed

Baseline Infection Rate Facial Bites (FaceNoRxInf)

10% Retrospective Review 5% - 10%

5.60%(95% CI 3.8 –8.2%)

Baseline Infection RateHand Bites (HandNoRxInf)

8.30%(95% CI 1.8 – 34.2%)

Baseline Infection RateOther Area (extremity and trunk)

(OtherNoRxInf)9.50%

(95% CI 6.5% - 13.95)% of infected wounds considered for

outpt Rx (OutPtRx)90% Physician Opinion 50% - 90%

Failure Rate of outpt antibiotics (Fail) 5 % - 15%RCT for outpt treatment of

cellulitis10% - 30%

Risk of Complicated Inpt Course (1-Inptunc)

1- 4% Retrospective review 2% - 10%

Risk of disability after hand infection/surgery (CompDis)

5% Surgeon’s opinion 1% - 10%

Risk of Death w/ complicated infection (1-CompRec)

3% Survey Estimate 2% - 3%

Antibiotic Side Effects (ShortSE; LongSE)

5% - 34% RCT 5% - 34%

Infection Rate - prophylactic antibiotics – facial bites (FaceRxInf)

Meta-analysis of RCT 3.2% - 10%

36% Retrospective Review 10% - 50%

Infection Rate - prophylactic antibiotics – other areas

Meta-analysis of RCT 6.5% - 14%

Infection Rate - prophylactic antibiotics – Hand Bites (HandRxInf)

Meta-analysis of RCT 18% - 34.2%

17% Retrospective Review 10% - 30%

Location of StudyDouble Blind? Antibiotic Exclusions*

Potential No. of Subjects

Entered in Trial

No. With Known Results (%)†

No. Infected

(%)

Fresno, California15 Yes Penicillin Wound > 24 hr old 569 98 62 (63.3) 11 (17.7)

Kansas City, Missouri5 Yes Oxacillin Wound > 24 hr old Children Hospitalized patients Bone involvement — 63 47 (73.0) 2 (4.3)

Philadelphia6 No Penicillin Wound > 24 hr old Adults Sutured wounds Facial wounds 80 58 55 (94.8) 2 (3.6)

Pittsfield, Massachusetts7 Yes

Cloxacillin Dicloxacillin Erythromycin

Wound > 8 hr old Involved, bone, tendon, nerves Cannot take capsule medication 150 36 33 (91.7) 3 (9.1)

Manchester, United Kingdom16 Yes

Trimethoprim/sulfamethoxazole Age < 3 yr Problems requiring antibiotics — 113§ 78 (69.0) 11 (14.1)

Chicago17 Yes Penicillin Adults Hospitalized patients — 39 39 (100.0) 3 (7.7)

Middlesbrough, United Kingdom18 Yes

Amoxicillin/clavulanate Wound > 24 hr old Age < 6 yr Tendon or joint involvement 1,334 185¶ 96 (51.9) 44 (45.8)

Fort Hood, Texas19 No

Dicloxacillin Cephalexin Erythromycin

Wound > 12 hr old Age < 1 yr Wounds of hands or feet Puncture wounds Immunocompromised host Immunosuppressive medication 759 191 185 (96.9) 6 (3.2)

783 594 (75.9) 82 (13.8)

Antibiotic Control

Location of Study Total Infected (%) Total Infected (%) Relative Risk 95% CI

Fresno, California15 30 3 (10) 32 8 (25) 0.40 0.12-1.37

Kansas City, Missouri5 22 2 (9) 24 0 (0)                             —

Philadelphia6 25 1 (4) 30 1 (3) 1.20 0.08-18.23

Pittsfield, Massachusetts7 15 1 (7) 18 2 (11) 0.60 0.06-5.99

Manchester, United Kingdom16 55 3 (5) 58 8 (14) 0.40 0.11-1.41

Chicago17 19 2 (11) 20 1 (5) 2.11 0.21-21.36

Middlesbrough, United Kingdom18 51 17 (33) 45 27 (60) 0.56 0.35-0.88

Fort Hood, Texas19 89 1 (1) 96 5 (5) 0.22 0.03-1.81

Summary ' 0.56 0.38-0.82

Willingness to Pay DataSurvey

QuestionBite

LocationWillingness to Pay to: Median Range

2Hand, Face,

OtherAvoid side effects of a 10-day course of antibiotics $40

$7 to $250

3 Face Lower risk from 4% to 2% that face wound will scar and need plastic surgery $50 $10 to $400

4 Other Lower risk from 2% to 1% that leg wound will scar and need plastic surgery $10 $0 to $20

5 Hand Lower risk from 4% to 1% that hand wound will scar and need plastic surgery $20 $0 to $250

6 Hand Decrease chance of chronic pain in dominant hand from 4% to 1% $200 $0 to $3000

7 Hand Decrease chance of chronic pain in non-dominant hand from 4% to 1% $100 $0 to $2500

8 Face, Other Decrease 0.0006% risk of death from extremely rare infection by 50% $10 $0 to $200

9 Hand Decrease 0.002% risk of death from rare infection by 75% $20 $0 to $200

10 Face Take pills for 3 days to decrease need to take the same pills for 10 days from 10% to 6% $5 $0 to $40

11 Hand Take pills for 3 days to decrease need to take the same pills for 10 days from 16% to 10% $25 $0 to $100

12 Other Take pills for 3 days to decrease need to take the same pills for 10 days from 36% to 8% $10 $0 to $30

13 Face Take a pill to decrease chance of hospitalization for 3 days from 2% to 1% $15 $0 to $50

14 Other Take a pill to decrease chance of hospitalization for 3 days from 4% to 2.2% $15 $0 to $150

15 Hand Take a pill to decrease chance of hospitalization for 3 days from 10% to 2.3% $100 $15 to $300

16 Face Take a pill to decrease chance of hospitalization for 10 days from 0.4% to 0.2% $3 $0 to $100

17 Other Take a pill to decrease chance of hospitalization for 10 days from 0.8% to 0.5% $4 $0 to $100

18 Hand, Other Take a pill to decrease chance of hospitalization for 3 days from 2% to 0.5% $25 $0 to $200

Cost DataDiagnosis / Procedure Charge

(1997)Charge (2000)

Payment (2000)

Range Source In 2000 Dollars

Cellulitis of the face $4,357 $4,624 $2,312 $1,849-$4,624

HCUP Nationwide Inpatient Sample

4623.52

Cellulitis of the hand $4,153 $4,407 $2,204 $1,763-$4,407

HCUP Nationwide Inpatient Sample

4407.04

Other Skin & Subcutaneous Locations

$4,776 $5,068 $2,534 $2,027-$5,068

HCUP Nationwide Inpatient Sample

5068.15

Septicemia (except in labor) $10,540 $11,185 $5,593 $4,474-$11,185

HCUP Nationwide Inpatient Sample

11185

Incision and Drainage of skin and subcutaneous tissue

$7,141 $7,578 $3,789 $3,031-$7,578

HCUP Nationwide Inpatient Sample

7577.82

Other skin and breast procedures, OR

$10,991 $11,663 $5,832 $4,665-$11,663

HCUP Nationwide Inpatient Sample

11663

Medication Augmentin (per pill) $4.76 $2-$6 www.pillbot.com

Branch WTP Cost WTP - CostIncremental Difference

Rx 44.1 70.8 -26.7 28.3NoRx 10.7 65.7 -55

Rx 75.3 87.4 -12.1 125NoRx 92.3 229.4 -137.1

Rx 49.4 100.1 -50.7 49.2NoRx 18.9 118.8 -99.9

Branch WTP Cost WTP - CostIncremental Difference

Rx 12.9 116 103.1 37.1NoRx 6.2 146.4 140.2

Rx 44.1 253.2 -208.8 257.5NoRx 39.9 506.2 -466.3

Rx 18.2 184.2 -166 92.7NoRx 10.6 269.3 -258.7

Base Case

Extreme Scenario (based on sensitivity runs)

Hand

Other

Face

Hand

Other

Face

Sensitivity Analysis on FaceRxinf

FaceRxinf

Exp

ecte

d V

alue

0.0320 0.0490 0.0660 0.0830 0.1000

-10.0

-15.0

-20.0

-25.0

-30.0

-35.0

-40.0

-45.0

-50.0

-55.0

Rx

NoRx

Are Prophylactic Antibiotics Beneficial in Dog Bites?Value of Cost- Benefit Models

• Model determined 1% difference may be important as far as cost

• An RCT to determined this would not be reasonable

But….

- The model is based on assumptions and best available data.

- Sensitivity analysis can determine the errors associated with assumptions

- Better model estimates will improve the accuracy of the results.

Are Prophylactic Antibiotics Beneficial in Dog Bites

• Funding – NIAMS• Design Cost Benefit Analysis with Clinical Trial Data• Start Aug 2003 UCSF add Stanford November 2004,

Study will run through June 2006• Patient randomized to 3 days of Augmentin or Placebo• Goal 100 – 125 patients outcomes (the largest trial)- 37 patients with complete F/U to date• Goal is to define and measure accurately all outcomes

(infections, side effects, hospitalizations etc.) in the model• Re-run the model and sensitivities to come up with the best

recommendations.

Are Prophylactic Antibiotics Beneficial in Dog Bites?Outcomes

• Infection- Defined as to whether the patient on follow-up

was treated for an infection with antibiotics• Related physician/hospital: visits, admissions,

treatments• Side effects- Self limited: patient self treated- Required physician visit/treatment

How Can I Help?• Expect a call when a dog bite comes in

• Volunteers will do the enrolment if they are present, but will ask you some questions and will need physician help with attaining the consent

• We will walk you through enrollment if the volunteers are not present

• If you are too busy we will come in

Summary

• Academic Emergency Medicine is growing

• Research is an integral part

• Division of Emergency Medicine at Stanford has made a commitment to the research program

Questions and Answers