North CarolinaInstitute for Public

Health

Measuring Public Health Agency Preparedness

Mary V. Davis, DrPH, MSPH

December 20, 2006

What is the Role of Public Health Agencies in Disasters/Bioterrorism?

Feedback Poll

Is it possible for me to know whether my health department is prepared?

A. Yes

B. No

Measuring Preparedness

• What does it mean to be prepared?

• How can preparedness be measured?– Definitions– Metrics– Instruments

• What is the relationship between individual competency and agency preparedness?

Measurement 1.0

Evaluation of Public Health Response to Hurricanes Finds North Carolina Better Prepared for Public Health Emergencies

January–June 2004

Purpose of NC Study

Identify how BT funds improved public health capacity in North Carolina to prepare for and respond to bioterrorism events and other public health emergencies.

Examine how improvements in public health capacity for a specific public health emergency can translate to all public health emergencies.

NC Context

• Capitalize on “natural experiment”– 1999 Hurricane Floyd– 2002 Infusion of Federal funds to address

bioterrorism– 2003 Hurricane Isabel

• Strong partnership history between NC DPH and UNC SPH

• No standard definition of “preparedness”

Evaluation Questions—Resources

• Phases of Disaster Framework—Noji

• Public Health Management of Disasters—Landesman

• Criteria for evaluating management of disasters—Quarentelli

NC Hurricane Evaluation Study

Handout Page 1Evaluation Questions

Data Collection

• Key informant interviews—Professional role involvement in Floyd and/or Isabel

• Document review—state and federal protocols, documents, records

Results Floyd

• Planning—no specific activities

• Preparation– Communication with local health departments

• Response– Command and Control– Surveillance and Assessment

Floyd and “The Box”

DPH Use of Federal Funds

• Created Infrastructure– Office of Public Health Preparedness & Response– Public Health Command Center– Public Health Regional Surveillance Teams

• Employed Staff– BT Coordinator– Office of Public Health Preparedness & Response staff– Public Health Regional Surveillance Team staff: 28

in 7 regions statewide– Public health epidemiologists: 12 statewide

• Implemented Health Alert Network

• Implemented NC Emergency Department Database

Actions (cont’d)

• Enhanced Laboratory Capacity– Biosafety Level 3 and Level 2

• Purchased Equipment– 8 vehicles– Fixed vehicle-mounted and hand-held radios

• Strengthened Communication– Bioterrorism Steering Committee, reps from multiple agencies

• Provided Training– Emergency preparedness and core public health competencies

– Exercises and drills

Resulted in Capacities Used in Isabel

• Effective, centralized communication through Public Health Command Center– Between state agencies

– With local public health departments

– With community and academic partners

– With citizens

Capacities (cont’d)

• Command and Control Incident Management– Operational and functional plans

– Systematic response

– Decision making in real-time emergency conditions

– Epidemiology and surveillance• Rapid needs assessment

• Medical surveillance

Preparedness Logic Model

Handout Page 2

NC Model

Preparedness Logic Model

Handout Page 3

Preparedness Logic Model

RAND Corporation. RAND Health. Learning from Experience, Page 9

• Goal: mitigate morbidity, mortality, stress

• Objectives– Early identification– Early and effective response– Earliest return to normal function

• Functional capabilities meet objectives

• Capacity-building activities result in capabilities

Measurement 2.0

Mississippi Department of Health Hurricane Katrina After Action Report

January–June 2006

Target Capabilities List

• U.S. Department of Homeland Security Target Capabilities List version 1.1

• Defines 36 prevention, protection, response, and recovery capabilities (measures)

• Capability measures: capacity (processes, infrastructure)

• Performance measures (demonstrate task execution)

Why Target Capabilities?

• Becoming benchmarks for state performance

• Provide a framework for performance measures

• Create a performance plan– Specific– Measurable– Obtainable

TCL Pilot Test

• MS Department of Health Katrina After Action Review– Specific type of evaluation– Identifies strengths and weaknesses in

response– Defines specific action plan for improvements

• Examined response in 21 TCLs– Emergency Support Function 8—public health– General TCL’s—planning

Target Capabilities Examined in MDH Katrina AAR

• Handout 4

NC Involvement

• MS requested NC to construct a team to assess ESF 8 response

• MDH planning with Region IV, specifically NC OEMS prior to Katrina

• NC mobilized SMAT to Waveland, MS during Katrina response

• NC personnel familiar with gulf communities and MDH structure

MDH Katrina AAR

• Identify major points what worked/did not work well during MDH Katrina response Katrina in Emergency Support Function 8

• Data collection (all new instruments)– Responder Online Assessment– Responder Interviews– Community Assessment

• Results used to:– Inform Performance Improvement Plan– Revise MS State Emergency Operations Plan

Online Survey Example

1.2.0 Interoperable Communications (Communications and Information Management)

* 1.2.0: Were you a part of Interoperable Communications (Communications and Information Management)?

Please choose only one of the following:YesNo

[Only answer this question if you answered 'Yes' to question '1.2.0 ']

1.2.2: Were sufficient back-up equipment and power sources available?

Please choose only one of the following:YesNoUnknown

[Only answer this question if you answered 'No' to question '1.2.2 ']

1.2.2n: Please explain why you answered no.

Online Survey and Interview Data

• Feedback available for all TCLs– Least for Animal Health Emergency Support

– Most for Planning, Mass Care

• Substantive open-ended feedback from survey “no” responses

• Interviewees provided substantive information on what worked and areas for improvement

AAR Recommendations

• Overarching capability and specific TCL

• Results from online survey, interviews, community assessment

• Experience and knowledge of MDH and NC team

• Federal guidelines

Limitations

• First use of TCL Performance Measures as data collection instruments

• Limited time to create and validate data collection instruments

• Respondents lacked knowledge of performance measure

• MDH identified participant samples

• Challenge to digest the wealth of information from the results

Take Home Points

• First use of Target Capability List as an AAR framework

• First instance of one state government conducting an AAR for another state

• NC team drew upon EM, OEMS, PHP&R, UNC SPH, and other expertise

• Approach yields data in performance measure framework

Preparedness Measurement Next Steps

• Modify Target Capability measurement instruments

• Pilot in an After Action Review

• Continuously Refine Measures

• Use RAND Public Health Preparedness Logic Model as guide

• Measures should be – specific about responder function– specific about activities and capabilities

Public Health Preparedness

• Continuous process—constant quality improvement

• Agencies need to test preparedness in real conditions– Exercises and drills

– Outbreak investigations

– Disaster, BT responses

Individual Competency and Agency Preparedness

• Using RAND Public Health Preparedness Logic Model

• Individual Competency– Conduct capacity-building activities– Demonstrate competency to perform

functional capabilities

• Agencies– Implement functional capabilities– Achieve preparedness objectives and goals

Resources

• NC Hurricane Evaluationhttp://www2.sph.unc.edu/nccphp/hurricane_10_19_04.pdf

Article in PH Reports in Jan/Feb 2007 issue

• MDH AAR Reporthttp://www.msdh.state.ms.us/msdhsite/_static/resources/1676.pdf

• Contact mvdavis@email.unc.edu