march 2006 addressing the limited data dilemma non-traditional sources of safety data presented by:...
TRANSCRIPT
March 2006
Addressing the Limited Data Dilemma
Non-Traditional Sources of Safety Data
Presented by: Terecia Wilson
Director of Safety
March 2006
Traditional Data Collection
Crash Data
Driver Records
Vehicle Records
Roadway Inventory Data
Citation Data / Adjudication Data
EMS Run Reports
March 2006
Alternative Data Collection
Road Safety Audits Opinion Surveys Observational Surveys Program Assessments Interviews SCP Forum (Data Guide)
CODES Data Data Cube Complaint Files Program Evaluations Professional Judgment
March 2006
A formal examination of an existing or future road or traffic project, or any project which interacts with road users, in which
an independent, qualified examiner reports on the project’s crash potential
and safety performance.
What is a Road Safety Audit?
March 2006
Why Do RSAs?
Proactive approach to highway safety. Widely used in other countries - highly
effective. Possible even with limited resources. Supports Strategic Plan Goal of improving
safety.
March 2006
Future Roads Stage 1 – Planning Stage 2 – Preliminary Design
Roads Under Construction Stage 3 – Final Design Stage 4 – Pre-opening
Existing Roadways Stage 5 – Operations Review
When Can an Audit Be Done?
March 2006
Types of Audit Data
Information collected for use in audit
Traffic counts
Public hearing information
Detailed designs
Crash Data
March 2006
Types of Audit Data
Information collected from audit report
Prioritized findings and recommendations
Multi-disciplinary report
Comments from special road user groups
March 2006
Geographic Representation from across state.
Representation from various disciplines: Traffic Engineering, Planning, Engineering, Construction, Pre Construction, Special Interest groups.
Special Interest groups (ie, local law enforcement, EMS, Disabilities and Special Needs, AARP, etc.)
RSA Team Participants
March 2006
RSA Follow Up
Conduct follow up study to determine impact on traffic safety 3 years after final audit report.
Examine traffic collision data 3 years before and 3 years after audit.
Include RSA team to assist in evaluation as needed.
March 2006
Method Advantages Disadvantages
· Can survey many people · Difficult to get much detail· Not time-consuming · Sometimes difficult to get correct addresses
· Relatively inexpensive · May be problems with interpreting questions
· Everyone gets the same instrument · Sometimes a problem getting surveys completed & returned
· Objective Interpretation
· Able to ask for more detail when needed · Sometimes difficult reaching people
· Everyone gets the same instrument · Lack of anonymity
· Researcher knows how people are interpreting questions · Time-consuming time, can limit sample size
· Able to ask for more detail when needed · Subjective interpretation
· Provide detailed data · Can be expensive
· Can be difficult to analyze
· Researcher knows how people are interpreting questions · Group setting may inhibit some individuals from providing information
· Able to ask for more detail when needed · Sometimes hard to coordinate multiple schedules
· Able to interview multiple people at one time, thus, more cost-effective
· Responses from one person provide stimulus for other people
· Responses from one person provide stimulus for other people
· Objective interpretation · Time-consuming
· Low burden for people providing data · Some items are not observable
· Can be expensive
· Participant behavior may be affected by observer presence
Focus Group
Observation
Surveys
Telephone
Interview
March 2006
Opinion Surveys
Telephone Surveys Random Digit Dialing
Unbiased
Varied Demographics (Age, Race, Education, Income)
Representative sample of licensed drivers
March 2006
Opinion Surveys
Examples of Telephone Surveys Follow-up evaluation of public information
campaigns
Determine public opinion on safety issues
• Primary Safety Belt Law
• .08 BAC
• Motorcycle Helmet Legislation
Determine changes in public opinion on safety issues (Baseline/Ongoing)
March 2006
Opinion Surveys
Focus Groups Used for more qualitative information Used in a wide range of applications in nearly
every field of market research. Encourages participants to express their feelings
freely and without inhibitions Probes more deeply into issues in a relaxed,
uncontrolled atmosphere.
March 2006
Opinion Surveys
Focus Groups Discussion leaves participants feeling, justifiably,
that their opinion is important Gives clients a clearer perspective on how their
customers feel and why they feel that way. Information gained may lead to changes in
program and product development and implementation.
Include demographic representation of target population.
March 2006
Opinion Surveys
Used as part of dispute resolution, or to solicit opinions before making significant changes.
Useful in developing consensus when community opinion is not immediately obvious in normal discussion.
Especially useful during legislative debate on controversial safety issues.
Also, useful in developing funding priorities.
March 2006
Observational Surveys
Sometimes the best way to collect information about people's behavior is to watch them.
Observation allows the researcher to collect information without being a burden on the person providing the information.
Typically evaluators develop guides that structure the observation process.
March 2006
Observational Surveys
Drawbacks:
Measures only what you can see. Other types of data (e.g., opinions, reasons behind behavior) cannot be collected in this fashion.
Time-consuming as multiple observations are often required.
Presence of collectors may influence behavior.
Safety of observers.
March 2006
Program Assessments
Help determine ways to improve effectiveness and efficiency of programs.
Provide tools and documentation by which additional steps can be taken to make programs better and/or safer.
Identify gaps in services.
Provide support for additional financial and human resources.
March 2006
Program Assessments
Peer reviews of programs. NHTSA currently offers program assessments for the following:emergency medical services impaired driving traffic records motorcycle safety occupant protection
March 2006
Examples : Studies on the effectiveness of program activities
Identifies steps to be taken to enhance existing programs
Evaluates the implementation of new programs
Assists in justifying additional funding and program support
Considers current legislation and the direction for legislative action
Provides documentation to be used as National and State input for policy, training and program development.
Program Assessments
March 2006
How Do You Arrange for a Program Assessment?
The State Highway Safety Offices obtain program assessments by writing and requesting an assessment from one of the NHTSA Regional Offices.
If information or assistance is needed regarding the Highway Safety Program Assessments, please contact the NHTSA Regional office for your state.
March 2006
Interviews
Interviews with local agencies personnel (Police, EMS Responders, Local Engineers, Coroners) provide invaluable data.
Insight on perceived needs (more enforcement, engineering improvements, etc.)
Opportunity to speak with someone that may have been first on the scene at a particular incident or fatality
The benefit in some cases of a play by play account of what happened at a particular crash
March 2006
Interviews
Provide insight on local uses for roads (cut through, alternate route to avoid traffic, racing, truck route, etc.)
Offer insight as to how areas surrounding road may change with the various seasons
Is there anything planted that might impede sight distance at an intersection?
Are there any streams that deer gravitate toward which might increase the need for them to cross roads?
Become aware of planned projects that may impact the road
March 2006
Interviews
When do you conduct interviews? Site visits
Monitoring visits
Roundtable discussions
Development meetings
Public Hearings / Town Hall Meetings
Annual Professional Conferences
Individual Interviews
March 2006
Interviews
Questions asked in interviews
What is the most common type of crashes?
What happens when it rains? Does the road flood in particular area?
Did something change in the landscape recently that might affect why crashes increased (i.e., cutting down trees that block the sun)?
March 2006
Safety Conscious Planning
“...a proactive approach for the prevention of motor vehicle crashes and unsafe transportation conditions.”
Improving Safety on Our Highways
March 2006
Safety Conscious Planning
... a comprehensive, system wide, multi-modal, proactive process that better integrates safety into surface transportation decision making.
March 2006
Considers all aspects of highway safety – engineering, education, awareness, enforcement & emergency response
Uses a system-wide approach including sites, corridors & entire state, regional & local transportation systems
Safety Conscious Planning
March 2006
Safety Conscious Planning
Multi-modal including transit, pedestrian & bicycle safety needs
Proactive - addresses current safety problems & looks for opportunities to prevent them in the future
March 2006
Forum Participants
Broad cross section of planning and safety communities
Statewide representation
Multidisciplinary group (including MPO’s, COG’s)
Federal partners (FHWA, NHTSA, FMCSA)
March 2006
Plans Provided to Participants
AASHTO Strategic Highway Safety Plan
Emergency Medical Services State Plan
Federal Railroad Administration Action Plan
402 Highway Safety Plan
Injury Control Plan
SCDOT Strategic Plan
SC Long Range Transportation Plan
Others
March 2006
Data Guide
SCDOT (Road Inventory, Traffic Counts, Mileage Reports)
SCDPS (Collision File)
SCDMV (Driver & Vehicle Files)
EMS (Run Reports, Trauma Registry)
DAODAS (School Age & Adult Surveys)
Office of Research & Statistics (CODES, Census, Hospital Discharge)
March 2006
Forum Accomplishments
Brought over 200 partners together, many 1st time
Adopted several goals & strategies to improve safety – all willing to support in their plan
Improved communications among partners (E-mail group)
Enlightened participants on available data sources
March 2006
CODES Project
Crash Outcome Data Evaluation System
Multi-agency effort which includes: SC Department of Public Safety
SC Department of Transportation
SC Department of Health and Environmental Control
SC Emergency Medical Services
SC Budget and Control Board Office of Research and Statistics
March 2006
CODES: Goal
Provide a comprehensive view of motor vehicle crashes and their resultant impact on morbidity, mortality, health care services and associated costs.
March 2006
CODES Project
Collaborative approach to obtain medical and financial outcome information related to motor vehicle crashes for highway safety and injury control decision making.
Evolved as the result of the Intermodal Surface Transportation Efficiency Act (ISTEA) of 1991
Report to Congress about the benefits of safety belts and motorcycle helmets for persons involved in motor vehicle crashes.
March 2006
CODES Project
Measure benefits in terms of reducing death, disability, and medical costs
Includes statewide data for all persons involved in police-reported crashes
Includes those who were injured or who died as well as those who were not injured.
Allows comparisons between those using and not using safety belts or motorcycle helmets
Identifies and contrasts characteristics of injured and uninjured persons within each of the restraint use groups.
March 2006
CODES: Linked Data Sets
Crash (DPS)
Emergency Medical Services (DHEC EMS)*
Hospital (ORS)
*Prior to 2001 only
March 2006
Data Collected: Crash
Demographic Data Driver / Pedestrian / Pedalcyclist
Passengers
Restraint Usage
Crash Location / Type of Crash
Contributing Factors
Injuries / Fatalities / Transported
Alcohol or Drug Involvement
March 2006
Health Care Utilization Databases
Hospital Inpatient Discharges
Ambulatory Surgery Episodes
Emergency Room Visits
March 2006
ABOUT THE PATIENT: age, race, gender, geographic location codes
ABOUT THE EPISODE: Hospital & Physician Characteristics
primary diagnosis and nine related diagnoses
primary and nine secondary procedures with dates, admission and discharge dates, length of episode
destination at discharge (home, home health referral, death, etc.)
Data Collected: Health Care Utilization Databases
March 2006
Data Collected: Health Care Utilization Databases
COST OF CARE:
Detailed charges by revenue center (e.g., pharmacy, lab, respiratory therapy, etc.)
Primary and secondary payer class (e.g., Medicare, Medicaid, Private Insurance, Self-pay/Indigent)
*Hospital charges used as proxy
March 2006
CODES Project
Links databases containing information about individual persons collected from police crash reports, emergency medical transports, emergency room visits and inpatient hospitalization records.
Uses probabilistic linkage methods
SC has been a CODES state for 7 years.
March 2006
CODES Project
System helps in analyzing crash patterns.
Includes a mapping system to locate crashes based on such characteristics as crash severity, volume of crashes, age of driver and involvement of alcohol and/or drugs.
Developed CODES Internet site to facilitate the dissemination of information from the project.
March 2006
CODES Project
Statewide data collected on all persons: Involved in police-reported crashes Transported by emergency medical services due to crashes Visits to emergency room due to injuries Hospitalized due to injuries.
Analyses of data used to measure impact of crashes by communities in terms of reducing injuries, deaths, and medical costs.
Comparisons can be made between the characteristics of those using and not using safety belts, helmets, and other restraints.
PDO crashes were not included in the linkage or analysis.
March 2006
Uses of CODES Data
Provides economic argument for safety legislation Primary Belt Law
.08
.10 Per Se
Automated Enforcement at Red Lights
Develops profiles for Safe Communities Programs
Provides demographic data to develop educational / enforcement programs.
March 2006
Impact of ED Data on CODES
Policy decisions can be made based on better estimates of medical cost data
Population of crash victims injured and treated in a hospital setting more accurate
Surrogate SC Trauma Registry
Complete look at crash and injury patterns for Community Needs Assessments (high volume vs. high injury)
March 2006
Data Request / Fact Sheet
Restraint use by pay source and treatment type (ED and inpatient)
Number and rate of injuries
Total and average charges
Length of stay (inpatient only)
Also included the total numbers of injured, number and percent linked from crash to hospital data.
March 2006
What is a Data Cube?
While ORS answers requests now using information from the Data Warehouse, our dream was to create a
“WEB-Accessible User-Driven Query Based System that agencies can access and explore their own questions”
Cubes would be for statistical / aggregate analyses
March 2006
SC Data Warehouse
Build off of existing systems (legacy systems from state agencies and private sector)
Create a Unique ID (not related to any other number)
Identifiers are pulled off of the statistical data. Use only the statistical data
Data is always “owned” by the originating agency. Must have permissions to use and/or link any data
March 2006
Disabilities & Special Needs
Vocational Rehabilitation
Law Enforcement
Health Department
Education
Outpatient Surgeries
State Employee
Health Services
Emergency Room Visits
Hospitalizations
Environmental Conditions
Home Health Care
Medicaid Services
Social Services
Public Safety
Mental Health
Juvenile Justice
Integrated Data
System
Free Clinic Visits
Alcohol & Drug Services
Child Care
Community Health Centers*
Medicare
Legal/Safety Services
Social Services
Claims Systems
All Payer Health Care Databases
Behavioral Health
Health Department
Education
Other State Support Agencies
LEGEND
Disease Registries
Disease Registries
Elder Services &
Assessments
*Still in contract negotiations.
ProbationParole & Pardon
Corrections*
March 2006
Data Warehouse Allows Agencies & Other Entities to: Evaluate their programs
Look at Outcomes
Understand better how their programs interact with other agency & other entity programs
Study Health, Human Service, Education, and Law Enforcement Issues
Analyze Statistical – Aggregate Information
Access Analytic Data Cubes
Partner in the Development of Customized Software Applications
SC Data Warehouse
March 2006
Disabilities & Special Needs
Vocational Rehabilitation
Law Enforcement
Health Department
Education
Outpatient Surgeries
State Employee
Health Services
Emergency Room Visits
Hospitalizations
Environmental Conditions
Home Health Care
Medicaid Services
Social Services
Public Safety
Mental Health
Juvenile Justice
Injury and Violence
Cube
Free Clinic Visits
Alcohol & Drug Services
Child Care
Community Health Centers*
Medicare
Legal/Safety Services
Social Services
Claims Systems
All Payer Health Care Databases
Behavioral Health
Health Department
Education
Other State Support Agencies
LEGEND
Disease Registries
Disease Registries
Elder Services &
Assessments
*Still in contract negotiations.
ProbationParole & Pardon
Corrections*
March 2006
Linking Data Sets
Records are linked for the same individual using a unique tracking number
Tracking number is random so cannot be unencrypted to identify the individual
An individual is assigned the same number over time
March 2006
What Is an Analytic Cube?
A way to “slice and dice” through large amounts of data
Define “slicers,” characteristics that are important to analyzing the subject population
Pre-aggregate the linked data by all possible combinations of “slicers”
Data shown as an example - not for analytical use
March 2006
Data shown as an example - not for analytical use
March 2006
Data shown as an example - not for analytical use
59
March 2006
Data shown as an example - not for analytical use
March 2006
March 2006
Cubes Under Development
Injury Cube (funded through a CDC grant)
Mental Health/Alcohol Cube funded through DMH funds
Medicaid Cube (all ages)
Elderly Cube (just received funding to development this cube!)
March 2006
Complaint File
Track all complaints
News stories, editorials, news articles
Be Responsive to public concerns
Is it a real problem or just perceived?
Coordinate site visit
Prepare response
Include in future planning
Compile crash data for complaint site
Do not ignore the public!
March 2006
Program Evaluation Reports
Assess how well the program has been implemented.
Assess the extent to which the activities have achieved the project’s goals.
Identify gaps in services.
Identify spin-off efforts.
March 2006
Establish an Evaluation Plan
How will you know you’re achieving goals?
What will you measure?
How will you evaluate?
Who measures? When?
What documentation will you maintain?
When and what evaluation report?
March 2006
Process Evaluation
Why a program succeeds or fails.
Compares program design with implementation.
Describes and documents life of program.
March 2006
Outcome Evaluation
Deals with short-term, direct effects of program.
Identifies the results of a program's/initiative's effort.
It seeks to answer the question, "What difference did the program make?“
It provides information about effects of a program after a specified period of operation.
March 2006
Impact Evaluation
Deals with long-term, ultimate effects of program.
Assess program effectiveness in terms of end results, including intended and unintended results.
Also assess the net effect of a program by comparing impacts with an estimate of what would have happened in the absence of the program
March 2006
Professional Judgment
Allows for a multi-disciplinary approach.
Assimilates data from variety of sources to determine strategies.
Provides benefits from years of knowledge and experience.
Essential to any planning process.
March 2006
For More Information Contact:
Terecia Wilson
Director of Safety
803-737-1161