evaluation of the iaff hazmat and wmd training programs - … · 2020-05-14 · evaluation of the...
TRANSCRIPT
![Page 1: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/1.jpg)
Evaluation of the IAFF Hazmat and WMD Training ProgramsEvaluation of the IAFF Hazmat Evaluation of the IAFF Hazmat and WMD Training Programsand WMD Training ProgramsWETP Fall Technical Workshop, 10-18-07, Sheraton Chapel Hill, Chapel Hill, NCPatrick MorrisonAssistant to the General PresidentInternational Association of Fire Fighters
Bruce Lippy, Ph.D., CIH, CSPPresident, The Lippy Group, LLC
![Page 2: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/2.jpg)
Introductory CommentsIntroductory CommentsPatrick Morrison
Assistant to the General PresidentIAFF
![Page 3: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/3.jpg)
Bruce LippyBruce Lippy
President, The Lippy Group
![Page 4: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/4.jpg)
Lippy Group Lippy Group Company Picnic 2007Company Picnic 2007
First row, left to right: Lippy, B.
![Page 5: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/5.jpg)
Lippy Group Corporate MetricsLippy Group Corporate Metrics
![Page 6: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/6.jpg)
Evaluation TeamEvaluation Team
Bruce Lippy, Ph.D., CIH, CSPJames Weeks, Sc.D., CIHBernard Mizula, MS, CHS-V, CET, RPIH
IAFF provided us an office and full access
![Page 7: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/7.jpg)
Bruce Bruce Lippy’sLippy’s experience with experience with fire fightingfire fighting
Setting duckpins for the Manchester, MD Fire
Company
![Page 8: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/8.jpg)
Evaluation approachEvaluation approach1. Interviews of board members,
headquarters staff, and students2. Evaluation of NIEHS and DHS
progress reports3. Focus groups with IAFF instructors 4. Analysis of IAFF student database5. Comparison of findings against
recommendations from guidance documents
6. Onsite evaluation of training program
7. Follow-up interviews of federal, state and county officials
![Page 9: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/9.jpg)
Final Issues for evaluationFinal Issues for evaluation1. Methods of communication 2. Student database3. Student registration process4. Class records (storage, accuracy, etc.)5. Class sizes 6. Make-up of instructor cadre 7. Current programs 8. New programs9. Instructor resource center10. Blended learning
![Page 10: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/10.jpg)
Evaluation timelineEvaluation timelineInitial discussions March 06
Interviews, observations and data analysis
Sep to Nov 06
Presentation to General President and Advisory Board of initial findings
Oct 06
Evaluation of presentation at Trainers Exchange
March 07
Draft reports March to May 07
Final report (v. 12) July 07
![Page 11: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/11.jpg)
Broad Finding:Broad Finding:
NIEHSNIEHSMinimum Minimum Criteria is Criteria is excellent basis excellent basis for program for program evaluationevaluation
![Page 12: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/12.jpg)
Is there evidence that the program is accomplishing its objectives?
IAFF ranked #2 in cost per contact hr.IAFF ranked #2 in cost per contact hr.(NIEHS Annual Report 2003)(NIEHS Annual Report 2003)
![Page 13: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/13.jpg)
IAFF ranked 7/17 in cost per studentIAFF ranked 7/17 in cost per student(NIEHS Annual Report 2003)(NIEHS Annual Report 2003)
![Page 14: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/14.jpg)
IAFF Number of traineesIAFF Number of traineesProjected v. actual, Projected v. actual, (FY96 (FY96 –– FY04)FY04)
![Page 15: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/15.jpg)
Are appropriate facilities and staff available and committed to the program?
YES!“Training coordinators” are key to effectivenessStaff meetings short, focused and productive“Lead” instructor facilitates communication with the training site and with headquartersTraining at the host site with their equipment
![Page 16: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/16.jpg)
Methods of CommunicationsMethods of CommunicationsRecommendationRecommendation
Emphasize prerequisites for courses, particularly T-t-T, and don’t rely solely on the
requesting organization
![Page 17: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/17.jpg)
Methods of CommunicationsMethods of CommunicationsRecommendationsRecommendations
Include more actual case studies in reports to NIEHS
NIEHS official:*“Firefighters save lives every day, but their progress reports don’t reflect that sufficiently, they can be boring.”
*Ted
![Page 18: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/18.jpg)
Student DatabaseStudent DatabaseImpressive File-Maker Pro database maintained by an outside contractor for the last 8 years38,000 records in database described as a “learning management system”
![Page 19: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/19.jpg)
System covers shipping, System covers shipping, finance, and classesfinance, and classes
![Page 20: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/20.jpg)
IAFF Shipping ProcessIAFF Shipping Process
Unlike other NIEHS grantees, IAFF relies upon host for equipment Student notebooks and reference materials (Pocket Guide, DOT ER book) Shipped out of headquartersRecommended positioning Guides in active locations, like AZ, rather than constantly shipping from HQ
![Page 21: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/21.jpg)
Gender
Male91%
Female6%
NA0%
Missing3%
Student Demographics from Student Demographics from database: Genderdatabase: Gender
![Page 22: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/22.jpg)
0.010.020.030.040.050.060.070.0
Percent
Cauca
sian
Hispan
ic
African
America
n NA
Asian/Pac
ific Is
lande
r
Native
America
n
Student Demographics: Student Demographics: EthnicityEthnicity
![Page 23: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/23.jpg)
Age of IAFF StudentsAge of IAFF StudentsMean age is 33 yearsMean age is 33 years
Histogram: Age
-10 0 10 20 30 40 50 60 70 80 90 1000
2000
4000
6000
8000
10000
12000
14000
16000
18000
No.
of o
bs.
![Page 24: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/24.jpg)
Schooling of IAFF StudentsSchooling of IAFF Students
Hig
h Sc
hool
Som
e C
olle
ge NA
Gra
de S
choo
l
Bach
elor
's D
egre
e
Asso
ciat
e D
egre
e
Post
Bac
helo
r's D
egre
e
Mas
ter's
Deg
ree
02000400060008000
10000120001400016000180002000022000240002600028000
No
of o
bs
![Page 25: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/25.jpg)
0.05.0
10.015.020.025.030.035.040.045.0
Percent Completed
FRO
ERT:OpsCSO
Tech
TtT FRO
WMD TtT
zOther ID
Inc. Mgt.
Courses ProvidedCourses Provided
![Page 26: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/26.jpg)
0102030405060708090
Percent Employed
Fire Serv
iceOthe
rNA
Private
Indu
stry/C
onsul
tant
Law Enfo
rcement
Public
Safety
Private
EMS
Indus
trial F
ire Brig
ade
Emergency
Man
agemen
t
Public
EMS
Employment of studentsEmployment of students
![Page 27: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/27.jpg)
05
101520253035
Percent
Less
than
one y
ear
1 - 5
year
s
Over 2
0 yea
rs6 -
10 ye
ars
11 - 1
5 yea
rs16
- 20 y
ears
Not App
licable NA
Years of employment Years of employment experience of studentsexperience of students
![Page 28: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/28.jpg)
What are the program’s strengths?
The instructor cadre. Currently 91 instructors (84 at time of survey)Headquarters staffTechnical quality of the curriculaSupport from General PresidentGuidance from Advisory BoardCoordination with Safety and Health staff
![Page 29: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/29.jpg)
What are the program’s What are the program’s weaknesses?weaknesses?
Feedback to instructorsQuality Assessment Reports are generated by headquarters from student evaluations, but provision to instructors needs to be more consistent and timely
![Page 30: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/30.jpg)
What are the program’s What are the program’s weaknesses?weaknesses?
Annual evaluation of instructors.A point for discussion by all Awardees:
“The annual review of instructor competency shall include, at a minimum, observation by the Training Director or his or her designee of instructional delivery, review and discussion of observations with the instructor, and an analysis of the instructor performance based upon evaluations completed by trainees during the previous year.”
Minimum Criteria guidance (9.3.3)
![Page 31: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/31.jpg)
Frontline SafetyFrontline SafetyCurriculum development meetingCurriculum development meeting
September 21, 2006September 21, 2006
![Page 32: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/32.jpg)
Is the evaluation tool current and appropriate for program
content?
Student evaluation form unchanged in 8 yearsExcellent and consistent efforts by Alexander Cohen, Ph.D.New efforts underway by Dr. Harold Stolovitch look quite promising
![Page 33: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/33.jpg)
Harold Stolovitch in venturing into Kirkpatrick Level 4
evaluations
![Page 34: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/34.jpg)
Blended LearningBlended Learning
IAFF has a talented individual leading the charge: Vilma Perez-AtwoodIAFF has created impressive web-based courses, such as the Avian Flu unitGiven the importance of skills training, not as much emphasis on web-based training as other granteesAvailability of computers and access to web still issues
![Page 35: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/35.jpg)
IAFF NearIAFF Near--Miss ProgramMiss Program
![Page 36: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/36.jpg)
IAFF NearIAFF Near--Miss ProgramMiss Program
An important program focused on an area that can have significant impact– OSHA insists on a near-miss program in
VPP– DOE’s Lessons Learned culture has made
a differenceReaching out to the airline industry is bold and can pay real dividends
![Page 37: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/37.jpg)
Do you enjoy being an Do you enjoy being an instructor in the IAFF program?instructor in the IAFF program?
“EXTREMELY! Any of the problems/challenges outlined here are normal organizational stuff. They are
overshadowed by the chance to share with fellow responders in their own
environment. You really get the feeling (most of the time) that you’re making a positive difference for these students
and that’s what it’s all about .”
![Page 38: Evaluation of the IAFF Hazmat and WMD Training Programs - … · 2020-05-14 · Evaluation of the IAFF Hazmat and WMD Training Programs WETP Fall Technical Workshop, 10-18-07, Sheraton](https://reader035.vdocuments.mx/reader035/viewer/2022070718/5ede2d00ad6a402d66697a86/html5/thumbnails/38.jpg)
Questions?Questions?