standard 4 breakout session inspection audit program russell lilly manager, manufactured food...
TRANSCRIPT
Standard 4 Breakout SessionInspection Audit Program
Russell LillyManager, Manufactured Food Program
Missouri Department of Health and Senior Services
Wednesday, March 12, 20148:00 – 10:00
Elm Fork I
Patrick KennellyChief, Food Safety Section
California Department of Public Health
Peter Haase Director, Bureau of Food Safety and Inspection Wisconsin Department of Agriculture, Trade and
Consumer Protection
Skya MurphyProgram and Policy Analyst
Wisconsin Department of Agriculture, Trade and Consumer Protection
Angela KohlsStandards Implementation StaffFDA ORA, Office of Partnerships
AgendaStrengthening Compliance and
Enforcement
• Welcome and Introductions• Current Status of Standard 4 Implementation• Presentations on Implementing Standard 4• Q & A/Group Discussion• Summary and Challenge
Housekeeping• Honor time schedule• Stay on topic • Get input from everyone• Be willing to share• Respect different program cultures• Do not reference specific audits• Be creative, ask questions – parking lot• Facilitators – responsible for process• Participants – responsible for content
Session Objectives
Generate a list of resources, best practices, lessons learned and challenges/recommendations for developing, implementing and monitoring an inspection protocol.
(Group Exercise Instructions)1. Listen to presentations2. Discuss and list suggestions on flip charts 3. Share your group’s suggestions with room
Standard Four
Full Conformance
(100%)
80-99% 60-79% 40-59% <39%
PE Conformance Rating
4 4 2 2 2710 10 5 5 69
Total StatesPercentage (%)
Standard Four
1.a 1.b 2.a 2.b 2.c 3.a 3.b 3.c 4.a 4.b 4.c 5.a 5.b 6.a 6.b
Field Inspection Audits Inspection Report Audits Sample Report AuditQuality Assurance
ProgramCorrective Action
PlanDocumentation
No. of YES 8 10 11 18 16 8 12 12 8 9 8 11 8 15 12No. of NO 30 28 28 21 23 31 27 27 30 27 27 28 31 24 27No. of NA 0 0 0 0 0 0 0 0 1 3 3 0 0 0 0
Breakout Session Focus
• Elements 4.2b &c, 4.3b & c, and 4.4b & c Implementation of the 3 Types of Audits– Field Inspection– Inspection Report– Sampling
Standard 4: Field Inspection Audits
Patrick Kennelly, ChiefFood Safety Section
California Department of Public Health
March 12, 2014
Field Inspection Audits
• Quality assurance review to assess the effectiveness of its inspection program.– 2 audits of each inspector every 36 months– Conducted by a qualified auditor– Documented on Audit form that is equivalent to
the FDA Contract Audit form– Need an 80% audit rating to be considered a
successful audit
Field Inspection Audits
• Then and Now– Better Defined Process– Uniform Documentation– Ability to Compare Ratings Across the Program
Field Inspection Audits
DEPARTMENT OF HEALTH AND HUMAN SERVICES FOOD AND DRUG ADMINISTRATION
CONTRACT AUDIT ☐ FDA CONTRACT AUDIT ☐ FDB INSPECTION AUDIT
AUDITOR STATE INSPECTOR
FIRM CFN / FEI NUMBER OR FIRM ID NUMBER
FIRM ADDRESS
PRODUCT(S) COVERED DATE
TIME IN TIME OUT OVERALL RATING
☐ ACCEPTABLE ☐ NEEDS IMPROVEMENT
I. PREINSPECTION ASSESSMENT
1. DID THE INSPECTOR REVIEW THE STATE’S ESTABLISHMENT FILE FOR THE PREVIOUS INSPECTION REPORT AND POSSIBLE COMPLAINTS OR ACCESS OTHER AVAILABLE RESOURCES IN PREPARATION FOR THE INSPECTION?
☐ Acceptable ☐ Needs Improvement
COMMENTS (required for Needs Improvement)
2. DID THE INSPECTOR HAVE THE APPROPRIATE EQUIPMENT AND FORMS TO PROPERLY CONDUCT THE INSPECTION? ☐ Acceptable ☐ Needs Improvement
COMMENTS (required for Needs Improvement)
Field Inspection AuditsII. INSPECTION OBSERVATIONS AND PERFORMANCE (Continued)
6. DID THE INSPECTOR DEMONSTRATE THE ABILITY TO DISTINGUISH BETWEEN SIGNIFICANT VERSUS INSIGNIFICANT OBSERVATIONS AND ISOLATED INCIDENTS VERSUS TRENDS?
☐ Acceptable ☐ Needs Improvement
COMMENTS (required for Needs Improvement)
7. DID THE INSPECTOR REVIEW AND EVALUATE THE APPROPRIATE RECORDS AND PROCEDURES FOR THIS ESTABLISHMENT’S OPERATION AND EFFECTIVELY APPLY THE INFORMATION OBTAINED FROM THIS REVIEW?
☐ Acceptable ☐ Needs Improvement
COMMENTS (required for Needs Improvement)
8. DID THE INSPECTOR COLLECT ADEQUATE EVIDENCE AND DOCUMENTATION IN ACCORDANCE WITH STATE PROCEDURES GIVEN THE NATURE OF THE INSPECTIONAL FINDINGS?
☐ Acceptable ☐ Needs Improvement
COMMENTS (required for Needs Improvement)
Field Inspection Audits
• Things to Consider– How do you select facilities for staff audits?– What constitutes a “qualified auditor”?– Do you want to vary or alternate auditors for
each staff?– How do you address individual “Needs
Improvement” Ratings?
Worksheet 4-B
Performance rating for the field inspection audits
State program: Performance period:
Field inspection audit perfomance rating (4):
Name/title of reviewer: Office: Date:
Auditor's initials, firm identification number, and date of inspection (1)
Performance Factors (5)
At
(3)
NIt (3)
Performance Ratings (2)I.1 0 0I.2 0 0II.1 0 0II.2 0 0II.3 0 0II.4 0 0II.5 0 0II.6 0 0II.7 0 0II.8 0 0II.9 0 0
II.10 0 0IIA.1 0 0IIA.2 0 0IIA.3 0 0IIA.4 0 0III.1 0 0III.2 0 0III.3 0 0III.4 0 0III.5 0 0III.6 0 0
Subtotal Enter the sum of the totals from all continuation sheets. 0 0
Total Enter the final sums (subtotal + sums of (3) on this form). 0 0
(5) USE THIS SPACE TO IDENTIFY AND MAKE NOTES ABOUT SINGLE PERFORMANCE FACTORS RATED AS “NEEDS IMPROVEMENT” IN MULTIPLE AUDITS.
0
Field Inspection Audits• Scoring across the spreadsheet identifies areas of
“Needs Improvement” ratings within a rated category.
– Need 80% “Acceptable” ratings within each rated category across the program to be in conformance.
– How do you address Non-Conformance ratings?
Field Inspection Audits
• Benefits– More Knowledgeable Workforce– Improved Inspection Procedures– Identifying Training Gaps– Findings Drive Quality Improvement
Contact Information
Patrick Kennelly, ChiefFood Safety Section
California Department of Public Health
(916) 650-6598
Inspection Report Audits
Russell LillyManufactured food program manager
Missouri Department of Health
March 12, 20148:00 AM
Because…
• I had three inspectors and reviewed every inspection report
• It is wildly prescriptive – At least half of the widgets don’t apply
• If the shoe doesn’t fit it is uncomfortable
Our situation changed
• 18% of 22 inspectors
• A quality assurance program becomes necessary and valuable
We took the money…
• We owe an honest effort to comply with the standard
We did it…
• My best advice– Hire someone smart and assign it to them
Was it worth it?
• I think so
– We have sent a number of “corrective emails”• Mark this here not there• Be sure to put your “time out” on the form• Turn in your reports on time
I would still like
• To have a quality system designed from the ground up to fit our program
Contact Information
Russell Lilly
Manufactured food program manager
Missouri Department of Health
Email: [email protected]
Steps to Implementing Standard 4 Sample Collection Record Audit
Peter Haase Director, Bureau of Food Safety and Inspection
Skya MurphyProgram and Policy Analyst
Wisconsin Department of Agriculture, Trade and Consumer Protection
Sample Collection Record Audit Outline
• Introduction and Challenges • Steps to customization • Conclusion
• Documents on flash drive
Intro: Wisconsin Department of AgTrade & Consumer Protection
• Enrolled in MFRPS in 2007 • Challenges: Full Standard 4
Implementation Planned for 2015 – Phase III Contract Audit since 2011
• Currently Piloting Non-Contract Field Inspection Audit
• Inspection Report Audit 2013• SCR Audit 2014
Intro: Wisconsin Department of AgTrade & Consumer Protection
Current Surveillance Sampling Program for Manufactured Food Establishments>Sampling at 2,400 establishments annually:
– 2 product samplesand
– 5-10 environmental swabs at each routine inspection
Intro: Wisconsin Department of AgTrade & Consumer Protection
• Standard 10 Full Conformance– ISO Certified since 2007
• Sampling Policies and Procedures • Two Sample Collection Record Forms,
Food and Dairy
Challenges of Auditing our Sample Collection Records: Step 1 Test drive of Appendix 4.7.
Challenges of Auditing our Sample Collection Record (SCR): Step 1 Results of Test drive of Appendix 4.7.
– Findings: • 4.7 is more easily
managed in table format
• FDA Criteria:– Some not applicable or
redundant for our reports
– Very general: guidance specific to our SCRs required
Audited against our Policy
Added guidance specific to our Sample Collection Record formand procedures
Audited against our SCRs
Customized 4.7 Form:
Added guidance specific to our Sample Collection Record formand procedures
Piloted new form and guidance by auditing 71 SCR’s from Calendar 2012
• Results: – Opportunities for improvement: Use of hash
marks and down arrows, form completeness• Redundant criteria:
– SCR and sample always together
Challenges of Auditing our Sample Collection Records (SCR): Problem detected but not an audit criterion: Use of Correct Form
Substituted alternate criteria: Use of correct form
Challenges of Auditing our Sample Collection Records (SCR): For redundant criteria, combined and substituted
alternate criteria
Sample Selection (included in SOP for Standard 4)
• 75 SCR’s chosen per calendar year from MF establishments for auditing: – at least 1 per sampler, – random with regard to date, establishment
SOP for Standard 4 Cont.
• Entering and tabulating results: Excel Spreadsheet
• 1 for acceptable, • 0 for needs improvement
SOP for Standard 4 Cont.
• SCR Audit completed in March for previous calendar year
• Needs Improvement Factors (<80%) become training topics at staff meetings
• Other improvements can be made immediately upon discovery:– Already identified from SCR 2013 audit:
“Foodborne Outbreak” reason for sampling on new Dairy SCR
Conclusion
– Adapting 4.7 to our Sample Collection Record
• Meaningful results for quality improvement
– Documents on AFDO-MFRPA website:• Lab SOP #2060-Receipt and Integrity of Samples• SCR Policy and Procedures• SCR Forms (Food and Dairy)• Our Audit Form 4.7 and Data Entry/Calculations Sheet• Description of sample selection and procedures for
Standard 4 including Sample Collection Record Audits
Contact Information
Skya Murphy
Program and Policy Analyst
Wisconsin Department of Agriculture, Trade and Consumer Protection
Email: [email protected]
Contact InformationPatrick Kennelly, Chief
Food Safety SectionCalifornia Department of Public Health
Email: [email protected]
Russell LillyManufactured Food Program Manager
Missouri Department of Health
Email: [email protected]
Skya MurphyProgram and Policy Analyst
Wisconsin Department of Agriculture, Trade and Consumer Protection
Email: [email protected]
Peter Haase, Director Bureau of Food Safety and Inspection Wisconsin Department of Agriculture,
Trade and Consumer Protection
Email: [email protected]