measuring the effectiveness of the nation’s foodservice and retail food protection system

1

MEASURING THE EFFECTIVENESS OF THE NATION’S FOODSERVICE AND RETAIL FOOD PROTECTION

SYSTEM

MEASURING THE EFFECTIVENESS OF THE NATION’S FOODSERVICE AND RETAIL FOOD PROTECTION

SYSTEM

2

PRESENTATION OBJECTIVES

PRESENTATION OBJECTIVES

• Purpose of FDA’s Study

• Review What the Study IS and IS NOT

• Review the Design of the Study

• Preview of 2003 Results

• Present Recommendations

• Purpose of FDA’s Study

• Review What the Study IS and IS NOT

• Review the Design of the Study

• Preview of 2003 Results

• Present Recommendations

3

BACKGROUNDBACKGROUND

• FDA National Retail Food Steering Committee

• Government Performance and Results Act (1993)“performance plans … measurable indicators”

• FDA National Retail Food Steering Committee

• Government Performance and Results Act (1993)“performance plans … measurable indicators”

4

INTRODUCTION AND PURPOSE


• Agency effort to change behaviors / practices related to foodborne illness

• Identify food safety practices and employee behavior most in need of attention

• Agency effort to change behaviors / practices related to foodborne illness

• Identify food safety practices and employee behavior most in need of attention

5



• Establish a national baseline on the occurrence of foodborne illness risk factors

• Effort to collect sufficient data to measure patterns in the occurrence of foodborne illness risk factors over time

• Establish a national baseline on the occurrence of foodborne illness risk factors

• Effort to collect sufficient data to measure patterns in the occurrence of foodborne illness risk factors over time

6

STUDY TIME LINESTUDY TIME LINE

• 1998 1ST Data CollectionReport of the FDA Retail Food Program Database of Foodborne Illness Risk Factors (2000)

• Established a Baseline for 9 different foodservice and retail food facility types

• 1998 1ST Data CollectionReport of the FDA Retail Food Program Database of Foodborne Illness Risk Factors (2000)

• Established a Baseline for 9 different foodservice and retail food facility types

7

STUDY TIME LINESTUDY TIME LINE

• 2003 2nd Data CollectionFDA Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional Foodservice, Restaurant and Retail Food Store Facility Types (2004)

• Begins the process for establishing multiple data collection periods

• 2003 2nd Data CollectionFDA Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional Foodservice, Restaurant and Retail Food Store Facility Types (2004)

• Begins the process for establishing multiple data collection periods

8

STUDY TIME LINESTUDY TIME LINE• 2008 3rd Data Collection will be

conducted

Report prepared for 2009 will begin the process of analyzing the three data sets (1998, 2003, and 2008) to assess improvement or regression changes from the 1998 Baseline

• 2008 3rd Data Collection will be conducted

Report prepared for 2009 will begin the process of analyzing the three data sets (1998, 2003, and 2008) to assess improvement or regression changes from the 1998 Baseline

9

WHAT THE REPORT IS AND IS NOT


• IS – a national assessment of the relative strengths & weaknesses of food safety systems designed to control the occurrence of foodborne illness risk factors

• IS NOT – a correlation of the occurrence of foodborne illness risk factors with the actual incidence of human illness

• IS – a national assessment of the relative strengths & weaknesses of food safety systems designed to control the occurrence of foodborne illness risk factors

• IS NOT – a correlation of the occurrence of foodborne illness risk factors with the actual incidence of human illness

10



• IS– an assessment that used the 1997 Food Code provisions as the standard upon which observations of food safety practices are based

• IS NOT– a determination of the industry’s OR a specific establishment’s regulatory compliance with prevailing laws & regulations

• IS– an assessment that used the 1997 Food Code provisions as the standard upon which observations of food safety practices are based

• IS NOT– a determination of the industry’s OR a specific establishment’s regulatory compliance with prevailing laws & regulations

11

METHODOLOGYMETHODOLOGY

• Scope: 3 Industry segments / 9 facility types:

1.Institutions • Hospitals• Nursing Homes• Elementary Schools


1.Institutions • Hospitals• Nursing Homes• Elementary Schools

12

METHODOLOGYMETHODOLOGY


2.Restaurants• Fast Food• Full-Service


2.Restaurants• Fast Food• Full-Service

17

BACKGROUNDCONTRIBUTING FACTORS

BACKGROUNDCONTRIBUTING FACTORS

• Food from Unsafe Source

• Inadequate Cooking

• Improper Holding/Time-Temp.

• Contaminated Equipment/Protection

from Contamination

• Poor Personal Hygiene

• Other “Chemical Hazards”

• Food from Unsafe Source

• Inadequate Cooking

• Improper Holding/Time-Temp.

• Contaminated Equipment/Protection

from Contamination


• Other “Chemical Hazards”

19

METHODOLOGYMETHODOLOGY• Data Collection – Records status

for all individual data items

IN – In Compliance

OUT – Out of Compliance

N.O. – Not Observed

N.A. – Not Applicable

• Data Collection – Records status for all individual data items

IN – In Compliance

OUT – Out of Compliance

N.O. – Not Observed

N.A. – Not Applicable

20

DATA ANALYSISDATA ANALYSIS• Three levels of data analysis are conducted

for each of the 9 facility types

A. Out of Compliance percentage for each individual data item

B. Out of Compliance percentage for each foodborne illness risk factor

C. Overall IN Compliance percentage for all 42 data items

• Three levels of data analysis are conducted for each of the 9 facility types

A. Out of Compliance percentage for each individual data item

B. Out of Compliance percentage for each foodborne illness risk factor

C. Overall IN Compliance percentage for all 42 data items

21

Determining the Out of Compliance percentage for

each DATA ITEM

Determining the Out of Compliance percentage for

each DATA ITEM

Percent Out of Compliance =Percent Out of Compliance =Percent Out of Compliance =Percent Out of Compliance =

Total Out of ComplianceTotal Out of ComplianceObservations for a Data ItemObservations for a Data ItemTotal Out of ComplianceTotal Out of ComplianceObservations for a Data ItemObservations for a Data Item

Total number of Observations(IN and OUT) for the Data ItemTotal number of Observations(IN and OUT) for the Data Item

X 100%X 100%X 100%X 100%

23

Determining Out of Compliance Percentages

FOODBORNE ILLNESS RISK FACTORS

Determining Out of Compliance Percentages


Observations of DATA ITEMS

FOODBORNE ILLNESS RISK FACTOR

Observations of DATA ITEMS

FOODBORNE ILLNESS RISK FACTOR

24

RISK FACTOR

Number of Data Items

for each Risk Factor

Referenced Sections

From Data Collection

Form Food from Unsafe Sources

7

Sections 1 – 3

Inadequate Cooking

12

Sections 4 – 5

Improper Holding/Time Temperature

10

Sections 6 – 9

Contaminated Equipment/Protection from Contamination

5

Sections 10 – 11

Poor Personal Hygiene

5

Sections 12 – 15

Other (Chemical Contamination)

3

Section 16

TOTAL NUMBER OF DATA ITEMS 42

25

Determining the Out of Compliance percentage

for each FOODBORNE ILLNESS RISK FACTOR

Determining the Out of Compliance percentage

for each FOODBORNE ILLNESS RISK FACTOR

Percent Out of Compliance =Percent Out of Compliance =Percent Out of Compliance =Percent Out of Compliance =

Total Out of ComplianceTotal Out of ComplianceObservations for a Risk FactorObservations for a Risk FactorTotal Out of ComplianceTotal Out of ComplianceObservations for a Risk FactorObservations for a Risk Factor

Total number of Observations(IN and OUT) for the Risk FactorTotal number of Observations(IN and OUT) for the Risk Factor

X 100%X 100%X 100%X 100%

26

ExampleOut of Compliance Percentage

Poor Personal Hygiene - Risk Factor

ExampleOut of Compliance Percentage

Poor Personal Hygiene - Risk Factor


5 DATA ITEMS – Data collection Form

12A 13A 14A 15A 15B

Poor Personal Hygiene - RISK FACTOR


5 DATA ITEMS – Data collection Form

12A 13A 14A 15A 15B

Poor Personal Hygiene - RISK FACTOR

27

Determining Overall IN Compliance Percentages

- FACILITY TYPES -

Determining Overall IN Compliance Percentages

- FACILITY TYPES -

6 RISK FACTOR CATEGORIES

Overall In Compliance %

Facility Types

6 RISK FACTOR CATEGORIES

Overall In Compliance %

Facility Types

28

Overall PERCENTAGE (%) of OBSERVABLE and APPLICABLE

data items found IN Compliance

Industry Segment

Facility Type

1998 Baseline*

% IN Compliance Observable Items

(rounded to nearest %)

2010 FDA

Improvement

Goal** (rounded to nearest

(%) Institutions Hospital 80% 85%

Nursing Home 82% 87% Elementary

School

80%

85%

Restaurants Fast Food 74% 81% Full Service 60% 70%

Retail Food Deli 73% 80% Meat & Poultry 81% 86% Produce 76% 82% Seafood 83% 87%

31

Foodborne Illness Risk Factorin Need of Priority Attention

(2000 Report)

Foodborne Illness Risk Factorin Need of Priority Attention

(2000 Report)

• Improper Holding/Time and Temperature


• Contaminated Equipment / Protection from Contamination

• Improper Holding/Time and Temperature


• Contaminated Equipment / Protection from Contamination

32

2004 ReportData Results and Discussion


• For each of the 9 facility types the Data Results will present:

A. Overall Out of Compliance percentage for each of the foodborne illness risk factors

B. Individual data Items that comprise each of the risk factors that are in need of priority attention


A. Overall Out of Compliance percentage for each of the foodborne illness risk factors

B. Individual data Items that comprise each of the risk factors that are in need of priority attention

33




C. A national strengths/weaknesses assessment of management systems

for controlling the occurrence of foodborne illness risk factors


C. A national strengths/weaknesses assessment of management systems

for controlling the occurrence of foodborne illness risk factors

38

SUMMARY – DELI DEPARTMENTS

FOODBORNE ILLNESS

RISK FACTOR in need of priority attention

INDIVIDUAL DATA ITEMS

In need of priority attention

Commercially-processed RTE, PHF Date Marked

PHF Held Cold at 41º F or Below RTE, PHF Date Marked After 24 Hours PHF Held Hot at 140º F or Above PHF Cooled to 70º F in 2 Hours/41º F in Total of 6 Hours

Improper Holding/ Time & Temperature

RTE, PHF Discarded After 4 Days/45º F or 7 Days/41º F

Proper, Adequate Handwashing

Handwashing Facility, Convenient/Accessible


Handwashing Facility, Cleanser/Dry Device

Surfaces/Utensils Cleaned/Sanitized

Contaminated Equipment/ Protection from Contamination Raw/RTE Foods, Separated

Other/Chemical

Poisons/Toxic ID Store/Use Properly

43

Hot Holding at 140 oF (Data Item 8A) vs.

Hot Holding at 135 oF (Data Item 18A)

Facility Type Total

IN 140 oF

Total Out

140 oF

% IN 140 oF

Total IN

135 oF

Total Out 135 oF

% IN 135 oF

Difference (%IN)

Hospitals 59 36 62.1% 65 30 68.4% 6.3% Nursing Homes 53 17 75.7% 54 16 77.1% 1.4% Elementary Schools 61 21 74.4% 62 20 75.6% 1.2% Fast Food Rest. 66 28 70.2% 71 23 75.5% 5.3% Full Service Rest. 45 42 51.7% 48 39 55.2% 3.5% Delis 38 59 39.2% 41 56 42.3% 3.1% Meat & Poultry* 0 2 0%* 1 1 50.0%* 50.0%* Seafood* 3 3 50.0%* 3 3 50.0%* 0%*

Produce* 6 0 100%* 6 0 100%* 0%* * The number of observations for these 3 facility types is too low to make comparisons

46

Risk Factors with Statistically Significant Differences Between Establishments

WITH and WITHOUT a Certified Food Protection Manager

Facility Type

Foodborne Illness Risk Factors

Fast Food

Restaurants

Improper Holding/Time and Temperature Contaminated Equipment/Protection from

Contamination

Full Service Restaurants

Poor Personal Hygiene Contaminated Equipment/Protection from

Contamination

Meat and Poultry Departments


Produce Departments


49

Average Inspection Time per Establishment for each of the 9 Facility Types (Total MINUTES per Establishment)

Facility Type Average Inspection Time

(In Minutes)

HOSPITALS 155 ELEMENTARY SCHOOLS 99 NURSING HOMES 129 FAST FOOD RESTAURANTS 87 FULL SERVICE RESTAURANTS 133 DELI 94 MEAT & POULTRY 48 SEAFOOD 56

PRODUCE 41

50

Pathways to Reach the Goals Pathways to Reach the Goals

Risk Factors +

InterventionStrategies

+ Performance

Measures

FDA Foodborne Illness Risk Factor Study

Standardization &Certification

Program

Standards

51

RECOMMENDATIONS FOR INDUSTRY


ACTIVE MANAGERIAL

CONTROL OF FOODBORNE

ILLNESS RISK FACTORS

ACTIVE MANAGERIAL

CONTROL OF FOODBORNE

ILLNESS RISK FACTORS

52

ACTIVE MANAGERIAL CONTROL


Purposeful incorporation of

specific actions or procedures

by industry management

to attain control of

foodborne illness risk factors

Purposeful incorporation of

specific actions or procedures

by industry management

to attain control of

foodborne illness risk factors

53



• Develop and implement Standard Operating Procedures (SOPs) to address FBI risk factors

• Develop and implement Standard Operating Procedures (SOPs) to address FBI risk factors

54



• Provide employees with specific training and equipment to implement the SOPs

• Provide employees with specific training and equipment to implement the SOPs

55



• Incorporate critical limits and measurable standards for control of FBI risk factors in SOPs

• Incorporate critical limits and measurable standards for control of FBI risk factors in SOPs

56



• Establish monitoring procedures that focus on critical processes and practices

• Establish monitoring procedures that focus on critical processes and practices

57



• Identify methods to routinely assess the effectiveness of the SOPs

• Identify methods to routinely assess the effectiveness of the SOPs

58



• FDA guidance document for industry

Managing Food Safety: A Guide for Voluntary Use of HACCP Principles for Operators of Foodservice and Retail Establishments

www.cfsan.fda.gov/~dms/hret-toc.html

• FDA guidance document for industry

Managing Food Safety: A Guide for Voluntary Use of HACCP Principles for Operators of Foodservice and Retail Establishments

www.cfsan.fda.gov/~dms/hret-toc.html

59

RECOMMENDATIONS FOR REGULATORY


• Use risk-based inspection methodology

- risk factors should be the primary focus of every inspection

• Provide flexible work schedules

• Use risk-based inspection methodology

- risk factors should be the primary focus of every inspection

• Provide flexible work schedules

60



• Properly train and equip field personnel

• Document compliance determination (IN; OUT; N.O.; N.A.)

• Properly train and equip field personnel

• Document compliance determination (IN; OUT; N.O.; N.A.)

61



• Establish a dialog with industry foodservice and retail food store operators

• Recognize existing industry Quality Assurance Systems and Training Programs

• Establish a dialog with industry foodservice and retail food store operators

• Recognize existing industry Quality Assurance Systems and Training Programs

62



• Take appropriate corrective action

– Obtain immediate corrective action at the time of inspection for risk factors found out of compliance


– Obtain immediate corrective action at the time of inspection for risk factors found out of compliance

63




– Assist in developing Standard Operating Procedure and Risk Control Plans designed to attain long-term managerial control of risk factors


– Assist in developing Standard Operating Procedure and Risk Control Plans designed to attain long-term managerial control of risk factors

64

IMPLEMENT A CONSISTENT AND EFFECTIVE

ENFORCEMENT PROTOCOL

IMPLEMENT A CONSISTENT AND EFFECTIVE

ENFORCEMENT PROTOCOL

• Develop enforcement procedures

• Ensure credibility

• Develop enforcement procedures

• Ensure credibility

65

VOLUNTARY NATIONAL RETAIL FOOD REGULATORY

PROGRAM STANDARDS

VOLUNTARY NATIONAL RETAIL FOOD REGULATORY

PROGRAM STANDARDS

Standards of Excellence for

Continuous Improvement

Standards of Excellence for


66

Voluntary National Retail Food Voluntary National Retail Food Regulatory Program StandardsRegulatory Program StandardsVoluntary National Retail Food Voluntary National Retail Food Regulatory Program StandardsRegulatory Program Standards1. Regulatory FoundationRegulatory Foundation

2. Staff Training2. Staff Training

3. HACCP Principles-based Inspection 3. HACCP Principles-based Inspection Program Program

4. Inspection Uniformity4. Inspection Uniformity

5. Foodborne Illness and Food Security5. Foodborne Illness and Food Security

Preparedness and ResponsePreparedness and Response

1. Regulatory FoundationRegulatory Foundation

2. Staff Training2. Staff Training

3. HACCP Principles-based Inspection 3. HACCP Principles-based Inspection Program Program

4. Inspection Uniformity4. Inspection Uniformity

5. Foodborne Illness and Food Security5. Foodborne Illness and Food Security

Preparedness and ResponsePreparedness and Response

67

Voluntary National Retail Food Voluntary National Retail Food Regulatory Program StandardsRegulatory Program StandardsVoluntary National Retail Food Voluntary National Retail Food Regulatory Program StandardsRegulatory Program Standards

6. Compliance and Enforcement6. Compliance and Enforcement

7. Industry and Community Relations7. Industry and Community Relations

8. Program Support and Resources8. Program Support and Resources

9. Self Assessment9. Self Assessment

6. Compliance and Enforcement6. Compliance and Enforcement

7. Industry and Community Relations7. Industry and Community Relations

8. Program Support and Resources8. Program Support and Resources

9. Self Assessment9. Self Assessment

68


• Identify program areas where an agency can have the greatest impact on retail food safety.

• Promote wider application of effective FBI risk factor intervention strategies

• Identify program areas where an agency can have the greatest impact on retail food safety.

• Promote wider application of effective FBI risk factor intervention strategies

69


• Assist in identifying program areas most in need of additional resources

• Assist in identifying program areas most in need of additional resources

70

Standards as a Tool for Continuous ImprovementStandards as a Tool for


*Self Assess Improve *Self Assess Improve

PlanPlan

MeasureMeasure

71

FDA Retail Food Program Resource Disk


Contains all FDA’s technical guidance for retail food protection programs:– National Retail Food Program

Standards

– FDA’s 2000 Baseline Report

– Software Program & Guidance documents for jurisdictions to use to conduct their own foodborne illness risk factor studies

Contains all FDA’s technical guidance for retail food protection programs:– National Retail Food Program

Standards

– FDA’s 2000 Baseline Report

– Software Program & Guidance documents for jurisdictions to use to conduct their own foodborne illness risk factor studies

72



– 2001 Food Code (AND 2003 Food Code Supplement)

– FDA Standardization Procedures

– HACCP @ Retail Guides

– Plan Review Guide

– Temporary Events Guides

– 2001 Food Code (AND 2003 Food Code Supplement)

– FDA Standardization Procedures

– HACCP @ Retail Guides

– Plan Review Guide

– Temporary Events Guides

73

PROGRAM GOALPROGRAM GOAL


OF



OF


measuring the effectiveness of the nation’s foodservice and retail food protection system

Documents

data collection report

retail food stores

data collection fda

sufficient data

data sets

food code provisions

individual data items

facility typesa