Integrated Performance Integrated Performance Information Systems: Florida’s Information Systems: Florida’s

ExperienceExperiencePhoenix, ArizonaPhoenix, ArizonaJune 14-16, 2005June 14-16, 2005

Jay Pfeiffer, Assistant Deputy CommissionerJay Pfeiffer, Assistant Deputy CommissionerDivision of Accountability, Research, and MeasurementDivision of Accountability, Research, and Measurement

Florida Department of EducationFlorida Department of Education850-245-0437850-245-0437

Jay.Pfeiffer@FLDOE.orgJay.Pfeiffer@FLDOE.org

Francis

Jeanne

Charley

IvanBonnie

Source: Florida K20 Education Data Warehouse

Florida: The Home of Integrated Performance

Additional role: Department-wide conduit for LMS, LED, Census, BEBR publications

Office of K-20 EducationInformation & Accountability

Public SchoolsCommunityCollege &

Technical CenterUniversity

Source InformationSystems

K-12 Adult EducationCareer/TechnicalAssociate Degree

BaccalaureateMastersPh.D./Professional

IntegratingSystems

Source SystemsEmploymentAssessmentFinancial AidTeacher CertificationFacilities

EmploymentEarningsPostsecondary Ed.WelfareIncarcerationLicensure

K-20 EducationData Warehouse

FETPIP

K-20 Governance Organization: Alignment of Data Systems

FETPIP – The Florida Education and Training

Placement Information Program

Florida Education and Training Placement Information Program Florida Education and Training Placement Information Program (FETPIP)(FETPIP)

Collection CycleCollection CycleHigh

School

Vocational Center

Community College

Public University

Independent University

Other Programs

Federal Programs

Welfare Transition Services

FL CorrectionsReleases

Special Projects,Longitudes

FL UC CoveredEmployment

Fed Government& Postal Service

Uniformed MilitaryService

Out-of-StateEmployment

Out-of-StatePostsecondary Ed.

Public UniversityEnrollment

Public Adult Ed. &Vocational Enrollment

Private College &University Enrollment

Public CommunityCollege Enrollment

FL CorrectionsIncarcerations

Performance and Accountability

Reports

Workforce FundingFormula

Ad HocReports

“CHOICES”Career

CounselingSoftware

InternetEducation &WIA Reports

Merge FilesUnduplicateEdit Names

& SSN

Create MasterFollow-up File

Create Tapes& Files for

Record LinksPublic Assistance

Receipt

June,2005

Input DataInput Data

Student/ParticipantStudent/Participant - demographic, socio-economic, - demographic, socio-economic, enrollment/services, educational programs, promotion, enrollment/services, educational programs, promotion, attendance, GPA, test scores…attendance, GPA, test scores…

Educational InstitutionEducational Institution - types, location, performance - types, location, performance statistics, private/public…statistics, private/public…

Financial AidFinancial Aid - federal/state/local, loans/grants, - federal/state/local, loans/grants, amounts disbursed…amounts disbursed…

CoursesCourses - offerings, student & teacher participation, - offerings, student & teacher participation, instruction type, grades…instruction type, grades…

Educational StaffEducational Staff - demographic, awards, certification, - demographic, awards, certification, teaching & work activityteaching & work activity

Educational AwardsEducational Awards - type, program, institution, date - type, program, institution, date

ExamplesExamples

Postsecondary Enrollments

Starting an Integrated Performance Information Starting an Integrated Performance Information SystemSystem

High Schoo

l

Vocational Center

Community College

Welfare Transition Services

State UC - CoveredEmployment

Fed Government& Postal Service

Uniformed MilitaryService

.

StatePerformance and

Accountability Reports

Workforce Reporting

Ad HocReports

CareerCounselingSoftware

WIA Reports

Merge FilesUnduplicateEdit Names

& SSN

Create MasterFollow-up File

Create Tapes& Files for

Record Links

Public AssistanceReceipt

June,2005

WIA Participants

Adult Education

FETPIP DataFETPIP Data

Public Education FundingPublic Education Funding Performance-Based BudgetingPerformance-Based Budgeting Federal Performance Measurement: WIA, Wagner-Peyser, Voc Federal Performance Measurement: WIA, Wagner-Peyser, Voc

Rehab, Blind Services, Perkins, NRS (Adult Education)Rehab, Blind Services, Perkins, NRS (Adult Education) Private School LicensurePrivate School Licensure School District Accountability ReportsSchool District Accountability Reports Community College Performance ReportsCommunity College Performance Reports University System Program ReviewsUniversity System Program Reviews ICUF Accountability ReportingICUF Accountability Reporting Legislative Performance ReviewsLegislative Performance Reviews Workforce Estimating ConferenceWorkforce Estimating Conference Policy ResearchPolicy Research Career Counseling, Consumer and Guidance InformationCareer Counseling, Consumer and Guidance Information

Major UsesMajor Uses

FETPIP-Type OperationsFETPIP-Type Operations

Use existing data resourcesUse existing data resources Cheaper, more accurate, more complete, and with Cheaper, more accurate, more complete, and with

more capabilities, than traditional surveysmore capabilities, than traditional surveys Data are regarded as objective- non-judgmentalData are regarded as objective- non-judgmental Provide a basis for common core sets of outcome Provide a basis for common core sets of outcome

measures, comparisons, discussions of issues across measures, comparisons, discussions of issues across many programs & operationsmany programs & operations

BenefitsBenefits

Challenges (part 1)Challenges (part 1) Administrative data are designed for Administrative data are designed for

purposes other than performance purposes other than performance measurementmeasurement

Definitions, taxonomies may differDefinitions, taxonomies may differ Time periods may differ Time periods may differ What is measured may not be exactly what is desiredWhat is measured may not be exactly what is desired Administrative data change over timeAdministrative data change over time

Owners of Administrative Data have legal, Owners of Administrative Data have legal, ethical, proprietary obligationsethical, proprietary obligations

They won’t naturally want to shareThey won’t naturally want to share ““Legal” beliefs may be conjured upLegal” beliefs may be conjured up Program interests may be different from those of technical staffProgram interests may be different from those of technical staff The “owners” change over timeThe “owners” change over time

Challenges (part 2)Challenges (part 2) Confidentiality of InformationConfidentiality of Information

Requirements may be different for each data sourceRequirements may be different for each data source The issue is complicated by legal issues as well as beliefsThe issue is complicated by legal issues as well as beliefs Beliefs/requirements may change over timeBeliefs/requirements may change over time

Weakest LinksWeakest Links The strength of a system of linked data sets is determined by the The strength of a system of linked data sets is determined by the

weakest of the data resources weakest of the data resources Opponents will focus on the weaknessesOpponents will focus on the weaknesses The data lag actual performanceThe data lag actual performance

SupportSupport On going, reliable fundingOn going, reliable funding On going, reliable political supportOn going, reliable political support Cooperative attitude among stake holdersCooperative attitude among stake holders All involved will changeAll involved will change

What has worked for FETPIP?What has worked for FETPIP?

Statutory authority, provisions, including cross referencesStatutory authority, provisions, including cross references Core funding from State General Revenue appropriated by the Core funding from State General Revenue appropriated by the

LegislatureLegislature Persistently collaborate with legislative, executive staffPersistently collaborate with legislative, executive staff Persistently collaborate with data owners, stakeholders in decisions Persistently collaborate with data owners, stakeholders in decisions

about the mechanics of exchanges, reporting of resultsabout the mechanics of exchanges, reporting of results Constantly anticipate roles that the system can play in Constantly anticipate roles that the system can play in

accountability, performance measurement, cost return analysis, accountability, performance measurement, cost return analysis, evaluationevaluation

Consistently work with data owners on changes, data anomalies, Consistently work with data owners on changes, data anomalies, problemsproblems

Steadfastly make it easy for stake holders to participate, reduce Steadfastly make it easy for stake holders to participate, reduce burdens, provide data in exchangeburdens, provide data in exchange

Always facilitate discussions about common data elements, Always facilitate discussions about common data elements, definitions, core performance measures, performance fundingdefinitions, core performance measures, performance funding

SupportSupport

What has worked for FETPIP?What has worked for FETPIP?

Location within the state education agencyLocation within the state education agency Always surpass all requirements, expectations regarding Always surpass all requirements, expectations regarding

data securitydata security Faithfully limit uses to statistical purposes for evaluation, Faithfully limit uses to statistical purposes for evaluation,

program improvementprogram improvement Persistently keep exchange agreements current and up-Persistently keep exchange agreements current and up-

to-date, continually brief employees, stake holders on to-date, continually brief employees, stake holders on security and confidentiality issuessecurity and confidentiality issues

Always refer inquiries about particular resources to the Always refer inquiries about particular resources to the resource ownersresource owners

Confidentiality/SecurityConfidentiality/Security

What has worked for FETPIP?What has worked for FETPIP?

Strive to meet changes, challenges head-onStrive to meet changes, challenges head-on Persistently refresh understanding of resources, Persistently refresh understanding of resources,

requirements, limitationsrequirements, limitations Persistently seek new data resources, expand Persistently seek new data resources, expand

coveragecoverage Always revise business rules as understanding, Always revise business rules as understanding,

conditions changeconditions change Establish short term and long term visions for Establish short term and long term visions for

the effortthe effort

ChangesChanges

What has worked for FETPIP?What has worked for FETPIP?

Establish archival data resourcesEstablish archival data resources Accommodate ad hoc interests and Accommodate ad hoc interests and

requirementsrequirements Seek out opportunities to demonstrate your Seek out opportunities to demonstrate your

wareswares Always collaborateAlways collaborate

OtherOther

– K20 EDW – Florida’s K20 Education Data Warehouse

Employee Identity

EducationalInstitutions

EducationalCurriculumEmployee Demographic

Includes:Ÿ TeachersŸ Faculty MembersŸ Support Staff

Course

K-20 Budget

Student Demographic

Certified Staff

Instructional activity

Financial Assistanceincludes:Ÿ Federal AidŸ State AidŸ Local AidŸ Bright Future

StudentTest Scores

(Ex-)studentEmployment

District

Institution'sFinances

Chart ofAccounts

Institution'sRevenues

Active Studentincludes:Ÿ All enrollmentsŸ AccelerationŸ AttendanceŸ DisciplineŸ Exceptionality Program

Educational Facilities

StudentAward

Student'sIdentityStudent'sIdentity

Student'sCourses

Includes:Ÿ Public SchoolsŸ CollegesŸ UniversitiesŸ Private

Institutions

IncludesŸ SUS ApplicationŸ Talented 20

Florida’s K20 Education Data Warehouse

Educational Institution

Demography

Educational Awards

Educational Staff

Courses

Grades, Standardized Test Scores

Financial Aid

Student Employment

K20 Data Warehouse: Student-Centered

Transform & Loadprocesses

M atching P rocess

D ata warehousein ternal identifier

M ainfram e System s

Other System s

Data fo r load in the warehouse

Personal Identifiable

Data

Data warehouse

Matching Individuals in Florida’s Education Data Warehouse: Sophisticating the

Matches, Anonymizing Data