Strengthening Measures of College and Career Readiness and Success: State Longitudinal Data Systems,

Governance, & Privacy

March 23, 2015

@CCRSCenter@AYPF_Tweets

#CCRSMeasures

GoToWebinar Technical Assistance: 1-800-263-6317

To submit live questions, please use the “Questions” box

A recording of the webinar and other resources will be available at www.ccrscenter.org and www.aypf.org.

Webinar Technical Support

#CCRSMeasures 2

https://www.streamtext.net/player?event=AYPF_March23

Please click to launch this link as a separate window that will run simultaneous to the webinar

slides

Closed Captioning Available

#CCRSMeasures 3

• Laura Jimenez, Director, CCRS Center

• Dorothyjean (DJ) Cratty, Senior Researcher, AIR

• Elizabeth Dabney, Associate Director, Research and Policy Analysis, Data Quality Campaign

• Rachel Anderson, Senior Associate, Policy and Advocacy, Data Quality Campaign

• Jean Osumi, Senior Associate for Academic Policy and Evaluation, Hawai‘i P-20/Hawai’i Data Exchange Partnership

• Melissa Beard, Data Governance Coordinator/Higher Education Analyst, Washington Education Research and Data Center (ERDC)

Today’s Presenters

#CCRSMeasures 4

CCRS CenterWebinar Series: Strengthening Measures of College and Career Readiness and Success

American Institutes for ResearchLaura Jimenez, DirectorMarch 23, 2015

#CCRSMeasures

5

CCRS Center as a Technical Assistance Hub

6

#CCRSMeasures

7

#CCRSMeasures

College and Career Readiness and Success Organizer

8

#CCRSMeasures

CCRS Organizer Outcomes and Measures Strand

Ch. 1 – Specify measurement goals

Ch. 2 – Secure and sustain stakeholder investment

Ch. 3 – Define college and career readiness and success goals and expectations

Ch. 4 – Select measures aligned with these goals and expectations

Measurement Practice Guide

9

#CCRSMeasures

Goal

Support state education agencies in their expanding use of P-20W state longitudinal data systems to increase college and career readiness and success.

Data Use Series

10

#CCRSMeasures

College & Career Readiness & Success Center

1000 Thomas Jefferson Street NWWashington, DC 20007800-634-0503www.ccrscenter.org | ccrscenter@air.org

11

#CCRSMeasures

Overview of State P20 Data Systems

March 23, 2015

Elizabeth Dabney and Rachel Anderson

12#CCRSMeasures

About DQC: Just the Facts

• Nonprofit, nonpartisan, national advocacy organization based in Washington, DC

• Launched in 2005 by 10 founding partners• Partnership network now comprises nearly 100

organizations• Governed by 11-member Board of Directors

13#CCRSMeasures

About DQC: Vision and Mission

Vision: DQC envisions an education system in which all stakeholders—from parents to policymakers—are empowered with high-quality data from the early childhood, K–12, postsecondary, and workforce systems to make decisions that ensure every student graduates high school prepared for success in college and the workplace.

Mission: DQC supports state policymakers and other key leaders to promote effective data use to improve student achievement.

14#CCRSMeasures

Start with Your Questions

15#CCRSMeasures

Start with Your Questions

16

What is the graduation rate by high school?

Which early childhood programs best prepare students for kindergarten?

To what degree are high school math grades predictors of readiness for college math?

What industries are most employing high school and college graduates?

How successful are college graduates in the workforce by major or credential?

Early Childhood K-12 Postsecondary Workforce

#CCRSMeasures

10 State Actions for Effective Data Use

17#CCRSMeasures

10 State Actions

LINKING P–20W Data1. Link P–20W data systems2. Create stable, sustained support3. Develop governance structures4. Build data repositories

ENSURING DATA ACCESS5. Provide timely data access6. Create individual student progress

reports 7. Create longitudinal reports

BUILDING CAPACITY TO USE DATA8. Develop research agenda 9. Build educator capacity10.Raise awareness of available data

18

Data For Action 2011

19#CCRSMeasures

Data For Action 2014

20#CCRSMeasures

Actions 2011 and 2014

UPDATED 2011 AND 2014 STATE

ACTIONS

21#CCRSMeasures

Action 1: Link P–20W data systems

22#CCRSMeasures

Action 1: 2011 and 2014

UPDATED 2011 AND 2014 STATE

ACTIONS

0

10

20

30

40

5038

11 14

43

1927

Number of states in 2011 Number of states in 2014

23#CCRSMeasures

Action 3: Develop governance structures

24#CCRSMeasures

Action 3: 2011 and 2014

UPDATED 2011 AND 2014 STATE

ACTIONS A cross-agency data governance committee/council is established with authority

0

10

20

30

40

50

39

43

Number of states in 2011 Number of states in 2014

25#CCRSMeasures

Key focus areas for high quality implementation of cross-agency data governance

Vision and missionComposition and membershipRoles and responsibilitiesData decisionsCommittee processesSustainability

26#CCRSMeasures

Barriers to Ensuring Effective Data Use:“The Four Ts”

Turf: The current culture and structures in education do not support working across

traditional boundaries

Trust: Skepticism about the quality and use of data

persists because data were previously used as a hammer

to punish rather than a flashlight to illuminate

Technical issues: Technical issues remain; however,

solutions are emerging and require the leadership and political will to implement

Time: Competing priorities and scarce resources present challenges to continuing to allocate adequate time to

building and using P20 data systems

27#CCRSMeasures

Privacy

28#CCRSMeasures

Legislative Activity in the States: 2013

1bill passed related

to student data privacy in

1 state

29#CCRSMeasures

Legislative Activity in the States: 2014

110bills related to student data privacy

in 36 states

30

Continuing Priorities

• Persistence of some myths (e.g., PARCC) • Concerns about commercialization of data• Congressional action• Increasing demands from districts for

legal/technical support• Concerns about linking data across

systems/sectors• The use of big data and analytics• Unintended consequences from 2014

31#CCRSMeasures

Legislative Activity in the States: 2015 As of March 20, most states have introduced privacy legislation.

We have seen more bills introduced in the first half of the 2015 legislative session than in the entire 2014 legislative session.

32#CCRSMeasures

States Are Not Wasting Time!

2015 legislation:• Picking up themes from last year: governance,

vendors, Common Core• Strong prohibitions could disrupt key

education services• Some encouraging news too!

33#CCRSMeasures

Please use the Questions box to ask questions of our presenters

Questions?

#CCRSMeasures 34

Hawai‘i Data eXchange PartnershipData Governance

CCRS Center and AYPF webinarMarch 23, 2015

Jean Osumi, EdD

35#CCRSMeasures

Infancy to early learning K-12 Postsecondary Workforce

Hawaiʻi Data eXchange Partnership

Kamehameha

Schools(private K-

12)

Hawai‘i private

postsecondariesDept of

Human Services

Dept of Health

GED

Hawai‘i Department of

Education

University of Hawai‘i System

Dept of Labor & Industrial Relations

HCAP Head Start, KCAA Preschools

POTENTIAL NEW DATA SOURCES:

Hawai‘i private

self-insured entities

36

#CCRSMeasures

Hawaiʻi DXP Underlying Assumptions

• Data used for research, evaluation, and audit purposes to improve the educational and workforce outcomes that benefit the citizens of Hawai‘i– “Pipeline” or “cross-sector” data– De-identified or aggregate data– Personally identified information only

used for matching individuals within and across sectors

37

#CCRSMeasures

Hawaiʻi Data eXchange Partnership

• General timeline

3-Party Data Sharing MOU (DOE-UH-DLIR)

Added Dept of Health & Dept of Human Services to Data Sharing MOU

Start of Hawaiʻi DXP infrastructure build

1st Qtr: rollout of public reports

2009 2011 2013 2016

2010 2012 2014Act 41 Data Sharing Across State Agencies

Awarded Hoʻokele Grant

4th Qtr: start transitioning Hawaiʻi DXP infrastructure from vendor to UH

38#CCRSMeasures

Structure

• 5 Party MOU (Partner agencies)– Hawai‘i State Dept of

Education– University of Hawai‘i System– Dept of Labor & Industrial

Relations– Dept of Health– Dept of Human Services

• Managing partner– Hawai‘i P-20 Partnerships

for Education

Data not currently shared with Hawai‘i DXP

39#CCRSMeasures

Data Governance and Access Committee

Data Governance

Process

What data?

How to request data?

Request approval process

Confidentiality training

Request forms

• Initial discussions all about processes

• Process meetings devolved into questions about access to data

40#CCRSMeasures

Initial Data Governance Discussions

• Data Access:– Eventually one representative proposed

that Partners should have access to identified data

• No committee member objected to the request

41#CCRSMeasures

Regrouping Data Governance

LACK OF UNDERSTANDING OF MOU ASSUMPTIONS

• How do we get everyone on the same page?

ACTION STRATEGY FOR COMMUNICATION AND TRANSPARENCY• Suspended further Data Governance & Access Committee

meetings• Held a series of meetings:

– Internal “data team”– Data governance representatives from Partner agencies

• Prepare for meetings by reading 2009 (3-party) and 2011 (5-party) MOUs– Intent and legal justification for data sharing across agencies

42#CCRSMeasures

Data Governance Clarification Meetings • Each group met to document their understanding

of data governance based on the MOUs– Internal “data team” meetings were critical

• Collaborated on a single written data governance document– Fostered “ownership” by Partner data governance

representatives– Built relationships and trust between Partner agencies– Identified each agency’s “hot button” confidentiality

issues

43#CCRSMeasures

Data Governance Policy

• Published on Hawai‘i DXP websitehttp://www.hawaiidxp.org/files/Hawaii_DXP_Data_Governance_Policy_2014-03-04.pdf

Includes:– Purpose– Underlying Assumptions– Partner and Affiliate

Definitions/Obligations/Access to Data– Roles & Responsibilities

44#CCRSMeasures

Lessons Learned

•The MOUs that establish the data sharing were critical to forming a common vision and understanding of this new resource

Policy First

•There needs to be a “gatekeeper” to enforce the intentions behind the MOUs – particularly around protecting confidentiality and privacy

“Gatekeeper” to Enforce MOU

•Anticipate constant review and changes to policy•e.g.,

role of DXP as only viewing “transition” points

Review and Change

45#CCRSMeasures

Please use the Questions box to ask questions of our presenters

Questions?

#CCRSMeasures 46

P-20W Data Governancein Washington

Melissa Beard, Data Governance CoordinatorMarch 23, 2015

47#CCRSMeasures

ERDC created in statute in 2007Mission: To develop longitudinal information spanning the early learning through

workforce system in order to facilitate analyses, provide meaningful reports, collaborate on education research, and share data.

K-12 Postsecondary Education Workforce

Early Learning

Social Services

#CCRSMeasures 48

Data Governance Structure

Education Research& Data Center

Data Steward Committee

Data Custodian Committee

Research & Reporting Coordination Committee

Office of Financial Management

#CCRSMeasures 49

Process resulted in an MOU with five principles

• ERDC provides linked, cross-sector data• ERDC maintains the P-20W data warehouse• Protecting privacy of individuals is a priority• Data contributors are the data experts• Common understanding and use of data

increases its value

#CCRSMeasures 50

Free the data!

• Protecting privacy is paramount• Focus on sharing de-identified data• Data contributors provide advice, not approval• Request process allows for two types of

review, which helps in protecting privacy– Data requests are reviewed by data contributors

prior to data being sent– Requestors provide data contributors report to

review 10 days before going public

#CCRSMeasures 51

Lessons Learned

• Each agency should have a data governance coordinator, preferably from business

• Data belongs to the state, not program managers

• DG process is to build trust and relationships• Communication was key; monthly large group

and individual meetings• Understand how each partner can benefit

#CCRSMeasures 52

Questions?

Melissa BeardState of Washington Office of Financial Management

Melissa.beard@ofm.wa.gov360-902-0584

www.erdc.wa.gov

53#CCRSMeasures

Please use the Questions box to ask questions of our presenters

Questions?

#CCRSMeasures 54

• Laura Jimenez, Director, CCRS Center

• Dorothyjean Cratty, Senior Researcher, AIR

• Elizabeth Dabney, Associate Director, Research and Policy Analysis, Data Quality Campaign

• Rachel Anderson, Senior Associate, Policy and Advocacy, Data Quality Campaign

• Jean Osumi, Senior Associate for Academic Policy and Evaluation, Hawai‘i P-20/Hawai’i Data Exchange Partnership

• Melissa Beard, Data Governance Coordinator/Higher Education Analyst, Washington Education Research and Data Center (ERDC)

Questions for our Presenters

#CCRSMeasures 55

Dorothyjean (DJ) Cratty, Senior Researcher, AIR

Closing Remarks

56#CCRSMeasures

Please fill out the survey upon exiting the webinar. Your feedback is important to us!

To learn more and access copies of the slides and event recording, please visit

www.ccrscenter.org

www.aypf.org

Thank you for attending

#CCRSMeasures 57