trend data guest speakers: pattie johnson, tri sally simich, ode

26
Trend Data Guest Speakers: Pattie Johnson, TRI Sally Simich, ODE

Upload: jovan

Post on 22-Feb-2016

57 views

Category:

Documents


0 download

DESCRIPTION

Trend Data Guest Speakers: Pattie Johnson, TRI Sally Simich, ODE. Why Use Trend Data . Visual representation of multiple years of data to reveal a pattern of gradual change over time To convey data clearly and accurately To reveal or see patterns in the data - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Trend DataGuest Speakers:

Pattie Johnson, TRISally Simich, ODE

Page 2: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Why Use Trend Data • Visual representation of multiple years of data to

reveal a pattern of gradual change over time• To convey data clearly and accurately • To reveal or see patterns in the data • To motivate an audience to use data • To facilitate accurate and efficient interpretations

of the data while minimizing the likelihood of misrepresenting data

• Many individuals learn better through pictures than words; graphs help people remember the information

Page 3: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Graphs are Common • Inaccurate scale or axis• Wrong type of graph • Too much or too little data

being displayed

Bad Graph Contestants Forbes.com 2012

Page 4: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Considerations for Displaying Data

• Scale should be to 100 when showing percents • Avoid 3D graphs • Consider the audience

▫ More sophisticated audience = more sophisticated graph • Consider the task

▫ What do you want the audience to do with the information Know a specific percentage or number

(e.g., 79% of youth with disabilities are engaged one year after leaving high school)

Make comparisons (e.g., males are more engaged than females)

See trends and patterns over time (e.g., since 2010 the percent of youth engaged has steadily increased).

Page 5: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Questions Guiding a Trend Analysis

•How representative are these data? •What direction are our outcomes going?•Are there differences in outcomes by subgroups? ▫Gender, Disability, Method of Exit, and

Ethnicity•What is contributing to our outcomes?•How can we use the information?

Page 6: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

» Let’s look at some overall data trends across the four years of consistent data collection

Page 7: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

PSO Data Collection

» All districts are required to participate each year. Student selection is based on a stratified sample designed to generate a representative sample of leavers to ensure generalizable information.

» Districts select who will conduct the interviews, and data entry is open from June through September. Interviews can be completed with young adult or their family members.

» Oregon’s PSO data have consistently been representative on all subgroups except dropouts

7

Page 8: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Response Rate2010 2011 2012 2013

0

500

1000

1500

2000

2500

3000

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

27182779 2714 2660

1,9111,989

1,7481,914

Sample Completed Response Rate

Sample Leavers, Response Rate, and Completed Interviews for Four Years

Page 9: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Hi Ed Comp Empl Other School Other Emp Not Engaged0

20

40

60

80

100

2426

7 8

34

25

29

6 7

32

25

30

79

2826

29

69

30

Trend PSO Data for Oregon

2010 2011 2012 2013

Outcome Category

Perc

ent o

f Res

pond

ents

Page 10: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

What do we see in the trends?•Higher Education initial increase, then static•Competitive Employment Increasing

▫More Oregon leavers employed than in education

•Other School and Other Work relatively unchanged

•Not Engaged rate decreasing – right direction – until last year

•Why look further? ▫To target resources to specific needs

Page 11: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Are there differences in outcomes by subgroups?

Gender x 3 years Disability categories x 3 years Ethnicity categories x 3 years Method of Exit x 3 years

Page 12: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Very Small Small Medium Large0%

20%

40%

60%

80%

100%

52%

62%64%

69%

52%

62%64%

69%71% 71% 71%

73%

80%

71% 72%

67%

Overall Engagement Rate by District Size2010 2011 2012 2013

District Size Breakdown

Perc

ent o

f Res

pond

ents

Page 13: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Looking at Data: Process summary » How representative are these data?

˃ Explore the response size and how the subgroups matched the population

» What direction are the outcomes going?˃ Look at graphs showing performance, trends, and

comparisons

» Examine outcomes by subgroups.• Work from general overview to more specific components

• Explore what is contributing to the outcomes.• Look at a combination of components; extra questions asked as

part of the survey; collect or examine other data

Page 14: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Two Follow Up Report Formats

District level summary reports

» Engagement Report includes information on who is engaged by subgroup and if the sample of students interviewed matches the total district leavers

» Interview Summary Report summarize your interview data, including the open ended responses given by the interviewee.

Page 15: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Using the Reports

» Monitor your progress on the 2014 data collection – constantly updated

» Compare current district response rate, engagement rate, and representativeness to final results from 2012, 2011, and 2010

» Examine results of changes the district is making in transition services over time

Page 16: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

2013 Engagement Report

.

16

Goal

Page 17: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Sample District – subgroup 17

Page 18: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Sample District Data– Representative? 18

Page 19: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

District and State comparisons

Compare results with like sized

districts

Page 20: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Questions for the district to consider when looking at reports

• What do the most recent data show?• Can we make district-wide statements using our results,

or have we missed dropouts or other groups?

• Are our students going on to school within a year?

• Are they able to get and keep jobs for at least 3 months?• What percentage of our students are actively working or

learning within a year of leaving?• Do we have more than a third of our leavers who haven’t

done any work or training after leaving high school?

• Are we getting better at launching successful students?

Page 21: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

» How is ODE holding districts accountable for student PSO?

» PSO is separate application available to secondary transition staff at district level.

» Needed administrative audience

» Now PSO will be included with monitoring indicators with notification if district failed to meet state targets

Page 22: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

SPR&I Monitoring Report on Secondary Transition Indicators: PSO

Overall engagement rate: Percent enrolled in higher education, competitively employed within one year of leaving high school, in some other postsecondary education or training program or in some other employment. [(# of youth who are no longer in secondary school, had IEPs in effect at the time they left school and were enrolled in higher education, or in some other postsecondary education or training program; or competitively employed or in some other employment) divided by the (# of respondent youth who are no longer in secondary school and had IEPs in effect at the time they left school)] times 100.

Page 23: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

SPR&I Monitoring Report on Secondary Transition Indicators: PSO

Page 24: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Accountability• What Is working for the district? • Where is more effort needed?

Administrators will need to explore the reports on the Post School Outcomes Application to learn about subgroup results and trends:

• Disability • Method of exit • Gender • Race/Ethnicity • Like-sized districts

Page 25: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

Next Steps for Districts •Share and discuss data with

stakeholders • Identify areas of strength and areas

for improvement •Use data to determine what changes

are needed

Page 26: Trend Data Guest Speakers:  Pattie Johnson, TRI Sally Simich, ODE

For more information:

Sally SimichTransition Specialist, Oregon Department of [email protected] 503-947-5639

Pattie JohnsonTeaching Research Institute, Western Oregon [email protected] 503-838-8779

Charlotte Y. AlversonNational Post School Outcomes Center, University of [email protected] 541-346-1390