panayiota (pani) kendeou susan slater ted christ mary jane white · 2019. 8. 6. · the institute...

Panayiota (Pani) Kendeou Susan Slater Ted Christ Mary Jane White

Britta Bresina Reese Butterfuss Rina Hirsch Kyle Wagner Jasmine Kim Cristina Umana Soo-hyun Im Joseph Aubele

Kristen McMaster

Minnesota:Elizabeth LamAhmed AlghamdiEmma ShanahanNicole McKevettStacy BransjordJena HittSeyma Birinci

Missouri:Erica LembkeCarol GarmanKim MooreLizzie TiptonLizzie McCollomJessica SimpsonLaura Browning

What evidence do

you work with? Why?

“…practices or programs that have evidence to show that they are effective at

producing results and improving outcomes when implemented”

(Every Student Succeeds Act; https://www.cde.ca.gov/re/es/evidence.asp)

Strong Evidence: ≥ 1 well-designed, well-implemented randomized control experimental studies.Moderate Evidence: ≥ 1 well-designed and well-implemented quasi-experimental studies.

Promising Evidence: ≥ 1 well-designed and well-implemented correlational studies.

Evidence collected in research – generalizable outcomes from “well designed, well implemented” studies showing what works, for whom, and under what conditions

Evidence collected in practice – particular data showing whether a practice works for specific students in a specific context

•

•

-24.64

-4.6

-20.69

-25.28

8.37

-3.87

-27.98-25.65

-18.84

-30

-25

-20

-15

-10

-5

0

5

10

15

MN ACHIEVEMENT GAP: READING MCA (SCORES COMPARED TO AVERAGE)

Source: MDE; https://www.twincities.com/2018/08/30/another-new-way-of-grading-schools-but-achievement-gaps-persist/

Evidence collected in research – generalizable outcomes from “well designed, well implemented” studies showing what works, for whom, and under what conditions

Context

Theory

Empirical Data Empirical Data Empirical Data

Empirical Data Empirical Data

Empirical Data Empirical Data Empirical Data

Empirical Data

Empirical Data

“There’s nothing so practical as good

theory.” ~ Kurt Lewin (social psychologist)

“In god we trust, others must

provide data.” ~ Edwin R. Fisher

(Professor of Pathology)

“Always design a thing by considering it in its next larger context –

a chair in a room,a room in a house,

a house in an environment….”

~ Eliel Saarinen (Finnish architect)

PI: Panayiota (Pani) Kendeou

The research reported here is supported by the Institute of Education Sciences, U.S. Department of

Education, through Grant R324A160064 to the Regents of the University of Minnesota.

•

•

•

•

•

Theory

Theory

•

•

Theory

Year 1: developed, tested usability

Year 2: looked at feasibility for

classroom use

Year 3: pilot study

Sep Oct Nov Dec Jan Feb Mar Apr May

Eligible n = 124

CELF-

USP< 8

Teacher Training n = 11

TeLCI n = 61

Control n = 60

TeLCI 8 weeks of intervention

4x 25 min

Control 8 weeks of BAU instruction

Mage= 6.97 50% Female 48% Hispanic 17% Special Ed 46% ELL 77% FRL

MIA – Proximal

MIA – Proximal CELF – USP CELF –WC Early Reading GMRT

CELF –USP MEFS

Fidelity Observations 74.5%

Kendeou, McMaster, & the TeLCI Team, 2019

Time1 Time2 Time3 Time4

Attempt 1 accuracy (%) across Time

STEM 45.47 45.73 50 50.4

Centennial 46.22 44.96 51.44 55.5

Overall 49.91 45.27 50.86 53.44

30

40

50

60

70

80

90

Overa

ll A

ve

rag

e A

ccu

racy (

%)

TeLCI Yr3 Scores without Scaffolding

Time1 Time2 Time3 Time4

STEM 74.67 75.87 74.8 74.13

Centennial 74.96 74.5 75.77 80.09

Overall 74.84 75.05 75.38 77.69

30

40

50

60

70

80

90

Overa

ll A

ve

rag

e A

ccu

racy (

%)

TeLCI Yr3 Scores with Scaffolding

Kendeou, McMaster, & the TeLCI Team, 2019

Kendeou, McMaster, & the TeLCI Team, 2019

Evidence collected in practice – particular data showing whether a practice works for specific students in a specific context

Co-PI: Erica Lembke, University of Missouri, Columbia

The research reported here is supported by the Institute of Education Sciences, U.S. Department of

Education, through Grants R324A130144 and R324A160064to the Regents of the University of Minnesota.

Data-Based Instruction

Is… Is not…

• A framework for making instructional decisions

• A dynamic process of ongoing assessment and intervention

• A curriculum• An

assessment• A single

intervention

0

5

10

15

20

25

30

35

40

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

Co

rre

ct W

ord

Se

qu

en

ces:

3 m

in

Molly's Picture-Word Progress

Baseline

0

5

10

15

20

25

30

35

40

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

Co

rre

ct W

ord

Se

qu

en

ces:

3 m

in

Molly's Picture-Word Progress

Baseline

Goal

Goal Line

0

5

10

15

20

25

30

35

40

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

Co

rre

ct W

ord

Se

qu

en

ces:

3 m

in

Molly's Picture-Word Progress

Intervention 1

Baseline

Goal

Goal Line

0

5

10

15

20

25

30

35

40

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

Co

rre

ct W

ord

Se

qu

en

ces:

3 m

in

Molly's Picture-Word Progress

Intervention 1

Baseline

Goal

Goal Line

Trend line

0

5

10

15

20

25

30

35

40

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

Co

rre

ct W

ord

Se

qu

en

ces:

3 m

in

Molly's Picture-Word Progress

Intervention 1

Baseline

Goal

Goal Line

Trend line

Examine the trend line Select one of three

decisions:

The trend line is steeper than the goal line

The trend line is flatter than the goal line

The trend line is even with the goal line

Increase the goal

Change instruction

Continue as is

0

5

10

15

20

25

30

35

40

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

Co

rre

ct W

ord

Se

qu

en

ces:

3 m

in

Molly's Picture-Word Progress

Intervention 1

Baseline

Goal

Goal Line

Trend line

0

5

10

15

20

25

30

35

40

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

Co

rre

ct W

ord

Se

qu

en

ces:

3 m

in

Molly's Picture-Word Progress

Intervention 1 Intervention 2

Baseline

Goal

Goal Line

Trend line

0

5

10

15

20

25

30

35

40

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

Co

rre

ct W

ord

Se

qu

en

ces:

3 m

in

Molly's Picture-Word Progress

Intervention 1 Intervention 2 Intervention 3 Intervention 4

Baseline

Goal

Goal Line

Trend line

•

•

•

•

Theory of Change

Providing PD in DBI

improves…

Teachers’ DBI

knowledge & skills

Teachers’ self-efficacy

Teachers’ fidelity of:

*Assessment

*Intervention

*Decision-making

Student Early Writing

Outcomes

Teac

he

rs (

n=

20

) S

tud

ents

(n

= 5

7) DBI-TLC

Control

Pre

-tes

t

Business as usual

Po

st-t

est

September October - March March June

Minnesota &

Missouri

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

Pre Post

Pre-Post Change on DBI Knowledge & Skills

Control MN Control MO Control Total

DBI-TLC MN DBI-TLC MO DBI-TLC Total

Time p = .016

ES = 3.05

Theory of Change

Teachers’ DBI Knowledge and Skills

Theory of Change

0

1

2

3

4

5

6

Personal efficacy General efficacy Writing orientationCorrect

Writing orientationExplicit

Writing orientationNatural

Teacher Efficacy & Writing Orientation at Posttest (No significant pre-test differences)

DBI Control

p = .052ES = 1.02

p = .726ES = -.17

p = .512ES = .32

p < .01ES = 1.67

p = < .05ES = -1.06

Teachers’ DBI Knowledge and Skills

Teachers’ Self-Efficacy

Theory of Change

Teachers’ DBI Knowledge and Skills

Teachers’ Self-Efficacy

Teachers’ Fidelity to DBI

•

•

•

Theory of Change

Teachers’ DBI Knowledge and Skills

Teachers’ Self-Efficacy

Teachers’ Fidelity to DBI

0

5

10

15

20

25

30

PW_WW PW_WSC PW_CWS PW_CIWS

Adjusted Posttest Mean on CBM-Picture-Word by Condition(Pretest scores were used as covariates)

control DBI

p = .448ES = .225 p = .435

ES = .232 p = .292ES = .312

p = .279ES = .328

Students’ early writing

outcomes

Teac

he

rs (

n=

140

) S

tud

ents

(n

~ 3

00

-40

0)

DBI-TLC

Control

Pre

-tes

t

Business as usual

Po

st-t

est

September October - March March May

Minnesota & Missouri

3 teacher/studentcohorts across 3 years

Evidence collected in research – generalizable outcomes from “well designed, well implemented” studies showing what works, for whom, and under what conditions

Evidence collected in practice – particular data showing whether a practice works for specific students in a specific context

Context

Theory

Empirical Data Empirical Data Empirical Data

Empirical Data Empirical Data

Empirical Data Empirical Data Empirical Data

Empirical Data

Empirical Data

What about

context?

The University of Minnesota is an equal opportunity educator and employer.

mcmas004@umn.edu

@kristenmcmaster