panayiota (pani) kendeou susan slater ted christ mary jane white · 2019. 8. 6. · the institute...
TRANSCRIPT
Panayiota (Pani) Kendeou Susan Slater Ted Christ Mary Jane White
Britta Bresina Reese Butterfuss Rina Hirsch Kyle Wagner Jasmine Kim Cristina Umana Soo-hyun Im Joseph Aubele
Kristen McMaster
Minnesota:Elizabeth LamAhmed AlghamdiEmma ShanahanNicole McKevettStacy BransjordJena HittSeyma Birinci
Missouri:Erica LembkeCarol GarmanKim MooreLizzie TiptonLizzie McCollomJessica SimpsonLaura Browning
What evidence do
you work with? Why?
“…practices or programs that have evidence to show that they are effective at
producing results and improving outcomes when implemented”
(Every Student Succeeds Act; https://www.cde.ca.gov/re/es/evidence.asp)
Strong Evidence: ≥ 1 well-designed, well-implemented randomized control experimental studies.Moderate Evidence: ≥ 1 well-designed and well-implemented quasi-experimental studies.
Promising Evidence: ≥ 1 well-designed and well-implemented correlational studies.
Evidence collected in research – generalizable outcomes from “well designed, well implemented” studies showing what works, for whom, and under what conditions
Evidence collected in practice – particular data showing whether a practice works for specific students in a specific context
•
•
-24.64
-4.6
-20.69
-25.28
8.37
-3.87
-27.98-25.65
-18.84
-30
-25
-20
-15
-10
-5
0
5
10
15
MN ACHIEVEMENT GAP: READING MCA (SCORES COMPARED TO AVERAGE)
Source: MDE; https://www.twincities.com/2018/08/30/another-new-way-of-grading-schools-but-achievement-gaps-persist/
Evidence collected in research – generalizable outcomes from “well designed, well implemented” studies showing what works, for whom, and under what conditions
Context
Theory
Empirical Data Empirical Data Empirical Data
Empirical Data Empirical Data
Empirical Data Empirical Data Empirical Data
Empirical Data
Empirical Data
“There’s nothing so practical as good
theory.” ~ Kurt Lewin (social psychologist)
“In god we trust, others must
provide data.” ~ Edwin R. Fisher
(Professor of Pathology)
“Always design a thing by considering it in its next larger context –
a chair in a room,a room in a house,
a house in an environment….”
~ Eliel Saarinen (Finnish architect)
PI: Panayiota (Pani) Kendeou
The research reported here is supported by the Institute of Education Sciences, U.S. Department of
Education, through Grant R324A160064 to the Regents of the University of Minnesota.
•
•
•
•
•
Theory
Theory
•
•
Theory
Year 1: developed, tested usability
Year 2: looked at feasibility for
classroom use
Year 3: pilot study
Sep Oct Nov Dec Jan Feb Mar Apr May
Eligible n = 124
CELF-
USP< 8
Teacher Training n = 11
TeLCI n = 61
Control n = 60
TeLCI 8 weeks of intervention
4x 25 min
Control 8 weeks of BAU instruction
Mage= 6.97 50% Female 48% Hispanic 17% Special Ed 46% ELL 77% FRL
MIA – Proximal
MIA – Proximal CELF – USP CELF –WC Early Reading GMRT
CELF –USP MEFS
Fidelity Observations 74.5%
Kendeou, McMaster, & the TeLCI Team, 2019
Time1 Time2 Time3 Time4
Attempt 1 accuracy (%) across Time
STEM 45.47 45.73 50 50.4
Centennial 46.22 44.96 51.44 55.5
Overall 49.91 45.27 50.86 53.44
30
40
50
60
70
80
90
Overa
ll A
ve
rag
e A
ccu
racy (
%)
TeLCI Yr3 Scores without Scaffolding
Time1 Time2 Time3 Time4
STEM 74.67 75.87 74.8 74.13
Centennial 74.96 74.5 75.77 80.09
Overall 74.84 75.05 75.38 77.69
30
40
50
60
70
80
90
Overa
ll A
ve
rag
e A
ccu
racy (
%)
TeLCI Yr3 Scores with Scaffolding
Kendeou, McMaster, & the TeLCI Team, 2019
Kendeou, McMaster, & the TeLCI Team, 2019
Evidence collected in practice – particular data showing whether a practice works for specific students in a specific context
Co-PI: Erica Lembke, University of Missouri, Columbia
The research reported here is supported by the Institute of Education Sciences, U.S. Department of
Education, through Grants R324A130144 and R324A160064to the Regents of the University of Minnesota.
Data-Based Instruction
Is… Is not…
• A framework for making instructional decisions
• A dynamic process of ongoing assessment and intervention
• A curriculum• An
assessment• A single
intervention
0
5
10
15
20
25
30
35
40
1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May
Co
rre
ct W
ord
Se
qu
en
ces:
3 m
in
Molly's Picture-Word Progress
Baseline
0
5
10
15
20
25
30
35
40
1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May
Co
rre
ct W
ord
Se
qu
en
ces:
3 m
in
Molly's Picture-Word Progress
Baseline
Goal
Goal Line
0
5
10
15
20
25
30
35
40
1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May
Co
rre
ct W
ord
Se
qu
en
ces:
3 m
in
Molly's Picture-Word Progress
Intervention 1
Baseline
Goal
Goal Line
0
5
10
15
20
25
30
35
40
1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May
Co
rre
ct W
ord
Se
qu
en
ces:
3 m
in
Molly's Picture-Word Progress
Intervention 1
Baseline
Goal
Goal Line
Trend line
0
5
10
15
20
25
30
35
40
1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May
Co
rre
ct W
ord
Se
qu
en
ces:
3 m
in
Molly's Picture-Word Progress
Intervention 1
Baseline
Goal
Goal Line
Trend line
Examine the trend line Select one of three
decisions:
The trend line is steeper than the goal line
The trend line is flatter than the goal line
The trend line is even with the goal line
Increase the goal
Change instruction
Continue as is
0
5
10
15
20
25
30
35
40
1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May
Co
rre
ct W
ord
Se
qu
en
ces:
3 m
in
Molly's Picture-Word Progress
Intervention 1
Baseline
Goal
Goal Line
Trend line
0
5
10
15
20
25
30
35
40
1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May
Co
rre
ct W
ord
Se
qu
en
ces:
3 m
in
Molly's Picture-Word Progress
Intervention 1 Intervention 2
Baseline
Goal
Goal Line
Trend line
0
5
10
15
20
25
30
35
40
1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May
Co
rre
ct W
ord
Se
qu
en
ces:
3 m
in
Molly's Picture-Word Progress
Intervention 1 Intervention 2 Intervention 3 Intervention 4
Baseline
Goal
Goal Line
Trend line
•
•
•
•
Theory of Change
Providing PD in DBI
improves…
Teachers’ DBI
knowledge & skills
Teachers’ self-efficacy
Teachers’ fidelity of:
*Assessment
*Intervention
*Decision-making
Student Early Writing
Outcomes
Teac
he
rs (
n=
20
) S
tud
ents
(n
= 5
7) DBI-TLC
Control
Pre
-tes
t
Business as usual
Po
st-t
est
September October - March March June
Minnesota &
Missouri
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
Pre Post
Pre-Post Change on DBI Knowledge & Skills
Control MN Control MO Control Total
DBI-TLC MN DBI-TLC MO DBI-TLC Total
Time p = .016
ES = 3.05
Theory of Change
Teachers’ DBI Knowledge and Skills
Theory of Change
0
1
2
3
4
5
6
Personal efficacy General efficacy Writing orientationCorrect
Writing orientationExplicit
Writing orientationNatural
Teacher Efficacy & Writing Orientation at Posttest (No significant pre-test differences)
DBI Control
p = .052ES = 1.02
p = .726ES = -.17
p = .512ES = .32
p < .01ES = 1.67
p = < .05ES = -1.06
Teachers’ DBI Knowledge and Skills
Teachers’ Self-Efficacy
Theory of Change
Teachers’ DBI Knowledge and Skills
Teachers’ Self-Efficacy
Teachers’ Fidelity to DBI
•
•
•
Theory of Change
Teachers’ DBI Knowledge and Skills
Teachers’ Self-Efficacy
Teachers’ Fidelity to DBI
0
5
10
15
20
25
30
PW_WW PW_WSC PW_CWS PW_CIWS
Adjusted Posttest Mean on CBM-Picture-Word by Condition(Pretest scores were used as covariates)
control DBI
p = .448ES = .225 p = .435
ES = .232 p = .292ES = .312
p = .279ES = .328
Students’ early writing
outcomes
Teac
he
rs (
n=
140
) S
tud
ents
(n
~ 3
00
-40
0)
DBI-TLC
Control
Pre
-tes
t
Business as usual
Po
st-t
est
September October - March March May
Minnesota & Missouri
3 teacher/studentcohorts across 3 years
Evidence collected in research – generalizable outcomes from “well designed, well implemented” studies showing what works, for whom, and under what conditions
Evidence collected in practice – particular data showing whether a practice works for specific students in a specific context
Context
Theory
Empirical Data Empirical Data Empirical Data
Empirical Data Empirical Data
Empirical Data Empirical Data Empirical Data
Empirical Data
Empirical Data
What about
context?
The University of Minnesota is an equal opportunity educator and employer.
@kristenmcmaster