the impact of writing letters to older peers
DESCRIPTION
The impact of writing letters to older peers. Shu-Ching Chang, Kun Chen 04/21/2010. Research Purpose. To examine the impact of evidence-based letter-writing tasks on students’ conceptual understanding of some physics concepts. - PowerPoint PPT PresentationTRANSCRIPT
The impact of writing letters to older peers
Shu-Ching Chang, Kun Chen04/21/2010
Research Purpose
• To examine the impact of evidence-based letter-writing tasks on students’ conceptual understanding of some physics concepts.
• In what way students benefit from the task of writing to different audiences.
Question
• Do students who engage in writing letters to older peers perform better than students who do not?
Research Design
• Two-year project.• SchoolClassStudents• Control and treatment groups.• Pre-/post-/delayed tests.• Tests: 20 multiple choice questions based upon 6
core concepts. ( Developed by Horizon Research Inc.)
Core ConceptsCore Concept Content
A (9 items) A force is any push or pull on an object.
B (3 items) A force can change the speed of an object.
C (2 items) Friction is a force that occurs when two surfaces rub together.
D (2 items) If the same force is applied to a lighter vehicle and a heavier vehicle, the speed of the lighter vehicle will change more that the speed of the heavier vehicle.
E (2 items) Energy can be stored in a rubber band and released to turn an axle or spin a propeller to make a vehicle move.
F (2 items) Air resistance is a force that can slow the speed of a moving vehicle.
Writing Task—Three letters exchange between Grade 4 and Grade 11
Grade 11: Design a question to ask based on the concepts that will be needed by the 4th graders to complete the final design challenge.
Grade 4: Make a hypothesis and/or claim to answer the question
Letter 1
Letter 2
Grade 11: This letter will evaluate, critique the rationale, and suggest revisions to the elementary students’ ideas.
Grade 4: Make a new hypothesis and/or claim to answer the question.
Letter 3
Grade 11: Continue to evaluate, critique the rationale, and suggest revisions to the elementary students’ ideas.Grade 4: Same as response 2.
Guidelines for Grade 4• Answer the question in the letter from your high school
partners and explain your thinking by using supporting evidence.
• Evidence can come from your experience and/or from the investigations you are conducting in science class. It can include the data you collect in experiments and your observations.
• Consider different ways to explain your thinking about the concepts of force and motion. How could you use diagrams, pictures, charts or graphs to help explain your ideas?
• If your high school partners suggest an experiment to test one of your ideas, think about how you can control the variable so that you have a fair test.
Data: participants
• 4 elementary Schools in a north-eastern state of the U.S.A• 27 Teachers. Overlap in two years. Not randomly assigned.• Students are different from year to year. 4th graders.
Control Group
Treatment Group Total Number
1st Year 309 145 454
2nd Year 68 316 384
Total 377 461 838
–Method (1: control 2: treatment)– School – Teacher– Student– Pretest Score– Posttest Score– Score type (0:pre 1:post 2:delayed)– Improvement: Post – Pre– Year (Combined)– Delayed test (Not available)
Data: variables
Data Analysis
• ANCOVA (From our client) ---combing two years as a whole ---design structure is totally ignored.
• Split-Plot Analysis --- can we combine the two years’ data? ---schoolteacher/classesstudents
ANCOVA Result
Control Group Treatment Group
n Mean S.D. n Mean S.D.
Pretest 377 7.98 2.874 461 7.93 2.663
Posttest 377 10.79 3.105 461 11.47 2.887
Control Group Treatment Group
n
Adjusted
Mean S.E. n
Adjusted
Mean S.E.
Student
Achievement377 10.77 .14 461 11.477 .127
F(1, 835)= 13.699, p<.000, effect size=0.25
Split Plot Analysis
• Design schoolteacher/classesstudents
• Combine two years’ data? --Students are different in two years. --Tests are standardized. -- Treatments are the same. -- Year effect is not significant.
SPA: Model 1
• Model1 :
studentzteacherrandom
treatmentcontrolmethodschoolfixed
effect
test scorecore - preposttest s diff y
iable: depedent
lkjiwhere
zy kjilklijkijjiijkl
::
):2:1(::
:
)(
var
838,,127,,12,14,,1
,),,()()(
SPA: Model 1
– SAS code:proc mixed data=data1; class School Teacher Student Method; model diff=School|Method ddfm = satterth outp=diags; random Teacher(School Method) Student(Teacher); LSmeans School*Method / pdiff adj=Tukey; LSmeans School;run;
SPA: Model 1
– Result:• there is no significant difference of the score
improvement for treatment and control groups in different schools.
Type 3 Tests of Fixed Effects
Effect NumDf
Den
Df F value Pr > F
SchoolMethod
School*Method
313
34.234.234.2
2.124.330.67
0.11610.04500.5777
SPA: Model 2
• Model2 :
studentzteacherrandom
treatmentcontrolmethodyearfixed
effect
test scorecore - preposttest s diff y
iable: depedent
lkjiwhere
zy kjilklijkijjiijkl
::
):2:1(::
:
)(
var
838,,127,,12,12,1
,),,()()(
SPA: Model 2
– SAS code:proc mixed data=data1; class School Teacher Student Method Year; model diff=Year|Method ddfm = satterth outp=diags; random Teacher(Method) Student(Teacher);run;
SPA: Model 2
– Result:• there is no significant difference of the score
improvement for treatment and control groups for two years.
Type 3 Tests of Fixed Effects
Effect NumDf
Den
Df F value Pr > F
YearMethod
Year*Method
111
43.8109109
1.270.760.08
0.26590.38640.7728
SPA: Model 3
• Model3:
studentzteacherrandom
postprescorecode
trtcontmethodfixed
effects
posttest)retest and scores (py
iabledependent
,l,,k,j,iwhere
zry kjilklikijjiijkl
::
):1:0(:
):2:1(:
:
:var
838,12712121
,)( ),,()()(
SPA: Model 3
– SAS code:proc mixed data=data2; class Teacher Method Student Scorecode ; model Score=Scorecode|Method ddfm = satterth outp=diags; random Teacher(Method) Student(Teacher); LSmeans Scorecode*Method / pdiff adj=Tukey; estimate "12-24" scorecode*method 1 -1 -1 1run;
SPA: Model 3
• From the least square means, holding method fixed, the difference of pre and post tests is significant for control and treatment groups.
Diff of Least Square
Means
Effect M S _M _S Estimate pvalueMethod(M)*Scorecode(S)Method(M)*Scorecode(S)
12
00
12
11
-2.809-3.538
<.0001<.0001
SPA: Model 3
– Results:• the interaction term of method and score-type turns
out to be significant.
Type 3 Tests of Fixed Effects
Effect NumDf
Den
Df F value Pr > F
ScorecodeMethod
Method*Scorecode
111
83637.4836
859.020.89
11.33
< .00010.35270.0008
SPA: Model 3
• the improvement for students in the treatment group is significantly higher than the improvement for students from the control group. • This implies that new method seems to increase the
critical thinking skills.
Estimate
Label Estimate Standard
Erroe DF t value Pr > |t|
12-24 0.7289 0.2166 836 3.37 0.0008
SPA: Model 4
• Model4(add school effect to model 3):
studentzteacherrandom
schoolfscorecodemethodfixed
effects
posttest)retest and scores (py
iable:dependent
lkmjiwhere
zfy mkjilklikmijjimlkji
::
:::
:
var
838,,127,,14,,12,12,1
)( ),,,()()(,,,,
SPA: Model 4
– SAS code:
proc mixed data=data2; class School Teacher Method Student Scorecode ; model Score=Scorecode|Method School/ ddfm = satterth outp=diags; random Teacher(Method) Student(Teacher); LSmeans Scorecode*Method / pdiff adj=Tukey; estimate "12-24" scorecode*method 1 -1 -1 1;run;
SPA: Model 4– Results:• the school effect is not significant . • after adjusting for school effect, the interaction
between method and score-type is significant.
Type 3 Tests of Fixed Effects
Effect NumDf
Den
Df F value Pr > F
ScorecodeMethod
Method*ScorecodeSchool
1113
83634.783634.4
859.021.02
11.330.92
< .00010.31860.
00080.4409
SPA: Model 4
• the improvement for students in the treatment group is significantly higher than the improvement for students from the control group. • This implies that new method seems to increase the
critical thinking skills.
Estimate
Label Estimate Standard
Error DF t value Pr > |t|
12-24 0.7289 0.2166 836 3.37 0.0008
Conclusion & Discussion
• Students who engage in writing letters to older peers performed better than students who do not.
• Teachers are not randomly assigned.• Delayed test.