sapsi
TRANSCRIPT
1
IL-ASPIRE Evaluation Project
Violeta CarriónStatewide ASPIRE Evaluation Coordinator
Phone: 312-915-7082
Hank BohanonHank BohanonStatewide ASPIRE Evaluation Director
Center for School Evaluation, Intervention and Training (CSEIT)Loyola University Chicago
820 N. Michigan Avenue, Chicago, IL 60611
Evaluation goals across ISBE projects
• Coordinating evaluation of technical assistance projects, with common evaluation…
• Questions– If people are trained, do they implement?– If people implement, do they do it with integrity?– If people implement with integrity, do they sustain?– If interventions sustain, what is the impact on students?
• Components• Processes
– Practice and training– Systems in place– Outcome data
Self-Assessment of Problem Solving Implementation
(SAPSI v2 1)(SAPSI v2.1)
2
3
Preliminary Results
4
5
9
68
10
quen
cy
SAPSI Total ScoresIn Progress AchievedNot Started
Ma
inta
inin
1 1
4
3
2
1
02
4Fre
q
0 8 16 24 32 40 48 56 64 72 80 88 96SCORE
ng
14
8
10
15
equ
ency
In Progress AchievedNot Started
Ma
inta
inin
3
1
05
Fre
0 2 4 6 8 10saps1
ng
13
10
15
requ
ency
In Progress AchievedNot Started
1
5 5
2
05
F
0 5 10 15 20saps2
5
7
11
51
0eq
uen
cy
In Progress AchievedNot Started
Ma
inta
inin
2
1
05
Fre
0 5 10 15 20saps3
ng
5
12
9
10
15
equ
ency
In Progress AchievedNot Started
2
3
05
Fre
0 2 4 6saps4
6
12
10
15
requ
ency
In Progress AchievedNot Started
Ma
inta
inin
g
3
6
4
1
05
Fr
0 2 4 6 8 10saps5
g
11
7
6
10
uen
cy
In ProgressNot Started
2
05
Fre
qu
0 2 4 6 8 10saps6
• Test scale = mean(unstandardized items)• average• item-test item-rest inter-item• Item | Obs Sign correlation correlation covariance alpha• -------------+-----------------------------------------------------------------• q1 | 26 + 0.7209 0.4451 .1821795 0.5931• q2 | 26 + 0.7975 0.5809 .1385897 0.4920• q3 | 26 + 0.7629 0.5370 .1592308 0.5286• q4 | 26 + 0.5207 0.2261 .2833333 0.7193• -------------+-----------------------------------------------------------------• Test scale | .1908333 0.6616• -------------------------------------------------------------------------------
• Test scale = mean(unstandardized items)• average• item-test item-rest inter-item• Item | Obs Sign correlation correlation covariance alpha• Item | Obs Sign correlation correlation covariance alpha• -------------+-----------------------------------------------------------------• q5 | 26 + 0.7772 0.6931 .1972527 0.8191• q6 | 26 + 0.8447 0.7757 .1833516 0.8069• q7 | 26 + 0.8526 0.7841 .180348 0.8052• q8 | 26 + 0.7624 0.6537 .1902381 0.8231• q9 | 26 + 0.5150 0.3674 .2300183 0.8569• q10 | 26 + 0.4404 0.3461 .247381 0.8548• q11 | 26 + 0.6558 0.5256 .2089194 0.8395• q12 | 26 + 0.6756 0.5528 .2067766 0.8360• -------------+-----------------------------------------------------------------• Test scale | .2055357 0.8495• -------------------------------------------------------------------------------
6
Test scale = mean(unstandardized items)
averageitem-test item-rest inter-item
Item | Obs Sign correlation correlation covariance alpha-------------+-----------------------------------------------------------------q13a | 26 + 0.7722 0.7023 .4587692 0.9248q13b | 26 + 0.8991 0.8547 .4027692 0.9097q13c | 26 + 0.8616 0.7982 .4049231 0.9159q13d | 26 + 0.8991 0.8547 .4027692 0.9097q13e | 26 + 0.8213 0.7642 .4481026 0.9198q13f | 26 + 0.7891 0.7051 .4321026 0.9249q13g | 26 + 0.8260 0.7566 .4254359 0.9196-------------+-----------------------------------------------------------------Test scale | .4249817 0.9289-------------------------------------------------------------------------------
Test scale = mean(unstandardized items)
averageitem-test item-rest inter-item
Item | Obs Sign correlation correlation covariance alpha-------------+-----------------------------------------------------------------q14 | 26 + 0.7678 0.5583 .4292308 0.7342q15 | 26 + 0.8128 0.5192 .3615385 0.7939q16 | 26 + 0.9022 0.7531 .2076923 0.4927-------------+-----------------------------------------------------------------Test scale | .3328205 0.7638-------------------------------------------------------------------------------
Test scale = mean(unstandardized items)
averageitem-test item-rest inter-item
Item | Obs Sign correlation correlation covariance alpha-------------+-----------------------------------------------------------------q17 | 26 + 0.7979 0.6284 .3771795 0.7634q18 | 26 + 0.7768 0.6225 .4097436 0.7693q19 | 26 + 0.8302 0.6685 .3438462 0.7438q20 | 26 + 0.7956 0.6073 .3707692 0.7751-------------+-----------------------------------------------------------------Test scale | .3753846 0.8113-------------------------------------------------------------------------------
Test scale = mean(unstandardized items)
averageitem-test item-rest inter-item
Item | Obs Sign correlation correlation covariance alpha-------------+-----------------------------------------------------------------q21 | 26 + 0.6275 0.4155 .184 0.7752q22 | 26 + 0.7779 0.6314 .152 0.7145q23 | 26 + 0.6560 0.4751 .1803077 0.7565q24 | 26 + 0.5665 0.4433 .2098462 0.7665q25 | 26 + 0.7616 0.6219 .1593846 0.7184q26 | 26 + 0.7432 0.5982 .1636923 0.7250-------------+-----------------------------------------------------------------Test scale | .1748718 0.7774-------------------------------------------------------------------------------
Test scale = mean(standardized items)
averageitem-test item-rest inter-item
Item | Obs Sign correlation correlation correlation alpha-------------+-----------------------------------------------------------------q1 | 26 + 0.5581 0.5230 0.3902 0.9520q2 | 26 + 0.5320 0.4954 0.3914 0.9522q3 | 26 + 0.5378 0.5016 0.3911 0.9522q4 | 26 + 0.3041 0.2586 0.4014 0.9541q5 | 26 + 0.6405 0.6102 0.3866 0.9513q6 | 26 + 0.6960 0.6694 0.3842 0.9508q7 | 26 + 0.7416 0.7183 0.3822 0.9504q8 | 26 + 0.7401 0.7166 0.3822 0.9505q9 | 26 + 0.4253 0.3839 0.3960 0.9531q10 | 26 + 0.4913 0.4527 0.3932 0.9526q11 | 26 + 0.6241 0.5928 0.3873 0.9515q12 | 26 + 0.6523 0.6227 0.3861 0.9512q13a | 26 + 0.7044 0.6784 0.3838 0.9508q13b | 26 + 0.8505 0.8360 0.3774 0.949513 | 26 + 0 8417 0 8265 0 3778 0 9496q13c | 26 + 0.8417 0.8265 0.3778 0.9496q13d | 26 + 0.8258 0.8092 0.3785 0.9497q13e | 26 + 0.7410 0.7176 0.3822 0.9504q13f | 26 + 0.6765 0.6485 0.3850 0.9510q13g | 26 + 0.8197 0.8025 0.3788 0.9497q14 | 26 + 0.3256 0.2807 0.4004 0.9539q15 | 26 + 0.7906 0.7711 0.3800 0.9500q16 | 26 + 0.6188 0.5871 0.3876 0.9515q17 | 26 + 0.6558 0.6264 0.3859 0.9512q18 | 26 + 0.6868 0.6595 0.3846 0.9509q19 | 26 + 0.7109 0.6853 0.3835 0.9507q20 | 26 + 0.6937 0.6669 0.3843 0.9509q21 | 26 + 0.6106 0.5784 0.3879 0.9516q22 | 26 + 0.7045 0.6785 0.3838 0.9508q23 | 26 + 0.3898 0.3471 0.3976 0.9534q24 | 26 + 0.3624 0.3187 0.3988 0.9536q25 | 26 + 0.6853 0.6580 0.3846 0.9509q26 | 26 + 0.7501 0.7274 0.3818 0.9504-------------+-----------------------------------------------------------------Test scale | 0.3868 0.9528-------------------------------------------------------------------------------
7
Test scale = mean(unstandardized items)
averageitem-test item-rest inter-item
Item | Obs Sign correlation correlation covariance alpha-------------+-----------------------------------------------------------------q1 | 26 + 0.5622 0.5254 .2305327 0.9537q2 | 26 + 0.5360 0.4987 .2315434 0.9539q3 | 26 + 0.5357 0.5001 .2320463 0.9539q4 | 26 + 0.2897 0.2484 .2387196 0.9556q5 | 26 + 0.6386 0.6120 .2308553 0.9530q6 | 26 + 0.6893 0.6634 .2285161 0.9526q7 | 26 + 0.7382 0.7150 .2268337 0.9522q8 | 26 + 0.7557 0.7325 .2254458 0.9520q9 | 26 + 0.4176 0.3800 .2357204 0.9547q10 | 26 + 0.4743 0.4526 .2379818 0.9542q11 | 26 + 0.6218 0.5914 .2300232 0.9532q12 | 26 + 0.6666 0.6394 .2291133 0.9528q13a | 26 + 0.7116 0.6889 .2289562 0.9525q13b | 26 + 0.8606 0.8456 .2214723 0.9510q13c | 26 + 0 8508 0 8338 2204946 0 9510q13c | 26 + 0.8508 0.8338 .2204946 0.9510q13d | 26 + 0.8394 0.8224 .2220744 0.9512q13e | 26 + 0.7431 0.7225 .228225 0.9523q13f | 26 + 0.6855 0.6553 .2264615 0.9527q13g | 26 + 0.8175 0.7989 .2232076 0.9514q14 | 26 + 0.3132 0.2741 .2383706 0.9553q15 | 26 + 0.7941 0.7711 .2218941 0.9516q16 | 26 + 0.6145 0.5808 .2291067 0.9533q17 | 26 + 0.6580 0.6257 .2272275 0.9529q18 | 26 + 0.6826 0.6557 .2283904 0.9527q19 | 26 + 0.7240 0.6951 .2243226 0.9523q20 | 26 + 0.7043 0.6733 .2247312 0.9525q21 | 26 + 0.6182 0.5863 .2296162 0.9532q22 | 26 + 0.7031 0.6774 .2277304 0.9525q23 | 26 + 0.3689 0.3289 .2367361 0.9551q24 | 26 + 0.3335 0.3075 .2398098 0.9548q25 | 26 + 0.6759 0.6505 .2295004 0.9528q26 | 26 + 0.7459 0.7252 .228038 0.9522-------------+-----------------------------------------------------------------Test scale | .229178 0.9544-------------------------------------------------------------------------------