Download - The effect of differential item functioning in anchor items on population invariance of equating
![Page 1: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/1.jpg)
The effect of differential item functioning in anchor items on population invariance of equating
Anne Corinne HugginsUniversity of Florida
![Page 2: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/2.jpg)
Introduction• Population invariance of equating– Test score level
• Differential item functioning– Item level
• Need to connect two facets of invariance• Certain types of DIF might cause equating
dependence across forms of a test.– Due to (1) a differential difficulty, particularly for anchor items(2) a second factor(s)(3) Item parameter drift
![Page 3: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/3.jpg)
• IRT true score equating– A three-step process:• Specify a true score on one form• Find the corresponding IRT ability estimate• Find the IRT ability estimate’s corresponding true score
on the other form
![Page 4: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/4.jpg)
• Place the latent ability scores on each test on the same scale
We can use the probability formula to get the relations.
![Page 5: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/5.jpg)
5
• Scaling– Mean-sigma
– Mean-mean
Here, measurement errors are ignored. If a and b were biased, A and B will be biased.
Q: How to estimate A under Rasch model?
![Page 6: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/6.jpg)
6
– The Stocking and Lord method• Minimize the difference in test characteristic curves
(TCCs) for anchor items
– The Haebara method• Minimize the difference in item characteristics curves
(ICCs)
![Page 7: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/7.jpg)
7
• Equating invariance– Compare the whole group to multiple subgroups
– Group to overall (compare one subgroup with the whole group)
Sigma Q is the unconditional SD of scores in population.
![Page 8: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/8.jpg)
8
• The factors that might influence population invariance of IRT true score equating– Differential anchor form DIF– The magnitude of DIF in anchor items– Number of DIF anchor items – Direction of DIF in anchor items– Ability differences between subpopulations
![Page 9: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/9.jpg)
9
Method
• Uniform DIF• R, BILOG-MG, SPSS• Design– Nonequivalent groups with anchor test (NEAT) design– Two forms contain 50 dichotomous items,
respectively, with 20% of common items– Ability ~ N (0,1)– 4500 participants (3000 in the references group)– 3PL model– 100 replications
![Page 10: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/10.jpg)
10
– Manipulations on DIF in anchor items• Magnitude (.3, .6, .9)• % of DIF items (20%, 49%, 60%)• Directionality of DIF• Mean ability differences• Differential anchor form DIF
– Null conditions• No DIF in anchor items and no mean ability differences• No DIF in anchor items but had mean ability differences
![Page 11: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/11.jpg)
11
Results—No DIF (RMSD(x))
![Page 12: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/12.jpg)
12
No DIF, Equal mean abilities (RSD(x))
![Page 13: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/13.jpg)
13
No DIF, Unequal mean abilities (RSD(x))
![Page 14: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/14.jpg)
14Why didn’t DIF show the impact here?
![Page 15: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/15.jpg)
15
![Page 16: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/16.jpg)
16
![Page 17: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/17.jpg)
17
![Page 18: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/18.jpg)
18
![Page 19: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/19.jpg)
19
![Page 20: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/20.jpg)
20
![Page 21: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/21.jpg)
21
![Page 22: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/22.jpg)
22
![Page 23: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/23.jpg)
23
![Page 24: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/24.jpg)
24
![Page 25: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/25.jpg)
25
Conclusions Differential anchor form DIF occurs, the
magnitude of equating dependence increases.IPD occurs for different subpopulations.
When differential anchor from DIF occurs:Direction of DIF in anchor items shows the largest
effect on equating dependence.The magnitude of DIF in anchor itemsNumber of DIF anchor items Ability differences between subpopulations
![Page 26: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/26.jpg)
Comments
• It is a good writing sample.• Errors:– P.9, since the study manipulated the direction of
DIF, the statement of magnitude of DIF should be revised properly.
– The author intended to design the sample size as the true subpopulations in the US, but the actual sample sizes did not reflect its intends.
![Page 27: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/27.jpg)
• When DIF or item parameter drift occurs in a specific anchor item, this item should not be used as an anchor.
• We usually do not talk about DIF in anchor items because DIF detections should be done before any equating or linking.
• When all anchor items are DIF items, the results should be very similar with the ones without DIF anchor items.
![Page 28: The effect of differential item functioning in anchor items on population invariance of equating](https://reader036.vdocuments.mx/reader036/viewer/2022062502/568156ce550346895dc4659e/html5/thumbnails/28.jpg)
28
Future studies
• Sample size differences across score levels• Test length, anchor length, ratio of anchor to
test length, mixed item formats