video-based assessment of a coach’s ability to fault-check performance
DESCRIPTION
Video-Based Assessment of a Coach’s Ability to Fault-Check Performance. John J. Lee, Girlie C. Delacruz, Jesse Elmore, & William L. Bewley. American Educational Research Association Annual Meeting - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/1.jpg)
American Educational Research AssociationAnnual Meeting
AERA 53.069 – TICL 6: Advancing the State of the Art: Computer-Based Assessment and Learning EnvironmentsNew York, NY - March 27, 2008
John J. Lee, Girlie C. Delacruz, Jesse Elmore, & William L. Bewley
Video-Based Assessment of a Coach’s Ability to Fault-Check Performance
![Page 2: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/2.jpg)
• Background
• Research base
• Assessments
• Design
• Analysis
• Demo
Overview
![Page 3: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/3.jpg)
Background1. Context: Fault Checking Performance on the Range
2. Two Weapon Manipulations Skills: Speed Reload and Tactical Reload
3. Using video-based assessment from four different angles
4. Main research question: Does assessment response format make a difference?
![Page 4: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/4.jpg)
Research Base• Expert-Novice
1. Experts have schemas to reduce cognitive load
2. Expertise Reversal Effect: some task formats may impede performance for experts when inconsistent or unaligned with existing schema
3. Scaffolding: helpful to novices who have little or no prior knowledge
4. Distributed Representations: task’s response format as scaffold
• Assessment Validation
1. Validity – reduce construct irrelevant variance as much as you can
2. Response fidelity more predictive validity than more abstract
![Page 5: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/5.jpg)
Video-Based Assessments
1. Shooter-Skill Diagnostic Task:
Checklist Version
Open-ended Version
2. Shooter-Skill Performance Task:
Proper / Improper Execution
Shooter-Skill Performance Task: Drag and Drop Order
![Page 6: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/6.jpg)
Video-Based Assessments
![Page 7: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/7.jpg)
Video-Based Assessments
![Page 8: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/8.jpg)
Video-Based Assessments
![Page 9: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/9.jpg)
Video-Based Assessments
![Page 10: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/10.jpg)
Video-Based Assessments
![Page 11: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/11.jpg)
Sensitivity to Expertise
Research Question Independent Variable
Dependent Variable
Prediction
1. Do experts perform better on the SSDT that is open-ended because it is more like what they do on the firing line?
Expertise level (four levels)
SSDT Open-ended scores
Experts (PMIs)> Unit Coaches > Coaches Course Post-test > Novices (SLR non-coaches)
2. Do novices perform better on the SSDT that uses a checklist because that format provides a scaffold for performance?
Expertise level (four levels)
SSDT Checklist scores
Novices (SLR non-coaches) > Coaches Course Post-test > Unit Coaches > Experts (PMIs)
3. Is the SSPT (proper/improper) sensitive to expertise?
Expertise level (four levels)
SSPT proper/improper scores
Experts (PMIs) > Unit Coaches > Coaches Course Post-test > Novices (SLR non-coaches)
4. Is the SSPT (drag and drop) sensitive to expertise?
Expertise level (four levels)
SSPT drag and drop scores
Experts (PMIs) > Unit Coaches > Coaches Course Post-test > >Novices (SLR non-coaches)
![Page 12: Video-Based Assessment of a Coach’s Ability to Fault-Check Performance](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56814645550346895db35160/html5/thumbnails/12.jpg)
Sensitivity to Instruction
Research Question Independent Variable
Dependent Variable Prediction
1. Is the SSDT open-ended sensitive to instruction?
Administration (pre- test versus post-test)
SSDT Open-ended scores
Post-instruction > Pre-instruction
2. Is the SSDT checklist sensitive to instruction?
Administration (pre- test versus post-test)
SSDT Checklist scores Post-instruction > Pre-instruction
3. Is the SSPT (proper/improper) sensitive to instruction?
Administration (pre- test versus post-test)
SSPT proper/improper scores
Post-instruction > Pre-instruction
4. Is the SSPT (drag and drop) sensitive to instruction?
Administration (pre- test versus post-test)
SSPT drag and drop scores
Post-instruction > Pre-instruction