high stakes testing on sakai with samigo and a locked-down os image
DESCRIPTION
Online writing tests, using an OS image that prevents access to any site other than Sakai.TRANSCRIPT
![Page 1: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/1.jpg)
June 10-15, 2012
Growing Community; Growing Possibilities
High Stakes Testing with Samigo
on a Locked-Down OS Image
Ken RomeoLanguage Center
Mike QueisserAcademic Computing Services
Stanford University
![Page 2: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/2.jpg)
2
Background: Why did we do this? Pieces: Where, what, and how?
◦ Hardware◦ Test settings◦ OS image◦ Preparation◦ Administration
Results: How well did it work? Lessons learned: What should/can we fix? The biggest victory: Collaborations
Roadmap
2012 Jasig Sakai Conference
![Page 3: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/3.jpg)
2012 Jasig Sakai Conference 3
CourseWork = Sakai ACTFL = American Council on the Teaching
of Foreign Languages WPA = Writing Proficiency Assessment IME = Input Method Editor
Terminology
![Page 4: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/4.jpg)
2012 Jasig Sakai Conference 4
Language Requirement
Accountability Speaking & Writing ACTFL Proficiency
guidelines
Background
![Page 5: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/5.jpg)
2012 Jasig Sakai Conference 5
Paper test distribution
![Page 6: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/6.jpg)
2012 Jasig Sakai Conference 6
Scans Project sites
![Page 7: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/7.jpg)
2012 Jasig Sakai Conference 7
Why not just do it in samigo?
More secureBetter organizationTyping produces more language
![Page 8: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/8.jpg)
2012 Jasig Sakai Conference 8
No space …
![Page 9: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/9.jpg)
2012 Jasig Sakai Conference 9
“Faculty agree to avoid academic procedures that create temptations to violate the honor code.”
![Page 10: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/10.jpg)
2012 Jasig Sakai Conference 10
Building 30
![Page 11: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/11.jpg)
2012 Jasig Sakai Conference 11
![Page 12: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/12.jpg)
2012 Jasig Sakai Conference 12
![Page 13: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/13.jpg)
2012 Jasig Sakai Conference 13
![Page 14: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/14.jpg)
2012 Jasig Sakai Conference 14
![Page 15: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/15.jpg)
2012 Jasig Sakai Conference 15
![Page 16: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/16.jpg)
2012 Jasig Sakai Conference 16
Lightweight, simple system Security Privacy
The OS image: Goals
![Page 17: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/17.jpg)
2012 Jasig Sakai Conference 17
Minimize running services and applications Sophos A/V, Python 2.6, and CopSSH Automatic login XML file >> IME settings CMD.exe and Notepad.exe Windows updates disabled
Lightweight, Simple System
![Page 18: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/18.jpg)
2012 Jasig Sakai Conference 18
Active directory Domain and local group policies Proxy server to allow access to only certain
sites Script read from imaging management
server
Security
![Page 19: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/19.jpg)
2012 Jasig Sakai Conference 19
Group Policy Screenshot
![Page 20: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/20.jpg)
2012 Jasig Sakai Conference 20
function FindProxyForURL(url, host) { //strings to return var allow = "DIRECT"; var block = "PROXY localhost:80"; if (shExpMatch(url,
"https://coursework.stanford.edu/*")) { return allow; } if (shExpMatch(url, "https://cw-tests.stanford.edu/*")) { return allow; } if (shExpMatch(url, "https://weblogin.stanford.edu/*")) { return allow; } if (shExpMatch(url, "*.microsoft.com/*")) { return allow; } if (shExpMatch(url, "*.windowsupdate.com/*")) { return allow; } if (shExpMatch(url, "*.microsoftupdates.com/*")) { return allow; } if (shExpMatch(url, "*.sophos.com/*")) { return allow; } return block; }
Proxy Configuration Script
![Page 21: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/21.jpg)
2012 Jasig Sakai Conference 21
Clear temporary files at logout Reset user account completely at login
Privacy
![Page 22: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/22.jpg)
2012 Jasig Sakai Conference 22
Typing: PC – IE8 (NOT a Mac, MSWord) Instructions
◦ Language bar◦ Don’t click the back button◦ Save after every sentence◦ Mistakes: ctrl-z / close browser◦ Submit when complete◦ Can re-open unsubmitted work◦ Can re-submit new work
User Preparation
![Page 23: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/23.jpg)
2012 Jasig Sakai Conference 23
Coordinator group and 1-on-1 meetings Multiple emails to instructors Coordinator meetings with instructors Consulting on preparation activities
Instructor preparation
![Page 24: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/24.jpg)
2012 Jasig Sakai Conference 24
Overview / schedule Preparation: steps and samigo export file Orientation
◦ Typing instructions: US International, Chinese, Japanese, Korean
◦ Test taking instructions Testing day instructions for administration
◦ Setup◦ Test start to finish◦ Error handling
http://wpa.stanford.edu
![Page 25: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/25.jpg)
2012 Jasig Sakai Conference 25
Introduction: Honor code and instructions 2 items: text prompt, short answer
responses Settings:
◦ No release, due, retract dates◦ No time limit◦ Allow multiple submissions◦ Allow late submissions◦ High security: UserID and password
Concept: security through physical location
Tests and Quizzes (aka samigo)
![Page 26: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/26.jpg)
2012 Jasig Sakai Conference 26
Previously observed Virtual server to avoid load balancer Certificate Vanilla Sakai In the end …
Unexplained errors
![Page 27: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/27.jpg)
2012 Jasig Sakai Conference 27
2011: pilot 155 tests in Western languages 2012: 493 on samigo / 238 on paper,
◦ 31 Chinese, Japanese, and Korean pilot tests◦ No major data loss: 11 students lost 1-2
sentences◦ One test fell back to paper (10 students)
2013: expand CJK Remaining languages: Russian and Arabic
Results
![Page 28: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/28.jpg)
2012 Jasig Sakai Conference 28
Students don’t read paper instructions Logistics are never, ever trivial Something was sketchy: load balancer? Overcautious instructions worked, sort of Non-soft controls (physical presence)
worked OS image totally worked (except notepad)
Lessons learned
![Page 29: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/29.jpg)
2012 Jasig Sakai Conference 29
Block Notepad and CMD IE >> Firefox ESR Point out language bar
Improvement Opportunities
![Page 30: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/30.jpg)
2012 Jasig Sakai Conference 30
Desktop (not web) app
Enrollment from Sakai Results to Sakai Cache results until
completely uploaded
Need: a better tool
SOPI application
![Page 31: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/31.jpg)
2012 Jasig Sakai Conference 31
The end of the blue book Mid-terms, finals: like paper, but online Instructional technology for the 80% The beginning of learning analytics
High stakes testing
![Page 32: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/32.jpg)
2012 Jasig Sakai Conference 32
The biggest victory
Imaging(Mike’s cube)
CourseWork
Team
Language Lab
(Ken’s office)
![Page 33: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/33.jpg)
2012 Jasig Sakai Conference 33
Cannot do instructional technology *at scale* in isolation
Collaborations across units work! Face-to-face meetings Professional cooperation to improve
instruction = incentive
n.b. My explicit role is to facilitate these collaborations for the Language Center
Collaborations
![Page 34: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/34.jpg)
2012 Jasig Sakai Conference 34
Proactively address needs Address problems systematically,
institutionally Cross institutional boundaries Generalize to contribute to the field
Instructional Technology should:
![Page 35: High Stakes Testing on Sakai with Samigo and a Locked-down OS image](https://reader031.vdocuments.mx/reader031/viewer/2022013100/54d2a8784a7959ba6d8b459b/html5/thumbnails/35.jpg)
June 10-15, 2012
Growing Community; Growing Possibilities
More Info:http://wpa.stanford.edu
Thank you!Ken Romeo – [email protected]
Language Center
Mike Queisser – [email protected] Computing Services
Stanford University