university of wisconsin placement testing with examsoft readying ourselves for the future jim...
TRANSCRIPT
University of Wisconsin Placement Testing with ExamSoft
Readying ourselves for the future
Jim WollackUniversity of Wisconsin-Madison
Amy SmithExamSoft Worldwide, Inc.
ExamSoft Company Overview
ExamSoft is a leading provider of
end-to-end CBT solutions
with
13 YEARSexperience delivering
ROBUST DESIGN,DELIVERY, SCORING
& ANALYSIS TOOLS
Used by 250+Academic, licensing, & certifying
institutions
Including:Health Professionals Financial ServicesAcademia
~1 MILLIONExams delivered
each year
38 of 50 State Bars
Leader in High-Stakes CBT
UW System and ExamSoft
The University of Wisconsin System has partnered with ExamSoft Worldwide, Inc. to offer computer based testing for freshman placement exams.
The new hybrid testing model maximizes:• Flexibility• Convenience• Efficiency• Validity• Security
Overview of the UW System
• 26 campus state systemo 13 4-year campuseso 13 2-year transfer institutionso Distinct from statewide technical colleges
system
Placement Testing and the UW System
• Placement testing is a critical element in establishing uniformity and facilitating transfers between institutions
The UW Placement Testing Model
• Math, English, French, German, and Spanish• Tests developed collaboratively across all 26 UW
campuseso Content of tests is reflective of UW curriculao New forms are published every year
• Same tests used for all campuseso Cutscores and other particulars are campus-specific
UW Center for Placement Testing
• Center coordinates test development, validation, and administration systemwide
• Center is funded by all campuseso Each campus’ contribution is proportional to the
size of campus’ incoming freshman classo Budgetary oversight by a Systemwide Advisory
Board 6 Provosts 5 chairs of placement test development committees 1 UW System Vice President Center Director, ex officio
Center’s Testing Cycle
JanMar FebApr May Jun Jul Aug Sep Oct Nov Dec
Regional Testing
Orientation Testing
Residual Testing
Regional Testing Orientation TestingResidual Testing
• Tests offered at all UW campuses on fixed dates (mostly on Saturdays)
• Students test at any campus on specified date
• 69% test at one campus but attend another
• Center’s budget covers costs for Reg. Testing
• Summer Orientation testing is a special case of Residual Testing
• 5 ½ hours during orientation
• At campus’s own expense
• Each campus handles their own Resid. Testing
• Conducted at campus’ own expense
• Models for Resid. Testing vary across campuses
Students: 13,047Exams: 30,760
Students: 10,538Exams: 20,525
Students: 7,049Exams: 10,512
Challenges Today
Capacity
• Late admits• Out-of-state students
& International students
• Student who missed regional testing
Logistics
• Scheduling 130 administrations between June & August
• Orientation impact (5 ½ hours)
Cost• Residual testing
costs extra
Access
• Limited to campus testing
• Not all campuses offer residual testing
Administrative Challenges
Security• Must have a
proctored environment
University System Total
Legend> 20,000> 1,000< 1,000< 100
Students: 30,634Exams: 61,797
Initial Concerns with CBT
• Security—exams must be proctored• Cost
o Exam is currently free for studentso Start-up/maintenance costs to campuses
• Switch to CBT may be disruptive for some campuseso Reliability/process with service failures
Changes for 2012
JanMar FebApr May Jun Jul Aug Sep Oct Nov Dec
Regional Testing (CBT and PP)
Computer Based Testing (Testing Centers)
Residual Testing (CBT & PP)
Convenient Year Round CBT
Expanded Regional Dates Enhanced Residual Testing
• Large scale CBT (reduced fee)
• Continued P&P (free) • Extended dates
through July on more campuses provides greater access to free tests
• More dates: CBT-only and P&P-only dates
• Test at a nearby testing center (additional fee)
• Students can test anytime
• All test are proctored
• Offer CBT (additional fee)
• Cover proctor costs (CBT)
• P&P still available
New International Options • Take proctored CBT exams at local locations (additional fee)
International Computer-based Testing
Overcoming CBT Concerns
• Flexibilityo Test on and off campus (domestic & international) o Deliver exams on any computer, including student-owned
laptops • Security
o all exams are fully secure and proctored• Costs
o Designed an equitable partnership to keep costs at a minimum for students
• Support & Services o Full support for students & administratorso Custom built exam registration site
• Reliability o Unique delivery method eliminates internet dependence during
the exam
Reasons out of state students can’t use regional testingFlight $382 (per person) $763
Hotel $99 (person) $297
Car $42 (per day) $126
Expenses $100 (per day) $300
Total $1486
Students: 30,634Exams: 61,797
University System Total
Student Access
Scheduling• Overcommitted • Tend to
procrastinate• Changing
schedulesCapacity• Limited dates• Limited seats
Access• Restrained by
campus testing only
Travel
Student Challenges
Legend> 20,000> 1,000< 1,000< 100
Incremental cost for out of state students ( Parent & Student).
Hotel $99 (person) $99
Car $42 (per day) $42
Expenses $100 (per day) $100
Total $241
• Convenience • Scheduling
Costs
Travel
Student Challenges
Significantly Expanded Coverage
On CampusOff Campus
Off Campus
Off Campus
Even Globally
How is this possible?
ExamSoft
Rethink Traditional CBT
How the Software Works
Auto UploadSECURE ANSWERS
Issue KeysFOR EXAMS
DownloadSECURE FILE
Creates & Posts the Exam
Wisconsin Students
Register online for their exams
Prior to the Exam
Encrypted exam file is downloaded
During the Exam
Internet, applications, and
local files are block.
After the Exam
Encrypted exam file is uploaded
Wisconsin Students
Results of exam delivered
Analyzes exam results
Change/Home
Registration/Home
Special/Home
Send to Human
Registration Form
Where would you like to take your exam
On Campus (CBT) On Campus (P&P) Off Campus (Testing Ctr) Off Int’l
Select Location
Select Date & Time
Shopping Cart
Confirm
Select Location
Select Date & Time
Confirm
Select Exam
Get Center (API)
Get Date (API)
Get Appointment (API)
Save/Another Exam
Shopping Cart
Confirm/Schedule Appointments (API)
Basic Form
Shopping Cart
Int’l Download Screens
Proctor Form/Download
Screen
Ch. Date/Time (Campus)
Ch. Date/Time
(ACT)
Ch. Location/ Channel -
Instructions
Buy Extra Exam Slots
(ACT)
Retake Instructions
Help/FAQs
Login
Update RSP
Other times
New Student Registration Site
New Placement Exam Site
1.Streamline the process for all methods of testing
2.Increase capacity and options3.Robust item banking tool for CBT and P&P
exams with item statistics4.Greater control and security5.Increase flexibility6.Easier scoring and reporting7.Enhance cost recovery without mandating8.Improve students’ first impression of UW
system by offering technology and convenience options for first assessments
Benefits to the University
1. Expanded regional testing gives greater access to the free test
2. Take test(s) at a location closer to home, reducing travel expenses for international and out of state students
3. Take test(s) at a convenient time, scheduling appointments for exams on dates and times that work within students’ busy academic and athletic schedules
4. CBT could provide an option for immediate feedback on exams
5. Take tests early - begin to register for courses earlier, which makes orientation easier and more fun
6. Use the latest technology to take the test(s) – and test with the comfort of your own laptop if you desire
Benefits to the Students
Why We Chose ExamSoft’s Model
• Proven security and reliability- No “server is down” concerns during the exams• Viable economic model for CBT migration• Robust item banking and scoring tools• Superior customer support• Personalized client services - above and beyond• Tremendous flexibility
o Students get comfort of testing on their own computerso Delivery model greatly reduces normal capacity issue associated
with CBTo Allows us to entertain moving towards inexpensive CBT for WI
students by changing some Regional Testing dates to be CBT sites
o Possibility of using ExamSoft for classroom testing
Appendix
How the Software Works
Auto UploadSECURE ANSWERS
Internet Availability(Testing Center or after exam)
Issue KeysFOR EXAMS
Secure Test Center or Campus Location
DownloadSECURE FILE
Internet Availability(home, dorm, hotel, Starbucks)
Unprecedented Test Site Flexibility230 State-of-the-Art Testing Centers
Deeper Look at Laptop Security
Over 60 security check processes running during the exam!!
Completely lock down the laptop at operating system level
Block internet access
System level tracking of exam taker activity
No access to local files, applications
Forced exit of exam and lock out of answer files
Easy, Intuitive Item Banking
Easy, Intuitive Item Banking
Question Navigation is easy by using these icons.
Easy, Reliable, Secure Exams
Item Validity and Reliability
Robust Reporting
Benchmarking Students
Benefits:Measure & drive learning outcomes with remediation, self directed learning, teaching/ curriculum changes, etc.
Provide valuable data for accreditation process
Web-Based Essay Scoring
View questions here
Review exam taker response here
Provide grader feedback here
Enter points here
Close the Loop – Item Analysis
Support From a Partner You Trust