2/15/2006. online testing initiatives in north carolina presenters: randy craven director for...
TRANSCRIPT
2/15/2006
Online Testing Initiatives in Online Testing Initiatives in North CarolinaNorth Carolina
Presenters: Presenters: Randy CravenRandy CravenDirector for Information TechnologyDirector for Information TechnologyTechnical Outreach for Public Schools (TOPS), NC State Technical Outreach for Public Schools (TOPS), NC State UniversityUniversity
Scott RagsdaleScott RagsdaleProject Manager, North Carolina Computer Skills Project Manager, North Carolina Computer Skills Assessments +Assessments +Technical Outreach for Public Schools (TOPS), NC State Technical Outreach for Public Schools (TOPS), NC State UniversityUniversity
2/15/2006
Modern science has been a voyage into Modern science has been a voyage into the unknown, with a lesson in humility the unknown, with a lesson in humility waiting at every stop. Many passengers waiting at every stop. Many passengers would rather have stayed home.would rather have stayed home.
Carl SaganCarl Sagan
2/15/2006
The purpose of this session is to The purpose of this session is to update the audience on NCDesk, the update the audience on NCDesk, the Online Test of Computer Skills, and Online Test of Computer Skills, and other online assessments used in North other online assessments used in North Carolina testing.Carolina testing.
2/15/2006
What Initiatives?What Initiatives?
• Online Test of Computer SkillsOnline Test of Computer Skills• End-of-Course Online Physics Field Test (mandatory)End-of-Course Online Physics Field Test (mandatory)• End-of-Course Tests (optional 06-07)End-of-Course Tests (optional 06-07)• Writing Online Special Study (Grades 7 and 10)Writing Online Special Study (Grades 7 and 10)• Online formative assessments (i.e., ClassScape)Online formative assessments (i.e., ClassScape)• Other online resourced assessmentsOther online resourced assessments
– NCEXTEND1NCEXTEND1– NCCLASNCCLAS
• Other web applications supporting test Other web applications supporting test development/maintenance:development/maintenance:– Online Item Writing SystemOnline Item Writing System– Experimental Section Review (ESR)Experimental Section Review (ESR)
2/15/2006
Online Assessments SuccessOnline Assessments Success
• The Online Test of Computer Skills was The Online Test of Computer Skills was successful in its initial administration year of successful in its initial administration year of 2005-06 and continues to be successful this 2005-06 and continues to be successful this year.year.
• NCCLAS online data collection was successful NCCLAS online data collection was successful in 2005-06 and continues to work this year.in 2005-06 and continues to work this year.
• Online Physics Field Test was administered to Online Physics Field Test was administered to over 3,700 students during the fall administration over 3,700 students during the fall administration with no significant issues.with no significant issues.
• For 2006-07, For 2006-07, NCEXTEND1NCEXTEND1 students have been students have been matched to their assessors and have had their matched to their assessors and have had their Decision Tree processes completed.Decision Tree processes completed.
2/15/2006
Elements of Online AssessmentsElements of Online Assessments
• More interactiveMore interactive• Test Simulations required to be availableTest Simulations required to be available• More data can be collectedMore data can be collected• Opportunity to expand the use of innovative Opportunity to expand the use of innovative
itemsitems– Virtual/performance-based itemsVirtual/performance-based items– Short and/or extended responseShort and/or extended response
• Quicker data processing and reportingQuicker data processing and reporting• Reduction of materials managementReduction of materials management• Students have a positive attitude for online Students have a positive attitude for online
testingtesting
2/15/2006
Elements of Online Assessments Elements of Online Assessments (continued)(continued)
• TechnologyTechnology– Internet aware applications (NCDesk)Internet aware applications (NCDesk)
• Java versionJava version
– Browser based (EOC & Physics)Browser based (EOC & Physics)• Session State CookiesSession State Cookies• JavascriptJavascript• Flashpaper / Interactive Flash Items / Flash playerFlashpaper / Interactive Flash Items / Flash player• VersionsVersions
2/15/2006
Performance ComponentsPerformance Components
• Online Test of Computer SkillsOnline Test of Computer Skills– 50% multiple choice50% multiple choice– 50% performance50% performance
• Physics OnlinePhysics Online– Mostly multiple choiceMostly multiple choice– Small number of performance-based itemsSmall number of performance-based items
• EOC OnlineEOC Online– All multiple-choiceAll multiple-choice
• Online Writing Special StudyOnline Writing Special Study– Single prompt with essay responseSingle prompt with essay response
2/15/2006
Preservation of StatePreservation of State
• Students expect responses to items to retain Students expect responses to items to retain state as they move from one item to another and state as they move from one item to another and if they return to an item, it will be as they had left if they return to an item, it will be as they had left it before.it before.
• Preservation of state is not always easy since Preservation of state is not always easy since online assessment systems must be able to online assessment systems must be able to recover from disconnection.recover from disconnection.
• At this time for the Online Test of Computer At this time for the Online Test of Computer Skills and the EOC Online Tests we are able to Skills and the EOC Online Tests we are able to manage state for all items except the item the manage state for all items except the item the student is currently working on.student is currently working on.
2/15/2006
Problems/OpportunitiesProblems/Opportunities
• Rate of technology changeRate of technology change
• Window of opportunity for testingWindow of opportunity for testing
• SchedulingScheduling
• Having enough…Having enough…– BandwidthBandwidth– ComputersComputers– SpaceSpace– TimeTime
2/15/2006
Managing ThingsManaging Things
• NCRegistrationNCRegistration– User AccountsUser Accounts– Student RegistrationsStudent Registrations– Scheduling SessionsScheduling Sessions– Receiving ReportsReceiving Reports
2/15/2006
"There is no reason anyone would want a "There is no reason anyone would want a computer in their home." computer in their home."
Ken OlsonKen Olson
President, chairman and President, chairman and founder of Digital Equipment founder of Digital Equipment Corporation (DEC), maker of Corporation (DEC), maker of business minicomputers, business minicomputers, arguing against the PC in arguing against the PC in
1977. 1977.
2/15/2006
Online Test of Computer SkillsOnline Test of Computer SkillsNotable Preliminary/Unofficial Numbers from Notable Preliminary/Unofficial Numbers from
Fall 2006Fall 2006• 205 students tested in one day at one school (for 05-06 205 students tested in one day at one school (for 05-06
this was 139)this was 139)• 52 schools tested 100 or more students in one day (for 52 schools tested 100 or more students in one day (for
05-06 this was 7)05-06 this was 7)• 5,068 maximum testing sessions used in a single day 5,068 maximum testing sessions used in a single day
(for 05-06 this was 3,063)(for 05-06 this was 3,063)• 23 days of testing in which volume of used sessions was 23 days of testing in which volume of used sessions was
3,000 or more3,000 or more• ~133,980 total students tested~133,980 total students tested• ~106,035 8~106,035 8thth grade students tested grade students tested• ~2,228 different test administrators with 52 logging in ~2,228 different test administrators with 52 logging in
300 or more students300 or more students• ~3,446 students had the Computer Reads Aloud option~3,446 students had the Computer Reads Aloud option
2/15/2006
Online Progress IndicatorsOnline Progress IndicatorsMaximum Number of Students Tested in a Day
0
1000
2000
3000
4000
5000
6000
2005-06 Fall 2006
Number of Schools Testing 100 or More Students in a Day
0
10
20
30
40
50
60
2005-06 Fall 2006
Maximum Number of Students Tested in a Day at a School
0
50
100
150
200
250
2005-06 Fall 2006
Average Minutes on Test for Students with One Login
0
15
30
45
60
75
90
105
120
2005-06 Fall 2006
2/15/2006
Platform ShiftingPlatform Shifting
Fall 2006
Mac OS X (10.3.3-9)1.544%
Windows Me (4.90)0.002%
Windows 2000 (5.0)9.178%
Mac OS X (10.4.1-8)5.734%
Windows 98 (4.10)5.495%
Windows XP (5.1)78.046%
Fall 2005
Windows XP (5.1) 65.22%
Windows Me (4.90) 0.01%
Windows 98 (4.10) 16.05%
Windows 2000 (5.0) 10.04%
Mac OS X (10.4.x) 1.88%
Mac OS X (10.3.x) 6.73%
Mac OS X (10.2.x) 0.07%
2/15/2006
Online Computer SkillsOnline Computer SkillsItems for Spring 2007Items for Spring 2007
• Testing Window 3/12/07 – 6/8/07 Testing Window 3/12/07 – 6/8/07 • NCRegistration accessible on 2/5/07 for NCRegistration accessible on 2/5/07 for
scheduling, etc.scheduling, etc.• Reports available to LEA Test Reports available to LEA Test
Coordinators through NCRegistrationCoordinators through NCRegistration• JAWS compatibility not tested/approved – JAWS compatibility not tested/approved –
also newer versions of JAWS released also newer versions of JAWS released since last tested – research/development since last tested – research/development ongoing processongoing process
2/15/2006
Even if you're on the right track you'll Even if you're on the right track you'll get run over if you just sit there.get run over if you just sit there.
Will RogersWill Rogers
2/15/2006
NCDesk Moving ForwardNCDesk Moving Forward
• Adjustments for accommodating delivery of Adjustments for accommodating delivery of Writing assessments onlineWriting assessments online– Assessment selection on login pageAssessment selection on login page– Programmatically remove spell check functionsProgrammatically remove spell check functions– Programmatically remove image insertion functionsProgrammatically remove image insertion functions– Other features deemed necessaryOther features deemed necessary
• Additional features for testing audio functionalityAdditional features for testing audio functionality• Additional features for viewing correct responses Additional features for viewing correct responses
for the Test Simulation possiblefor the Test Simulation possible
2/15/2006
EOC Online (Optional)EOC Online (Optional)
• Fall 2006Fall 2006– First time EOCs offered online as an optionFirst time EOCs offered online as an option– One school participated in Algebra IOne school participated in Algebra I
• No problems reportedNo problems reported• Students “enjoyed” itStudents “enjoyed” it
• Spring 2007Spring 2007– Registration & Scheduling now available via Registration & Scheduling now available via
NCRegistrationNCRegistration• Not included:Not included:
– Chemistry Field Test Chemistry Field Test – Physical Science Field Test Physical Science Field Test – Biology Field TestBiology Field Test
2/15/2006
EOCs Online EOCs Online Moving Forward (Next Year)Moving Forward (Next Year)
• Making the answer choices clickableMaking the answer choices clickable• Improvement of the calculatorImprovement of the calculator
– Moveable/draggable on screen (within test interface)Moveable/draggable on screen (within test interface)– Add graphing capabilityAdd graphing capability
• Improvement of online reference contentImprovement of online reference content– Moveable/draggable on screen (within test interface)Moveable/draggable on screen (within test interface)
• Attempt to improve item view state preservationAttempt to improve item view state preservation– PositionPosition– Zoom percentageZoom percentage
• Optional to continue until such time as it is no longer Optional to continue until such time as it is no longer feasible to maintain paper-and-pencil testing.feasible to maintain paper-and-pencil testing.
2/15/2006
NCEXTEND2NCEXTEND2 Online 2007-08 Online 2007-08
• Process under way to present all Process under way to present all NCEXTEND2NCEXTEND2 in an online format in an online format
• Format similar to EOC Online Format similar to EOC Online
• Format for Writing would be delivered Format for Writing would be delivered using NCDesk with a single editor item.using NCDesk with a single editor item.