2 – 6 september 2013 shape (mons) belgium petya georgieva georgi rakovski national defense academy...
Post on 21-Jan-2016
233 Views
Preview:
TRANSCRIPT
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
COMPUTER-ASSISTED TESTING
PETYA GEORGIEVA
BULGARIAN STANAG 6001 TESTING TEAM HEAD
2 – 6 September 2013SHAPE (Mons) Belgium
till 1 March, 2012 – approbation of the CAT “writing” module;
till 1 July, 2012 – analysis of the results; since exam session Autumn 2012 – optional CAT “writing”
module; March, 2013 – approbation of CAT modules for “reading”
and “listening”; till 1 July, 2013 – analysis of the results; since exam session Autumn 2013 – optional CAT “writing”,
“reading”, and “listening” modules; since exam session Autumn 2014 – all candidates are to be
tested on computers for the modules “writing”, “reading”, and “listening”(Action Plan 1 iaw Ministerial Order № 857/10.11.2011).
since exam session Autumn 2013 – parallel pen-and-paper and CAT in the three skills.
As of Spring 2015 - End of pen-and-paper STANAG 6001 tests (Action Plan 2 iaw Ministerial Order № 219/19.03.2013 ).
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
*BG CAT Background
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
Administrative Module main menus:ListsProtocolsTasksTestsEvaluation and Analysis
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
Candidates’ module: Reading Test Listening Test Writing Test
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
follows the current Specifications for STANAG 6001 exam for
the Bulgarian Armed Forces; starts with items for L1, then L2, then L3 ; a minimum of 12 and a maximum of 18 questions per
level; level is given for a minimum of 70% correct answers; a candidate can terminate the exam when a level is achieved or decide to go up; 50%-69% correct answers of the next level → (+)
level.
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
Paper-based
withdrawals -16.67%
Failed – Reading (1.5%) Listening (2.5%) Writing (1%)
CAT
withdrawals – 17.16%
Failed – Reading (2.56%) Listening (16.21%) Writing (0%)
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
CAT – 33% of candidates improved their results (R, L, or both).
Reading – 30.76% showed lower results;
33.33% confirmed the results;
35.91% scored higher.
Listening – 27.92% showed lower results;
39.63% confirmed the results;
32.45% scored higher.
Data refers to March 2013 CAT piloting session (R and L).
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
PROS:
candidates can manage their time;decreased candidates’ frustration;time saving (for both candidates and testers);anonymity criteria unconditionally complied with;immediate feedback;highly objective feedback;fewer items are compromised;new items can be piloted and statistics is facilitated.
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
Cons: interface problems for the test-takers
(computer screen clarity, font, size, color of the letters)
due to test-takers’ characteristics (computer literacy, motor skills needed to respond to test items, danger of bad time-management);
no option for test administrators to troubleshoot during the test process.
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
RECOMMENDATIONS FUTURE ACTION PLANS
(based on candidates retrospective written and oral feedback
and test administrators’ impressions);
Several groups:
technical (software perfection, interface layout);
methodological (revision of test components and test administration)
facility use and administrative support
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
Instructions – reviewed;
Layout of the reading material – black font; screen allowing for parallel viewing of the text and answer sheet;
Increase time allotted for looking through the options for L1 and L2 candidates at the Listening;
Reducing number of items if more than 3 per text (R and L).
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
2 – 6 September 2013SHAPE (Mons) Belgium Petya Georgieva
? ? ? ?
? ?
GEORGI RAKOVSKI NATIONAL DEFENSE ACADEMY - SOFIA
BULGARIA
top related