francesco gratton 2013 testing in the time of crisis bilc professional seminar stockholm, october 14...
TRANSCRIPT
![Page 1: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/1.jpg)
Francesco Gratton
2013
Testing in the time Testing in the time of crisisof crisis
BILC PROFESSIONAL SEMINARBILC PROFESSIONAL SEMINAR
Stockholm , October 14 - 17, 2013Stockholm , October 14 - 17, 2013
INNOVATIVE TEST DESIGNS AND FORMATSINNOVATIVE TEST DESIGNS AND FORMATS
Lt.Col. F. Lt.Col. F. GrattonGratton
![Page 2: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/2.jpg)
Francesco Gratton
2013
Summary:Summary:
• Past situation (up to Sept 2013)Past situation (up to Sept 2013)
• New Course Of Actions adoptedNew Course Of Actions adopted
• A do-it-all softwareA do-it-all software
• ProposalsProposals
![Page 3: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/3.jpg)
Francesco Gratton
2013
1: Multilevel test (level 1 to 4)
2: Multiple choice questions (60 for L & R)
3: No penalties for wrong answers
4: Duration: R 105’ / L 90’
5: Separate Sections (& levels)
6: # of correct answers multiplied for a coefficient
7: Potential use of “F” factor
RECEPTIVE SKILLS (Listening & Reading)RECEPTIVE SKILLS (Listening & Reading)
Stanag Proficiency Test 1.0 Stanag Proficiency Test 1.0
![Page 4: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/4.jpg)
Francesco Gratton
2013
How STANAG levels were awarded
(Stanag Proficiency TestStanag Proficiency Test 1.0)
# of correct answers
fixed coefficient (1,66)
Multiplied by
RECEPTIVE SKILLSRECEPTIVE SKILLS
![Page 5: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/5.jpg)
Francesco Gratton
2013
CORRECT % CORRECT % % % %
1 1,67 11 18,37 26 43,42 41 68,47 56 93,52
2 3,34 12 20,04 27 45,09 42 70,14 57 95,19
3 5,01 13 21,71 28 46,76 43 71,81 58 96,86
4 6,68 14 23,38 29 48,43 44 73,48 59 98,53
5 8,35 15 25,05 30 50,1 45 75,15 60 100,2
6 10,02 16 26,72 31 51,77 46 76,82
7 11,69 17 28,39 32 53,44 47 78,49
8 13,36 18 30,06 33 55,11 48 80,16
9 15,03 19 31,73 34 56,78 49 81,83
10 16,7 20 33,4 35 58,45 50 83,5
21 35,07 36 60,12 51 85,17
22 36,74 37 61,79 52 86,84
23 38,41 38 63,46 53 88,51
24 40,08 39 65,13 54 90,18
25 41,75 40 66,8 55 91,85
LEVEL 0 LEVEL 1
CORRECT % CORRECT % % % %
1 1,67 11 18,37 26 43,42 41 68,47 56 93,52
2 3,34 12 20,04 27 45,09 42 70,14 57 95,19
3 5,01 13 21,71 28 46,76 43 71,81 58 96,86
4 6,68 14 23,38 29 48,43 44 73,48 59 98,53
5 8,35 15 25,05 30 50,1 45 75,15 60 100,2
6 10,02 16 26,72 31 51,77 46 76,82
7 11,69 17 28,39 32 53,44 47 78,49
8 13,36 18 30,06 33 55,11 48 80,16
9 15,03 19 31,73 34 56,78 49 81,83
10 16,7 20 33,4 35 58,45 50 83,5
21 35,07 36 60,12 51 85,17
22 36,74 37 61,79 52 86,84
23 38,41 38 63,46 53 88,51
24 40,08 39 65,13 54 90,18
25 41,75 40 66,8 55 91,85
LEVEL 1
CORRECT % % CORRECT % % %
1 1,67 11 18,37 26 43,42 41 68,47 56 93,52
2 3,34 12 20,04 27 45,09 42 70,14 57 95,19
3 5,01 13 21,71 28 46,76 43 71,81 58 96,86
4 6,68 14 23,38 29 48,43 44 73,48 59 98,53
5 8,35 15 25,05 30 50,1 45 75,15 60 100,2
6 10,02 16 26,72 31 51,77 46 76,82
7 11,69 17 28,39 32 53,44 47 78,49
8 13,36 18 30,06 33 55,11 48 80,16
9 15,03 19 31,73 34 56,78 49 81,83
10 16,7 20 33,4 35 58,45 50 83,5
21 35,07 36 60,12 51 85,17
22 36,74 37 61,79 52 86,84
23 38,41 38 63,46 53 88,51
24 40,08 39 65,13 54 90,18
25 41,75 40 66,8 55 91,85
LEVEL 2
CORRECT % % % CORRECT % %
1 1,67 11 18,37 26 43,42 41 68,47 56 93,52
2 3,34 12 20,04 27 45,09 42 70,14 57 95,19
3 5,01 13 21,71 28 46,76 43 71,81 58 96,86
4 6,68 14 23,38 29 48,43 44 73,48 59 98,53
5 8,35 15 25,05 30 50,1 45 75,15 60 100,2
6 10,02 16 26,72 31 51,77 46 76,82
7 11,69 17 28,39 32 53,44 47 78,49
8 13,36 18 30,06 33 55,11 48 80,16
9 15,03 19 31,73 34 56,78 49 81,83
10 16,7 20 33,4 35 58,45 50 83,5
21 35,07 36 60,12 51 85,17
22 36,74 37 61,79 52 86,84
23 38,41 38 63,46 53 88,51
24 40,08 39 65,13 54 90,18
25 41,75 40 66,8 55 91,85
LEVEL 2 LEVEL 3
CORRECT % % % % CORRECT %
1 1,67 11 18,37 26 43,42 41 68,47 56 93,52
2 3,34 12 20,04 27 45,09 42 70,14 57 95,19
3 5,01 13 21,71 28 46,76 43 71,81 58 96,86
4 6,68 14 23,38 29 48,43 44 73,48 59 98,53
5 8,35 15 25,05 30 50,1 45 75,15 60 100,2
6 10,02 16 26,72 31 51,77 46 76,82
7 11,69 17 28,39 32 53,44 47 78,49
8 13,36 18 30,06 33 55,11 48 80,16
9 15,03 19 31,73 34 56,78 49 81,83
10 16,7 20 33,4 35 58,45 50 83,5
21 35,07 36 60,12 51 85,17
22 36,74 37 61,79 52 86,84
23 38,41 38 63,46 53 88,51
24 40,08 39 65,13 54 90,18
25 41,75 40 66,8 55 91,85
LEVEL 3 LEVEL 4
![Page 6: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/6.jpg)
Francesco Gratton
2013
1: Functional language assessed in a global manner
2: Structured interview
3: Tailored to the candidate
4: Checks & probes
5: 1 to 2 role plays
SPEAKING (SPEAKING (holistically assessed holistically assessed ))
PRODUCTIVE SKILLSPRODUCTIVE SKILLSPRODUCTIVE SKILLSPRODUCTIVE SKILLSProficiency Test 1.0Proficiency Test 1.0Proficiency Test 1.0Proficiency Test 1.0
![Page 7: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/7.jpg)
Francesco Gratton
2013
Three tasks (one for each level)
PRODUCTIVE SKILLSPRODUCTIVE SKILLSPRODUCTIVE SKILLSPRODUCTIVE SKILLSJFLT 1.0JFLT 1.0JFLT 1.0JFLT 1.0
WRITING (WRITING (holistically assessed holistically assessed ))
![Page 8: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/8.jpg)
Francesco Gratton
2013
Summary:Summary:
• Past situationPast situation
• New COAs adoptedNew COAs adopted
• A do-it-all softwareA do-it-all software
• ProposalsProposals
![Page 9: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/9.jpg)
Francesco Gratton
2013
•Specifications Specifications
•Cut-off score Cut-off score
• Joint DatabaseJoint Database
New COAs adoptedNew COAs adopted
![Page 10: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/10.jpg)
Francesco Gratton
2013
![Page 11: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/11.jpg)
Francesco Gratton
2013
• PURPOSEPURPOSE• PURPOSEPURPOSE
• ADMINISTRATION PROCEDURESADMINISTRATION PROCEDURES• ADMINISTRATION PROCEDURESADMINISTRATION PROCEDURES
• VALIDATION PROCEDURESVALIDATION PROCEDURES• VALIDATION PROCEDURESVALIDATION PROCEDURES
• TEST FORMATTEST FORMAT• TEST FORMATTEST FORMAT
• LEVELS OF LINGUISTIC KNOWLEDGELEVELS OF LINGUISTIC KNOWLEDGE• LEVELS OF LINGUISTIC KNOWLEDGELEVELS OF LINGUISTIC KNOWLEDGE
• TEST CONTENTTEST CONTENT• TEST CONTENTTEST CONTENT
![Page 12: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/12.jpg)
Francesco Gratton
2013
1: Multilevel test (level 1 to 4)
2: Multiple choice questions (60 for L & R)
3: No penalties for wrong answers
4: Duration: R 105’ / L 90’
5: Separate Sections (& levels)
6: # of correct answers multiplied by a coefficient
7: Potential use of “F” factor
RECEPTIVE SKILLS (Listening & Reading)RECEPTIVE SKILLS (Listening & Reading)
Stanag Proficiency Test 1.0 Stanag Proficiency Test 1.0 TWAS HIGH
TIME!!!
![Page 13: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/13.jpg)
Francesco Gratton
2013
Stanag Proficiency Test 2.0Stanag Proficiency Test 2.0New Key-factors New Key-factors
• Each section is a mini-test (L & R)
• Plus levels
• Percentages
• Each section is a mini-test (L & R)
• Plus levels
• Percentages
![Page 14: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/14.jpg)
Francesco Gratton
2013
•Specifications Specifications
•Cut-off score Cut-off score
• Joint DatabaseJoint Database
New COAs adoptedNew COAs adopted
![Page 15: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/15.jpg)
Francesco Gratton
2013
Section 1 –Stanag level 1 (questions from 1 to 15)
No.Correct answ.
Level awarded
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0 0+ 1Section 2 –Stanag level 2 (questions from 16 to 30)
No.Correct answ.
Level awarded
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1 1+ 2Section 3 –Stanag level 3 (questions from 31 to 45)
No.Correct answ.
Level awarded
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
2 2+ 3
Section 4 –Stanag level 4 (questions from 46 to 60)
No.Correct answ.
Level awarded
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
3 3+ 4
JFLT 2.0: RDS (Listening & Reading)
![Page 16: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/16.jpg)
Francesco Gratton
2013
EXAMPLE Level 1 (15 questions) Level 2 (15 questions) Level 3 (15 questions)
Max score: 45
Level 1 (15 questions) Level 2 (15 questions) Level 3 (15 questions)
Max score: 45
All candidates answer correctly to 30 questionsAll candidates answer correctly to 30 questionsAll candidates answer correctly to 30 questionsAll candidates answer correctly to 30 questions
Old test Old test Old test Old test WithWithWithWith they would all get thethey would all get thethey would all get thethey would all get the same scoresame scoresame scoresame score
![Page 17: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/17.jpg)
Francesco Gratton
2013
CANDIDATO CORRECT ANSWERS
LEVEL 1(15 Questions)
LEVEL 2(15 Questions)
LEVEL 3(15 Questions) FINAL LEVEL
BIANCHI 30 10 10 10 3ROSSI 30 11 10 9 2+VERDI 30 13 12 5 2GIALLI 30 15 8 7 1+ARANCIONI 30 15 6 9 1
![Page 18: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/18.jpg)
Francesco Gratton
2013
•Specifications Specifications
•Cut-off score Cut-off score
• Joint DatabaseJoint Database
New COAs adoptedNew COAs adopted
![Page 19: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/19.jpg)
Francesco Gratton
2013
NEW LISTENING & READING ITEMSNEW LISTENING & READING ITEMS
JDB (JOINT DATA BASE)JDB (JOINT DATA BASE) JOINT EFFORTJOINT EFFORT
HOW NOT TONOT TO MAKE JOINT EFFORTSHOW NOT TONOT TO MAKE JOINT EFFORTS
![Page 20: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/20.jpg)
Francesco Gratton
2013
Test-writers involvedTest-writers involved
• Accustomed to military environmentAccustomed to military environment
• Language Testing Seminar Language Testing Seminar
• QualifiedQualified
• Norming SessionsNorming Sessions
![Page 21: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/21.jpg)
Francesco Gratton
2013
THE JDB FLOW
THE JDB FLOW
CARABINIERICARABINIERIARMYARMY
AIR FORCE
AIR FORCE
NAVYNAVY
![Page 22: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/22.jpg)
Francesco Gratton
2013
SEP OCT NOV DEC JAN FEB MAR APR MAJ JUN JUL2012 2013
1st
Group
• PREP
ARATION O
F
FIRST
BATCH O
F
ITEM
S
•SENT T
O OTH
ER
GROUP
Phase1
TEST
ERS M
EET
FOR 1°
REVISI
ON
Phase2
PRE-
TEST
ING
REVISI
ON TRIA
LLIN
#
50 IT
EMS
(-30
%)
Phase3
FINAL
MODIFI
CATION
APPROVAZIO
NE
Phase4
AUGWG GdL GdL GdL
2nd
Group
FEBRUARY 201320 NEW ITEMS.
INTO JDB
•PREP
ARATION O
F
FIRST
BATCH O
F
ITEM
S
•SENT T
O OTH
ER
GROUP
Phase1
TEST
ERS M
EET
FOR 1°
REVISI
ON
Phase2
Phase3
PRE-
TEST
ING
REVISI
ON TRIA
LLIN
#
50 IT
EMS
(-30
%)
Phase4
FINAL
MODIFI
CATION
APPROVAZIO
NE
•PREP
ARATION O
F
FIRST
BATCH O
F
ITEM
S
•SENT T
O OTH
ER
GROUP
Phase1
TEST
ERS M
EET
FOR 1°
REVISI
ON
Phase2
Phase3
PRE-
TEST
ING
REVISI
ON TRIA
LLIN
#
50 IT
EMS
(-30
%)
3rd
Group
4th
Group •REP
ERIM
ENTO
• PREP
ARAZIONE
# 12
0 TTIV
AZIONI
•INVIO
AD A
LTRO
GRUPPO
Phase1
MAJ 201340 NEW ITEMS.
INTO JDB
NOVEMBER 201380 NEW ITEMS.
INTO JDB
TIMINGSTIMINGS
AUGUST 201360 NEW ITEMS.
INTO JDB
![Page 23: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/23.jpg)
Francesco Gratton
2013
Summary:Summary:
• Past situationPast situation
• COA (specs, JDB, cut-off score)COA (specs, JDB, cut-off score)
• A do-it-all softwareA do-it-all software
• ProposalsProposals
![Page 24: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/24.jpg)
Francesco Gratton
2013
WHAT’S THE DIFFERENCE ?WHAT’S THE DIFFERENCE ?
![Page 25: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/25.jpg)
Francesco Gratton
2013
PC-assessment-related terminologyPC-assessment-related terminologyTerm Definition Stakes
Assessment Any systematic method of obtaining evidence (through questions) for a purpose.
Quiz … measures for the purpose of providing feedback to the student. Low
Survey … to determine needs required to fulfill a defined purpose. Low
Test … measures knowledge for the purpose of informing the student on their current level
Medium
Exam … measures knowledge for the purpose of documenting the current level of knowledge High
![Page 26: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/26.jpg)
Francesco Gratton
2013
the software is used for:the software is used for:
• Needs analysis:(surveys) • Placement test• Any training activity• Assessment:
– First level survey – Post-course– Pre-certification – CertificationCertification
![Page 27: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/27.jpg)
Francesco Gratton
2013
Software’s system Software’s system Schema Schema
Software’s system Software’s system Schema Schema
Create questions
And organize them in
tests using a windows
based PC
![Page 28: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/28.jpg)
Francesco Gratton
2013
Assessment ……Assessment ……
via Browser
QuestionManager
AssessmentManager
AssessmentDefinitions
Questions
… allows to choose:
• Time limits
• Feedback to test-taker
• Styles (Template)
• Jumps
• Question shuffling
• Instructions to test-
takers
… allows to choose:
• Time limits
• Feedback to test-taker
• Styles (Template)
• Jumps
• Question shuffling
• Instructions to test-
takers
On Windows PC
QuestionManager
AssessmentManager
![Page 29: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/29.jpg)
Francesco Gratton
2013
AssessmentsAssessments
• … also created with
authoring manager by
selecting Qs previously
created
• Any question can be
chosen from the
database
• … also created with
authoring manager by
selecting Qs previously
created
• Any question can be
chosen from the
database
AssessmentDefinitions
Questions
su PC Windows
QuestionManager
AssessmentManager
via Browser
QuestionManager
AssessmentManager
![Page 30: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/30.jpg)
Francesco Gratton
2013
Software’s system Software’s system Schema Schema
Software’s system Software’s system Schema Schema
Create questions
And organize them in
tests using a windows
based PC
Set security parameters,
schedule assessment and link to
other (Learning
Management Systems
assessment published using any browser, secure browsers, or a PC/MAC
Result reports, CIA, graphs, gimmicks, you name it …..
![Page 31: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/31.jpg)
Francesco Gratton
2013
Types of Questions: Multiple ChoiceTypes of Questions: Multiple ChoiceTypes of Questions: Multiple ChoiceTypes of Questions: Multiple Choice
![Page 32: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/32.jpg)
Francesco Gratton
2013
Likert Scale (for questionnaires) Likert Scale (for questionnaires)
![Page 33: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/33.jpg)
Francesco Gratton
2013
Essay QuestionEssay Question
1. Candidate can write free text in the space provided
2. Testers will evaluate later
1. Candidate can write free text in the space provided
2. Testers will evaluate later
![Page 34: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/34.jpg)
Francesco Gratton
2013
![Page 35: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/35.jpg)
Francesco Gratton
2013
Summary:Summary:
• Past situationPast situation
• COA (specs, JDB, cut-off score)COA (specs, JDB, cut-off score)
• A do-it-all softwareA do-it-all software
• Proposals Proposals
![Page 36: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/36.jpg)
Francesco Gratton
2013
• Wide projectWide project
• Sharing experience & Sharing experience &
capabilitiescapabilities
• Optimization of resourcesOptimization of resources
• No alternative to B.A.T.No alternative to B.A.T.
Testing in the time Testing in the time of crisisof crisis
![Page 37: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/37.jpg)
Francesco Gratton
2013
![Page 38: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/38.jpg)
Francesco Gratton
2013
A Bilateral-based CDB
A multilateral-based CDB
COMBINED DATABASE
or …
or …
![Page 39: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/39.jpg)
Francesco Gratton
2013
![Page 40: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/40.jpg)
Francesco Gratton
2013
FLOWFLOW
TIME SCHEDULETIME SCHEDULE
SPECSSPECS1 2
3
![Page 41: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/41.jpg)
Francesco Gratton
2013
![Page 42: Francesco Gratton 2013 Testing in the time of crisis BILC PROFESSIONAL SEMINAR Stockholm, October 14 - 17, 2013 INNOVATIVE TEST DESIGNS AND FORMATS Lt.Col](https://reader036.vdocuments.mx/reader036/viewer/2022081603/56649dfe5503460f94ae647a/html5/thumbnails/42.jpg)
Francesco Gratton
2013
Thank youThank you
[email protected]@gmail.com
[email protected]@sclingue.esercito
.difesa.it.difesa.it