itab innovative assessments

ITABITAB IT Aptitude BatteryIT Aptitude Battery

© CogniMetrics Inc., 2014

• The ITAB is the world’s first battery of completely interactive tests of human intelligence.

• Artificial Intelligence is used to compute the ‘intelligence’ of each problem-solving action.

• The ITAB measures fluid intelligence with a proven validity for technical occupations (and math and science).

ITAB: innovative features

ITABITAB IT Aptitude BatteryIT Aptitude Battery

ITAB tests have a game-like character. This gives ITAB tests an advantage in testing populations that have developed an aversion against school math and the static logic of traditional tests of intelligence.

The tests provide an interactive task environment in which the examinee is free to act to achieve a goal. The tests measure:

• how well feedback is incorporated into follow-up problem-solving actions• how quickly this leads to a more or less efficient skill• the level of skill efficiency that the examinee has developed during the assessment


ITABITAB teststests

ITAB 1: Hidden Target Test

Task/Goal: The examinee has to locate a hidden target in as few steps as possible by moving a cursor across the grid using a set of arrow keys. To indicate his or her first guess the subject clicks at the <TEST> key. The feedback consists of the distance between the tested location and the location of the hidden target in City Block metric..

<START> bar

<MOVE> <TEST> keys

HELP menu


Feedback window

ITABITAB teststests

ITAB 2: Battery Test

Task/Goal: The test requires the examinee to identify two defective batteries in a set of nine batteries. The battery tester is designed such that batteries have to be tested in pairs.

HELP MENU

<REJECT> button<TEST> button

<START> button

Feedback window


Interactive Tests

Each ITAB test provides an interactive task-environment. An examinee action (user action) changes the environment (system action) and the examinee receives feedback about the new state of the task-environment (system output).

The diagnostic components of the ITAB tests measure these changes per user step and analyses:

• sequential aspects of the user actions

• The informational content of each new system state

ITABITAB teststests



workstyle

intel-ligence

test score

ITABITAB test scorestest scores

ITAB: TEST REPORT

The diagnostic components of the ITAB tests organize the many measurements into three scores that are intuitively accessible.


e10

e11


workstyle

intel-ligence

ITAB: TEST REPORT

solutionsteps

responsetime

test score


e10

e11


workstyle

intel-ligence

ITAB: TEST REPORT

solutionsteps

responsetime

data

test score


e10

e11


workstyle

intel-ligence

ITAB: TEST REPORT

solutionsteps

responsetime

test score

solutionsteps e10

response

time e11

Uncertainty 02e2

Uncertainty 01e1

Prob. SuccessAt Step 6e3

Working MemoryErrorse8

PartitioningErrorse7

Information Transfer (5)e6

Quality ProblemModele5

Strategy Usee4


ITABITAB battery test battery test

workstyle

intel-ligence

ITAB: TEST REPORT

test score

ITABITAB hidden target test hidden target test

Consistency 2e2

Consistency 1e1

Guessinge3

FeedbackIntegration 2e8


InformationTransfer 2e6


ProblemSolvinge4

work style

PP

intel- ligence

solutionsteps e10

response

time e11


ITAB: TEST REPORT

testscore

Analysis of data

workStyle

intel-ligence



testscore

ITAB: TEST REPORT

The standard ITAB report provides three scores:

•A prediction score•Two diagnostic scores



workStyle

testscore

intel-ligence

ITAB: TEST REPORT

Reports can be customized toclients’ special needs.

The next slides present twoexamples of possible additionsto the ITAB report

Uncertainty 02e2

Uncertainty 01e1

Prob. SuccessAt Step 6e3

Working MemoryErrorse8

PartitioningErrorse7

Information Transfer (5)e6

Quality ProblemModele5

Strategy Usee4


ITABITAB battery test battery test

workstyle

intel-ligence

ITAB: TEST REPORT

Example #1: if a job requires carefuldecision-making, because the conse-quences of a wrong decision may besevere, explicit reports of subtests U-01 and U-02 may be informative.

U-01 and U-02 are two strict parallel tests measuring the amount of risk examinee usually accepts with decision-making.

Very reliable assessment of this attribute: Split-half r: 0.86.Work Style factor loadings: 0.866

ITABITAB hidden target test hidden target test

Consistency 2e2

Consistency 1e1

Guessinge3





ProblemSolvinge4

work style

intel- ligence


ITAB: TEST REPORT

Example #2: Does examinee use feedback from the task-environmentfor his follow-up actions? Using feedback from the envirnment is the core of intelligent functioning, in particular in technology work environments.

URAT 03 and URAT 04 are two strictparallel tests measuring how well examinee uses feedback and acquiresefficient skills

Very reliable assessment of this attribute: Split-half r:= 0.951Intelligence factor loadings: 0.782


ITABITAB relation with ASVABrelation with ASVAB

ASVAB

0, 1

QUANTI-TATIVE

MK

0, v3

e2

1

AR

0, v1

e1

1

0, 1

VERBAL

PC

0, v1

e4

WK

0, v3

e3

11

0, 1

TECHNICALKNOWLEDGE

MC

0, v1

e7

GS

0,

e6

11

AS

0, v1

e8

1

EI

0,

e9

1

AO

0,

e5

1

0, 1

GENERAL

Selection & Classification

ASVAB has two functions:•Screening / selection (on trainability)•classification (optimal match)

ASVAB factor structure: a bi-factor model

•Each test score determined by the general factor and a group factor

•A bi-factor structure causes composite scores for selection and classification to be highly correlated (r’s > 0.85)

•For classification purposes additional tests are needed



Selection & Classification

ASVAB

0, 1

QUANTI-TATIVE

MK

0, v3

e2

1

AR

0, v1

e1

1

0, 1

VERBAL

PC

0, v1

e4

WK

0, v3

e3

11

0, 1

TECHNICALKNOWLEDGE

MC

0, v1

e7

GS

0,

e6

11

AS

0, v1

e8

1

EI

0,

e9

1

AO

0,

e5

1

0, 1

GENERAL

ITAB

0,

e10

1

Criteria for adding a test to improve classification:

•Test should possess unique and reliable variance

•This unique variance should have predictive validity

ITAB tests do meet these criteria:

•ITAB tests exclusively load on the general factor (0.446 - 0.512 over 5 Navy samples of N=300). None of the group factors loaded on the ITAB tests.

•Reliable and unique variance estimated between 65% and 72%.



Predictive Validity

Next page: results of a incremental validity study of ITAB over two ASVAB Selection Composites (here referred to as: ASC01 and ASC02) that are used for the classification of recruits for technical training.

ASC01 = VE + AR + MK + MC;ASC02 = MK + AR + GS + EI;

The trainings modules are part of the U.S. Navy Apprentice Technical Training program , a completely computer based modular training program (self-paced) for electronics and electrics vocations.


Predictive ValidityTable 1: Incremental validities of ITAB over ASVAB Selection Composites (ASC01 & ASC02 ) in a sample with unrestricted variance. Incremental validity is expressed as increase in percentages of explained

criterion variance. Criterion variables are the knowledge scores of ATT modules

ATT Training Module N (CMR)

name Reliability ASC ITAB R² ΔR²

ASC01

Intro to Electricity 539 1-D 0.51 0.238 0.158 0.109 0.052

Multi-meter Measurements 539 2-D 0.53 0.31 0.154 0.155 0.058

Basic DC Circuits 530 3-D 0.58 0.456 0.285 0.384 0.176

Intro to AC 539 6-D 0.51 0.326 0.158 0.169 0.062

AC Test Equipment 539 7-D 0.35 0.196 0.137 0.077 0.038

Transformers 539 12-D 0.47 0.299 0.169 0.155 0.065

Intro to Digital Circuits 539 23-D 0.55 0.135 0.03 0.022 0.004

Digital Logic Functions 539 24-D 0.46 0.003 0.088 0.008 0.008

ASC02

Intro to Electricity 548 1-D 0.51 0.269 0.172 0.135 0.063

Multi-meter Measurements 548 2-D 0.53 0.334 0.162 0.177 0.065

Basic DC Circuits 539 3-D 0.58 0.465 0.293 0.400 0.184

Intro to AC 548 6-D 0.51 0.416 0.181 0.260 0.087

AC Test Equipment 548 7-D 0.35 0.227 0.147 0.097 0.046

Transformers 548 12-D 0.47 0.27 0.19 0.146 0.073

Intro to Digital Circuits 548 23-D 0.55 0.125 0.038 0.020 0.005

Digital Logic Functions 548 24-D 0.46 -0.037 0.074 0.005 0.004

50.17%

88.43%

100.05%

31.07%

256.40%

86.79%

58.35%

84.95%

84.56%

58.77%

99.74%

73.09%

21.12%

88179.91%

criterion variables r (predictors) Incremental Validity

% Improvement

92.40%

60.84%


ATT Training Module N (CMR)name Reliability ASC.. ITAB R² ΔR²

ASC01

Intro to Electricity

Multi-meter Measurements 539 2-P 0.53 0.272 0.158 0.130 0.0563

Basic DC Circuits 530 3-P 0.63 0.42 0.275 0.336 0.1597

Intro to AC

AC Test Equipment 516 7-P 0.62 0.259 0.135 0.111 0.0437

Transformers 539 12-P 0.63 0.272 0.241 0.180 0.1058

Intro to Digital Circuits 524 23-P 0.61 0.27 0.146 0.123 0.05

Digital Logic Functions 503 24-P 0.54 0.144 0.128 0.051 0.0298

ASC02

Intro to Electricity

Multi-meter Measurements 548 2-P 0.53 0.293 0.166 0.148 0.0625

Basic DC Circuits 539 3-P 0.63 0.446 0.283 0.370 0.1707

Intro to AC

AC Test Equipment 523 7-P 0.62 0.26 0.141 0.114 0.0462

Transformers 548 12-P 0.63 0.282 0.252 0.194 0.1145

Intro to Digital Circuits 533 23-P 0.61 0.282 0.16 0.138 0.058

Digital Logic Functions 510 24-P 0.54 0.108 0.132 0.039 0.0277

68.35%

144.02%

72.93%

237.14%

72.78%

85.82%

65.11%

143.01%

68.61%

143.72%

criterion variables r predictors Incremental Validity% Improvement

76.03%

90.54%


Predictive ValidityTable 2: Incremental validities of ITAB over ASVAB Selection Composites (ASC01 & ASC02 ) in a

sample with unrestricted variance. Incremental validity is expressed as increase in percentages of explained criterion variance. Criterion variables are the skill scores of ATT

modules


Measurement claim: procedural skill learning(aspect of fluid intelligence)

Number of tests: 2 (HTT and BT)Administration time: 10 minutes per test (average)Languages: English, Spanish, German,

DutchTest Publisher: CogniMetrics Inc.Published: March 2004

Price: see: www.cognimetrics.com bulk discounts negotiable

Reliabilities: HTT (0.87 – 0.96)BT (0.85 – 0.92)ITAB (0.87 – 0.94)

FOR MORE INFO...

Download ITAB manuals: www.cognimetrics.com

ITABITAB informationinformation

ITAB: Technical Information


Thanks for reviewing thisCogniMetrics presentation

For further information,Please contact:

Dr. Martin J. Ippel

(210) 481 – [email protected]


itab innovative assessments

Documents

itabitab teststests

battery test taskgoal

hidden target test taskgoal

assessment cognimetrics

tests measure

examinee action user

innovative features

environment system action