lakin ability testing introduction - auburn...
TRANSCRIPT
Essentials of Ability Testing
Joni LakinAssistant Professor
Educational Foundations, Leadership, and Technology
Basic Topics
• Why do we administer ability tests?• What do ability tests measure?• How are they different from achievement tests?
• How can ability tests be used to improve instruction?
2
Why administer ability tests?
• VA state law requires it :|• Identifying students for gifted or enrichment
programs :)• Gives an additional perspective on students’
academic development from achievement tests :D
3
What do ability tests measure?
• Should ability scores change over time?• Are ability and achievement different or
related characteristics?
4
Misconceptions about ability testsMyths
• Ability tests measure “innate”, fixed intelligence.
• You’re either “intelligent” or not (a single trait).
• Ability tests are biased against low SES and many race/ethnicity groups.
• “Nonverbal” tests are culture- free and fair to students of all backgrounds.
Realities• Abilities are general skills
that are *less tied* to schooling, but must be practiced and developed.
• Humans have an array of abilities (correlated, but everyone has strengths).
• There is no evidence of bias, but we should be concerned about diversity.
• All tests are culturally bound. Nonverbal ability tests do not solve diversity issues.5
Defining key terms
• Identification vs. screening• Gifted vs. talented• Talent development• Potential vs. accomplishment• Opportunity to learn• Norms-based perspective
6
How can ability tests be used to improve instruction? • Gifted/talented identification• In-class differentiation and enrichment
• Support academic engagement by finding areas of strength and interest
• Build weaknesses using strengths
7
What different ability tests are out there?• Group administered tests
• Multiple format/battery• Single format
• Individually administered tests
8
Commonly used, group-administered ability tests• Multdimensional battery tests
• CogAT: 3 subtests; verbal, quantitative, and figural/nonverbal reasoning
• InView: 5 subtests; verbal reasoning (2 parts), nonverbal sequences and analogies, and quantitative reasoning (figure matrices)
• OLSAT: 5 subtests; verbal comprehension, verbal reasoning, pictorial reasoning, figural reasoning, and quantitative reasoning.
• Single-format nonverbal tests• NNAT: All items same item format (figure matrices)
9
Why do you need multiple batteries?
• “Intelligence” vs. “abilitieS”‐‐intelligence is not a unitary characteristic:
• Everyone has relative strengths and weaknesses in different content
• Learning depends on domain‐specific skills• Classroom activities and pull‐out programs emphasize different skills
• Alignment with services offered
10
CogAT Form 7
• Compared to Cogat 6—New Subtests and Item Types
11
CogAT Form 7• Group-administered abilities test, author: Dr. David Lohman• Provides insight into abilities not fully measured by achievement
tests across three domains• Verbal• Quantitative• Nonverbal
• Completely revised to be more “ELL friendly”• Levels 5/6-8 (grades K-2) bilingual primary battery
• Subtests mostly use pictures
• Reduced language load at Levels 9+• Spanish and English Directions
• Directions are brief and focused
• Online testing12
Picture Analogies
Number Analogies
Picture Classification
Figure Matrices
Figure Classification
Number Series
D/A
D/A
D/A
D/A
Sentence CompletionD/A
Number PuzzlesD/A
Paper FoldingD/A
V
Q
N
= Oral English
= Nonverbal= Oral English or Spanish
= Nonverbal
Levels 5/6 – 8 Subtests
CogAT 7 subtests
Verbal Analogies
Number Analogies
Verbal Classification
Figure Matrices
Figure Classification
Number Series
D/A
D/A
D/A
D/A
D/A
D/A
Sentence CompletionD/A
Number PuzzlesD/A
Paper FoldingD/A
Levels 9 – 17/18 Subtests
13
D/A
D/A
Form 7 New Design
14
15
Form 7
16
17
18
Using ability tests for gifted/talented screening• Have to consider the programs available
• In-class differentiation, enrichment, pull-out programs
• Need for special programming depends most importantly on the discrepancy between a child’s development and that of classmates
• Cast a broad net• Multiple sources of information • Initial screening of all students, then further
assessment• Measure critical aptitudes (later slide)19
Perspective on “gifted and talented”• “Giftedness” is often a label
• Encourages distinctions between “gifted” and “not gifted”
• Assumes that giftedness is a fixed quality
• Often a one-time identification leading to a permanent change in program
• Singular identification methods exclude those with varied profiles
• Encourages narrow concepts of enrichment
• Some parents covet the label to the detriment of their own children and the program
20
Think Aptitude (more than abilities)“The degree of readiness to learn and perform well in a particular situation.”
•Includes abilities and achievements•Not just cognitive (motivation and interest matter)•Not just positive (strengths and weaknesses)•Depends on the context and the activities involved•Not fixed
21
Best practices in screening for talent• Use multiple measures
• Academic: Domain knowledge and demonstrated achievement
• Abilities: Verbal, quantitative, and spatial
• Creativity and interest based on recommendations (teacher, parent, students’ own ratings)
• Take an “OR” perspective rather than an “and” perspective
22
Above Average Ability
Task Interest
Creativity
Adapted From Renzulli’s Three-Ring Conception of Giftedness. In Baum, S. M., Reis, S. M., & Maxfield, L. R. (Eds.). (1998). Nurturing the gifts and talents of primary grade students.
Combining multiple sources of information• Lohman-Renzulli Matrix
23
I = More intensive enrichment (pull out program?)II and III = Other enrichment (special events, in class activities)
Ongoing talent development• All children have special talents that can be developed
(not a binary decision)• Identification measures should indicate readiness for greater
challenge, not just current exceptional performance
• Ongoing identification (revolving door; Renzulli-Reis model)
• Expect that children will excel in some areas and not in all areas
• Take a school-based perspective, encourage identification and development for underprivileged and ELL students
24
Using ability test results for in-class differentiation and enrichment• Requires teachers to know
• Students’ overall ability level• Specific strengths and weaknesses• Appropriate instructional responses to this
information• Need profiles of scores, not a single score• Score Interpretation Guide can help
25
CogAT Profiles • Three Batteries:
• Verbal• Quantitative• Nonverbal
• Scores provided• Overall ability and battery-specific scores
• Pattern and level of scores = Profile• “A” Three battery scores about the sAme level• “B” One score aBove or Below others• “E” Extreme difference ( > 24 IQ-like points)
26
CogAT profile scores
8 B (Q-) 5A 4E (V+ N-)
A= Battery scores about the sAmeB= One score aBove or BelowE= Extreme difference
Median stanine score
Spread of scores
Relative strength/weakness
27
Overall Ability: CharacteristicsMedian Stanine Example characteristicsBelow‐average reasoning abilities(Stanines 1–3)
• difficulty learning abstract concepts• minimal or ineffective strategies for learning and remembering (Tend to rely on trial‐and‐error)
Average reasoning abilities(Stanines 4–6)
• likely to use only previously learned methods when faced with new tasks• difficulty transferring knowledge/skills
Above‐average reasoning abilities(Stanines 7–8)
• ability to learn relatively quickly• good memory, effective learning strategies
Very high reasoning abilities(Stanine 9)
• preference for discovery learning rather than highly structured learning environments (not necessarily solitary environments!
See page 4 of handout; Drawn from CogAT Score Interpretation Guide
28
Overall ability: Building on strengthsDrawn from CogAT Score Interpretation Guide
Median Stanine Example adaptationsStanines 1–3 Look for strengths in terms of specific interests and
achievements. Even more than other students, those who are behind their peers in reasoning abilities often learn more and sustain their efforts longer if the teacher discovers and builds on their interests.
Stanines 4–6 Help them develop the habit of analyzing new tasks to detect relationships with previously learned tasks. Do this by modeling the process for them.
Stanines 7–8 Recognize that these students generally profit most when allowed to discover relationships themselves. Guided discovery methods work better than more structured teaching methods.
Stanine 9 Carefully select challenging instructional materials, special projects, or other enrichment activities.29
Relative strengths: CharacteristicsRelative strength in verbal reasoning:• The students generally do best when they are encouraged
to talk and write about what they are attempting to learn.• These students often have remarkably good memories for
arbitrary sequences of sounds, letters, words, and events. Thus, they typically are above average in spelling; in their knowledge of syntax and grammar; in their ability to learn other languages; and in their ability to remember dialogue, prose, and poetry.
30
Build from relative strengthsDrawn from CogAT Score Interpretation Guide
Strength Example adaptationsV + Avoid pitfalls in math: Students with relatively strong
verbal abilities often find it easier to memorize formulas than to build more abstract conceptual systems. These abstract systems lead to the ability to transfer mathematical knowledge to unfamiliar domains.
Q + Provide opportunities for these students to contribute at high levels to group projects that require math skills. Group projects provide an avenue for building better verbal and spatial reasoning abilities.
N + Encourage students to create drawings when solving problems in mathematics, concept maps when taking notes, or mental models of a scene when reading a text.
31
Develop relative weaknessesDrawn from CogAT Score Interpretation Guide
Weakness Example adaptationsV ‐ Acquaint students with unfamiliar ways of conversing and
writing by providing opportunities to imitate the speaking and writing styles of individuals they admire. Drama, poetry, and storytelling are particularly useful in this regard.
Q ‐ If the difficulty is a lack of experience or the presence of anxiety, provide greater structure, reduce or eliminate competition, reduce time pressures, and allow students greater choice in the problems they solve. Experiencing success will gradually reduce anxiety; experiencing failure will cause it to spike.
N ‐ Provide simple drawings that encapsulate the essential features of the visual mental model required by the problem. Then give students time to examine the drawing and to label it or coordinate it with the text.
32
End of basics; time for questions
33
Advanced Topics (as time allows)
• Can I increase diversity in identified students by using Nonverbal Batteries?
• How do I appropriately interpret test results for English learners?
• What effect does practice have on CogAT scores?
• How long are CogAT scores still useful?
34
Can I increase diversity in identified students by using Nonverbal Batteries?
• Purported benefits:• Culture “fair”• Measure more innate ability
not affected by education• Increase diversity
• Problem: • Less related to most school content than Verbal and
Quantitative ability (less informative for instruction)• Often don’t actually increase diversity in selection!
35
Nonverbal task can actually show larger differences
36
Especially when controlling for other factors…
37
Nonverbal Fails on WISC-IV, too
38
Same “best” predictors of achievementELLstatus
Ethnicity Predictingachv’
Verbal Quant Non‐verbal
non‐ELL
White(n=114)
Math 0.74 0.82 0.73Reading 0.76 0.71 0.66
Hispanic(n=221)
Math 0.63 0.77 0.64Reading 0.69 0.57 0.48
ELL Hispanic(n=178)
Math 0.52 0.66 0.57Reading 0.54 0.50 0.44
CorrelationslowerforELLstudentsbut showSAMEPATTERNofbestpredictors
Meandifferencesnotafactor,Variancemaybe
How different is a correlation of .6 vs. .8?
r = .8 r = .6
Background of students selected
13.4
5.1
16
1.60
6
3.2
0.7
8
2.8
0.1
8
0
2
4
6
8
10
12
14
16
18
FRL ELL Minority
Perc
ent Whole school
VerbalQuantitativeNonverbal
Conclusions about Nonverbal Tests• Nonverbal tests are not a silver bullet solution• Decrease in correlations means you’d select a lot
students who wouldn’t benefit from the program as much as others
• Promised rewards of diversity often don’t materialize
• No clear connection between nonverbal skills and most enrichment options or school content
• More information from : Lakin, J.M., & Lohman, D.F. (2011). The predictive accuracy of verbal, quantitative, and nonverbal reasoning tests: Consequences for talent identification and program diversity. Journal for the Education of the Gifted, 34(4), 595-623.
Viable Alternatives
• Increasing size of program and variety of programs more effective for increasing diversity of program
• Renzulli and Reis’s Schoolwide Enrichment Model• Nonverbal tests can be used as part of a
comprehensive talent identification system • See Lohman & Lakin, 2007; Lohman & Renzulli,
2007
How do I appropriately interpret test results for English learners? (What you should do instead of using nonverbal tests.)
• Challenge: want useful and valid test scores• Some options are available that decrease
the verbal load of ability tests• Consider “readiness for additional
challenge” and “aptitude” continuous evaluations rather than “gifted” and “high achieving” labels
44
Option 1: Reducing the verbal demands of ability tests• Some districts use single-format Nonverbal
Tests (like NNAT), with the problems outlined in the previous section
• Better off using tests with verbal and quantitative domains with modifications to make appropriate interpretations
45
CogAT7: Alternative Verbal Scale
46
Option 2: Using Opportunity to Learn Norms• If the goal is to identify students ready for challenge,
need to see which students know the most given their opportunity to learn the content
• Using national age norms presumes that all students have had equal opportunity to develop the practiced skills• Uses age to control for opportunity to learn (OTL)
• Can use local and subgroup norms to control for clear differences in OTL within age groups
47
Identifying ELs ready for challenge: Opportunity to Learn Norms
• National norms (SAS) show only average ability among most able students
• Instead, can rank-order ELL students at a school
• These are students who can reason best given similar opportunity to learn
• Use multiple years of data to increase comparison group
Student Raw score out of 48
Verbal SAS
Rank within ELLs
1189 48 105 1001107 42 90 971111 41 90 971130 40 88 971145 37 80 951183 37 76 951105 36 75 931124 35 73 931108 34 71 911132 29 69 851118 29 69 851163 27 68 80
48
Using this information
• Subgroup norms show some students are high scoring compared to student with similar OTL
• Does not mean they are ready for the same types of enrichment or gifted programming as other students
• Does mean they are ready to be challenged
49
Programming options for EL students
• Current level of achievement is primary guide • Programming goal: to encourage interests and
improve achievement at a rate faster than would otherwise occur
• For on- and below-grade-level achievement options include: tutors, after-school or weekend classes/clubs, etc. Motivational component critical.
• For achievement well in advance of peers, consider single-subject acceleration
50
What effect does practice have on CogAT scores?• Scores are most valid when students clearly
understand what they are supposed to do • Unequal preparation often occurs (by
accident or well-meaning parents)• Practice
• Must go beyond basic test directions• Levels the playing field • Can involve activities that help teach important
thinking skills 51
CogAT7: Free practice activities
• Teacher guide and student practice booklet • By battery (V, Q, N) • Levels 5/6, 7, 8, 9, 10, 11, 12
52
Teacher’s guide (Reproduced with permission)
53
54
How long are CogAT scores valid?
• Remember: Ability is not fixed• Students develop at different rates• Tempting, but a bad idea to use “old” ability
test scores in later grades
55
Changes over time
Lohman, D. F., & Korb, K. (2006). Gifted today but not tomorrow? Longitudinal changes in ITBS and CogAT scores during elementary school. Journal for the Education of the Gifted, 29, 451-484.56
