Psy. 525BAdvanced Psychological

Measurement

Introduction to Testingand

Historical Perspective

Why do we test?• To measure ______________________

personalbrandingblog.wordpress.com/2008/12/23/

Types of Tests Individual vs. Group Tests

Single examiner gives a testto a group of people Example:

www.utsa.edu/counsel/Testing.htm academic.hws.edu/kendrick

Types of Tests

• Ability Tests • Personality Tests

Projective

Achievement

Intelligence

Objective

Types of Tests• Norm Referenced

– Individual scores can be compared to a _______

– Generate standardized scores • IQ = SS 102

– Allow ranking

• Criterion Referenced– Measure how well

individual is doing relative to a _____________________ level • Set criterion

– No standardized scores are obtained

– Ranges are used to describe performance• 18 to 30 months

Criterion Referenced

Norm Referenced

Tests Measuring Human “Ability”• Intelligence Tests • Tests of Memory and Learning • Achievement Tests

– Evaluate what an individual has learned• Reading, writing and math

Tests Measuring Personality• Rating scales

– Structured (Objective) Tests • MMPI

– Test taker is presented with a specific stimuli and asked specific questions • I feel sad N S O A• I feel alone N S O A• I don’t have any friends True False

MMPI-2 Sample Questions• I have a good appetite • There seems to be a lump in my throat much of the time • A person should try to understand his dreams and be guided by or

take warning from them • I enjoy detective or mystery stories• I am sure I get a raw deal from life • Evil spirits possess me at times • During one period when I was a youngster I engaged in petty thievery• It would be better if almost all laws were thrown away • A minister can cure disease by praying and putting his hand on your

head• I sometimes keep on at a thing until others lose their patience with me• I have often wished I were a girl.  (Or if you are a girl) I have never been

sorry that I am a girl• I am sad in most days• My life is getting worst every day

Clinical Scales1. Hypochondriasis (Hs)

• Identifies concerns with somatic symptoms and physical well being.

2. Depression (D)• Identifies depressive symptoms

3. Hysteria (Hy)• Identifies those who display need for attention and

affection from others, denial of problems 4. Psychopathic Deviant (Pd)

• Lack of concerns for social rules and lack of acceptance of authority and social alienation

5. Masculinity-Femininity (Mf)• Confusion of gender roles

Clinical Scales6. Paranoia (Pa)

• Suspiciousness, feelings of persecution and complaints about other people

7. Psychasthenia (Pt)• Identifies Obsessive-Compulsive Disorder

8. Schizophrenia (Sz)• Identifies schizophrenic patients and reflects a wide

variety of areas including bizarre thought processes, social alienation, difficulties in concentration and impulse control,

9. Hypomania (Ma)• Identifies Bipolar Patients

10. Social Introversion (Si)• Assess a person’s tendency to withdraw from social

contacts and responsibilities.

Tests Measuring Personality• Projective Tests

– Evoking a response by presenting an ambiguous stimuli• Inkblots, drawings, Sentence completion

Rorschach Inkblot Test

How do we discriminate between “normal” and

“abnormal” performance on tests?

• 13 year old got a raw score of 15 on the communication section of the Vineland. – Does she have age appropriate communication

for a 13 year old?

Norm Referenced Tests• The norm samples are selected randomly from a

given population • Tests get renormed on average every 10 years• Most current US Census is used when selecting a

comparison group – Example: WRMAL-2 was published in 2003,

used March 2001 US Census to obtain the norm group

• The Norm group must match as closely as possible the major demographic characteristics of the population:– Race\Ethnicity, Geographic areas, Gender and

SES

US Population of 8 year olds 400

400

Inferential Statistics

Population Sample for WRAML-2

Number Percent2001 US Census

TAPS-3

Norm Referenced Tests• Sample Size

– At lest 80 to100 subjects in each age group (Sattler, 2008).

– The larger the number of subjects in each norm group, the more stable the norm.

TAPS-3

Interpreting Results of Normed Reference Tests

Types of Standardized Scores • Standard Score (SS) (AKA Deviation IQ

Scores) – M of _______ and a Standard Deviation of ____

• Scaled Scores (ss)– Have Mean of ____ and a Standard Deviation of

___ • T-Scores (McCall’s T)

– Have Mean of ____ and a Standard Deviation of ___

Standard Scores (SS)M = 100 SD = 15

95 105

AverageRange

95 - 105High Av.106-115

Low Av.94-85

Below Av.94-85

Above Av.116-130

Significantly Below Av.

< 70

Superior> 130

Scaled Scores (ss)M = 10 SD = 3

9 11

AverageRange9 - 11

High Av.12-13

Low Av.7-8

Below Av.4-6

Above Av.14-16

Significantly Below Av.

< 4

Superior> 16

7 10 13 16 1941

T Scores M = 50 SD = 10

Normal Limits40 - 60

At Risk Limits

30 – 40

At Risk Limits

61-69

Clinical Limits> 30

Clinical Limits> 70

40 50 60 70 803020

For Adaptive Scales For Clinical Scales

v Scale Scores M = 15 SD = 3

AverageRange12 - 18

Moderately Low9-11

Moderately High19-24

Low< 9 High

> 24

12 15 18 21 2496

Percentile Ranks • Percentile Ranks are used

– To determine the individuals position relative to standardization sample

– Asks “What percent of scores fall below a given score (Xi)?”• Example: 54th % on a working memory test means

that the 54 % of the sample had lower scores than that person

• 46 % of the sample had higher scores than that person

TOMAL-2

Reliability and Validity of Psychological Tests

Validity and Reliability • Validity is the extent

to which a test measures what it is ______________. – Is the test measuring

the construct that we are trying to define and measure

– Example:

• What is Reliability? – In psychological

testing, the term reliability means ________________ or ________________

– A test is considered reliable if it gives us the same result over and over again (assuming that what we are measuring isn't changing).

Reliability

KTEA-II initial assessment KTEA-II given a month later

Test-Retest

What’s a good test reliability?

• ______ high reliability (decision making)• ___________ moderate to high reliability

(could be used with caution for diagnostic purposes)

• <.80 low reliability (should not be used for diagnostic purposes)

• All reliability coefficients (test retest, split half) are reported in the tests manuals

Which Test is more reliable? Test-Retest Reliability

TOMAL

Index r

Verbal Memory .85

Nonverbal Memory .79

Attention/Concent. .88

WRMAL-2

Index r

Verbal Memory .92

Visual Memory .89

Attention/Concent. .91

Validity

Predictive ValidityCorrelation of WJ-III Cognitive and WJ-III

Academic achievement testsWJ-III Academic Clusters

WJ-III Cog. Clusters Reading Decoding

Reading Comp.

Math Calculation

Written expression

General Intellectual Ability

.69 .74 .63 .49

Verbal Ability .61 .68 .47 .39

Thinking Ability .46 .55 .47 .30

In practice, validity coefficients in the range of .30 to .60 are considered high.

Correlation of WISC-IV and WJ-III Academic achievement tests

WJ-III Academic Clusters

WISC-III Scales Reading Decoding

Reading Comp.

Math Calculation

Written expression

Full Scale IQ .58 .60 .46 .42

Verbal IQ .61 .64 .49 .45

Performance IQ .36 .36 .29 .25

Which WISC-III scale is a better predictor of academic achievement?

Construct Validity • The extent to which the test may be said to

measure a theoretical construct.• Intelligence• Memory • Anxiety• Depression

• Construct validity is measured by correlating the test with other measures that– Are known to measure the same construct – Measure a different construct

Construct Validity of NNAT

r = .63

NNATWISC-IV Matrix

Reasoning

r=.65

Construct Validity of NNAT

Correlation of WRAML-2 and CMS Measure of Construct Validity

CMS Indexes

WRAML-2 Indexes Visual Immediate

Verbal Immediate

Attention/Concent.

General Memory

Verbal Memory .05 .36 .11 .21

Visual Memory .37 .55 .52 .48

Attention/Concent. .42 .12 .58 .28

General memory .44 .50 .64 .49

What Norm Referenced Tests Can you use? • It all depends on qualification level of the test established by the publisher

and training using the test – Qualification Level: A

• No special qualifications are required • The range of products eligible for purchase is limited.

– Qualification Level: B• Can purchase A and B Level products

– A degree from an accredited 4-year college or university in psychology, counseling, speech-language pathology, or a closely related field

– Satisfactory completion of coursework in test interpretation, psychometrics and measurement theory, statistics, or a closely related area; or

– license or certification from an agency that requires appropriate training and experience in the ethical and competent use of psychological tests.

What Norm Referenced Tests Can you use?

• Qualification Level: C– Can purchase A, B and C product

• All qualifications for Level B • Plus an advanced professional degree that

provides appropriate training in the administration and interpretation of psychological tests, or

• License or certification from an agency that requires appropriate training and experience in the ethical and competent use of psychological tests.