PSYCHOLOGICAL TESTS AND MEASUREMENT

PSY 425

A common complaint: Psychological variables can’t be measured.

We regularly make judgments about who is shy and who isn’t; who is attractive and who isn’t; who is smart and who is not.

Implicit in these statements is the notion that some people are more shy, for example, than others

This kind of statement is inherently quantitative.

Quantitative: It is subject to numerical qualification.

If it can be numerically qualified, it can be measured.

Why is Psychological Testing Important?Why is Psychological Testing Important?

1. 1. Allows us to make important decisions about people. e.g. Early School Placement, College Entrance Decisions, Military Job Selections

2. Allows us to describe & understand behaviour

3. Measures personal attributes

4. Measures performance

Why is Psychological Testing Important?Why is Psychological Testing Important?

4. Measures performance

5. Saves time

6. Most economical

7. It’s Scientific

Psychological Test DefinitionPsychological Test Definition

Is a measurement instrument that consists of a sample of behavior obtained under standardized conditions and evaluated using established scoring rules.

Characteristics of Psychological InstrumentsCharacteristics of Psychological Instruments

• Behaviour SamplingBehaviour Sampling

• StandardizationStandardization

• Scoring RulesScoring Rules

Behaviour SamplingBehaviour Sampling

• • It is a sample of behaviour.It is a sample of behaviour.

• • It is not an exhaustive measure - it is too difficult It is not an exhaustive measure - it is too difficult to evaluate every behavior.to evaluate every behavior.

• • Attempts to approximate the exhaustive Attempts to approximate the exhaustive procedure.procedure.

•• Does not necessarily require overt behavior.Does not necessarily require overt behavior.

• • The test must be representative of behaviors that The test must be representative of behaviors that would be observed outside the testing situationwould be observed outside the testing situation

StandardizationStandardization

• The behavior sample is obtained under standardized conditions.

• Each individual taking a psychological or educational test should be tested under essentially identical conditions.

For example, SAT administration instructions pertain to:

Seating Arrangements, Lighting Conditions, Noise Levels Interruptions, Answering common questions

• Standardization is vital because many test results are referential in nature: Your performance is measured relative to everybody else’s performance.

Standardization (cont.)Standardization (cont.)

• Standardization reduces between subject variability due to extraneous variables.

• Standardization is easier to obtain with tests designed to be administered en masse.

• Tests such as the Wechsler Adult Intelligence Scale, which are administered individually, are less standardized.

• The individual giving the test is an important variable. They take special training to standardize the way they give the test.

Scoring RulesScoring RulesThere are established scoring rules for obtaining quantitative information from the behavior sample.

Objective Scoring Rules : Most mass produced tests fall into this category. Different qualified examiners will all come to the same score for an identical set of responses.

Subjective Scoring Rules : When the judgement of the examiner is an important part of the test, different examiners can legitimately come to different conclusions concerning the same sample of behavior. There conclusions should be similar, however.

Good standardized psychological tests all have a set of rules or procedures for scoring responses to a test.

Types of TestsTypes of TestsMost psychological tests can be sorted into 3 general categories:

1. Tests in which the subject performs a task.

2. Tests that involve observations of the subject’s behaviour within a particular context.

3. Self-report measures

Tests of PerformanceTests of Performance•• Referred to as "Tests of Maximal Performance"

•• Subjects are given a well-defined task that they try to perform successfully.

•• Participant must know what he/she must do in response to the task.

•• The subject exerts maximal effort to succeed.

•• Performance tests are designed to uncover what an individual can do, given the specific test conditions.

Examples

- Intelligence Tests, language proficiency- Biology test, flight simulator

Behavior ObservationBehavior Observation

• • Naturalistic observationNaturalistic observation

• • Involves observing the subject’s behavior and responses Involves observing the subject’s behavior and responses in a particular context.in a particular context.

• • Differs from performance tests in that the subject does Differs from performance tests in that the subject does not have a single, well defined task.not have a single, well defined task.

• • The observer can record duration & intensityThe observer can record duration & intensity

Examples

- Examiner might observe children interacting or an individual having a conversation or some other social interaction.

- Companies recruit observers to pose as salespeople to observe employee’s behaviors. Subject’s may be unaware they are being tested.

Self Report InstrumentsSelf Report Instruments- - Participant is asked to report his or her feelings, Participant is asked to report his or her feelings,

attitudes, beliefs, values.attitudes, beliefs, values.

When self-report makes senseWhen self-report makes sense:: Self-report relies upon the test taker’s awareness and Self-report relies upon the test taker’s awareness and

honesty and minimal external contingencies. honesty and minimal external contingencies. It is the It is the bestbest method to measure internal states - things method to measure internal states - things

only the person themselves can be aware of and judge.only the person themselves can be aware of and judge.

People are not always good judges of their abilityPeople are not always good judges of their ability Provides an estimateProvides an estimate

Self Report Instruments (cont.)Self Report Instruments (cont.) Many personality inventories such as the MMPI and the Many personality inventories such as the MMPI and the

16PF measures are based on self-report. 16PF measures are based on self-report.

Clinicians include self-report measures as part of their Clinicians include self-report measures as part of their initial examinations of presenting clients. initial examinations of presenting clients.

Self-Report measures are frequently subject to self-Self-Report measures are frequently subject to self-censorship. censorship.

People know their responses are being measured and People know their responses are being measured and wish to be seen in a favorable light. (wish to be seen in a favorable light. (self-serving biasself-serving bias) )

Items are frequently included to measure the extent to Items are frequently included to measure the extent to

which people provide sociallywhich people provide socially desirable responses. desirable responses.

Application of Psychological MeasurementApplication of Psychological Measurement

Educational TestingEducational Testing

Personnel TestingPersonnel Testing

Clinical TestingClinical Testing

Educational Testing Educational Testing

• Intelligence tests and achievement tests are used from an early age in the U.S and Canada. From kindergarten on, tests are used for placement and advancement.

•Educational institutions have to make admissions and advancement decisions regarding students. e.g, SAT, GRE, subject placement tests

• Used to assess students for special education programs. Also, used in diagnosing learning difficulties.

•Guidance counselors use instruments for advising students.

•Investigates school curriculum.

Personnel TestingPersonnel Testing

• Following WW I, business began taking an active interest in testing job applicants. Most government jobs require some civil service examination.

•Tests are used to assess: training needs, worker’s performance in training, success in training programs, management development, leadership training, and selection.

• For example, at the Lally School of Management, the Myers -Briggs type indicator is used extensively to assess managerial potential. Type testing is used to hopefully match the right person with the job they are most suited for.

Clinical TestingClinical Testing

• Tests of Psychological Adjustment and tests which can classify and/or diagnose patients are used extensively.

• Psychologist generally use a number of objective and projective personality tests.

Neuropsychological tests which examine basic mental function also fall into this category. Perceptual tests are used detecting and diagnosing brain damage.

Testing Activities of PsychologistsTesting Activities of Psychologists

Clinical Psychologists - e.g. Assessment of Intelligence, Assessment of Psychopathology

Counseling Psychologists e.g. Career Interest Inventories, Skill Assessment

School Psychologists

e.g. Assessment of Academic progress, Readiness for School,

Social Adjustment

Testing Activities of PsychologistsTesting Activities of Psychologists

I/O Psychologists - e.g. Managerial potential, Training Needs, Leadership Potential

Neuropsychologists - e.g., Assessment of Brain Damage, neurological impairments.

Forensic Psychology - intersection between law and psychology --needed for legal determinations

e.g. Assessment for risk, competency to stand trial, child custody

Information About TestsInformation About TestsThe Mental Measurement Yearbook - A guide to all currently available psychological tests.

The MMY uses content classifications do describe tests:

1. Acheivement 2. Behavior Assessment

3. Developmental 4. Education

5. English & Language 6. Fine Arts

7. Foreign Languages 8. Intelligence and Aptitude

9. Mathematics 10. Neuropsychological

11. Personality 12. Reading

13. Science 14. Sensory-Motor

15. Social Studies 16. Speech and Hearing

17. Vocations

History of Test DevelopmentHistory of Test Developmentcirca 1000 BC. : Chinese introduced written tests to help fill civil service positions Civil Laws, Military Affairs, Agriculture, Geography

1850 : The United States begins civil service examinations.

1885 : Germans tested people for brain damage

1890 : James Cattell develops a "mental test" to assess college students . Test includes measures of strength, resistance to pain, and reaction time.

1905 : Binet-Simon scale of mental development used to classify mentally retarded children in France.

1914 : World War I produces need in U.S. to quickly classify incoming recruits. Army Alpha test and Army Beta test developed. Looked at psychopathology.

1916 : Terman develops Stanford - Binet test and develops the idea of Intelligence Quotient

History of Test Development (cont.)History of Test Development (cont.)

1920 - 1940 : factor analysis, projective tests, and personality inventories first appear.

1941-1960 : vocational interest measures developed

1961-1980 : item response theory and neuropsychological testing developed

1980 - Present : Wide spread adaptation of computerized testing. "Smart" Tests which can give each individual different test items develop

1990 - Present : Wide spread use of advanced statistical procedures to improve tests (e.g., confirmatory factor analysis).

Early Abuses of Tests in AmericaEarly Abuses of Tests in America• Goddard (1906) began testing 378 residents and categorized them asIdiot (ma below 2), imbecile (3-7), feebleminded (8-12), moron (foolish)

• MA four years behind, were feebleminded

• Goddard’s desire was to separate people out Believed feeble minded people were the cause of most social problems (thievery, laziness, alcoholism, prostitution, immorality).

• Called for the colonization of “morons” to restrict their breeding. Further, he believed that many immigrants were feeble minded.

• Went to Ellis Island, administered tests translated from French to English to Yiddish, Hungarian, , Italian, Russian, to farmers, laborers, who had just crossed the Atlantic. Then interpreted results based on French norms.

• Favored “deportation for low IQ immigrants” but then also in a “humanitarian

gesture” said we might be able to use “moron laborer” if only “we are wise

enough to train them properly.”

Early Abuses of Tests in America (cont.)Early Abuses of Tests in America (cont.)

• Robert Yerkes, a Harvard psychology prof. Convinced the Department of War that it should test all of its 1.75 million recruits for intelligence tests, so they could be classified and given appropriate assignments (Goddard and Terman also chaired this committee).

• Army Alpha

• Army Beta Examinations

• Produced evidence that supported segregation. Sounded dire warnings that racial intermixture would inevitably cause a deterioration of American intelligence. Later recanted: “without foundation” Probably the result of cultural and language differences.

Ethics In Psychological TestingEthics In Psychological Testing

• Given the widespread use of tests, there is considerable potential for abuse.

• A good deal of attention has therefore been devoted to the development and enforcement of professional and legal standards.

• The American Psychological Association (APA) has taken a leading role in the development of professional standards for testing.

American Psychological Association Ethical Guidelines:American Psychological Association Ethical Guidelines:

The investigator has the responsibility to make a careful evaluation of its ethical acceptability.

The investigator is obliged to observe stringent safeguards to protect the rights of human participants.

The researcher must evaluate whether participants are considered “Subject at risk” or “Subject at minimal risk” - No appreciable risk (physical risk, mental harm).

The principal investigator always retains the responsibility for ensuring ethical practice in research. That is, the principal researcher is responsible for the ethical practices of collaborators, assistants, employees, etc. (all of whom are also responsible for their own ethical behavior).

Except in minimal-risk research, the investigator establishes a clear and fair agreement with participants that clarifies the obligations and responsibilities of each. Must explain all aspects of the research that may influence the subjects decision to participate. Explains all other aspects that the participants inquire about.

American Psychological Association Ethical Guidelines (cont.):American Psychological Association Ethical Guidelines (cont.):In research involving concealment or deception, the research considers the special responsibilities involved.

Individual’s freedom to decline, and freedom to withdraw, is respected.Researcher is responsible for protecting participants from physical and mental discomfort, harm, and danger that may arise from research procedures. If there are risks, the participants must be aware of this fact.

After the data are collected the investigator provides participants with information about the nature of the study and attempts to remove any misconceptions that may have arisen.

The investigator has the responsibility to detect and remove any undesirable consequences to the participant that may occur due to the research.

The information obtained from the participant should be treated confidentially unless otherwise agreed upon with the participant.

Informed ConsentInformed Consent•Participants must be fully informed as to the purpose and nature of the research that they are going to be involved in.

•Participants must be fully informed about the procedures used in the research study.

•After getting this information, the participants must provide consent for their participation.

•Participants must be informed about their right to Confidentiality and their right to withdrawal without penalty.

DebriefingDebriefing

Post-administration debriefing should:-Restate purpose of the research.-Explain how the results will be used (usually. emphasize that the interest is in the group findings).-Reiterate that findings will be treated confidentially.-Answer all of the respondents questions fully.-Thank the participant!

Participant FeedbackParticipant Feedback

•In clinical research, or research with interpretive instruments, there may be the need to provide more in-depth feedback about individual’s responses (e.g., Research on Emotional Intelligence).

•In such cases, first and foremost, it is critical that this kind of detailed feedback be given by a qualified individual.

At Least 4 parties are involved in Professional Test Use:

(1) Testing professionals: the test developer and publisher (2) Testing professionals: the individuals who administer the testing procedure

(3) The user: the organization or practice that will eventually use the information to make certain decisions

(4) The test taker

DEVELOPING/SELECTING APPROPRIATE TESTS

Define what each test measures and what the test should be used for.

Describe the population(s) for which the test is appropriate.

Accurately represent the characteristics, usefulness, and limitations of tests for their intended purposes.

Describe the process of test development.

Provide evidence that the test meets its intended purpose(s).

Provide either representative samples or complete copies of test questions, directions, answer sheets, manuals, and score reports to qualified users.

DEVELOPING/SELECTING APPROPRIATE TESTS (Cont.)

Indicate the nature of the evidence obtained concerning the appropriateness of each test for groups of different racial, ethnic, or linguistic backgrounds who are likely to be tested.

Describe the population(s) represented by any norms or comparison group(s), the dates the data were gathered, and the process used to select the samples of test takers.

When feasible, make appropriately modified forms of tests or administration procedures available for test takers with handicapping conditions. Warn test users of potential problems in using standard norms with modified tests or administration procedures that result in non-comparable scores.

DEVELOPING/SELECTING APPROPRIATE TESTS (Cont.)

When a test is optional, provide test takers or their parents/guardians with information to help them judge whether the test should be taken, or if an available alternative to the test should be used.

Provide test takers the information they need to be familiar with the coverage of the test, the types of question formats, the directions, and appropriate test-taking strategies. Strive to make such information equally available to all test takers.

Provide test takers or their parents/guardians with information about rights test takers may have to obtain copies of tests and completed answer sheets, retake tests, have tests rescored, or cancel scores.

Tell test takers or their parents/guardians how long scores will be kept on file and indicate to whom and under what circumstances test scores will or will not be released.

Responsibility of The TesterResponsibility of The Tester1. Have competence in test administration, interpretation and feedback.

2. Have an understanding of basic psychometrics and scoring procedures and be competent in interpretation, and apply scientific knowledge and professional judgment to the results.

3. Take responsibility for the selection, administration, and scoring, the analysis, interpretation and communication of test results.

4. Be familiar with the context of use: the situation, purpose, setting in which a test is used.

Responsibility of The Tester (cont.)Responsibility of The Tester (cont.)

5. Have knowledge of legal and ethical issues related to test use

6 . Awareness of ethnic or cultural variables that could influence the results:

7. Have the ability to determine language proficiency

8. Have knowledge of important racial, ethnic, or cultural variables relevant for individuals or groups to whom tests are administered.

Issues to Address with the TesteeIssues to Address with the Testee

1. Informed consent - Assuring confidentiality, freedom’ to withdraw, purpose of assessment, What kinds of attributes are being measured?

2. Who is the client?Individual, Group, Employer3. What happens with results, who has access to it4. Where will the data be stored, how, and for how long5. Time frame in which results are to be considered valid6. Who will be the payer, and how much7. Where will the assessment take place8. Are the facilities appropriate, conducive for testing9. Will there be follow-up assessments or feedback

Factors Not Under the Tester’s ControlFactors Not Under the Tester’s Control

Some factors are not under the control of the administrator :

1. How fatigued a test taker is.

2. Motivation level of the test taker.

3. Physical Discomfort

4. Test Anxiety

Knowledge of attitudes of various racial, ethnic, or cultural groups toward testing.

Ability to determine language proficiency.

Ability to determine the potential effects of different test settings on different racial, ethnic, or cultural groups.

Knowledge of specific biases that have been demonstrated for particular tests for individuals or groups of individuals from particular racial, ethnic, or cultural minority groups.

People with different values often disagree over the fairness of some testing practices.

Factors that affect testing fairness:1. Obstacles that prevent people from performing

well 2. Test may provide unfair advantage to some people 3. Some tests are not valid and used in wrong situations 4. Some tests are used for purposes that are inherently objectionable

A test is most likely to be seen as unfair when:1. It is the sole basis for the decision.2. The consequences of doing poorly on the test is

harsh

Ways to reduce concerns over test unfairness:1. Multiple assessment procedures2. Use more intensive screening procedures for

those likely to be treated unfairly by a given test.

Two distinctions are very useful for classifying decisions:

1. Individual or Institutional

2. Comparative or Absolute

Websites www.apa.org and www.cpa.ca APA

American Psychological Association’s Code of Ethics for Psychologists

The Ethical Practice of Psychology in Organizations CPA

The Canadian Code of Ethics for Psychologists Companion Manual… Guidelines for Non-discriminatory Practice Guidelines for Educational and Psychological

Testing Practice Guidelines for Providers of Psychological

Service

Websites for publications / reports Rights and Responsibilities of Test Takers: Guidelines

and Expectations (www.apa.org/science/ttrr.html) Responsibilities in Providing Psychological Test

Feedback to Clients (http://kspope.com/sci/feedback1.shtml)

The Real World: It is Better to Receive than to Give--Practical Tips for Giving and Receiving Performance Feedback (www.siop.org/tip/TipOct00/12waclawski.htm)

Statement on the Disclosure of Test Data (www.apa.org/science/disclosu.html)

Test Security (American Psychologist, Dec. 1999, Vol.54, No.12, p1078)

Statement on the Use of Secure Psychological Tests in the Education of Graduate and Undergraduate Psychology Students