department of behavior analysis behv 5250: advanced topics in intervention summer 2003department of...

Summer 2003 Department of Behavior Analysis

Department of Behavior AnalysisBEHV 5250: Advanced Topics in Intervention

Overview of measurement, assessment, and evaluation

July 31, 2003



Today’s agenda

• Welcome (5 minutes)– Please remember to click– Bonus for reminding me about Free/Writes

• Questions & answers (5 minutes)• Student presentations (45 minutes)

– Difficult to discriminate stimuli– Model concept analyses

• Overview of Assessment



Some terms…

• Measurement – the gathering of quantitative information

• Assessment – the pacing of information within a context

• Evaluation – making a decision based on assessment information



Two Main Types of Assessment:

• Criterion referenced– Compares the student’s performance to a specified mastery

level– Instructional objectives

• Norm referenced– Compares the student’s performance on a task to a group of

students– Age equivalent, grade equivalents, percentile ranks



3 Levels of Assessment:

• Micro level• Assessment conducted on an on-going basis related to specific

instructional objectives

• Meta level• Assessment conducted on a regular basis related to broad skill

clusters within curriculum areas

• Macro level• Large-scale assessment to determine overall impact of

educational intervention

Intervals Corresponding to Levels of Assessment


Micro


Meta


Macro



The Micro Level of Assessment:

• Every skill being established or practiced is monitored• Monitoring occurs each time a skill is taught• Collects data along the same dimension as the instructional

objective• Is used to inform instructional decisions



Quality Intervention requires:

• Continuous MeasurementStudent responding to intervention practices is continuously measured.

• Visual Displays- A relevant dimension of the behavior to be taught is graphically displayed.- Graphic displays (indicating the progress of the student) are always available for viewing.- Graphs are used to make sound decisions about tactics for individuals.

Adapted from work by R.D. Greer



Quality Intervention requires:

• ResponsivenessIntervention driven by the moment to moment responses of each individual student and guided by existing research findings.

Adapted from work by R.D. Greer



The Micro Level of assessment

• Effects of instructional decisions are tracked relative to each skill

• Monitoring continues until essential learning outcomes are demonstrated

• Skill retention

• Skill endurance

• Skill application

• Skill stability



The Meta Level of Assessment:

• Clusters of interrelated skills are assessed together• Assesses student’s performance on a range of skills within a

curriculum area– E.g., reading, mathematics, social skills, language

• Helps determine whether intervention is being effective• Performance is compared to typical peers• Also called Curriculum Based Assessment (CBA)


Department of Behavior AnalysisBEHV 5250: Advanced Topics in InterventionExamples of CBA Measures:

• To assess effectiveness of:– Sounds instruction

– Words instruction

• We might measure the number of words read correctly per minute from prose samples

• To assess effectiveness of:– Expressive labeling of

objects instruction– Spoken sentence structure

(semantics) instruction

• We might measure the number of words and sentences spoken spontaneously during a five minute language sample

An Example:

Another Example:



The Macro Level of assessment:

• Skill clusters are assessed together on complex tasks• Uses norm-referenced, standardized, published tests

– e.g., Woodcock-Johnson, Preschool Language Scale-3, Photo Articulation Test, Child Development Inventory, etc.

• Allows comparison of client progress relative to the norm group• Emphasis is placed on rate of progress

• pre/post intervention

• compared to typical peers



An example of the importance of RATE of progress

“Emma has made significant progress since her last evaluation in her motor skills. She has gone from an age equivalent of 30 months to 54 months in her gross motor

development.”

- School Psychologist, in a triennial evaluation report

Did Emma progress as a result of educational intervention?

An example of why RATE of progress is so important...

Emma - Gross Motor Development

0

10

20

30

40

50

60

70

80

1 4 7

10

13

16

19 22

25

28

31

34

37 40

43

46

49

52

55

58 61

64

67

70

73

Chronological Age

Per

form

ance

Ag

e E

qu

ival

ent

Typical Child's Progress (1 monthper month)

Emma's Progress - Baseline (.83months per month)

Emma's Progress - DTT + preschool(.75 months per month)

Batelle Developmental Inventory

Daniel’s Expressive Language Progress with InterventionS

kill

Ag

e E

qu

ival

ent

(in

Mo

nth

s)

Chronological Age (in Months)

0

5

10

15

20

25

30

35

40

45

50

1 3 5 7 9 11

13

15

17

19

21

23

25

27

29

31

33

35

37

39

41

Age in Months

Daniel's Progress - .76 months of gain permonthTypical Child's Progress - 1 month gain permonthTherapy Progress - 3.2 months of gain permonth

Legend

Preschool Language Scale - 3

Daniel’s Auditory Comprehension Progress with InterventionS

kil

l A

ge

Eq

uiv

ale

nt

(in

Mo

nth

s)

0

10

20

30

40

50

601 3 5 7 9

11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41

Chronological Age (in Months)

Age in Months

Daniel's Baseline Progress - .63 months gainper month

Typical Child's Progress - 1 month gain permonth

Daniel's Therapy Progress - 4 months of gain inone month

Legend

Preschool Language Scale - 3

1.12

- 0.21

Sta

nd

ard

Dev

iati

on

s fr

om

Pee

rs

Pre-Therapy (11/00) Current (05/00)

Daniel’s Articulation Progress with Intervention

Photo Articulation Test



You should begin working on…• You will be responsible for completing a report related to assessment of children with autism.

Your report is to follow APA style guidelines and should have the following features:• 1. Describe at least three published assessment tools used with children with autism. In your

description, be sure to include descriptions of the skills and repertoires assessed by the tool, whether the tool is normatively or criterion referenced, a description of the referencing process used by the tool developers, and critical features of children for whom the tool would be appropriate (age, pre-requisite skills required, cultural background, etc.)

•2. For one of the assessment tools you describe, list the assessment items from the tool and identify the type of verbal operant tested (if the test items are, in fact, testing verbal behavior). For example, if a test item requires the student to answer the question, "Who lives in the Hundred Acre Wood?" by saying, "Winnie the Pooh", you would classify this as testing the intraverbal repertoire. NOTE: you will make me very happy if you also identify the type of intraverbal response tested (sequelic, codic, or duplic). As another example, if the test item asks the student to say "Winnie the Pooh" when shown a picture of Winnie the Pooh, you would classify this as testing the tact repertoire. If the test item is not testing verbal behavior, then categorize the item according to the taxonomy system you learn in class.



Tonight’s readings…• White, O.R. (1984).

Selected issues in program evaluation: Arguments for the individual. In B.K. Keogh (Ed.) Advances in Special Education: Documenting Program Impact. Greenwich, CT: JAI Press.

• Vargas, E.A. (1988). Verbally-governed and event-governed behavior. The Analysis of Verbal Behavior, 6, 11-22.

• Tiemann, P.W. & Markle, S.M. (1990). Principles (chapter 6). Analyzing Instructional Content. Champaign, IL: Stipes Publishing Company

• Tiemann, P.W. & Markle, S.M. (1990). Strategies (chapter 7). Analyzing Instructional Content. Champaign, IL: Stipes Publishing Company

• Tiemann, P.W. & Markle, S.M. (1990). Sets of related principles--some issue to consider (appendix 10). Analyzing Instructional Content. Champaign, IL: Stipes Publishing Company

• Tiemann, P.W. & Markle, S.M. (1990). Guidelines for the analysis of a set of related principles (appendix 11). Analyzing Instructional Content. Champaign, IL: Stipes Publishing Company



Tonight’s homework…• complete the in-text questions for Tiemann & Markle, chapters 6 and 7 (due

7/25/03) • identify four examples of instances of rule-governed behavior (due 7/28/03)

that children with autism often need to learn from each of the following areas:

• academic skills • social skills • self-help skills • leisure skills • of the examples you identified above, analyze one from each of the four

areas using the example formats Tiemann & Makle provide in appendix 11 ;please be prepared to present at least one of these in class tomorrow

department of behavior analysis behv 5250: advanced topics in intervention summer 2003department of...

Documents

levels of assessment

intervention summer

assessment information

assessment cba slide

micro slide

greer slide

advanced topics

instructional decisions