RTI: Response

RTI Assessment Tools for ELLs

ACHIEVEMENT TOOLSMeasuring Response to Intervention

CBM Considerations• Extent to which curriculum is culturally representative of

the student.• Student’s previous participation in bilingual education/

ESL• Known or suspected sensory or communicative

impairments• Information on formal former education.• How often has the child moved and how often does child

attend school?

CBM Considerations

• Students level of acculturation• English language proficiency• What skills are needed to answer the items that

are not related to item performance.• Experiences outside of the school setting that

support or detract from academic success.

CBM: Spanish Materials• Brigance Diagnostic Assessment of Basic Skills, Spanish– K-6– Readiness, Speech, Word Recognition, Oral Reading, Reading,

Comp., Word Analysis, Listening, Writing, Alphabetizing, Numbers and Computation, and Measurement

• DIBELS has a Spanish version that is like the English version. This can be found on the DIBELS website.

Suggestions: Non-English and Non-Spanish CBM

• Informal methods or CBM methods should be utilized in combination with parent/ teacher interviews, class observations, and child interviews.

• As very few CBM measures are available in languages other than Spanish, you may have to create your own CBM instruments.

Making Your Own CBM:Operationalize the Curriculum• Establish what is the MOST

IMPORTANT in the curriculum and change these concepts into questions from the materials.

• Determine what is most typical for most students.

• Attempt to avoid floor and ceiling problems by making items too easy or too hard (best to err on too easy).

Collect, Analyze, and Describe the Data

• For classroom, median is only stat needed.

• For larger data collection groups, organize scores by grade using Means and Standard Deviations.

• Use percentile ranks if N>100.

• Normal range is that 2 out of every 3 score in that group (16th - 84th percentile).

Local Norms: Classroom• Classroom:

– Usually a small number of students (5-10). – Only gross estimates of variability are available. – Limited to screening and prioritizing student w/in class needs.

• Sampling:– Obtain sample from class at large– OR Obtain a sample from students considered to be typical in that

classroom.– OR Obtain combination from class on group tasks and a smaller sample on

individual tasks.– Class of 40-45, sample of 5 is fine, 10 preferred.

Error Analysis• Finally, you need to

analyze the errors.• Determine if skills that

are missing can be made up in general education mainstreaming.

• Determine what objectives need addressing in the intervention.

Reading Assessments• Obtain basal readers for different levels in English and in

the other language• Have the child read in each language.• Score fluency by marking reading end point after 1 min

and 2 min.• Ask the child to retell the story to indicate passage

comprehension.• Basic reading can be evaluated by the number of errors

made while reading the passage.

Math Assessments

• Basic math: direct translation of instructions and actual items from a normative test (do not use norms).

• Applied math: direct translation of items from a normative test (do not use norms).

• Do an error analysis of types of problems and see if these carry across languages.

Written Expression• Develop an age appropriate writing prompt (e.g., “I was

walking down the street and suddenly I saw …”).• Have the child write a story based on this prompt.• Have a translator given a similar prompt in the native

language.• Work with the translator to determine total number of

words, grammar spelling, and understandability for BOTH language tasks.

BEHAVIORAL/ EMOTIONAL TOOLSMeasuring Positive Behavioral Supports

PBS: Data Collection

Internalized Behaviors• Rating Scales• Checklists

Externalized Behaviors• Time Sampling• Event Sampling

Rating Scales & Checklists

• Rating Scales: Use quality judgments (e.g., good, poor, etc.) instead of concrete descriptors.

• Checklists: Use yes/no to describe whether or not a criteria was met.

• You can use existing checklists/ rating scales but choose specific items for what you are measuring (Volpe & Gadow, 2010)

Systematic Observations

• Time/event appropriate• Operationally define the behavior• Have a planned recording technique• May want to have more than one setting.• May want to have more than one observer.• Generally works best for high incidence behaviors.• Can take place in a planned (forced) situation or in

a natural situation.

Designing an Observation

• How many times will you observe?• How long will you observe?• When/ where will you observe?• What behaviors will you observe?• How will you record the data?

Recording Data

• Use a specific form to collect the data.– Anecdotal record form: You write the time and a

description of each significant behavior following a narrative section on settings, patterns, and opportunities.

– Quantitative record form• Interval recording (time sampling, interval sampling, &

interval time sampling).• Event recording (event sampling)

Time Sampling: Types

• Partial Interval Time Sampling– Record the behavior if it happens once during a set time

period. For example, if it occurs at all during a 60 second period.

• Whole-Interval Time Sampling– Record the behavior if it occurs during the whole

interval. For example, if it occurs lasts from the first second to the last in a 60 second interval.

• Other time sampling methods– Other methods are available but not often used.

Time Sampling: Example

TargetedBehaviors

@10sec

@20Sec

@30Sec

@40Sec

@50Sec

@60sec

Time Sampling: Example

TargetedBehaviors

@10sec

@20Sec

@30Sec

@40Sec

@50Sec

@60sec

Out of his seat X X X X

Making noises X X

Not working X X X X X X

Partial-interval

Event Sampling

• Record every time the client performs a certain action.

• For example:TargetedBehaviors

Number of timesIn 3 min period

Out of his seat

|||

Making noises

|||| |||

Not working ||

Observation: Data

• Frequency: How many X in Y minutes (e.g., 3 times per min).

• Duration: How long is this happening (e.g., he cried for 3 min).

• Duration: What percentage of time is this happening (e.g., he cried for 45% of the time).

• Latency: How long does it take between the stimulus and the response.(e.g., he started crying 20 seconds after his mother left)

BEYOND RTI/PBS ASSESSMENTEvaluating English Language Learners

ELL and Assessment

• Cognitive Assessments:– Use the MAMBI to help guide your assessment plan.– Use confidence intervals with “normative” data

instead of just the score because of the reliability issues for ELL.

– Consider all normative data as suspect. Verify any test data against observation and interview/records review data.

– Even nonverbal assessments can have biased content.

ELL and Assessment

• Achievement:– You need language proficiency data. It needs to be

current.– If applicable, you need some estimate of achievement

in L1 and L2• YOU CANNOT HAVE SLD IN L2 AND NOT HAVE IT IN L1

– Sometimes good normative data is not available. You may have to change the basic structure of how you assess and diagnose.

ELL and Assessment

• Social, Emotional, Behavioral, and Adaptive– These constructs are culturally loaded. – Be careful when using translations that do not have

normative changes as well.– Parental and child interviews may not illicit much

information unless they feel comfortable with you.– For informal information, you may need a “cultural

ambassador” to help you talk with clients and families.– Keep in mind that mental illness manifests differently in

different cultures.

Psychologists and SST Report Writing

• What will be in a report:– Reports need to be written to include all of the data collected.– Your reports will often feel more “qualitative” instead of

“qualitative” in nature.– You may have to go into specific tests to examine response

patterns down to the item level.

• Language of a report:– Most districts do not require a “home language” translation of

a report or meeting notes.– You may want to record the translator’s description of the

report during the meeting for the family to review later.

Final Thoughts

• Most assessments for ELL will take 2-3 times as long as their English-only equivalents.

• Most meetings for ELL will also take 2-3 times longer.• When making a diagnostic decision, examine the

preponderance of the evidence.• Parents and families play a crucial role in these

assessments BUT they are often unavailable, difficult to reach, and/or not comfortable with school personnel.

• Families across cultures are more alike than different.– There is not a single cultural group that does fails to love their

children.