challenges of assessing young areas of...

1

Assessing Communication and Learning Skills in

Young Children Who are Deaf blind

Charity Rowland, Ph.D.

[email protected] Webinar, NCDB: September 24, 2010

Validation of Evidence-based Assessment Strategies to Promote Achievement

in Children who are DeafblindU. S. Department of Education Grant # H324D030001

� Dr. Deborah Chen California State University, Northridge

� Dr. Robert Stillman, University of Texas at Dallas

� Dr. Harvey Mar, Columbia University

� Dr. Charity Rowland, Oregon Health & Science University

� National Family Association for Deafblind

2

Areas of Focus

� Deafblind

� 2-8 year old children who are nonspeaking and have significant intellectual limitations

� Communication and cognitive development

3

Challenges of Assessing Young Children Who Are Deafblind

� May not use conventional forms of communication

� May use very subtle means of expression

� Professionals often inexperienced

� Appropriate assessment instruments may be hard to find

4

Activities/Studies

� Nationwide survey of parents

� Nationwide survey of professionals

� Expert query

� Family specialist query

� Poll of state projects

� Focus groups

� Collection of assessment and IEP data on 0-5 year olds

5

Parent Survey (n – 74)

6

C. Rowland, OHSU Assessing Communication and Learning Skills

Webinar, September 24, 2010

2

Parents’ Mean Ratings on Questions about the Assessment Process

7

Survey Question Mean Ratinga

1. How useful were the results of the latest evaluation report to you in terms

of identifying the specific needs of your child?

3

2. How easy was it for you to understand the results of the latest evaluation

report? That is, were the results presented in terms that you could

understand?

4

3. Do the results of the latest evaluation report match your own

observations about your child? That is, do they reveal the strengths and

weaknesses that you see in your child?

3

4. How useful were the results of the latest evaluation report in terms of

helping you participate in developing educational goals for your child?

3

5. How useful was the latest evaluation report in terms of describing your

child’s progress over time?

3

6. To what extent were you involved in the assessment of your child for the

latest evaluation?

3

Least satisfactory aspects of my child’s latest evaluation

� 40% - Tests inappropriate, underestimated child

� 20% - Evaluation didn’t provide helpful facts or suggest things to try

� 15% - Lack of progress of child

� 13% - Tests focused on what child can’t do,

rather than progress

� 10% - Evaluator didn’t know child

� 5% - Didn’t like how child was handled

8

Parents’ most frequent suggestions

� 40% - Evaluators should get to know child

� 35% - Get input from parents and classroom

� 28% - Evaluations should help parents understand how to set goals and move forward

� 20% - Use tests appropriate for child who is deafblind

� 15% - Tell parents names of assessments, what for, what doing during assessment

� 10% - Start with child’s strengths and reasonable goals

� 5% - Child’s needs should be paramount—not costs or difficulty of trying new things

9

Qualitative Data Sources

In-depth Interviews (14)

Focus Group Interview (10)

Survey of Family Specialists (12)

In-depth Interviews: Themes

N=14

� Accuracy of assessment data

� Applicability to intervention

� Modification of standardized tests

� Authentic assessment practices

Focused Interview: Themes

N = 10

� Appropriate assessment tools

� Expertise of assessment team

� Time to conduct assessment



3

Focused Interview: Findings

Family concerns

� Use of test scores

� Knowledge of child

� Parent view of professionals

Professional concerns

� Assessment tools

� Relationship to intervention

� Assessment report

Focused Interview: Common Questions of Parents

� Purpose of the assessment and procedures

� Appropriateness of assessment tools;

� Qualifications of the assessment team;

� Outcomes of the assessment process;

� Accuracy of assessment results; and

� What parents can do if they disagree with assessment reports.

14

Focused Interview: Helpful Strategies

� Multiple and skilled observations

� Natural environments

� Familiar people

� Interview skills

� Team approach

� Family-centered approach

Survey of Family Specialists

Parent questions

� Purpose of assessment and procedures

� Appropriateness of tools

� Qualifications of team

� Outcomes

� Accuracy

� Disagreements

Survey of Family Specialists

Recommendations for parents

� Make list of questions

� Ask about tools and procedures

� Prepare list of own observations

� Draw on relevant professionals and resources

Specific Assessments:Criteria for “Short List”

� High ratings by 135 professionals who completed survey

� Designed for assessment of 2-8 year olds

� Include sections on cognitive and/or social-communication skills

� Readily available

� Other instruments developed specifically for population also included

18



4

“Short List” of Assessments

� Callier-Azusa (Stillman) *

� Carolina Developmental Profile (Johnson-Martin, Attermeir & Hacker)

� HELP-Hawaii Early Learning Profile (Parks & Furuno)

� Infused Skills Assessment (Hagood) *

� INSITE (Morgan & Watkins) *

19

“Short List” of Assessments

� Oregon Project (Anderson, Boigon, Davis & deWaard)

� Vineland (Sparrow, Balla & Cicchetti)

� Communication Matrix (Rowland)

� Dimensions of Communication (Mar & Sall) *

� Home Talk (various authors) *

� School Inventory of Problem Solving Skills (Rowland & Schweigert)*

20

21

InstrumentDeveloped

for

deafblind?

Domains

Assessed

Age range

targeted

Focus on pre-symbolic

skills

Callier-Azusa-G Yes All domains 0-10 yr. High

Callier-Azusa-H Yes Communication 0-10 yr. High

CarolinaNo All domains 0-3 yrs (Infant &

Toddler edition)

2-5 yrs (Preschooler

edition)

Low

Communication Matrix

Yes Communication All ages: only earliest stages of

communication

High

Dimensions of Communication

Yes Communication All ages High

HELP No All domains 0-3 yr. (0-3

edition)

3-6 yr. (Preschooler ed.)

Low

22

InstrumentDeveloped

for

deafblind?

Domains

Assessed

Age range

targeted

Focus on pre-symbolic

skills

Home Talk Yes CommunicationLearning/concept

development

School-age (3+)

High

*Infused Skills Blind Social competenceOrganization

Early childhood-

secondary

Low

INSITE Yes All domains 0-2 yr. (short version)

0-6 yr. (long version)

High

*Oregon Project

Blind All domains 0-6 yr. Low

SIPSS/.HIPSS Yes Object interaction related to

cognitive and social domains

All ages High

*Vineland No Adaptive Behavior (Communication,

Daily Living Skills, Motor Skills)

All ages Low

Parent Ratings of Selected Instruments

23

Instrument

Callier-

Azusa

Communication

Matrix HELP HIPSS Home Talk

Home Talk

Section 3 only Vineland

This instrument uses clear language

4.3 4.0 4.3 4.6 3.3 4.0

This instrument is easy for me to

understand

3.9 4.0 4.1 4.5 4.0 4.0

This instrument is useful for evaluating my

child's needs 2.5 4.1 3.5 4.0 4.8 3.0 2.9

This instrument encouraged me to be

involved in my child's assessment process. 4.3 3.0 4.5 4.4 4.0 3.0

It is easy for me to understand the results

that this instrument provides. 4.0 3.4 2.5 3.4 4.1 3.7 3.4

This instrument provides results that match

my own observations of my child. 4.0 3.7 3.0 3.7 4.1 3.3 3.1

This instrument is useful for developing

educational goals for my child. 2.5 4.1 3.5 4.3 4.4 3.7 2.9

This instrument is useful for describing my

child's progress 2.5 4.1 3.5 3.9 4.0 3.0 2.8

This instrument has clear instructions (only

for instruments that you completed

yourself) 3.3 3.0 4.2 4.3 3.7

This instrument is easy for me to use (only

for instruments that you completed

yourself)

3.3 3.0 4.6 4.4 3.7

24

5 Composite Qualities RatedQ ua lity Q ue stion s

Con tent

U se f ul to a ss e s s com m unica tio n

U se f ul to a ss e s s so cia l in te ra ction

U se f ul to a ss e s s cog nit ive / lea rning

A ppr opr iaten e ss

U se f ul for 2-8 y ea r o ld s

U se f ul for ch ildre n w ith se ve re cog n it iv e im p airm e n t

U se f ul for ch ildre n w ithout lan gua ge

U se f ul for ch ildre n w ith se ve re vis ion im p airm en t

U se f ul for ch ildre n w ith se ve re h e a rin g im pa irm e n t

U se f ul for ch ild re n w ith se ve re ortho p e dic im p airm e n t

A cc ura cy

U se f ul to a ss e s s a v arie ty of ch ildre n with broa d ra n ge o f ski lls/ ne e ds

U se f ul to e va lu a te w id e ra nge of sk il ls in ea ch dom ain

U se f ul for ge n er ating a ccu ra te p ic tur e o f ch ild s ki lls

A pplica bility

U se f ul to g e ne rate instr uctio na l go als /for e du ca tion al p lann in g

U se f ul for e va luating p rogr es s

U se f ul for de s crib in g chi ld 's s tre ng th s a n d we a kne s s to pa re nts

U se f ul to e ncou rag e p are n t invo lv e m e nt

Usa bility

U se s cle a r la ng uag e

H a s c le a r ins truction s

Is e a sy for m e to und er sta nd

Is e a sy for m e to us e

Is u se r- fr ie nd ly



5

Professional Ratings of Selected Instruments

25

Instrument

Callier

-Azusa

Caro-

lina

Communi-

cation

Matrix

Dimensions

of Communi-

cation HELP

Infused

Skills

Home

Talk INSITE

Oregon

Project SIPSS VIneland

n 29 16 23 8 25 7 10 16 26 9 11

Content 1: Useful

to assess

communication

4.0 4.2 4.8 4.1 3.2 5.0 4.2 4.4 3.6 NA 3.2

Content 2: Useful to

assess social

interaction

4.0 3.9 4.4 4.0 3.4 4.8 4.1 4.4 3.7 NA 3.5

Content 3: Useful to

assess cognitive,

learning skills

3.9 4.3 NA NA 3.6 4.0 4.1 4.4 3.9 4.0 NA

MEAN

Appropriateness

4.0 3.1 4.4 3.8 2.7 4.9 4.3 4.2 3.4 3.8 2.8

MEAN

Accuracy

3.4 2.9 3.9 3.6 3.2 4.6 4.4 4.1 3.7 3.8 2.9

MEAN

Applicability

4.1 3.6 4.4 4.0 3.2 4.7 4.3 4.4 4.0 4.0 3.4

MEAN

Usability

4.3 4.5 4.5 3.5 4.3 3.0 4.5 4.5 4.3 4.7 4.3

Results of State Projects Query (n=18)

26

Instrument

Am familiar with

instrument

Is used widely in

my state

I would

recommend its use

Callier-Azusa 100% 41% 94%

Carolina Developmental

Profile 62% 15%

63%

Hawaii Early Learning

Profile 67% 33%

100%

Infused Skills

Assessment 14% 7%

33%

INSITE Developmental

Checklist 75% 50%

92%

Oregon Project 87% 33% 71%

Vineland 75% 33% 60%

Communication Matrix 83% 33% 83%

Dimensions of

Communication 60% 0% 100%

Home Talk 90% 20% 100%

School Inventory of

Problem Solving Skills 56% 6% 91%

Qualitative Findings from AllInterviews and Surveys

Challenges:

� Appropriate assessment tools

� Expertise of assessment team

� Time to conduct assessment

� Accuracy of assessment report

� Relationship to program planning

27

Qualitative Findings from AllInterviews and Surveys

Recommendations:

� Family involvement

� Multiple and skilled observations

� Natural situations

� Familiarity with child

� Interview skills

� Interdisciplinary approach

28

Final Products

� Guidelines for the

Assessment

of Young Children

who are Deafblind

� Data Summary and Assessment Forms available at:

http://www.ohsu.edu/oidd/d2l/com_pro/db_assess_ab.cfm

29

Evidence Based Practices (EBP)

Combination of:

� Best available research

� Craft knowledge and community of practice

� Family wisdom and experience



6

Authentic Assessment

A multidimensional, team-based approach which seeks to:

� Capture the richness and complexity of a child’s cognition and behavior

� Understand the child in the context of his/her natural social and physical environment

� Integrate the family’s perspective with those of professionals

� Apply the information to enhance the child’s development and acquisition of skills.

Steps of an Authentic Assessment Process

� Identify assessment team

� Arrange for family involvement

� Select appropriate instruments

� Conduct observations in multiple situations

� Arrange structured observations

� Request evaluations by specialists

� Emphasize strengths and interests

� Provide recommendations

The Planning Phase

� Assembling the Team

� Getting the Family Involved

� Selecting Appropriate Assessment Instruments

� Scheduling Observations in Natural and Staged Contexts

� Involving Specialists

33 34

Assembling the Team

� Define the questions and the scope of the assessment

� Multidisciplinary

� Include individuals with deepest knowledge of child

35

Getting the Family Involved

What the assessment team can do:

� Meet with the family in advance

� Address their questions and concerns

� Identify any obstacles to participation

36



7

How Families May Prepare for an Assessment

� Prepare your own questions and identify your goals and concerns in advance.

� Prepare a list of your child’s special skills and interests.

� Discuss what instruments will be used to assess your child and who will administer them.

� Provide the evaluator with possible times to observe your child at home or in other familiar settings.

� Prior to the assessment, complete any parent questionnaires or assessments so that the evaluator can better understand your child’s abilities.

37

Selecting Appropriate Assessment Instruments

� Helps structure observations:

� What to look for

� How to interpret

� Compare across observers

� Accuracy, Applicability, Convenience

� May use parts of several instruments

� See reviews in our assessment guide

� What about standardized tests? 38

Considerations for Selecting an Assessment Instrument

� Do the items describe behaviors a child with vision and hearing losses or motor impairments could be expected to show?

� Are there enough items at the early developmental levels to clearly identify a child’s current skills and measure progress in small steps?

� Are the items appropriate to the child’s chronological age or do they describe the behavior one would only expect to see in infants and toddlers?

� Are there a sufficient number of examples to clarify the items and to determine how a particular skill might be observed in a child who is deafblind?

39

Considerations for Selecting an Assessment Instrument

� Does the assessment include information derived from observations in natural settings?

� Do the results provide applicable information for program planning or are they primarily numerical scores? Does the instrument give you ideas about the “next step” for the child?

� Are the results in a format that can be easily communicated to and understood by families?

� Who can use the instrument?. Does it require specialized training or professional credentials?

� Is there a parent version or a way for parents to provide their input and perspectives?

40

Observing in Natural Contexts and Staged Situations

Natural

� Gather information about the child in everyday contexts and routines

� Identify typical and optimal skills

� Multiple perspectives

Staged

� Manipulate the natural context to encourage specific behaviors

� Fill in gaps in knowledge and validate impressions

41

Involving Specialists

� Vision specialists

� Hearing specialists

� Physical therapist

� Occupational therapist

� Speech-language pathologist

� Health professionals

42



8

What Questions Should an Authentic Assessment Address?

43 44

45

Expressive Communication

What existing communication behaviors should be promoted and what new skills should be targeted?

46

47

Expressive Communication

What behaviors does this child use to make his/ her needs and wants known (body movements, gestures, facial expressions, vocalizations, words, sign language, picture

symbols, object symbols, etc.)?

Does this child’s expressive behavior appear to be intentional? Is it directed toward a goal? Does it appear that the child anticipates a response to the communication?

How frequently does this child communicate?

What specific messages or communicative functions does this child express (protests, requests, greetings, etc.)?

Under what circumstances is this child most communicative (with whom? when? where?)?

Does the child need prompting or support to communicate clearly or consistently? What type of support?

Receptive Communication

What forms of communication does the child understand? How can we best communicate with this child?

48



9

49

Receptive Communication

What types of communicative behavior do you think this child understands (spoken words, manual signs, gestures, facial expressions, vocal intonation, picture symbols, object symbols, etc.)?

What messages or communicative functions does this child appear to understand(directives, greetings, requests, etc.)? Is prompting and support needed for the childto respond to a communication?

Who communicates effectively with this child? Are there particular activities in whichthe child seems most likely to respond?

Social Interaction

What opportunities exist to enhance the child’s social interactions, play, and communication with others?

50

51

Social Interaction

Does this child enjoy interacting with adults? If yes, under what

circumstances?

Does this child enjoy interacting with peers? If yes, under what

circumstances?

Hearing/Vision

What adaptations and accommodations are needed to support the child’s use of hearing and vision for communication?

52

53

Hearing

Is this child able to perceive speech, vocalizations or other auditory input?

Would this child benefit from hearing aids or amplification devices (such as an FM system)?

Vision

Is this child able to use visually-based symbols for expressive communication (can s/he perceive, discriminate between, and understand visually-based symbols)? If

not, is a tactile system indicated (e.g., object symbols, Braille)?

Is this child able to perceive visually-based symbols for receptive communication (e.g., picture symbols, sign language, print)? If not, is tactile input required (e.g.,

tactile signs, object symbols)?

What accommodations are needed to help this child perceive visually-based communication (positioning of child and partner, illumination, position and spacing

of stimuli or symbols, size and color of symbols, etc.)?

Motor/Fine Motor Skills

What adaptations are needed to support the motor behaviors and tactual experiences needed for communication?

54



10

55

Motor/Fine Motor Skills

Does this child have the fine motor skills needed to use gestures or manual

signs, to indicate choices or to activate communication devices?

What accommodations need to be made to allow physical access to

expressive and receptive communication systems?

Preferences

What activities, materials, people, and contexts motivate the child to communicate and learn?

56

57

Preferences

What people, things, and activities does this child prefer?

What people, things, and activities does this child dislike?

Is this child mostly engaged by objects or by people?

Where and when is this child at his/her best?

To what extent does this child tolerate direct assistance, such as hand-

over-hand or hand-under-hand support?

What is this child’s primary mode of exploring new things (tactile, visual,

oral)?

Temperament

How does the child’s temperament affect learning and communication?

How can we use this information to develop effective interventions?

58

59

Temperament

Is this child easily excited or passive and quiet?

Does this child show sustained attention to tasks or quickly lose interest?

Is this child calm or irritable?

Is this child sociable or shy?

Does this child accept or reject changes in routines, materials, people,

and situations?

60



11

61

Generic Expressive Communication Outcomes

� Communicate more frequently using existing communication behaviors

� Communicate more independently (with less assistance)

� Use new (more conventional or sophisticated) forms of communication

� Express new messages/communicative functions (e.g., request “more”, ask questions)

� Communicate about a broader range of topics

� Communicate with more people and across settings and activities

� Initiate communication62

Generic Receptive Communication Outcomes

� Respond to communication from others with less cueing/assistance

� Understand new (more conventional or sophisticated) forms of communication

� Understand new messages/communicative functions (e.g., requests for information, greetings)

63

Generic Social Interaction Outcomes

� Interact comfortably with adults

� Interact comfortably with peers

� Initiate social interactions

64

Sample Expressive Communication Outcomes & Goals

� Outcome 1: Communicate more frequently and independently using

current behaviors.

� Goal 1: Maria will increase independent use of gestures.

� Suggestions: Provide frequent opportunities to make choices in snack, circle, play; promote the use of gestures to make choices/requests; fade

physical assistance; record frequency of independent responses.

� Outcome 2: Use new symbolic forms of communication.

� Goal 2: Maria will use object symbols to make simple choices.

� Suggestions: Use familiar objects associated with preferred items and

activities. Probe ability to choose correct one (out of two) associated with preferred snacks, toys.

65

Sample Receptive Communication Outcomes & Goals

� Outcome 1: Increase responsiveness to current communication forms.

� Goal 1: The consistency of Maria’s awareness of and reactions to

auditory input will increase.

� Suggestions: Provide FM system; track responsiveness to spoken cues.

� Outcome 2: Understand new forms of communication.

� Goal 2: Maria will demonstrate understanding of object symbols for major activities.

� Suggestions: Provide daily schedule system with one familiar object to

represent each major activity in school schedule.

66



12

Sample Social Interaction Outcomes & Goals

� Outcome 1: Increase ability to interact comfortably with peers.

� Goal 1: Maria will have more opportunities to interact with peers during regular classroom routines.

� Suggestions: Include Maria in all group activities at school; provide consistent structure for simple interactions with peers; sit Maria next to favorite peers; where appropriate, involve peers instead of (or in addition to) one-to-one assistant.

67

Qualitative Indicators of Progress

� Increased consistency of response

� Increased independence of response/decreased assistance required

� Increased frequency of response

� Increased duration of response

� Increased clarity of response

68

Qualitative Indicators of Progress, contd.

� Faster response time

� Partial participation in one step of a routine

� Ability to participate in an activity for longer periods of time

� Sustained attention to a task

� Increased stamina

� Improved strength or accuracy of a motor response

69 70



challenges of assessing young areas of...

Documents