osha presentation 1 - oregon speech-language hearing ......osha presentation 1 ron gillam, phd, and...

36
Ron and Sandi Gillam 10/2/18 OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children Disclosure Ron Gillam receives royalties from the sale of the Test of Narrative Language - 2, which is distributed by Pro-Ed. Today’s Presentation 1. Summarize key issues in measuring narrative comprehension and narrative production 2. Explain the differences between a norm-referenced test of narration (TNL-2) and a narrative progress monitoring tool (MISL) a. Purposes b. Methods c. Reliability and validity d. Use of scores 3. Clinical applications

Upload: others

Post on 29-May-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 1

RonGillam,PhD,andSandiGillam,PhDUtahStateUniversity

AssessmentofNarrativesinSchool-ageChildren

Disclosure

• RonGillamreceivesroyaltiesfromthesaleoftheTestofNarrativeLanguage- 2,whichisdistributedbyPro-Ed.

Today’sPresentation

1.Summarizekeyissuesinmeasuringnarrativecomprehensionandnarrativeproduction2.Explainthedifferencesbetweenanorm-referencedtestofnarration(TNL-2)andanarrativeprogressmonitoringtool(MISL)

a.Purposesb.Methodsc.Reliabilityandvalidityd.Useofscores

3. Clinicalapplications

Page 2: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 2

StoryExample(ChildwithDLD)

(Umumtheleaf)theleavesiscomingout.Thinkitwasfun(becausebecause)becauseshejustneedtowanttoplayleaf.Andthenshejumpintheleaf.And(they)shewastrashX.Andthensheplayinthe>(Aagirl)agirldon'tknowthatshewanttogotothe(um)leafs.Andthen(andthenandthen)the(trees)treescoming.Andthenshejumping.Andthenshewillgetitalittledown.

What is Narration?nType of discourse involving orderly accounts of

real or imagined eventsnCoherent sequences of utterances with a

common themenCharacters, actions, complications, resolutionsnActions occur in temporal ordernCausal relationships between characters and actionsnCharacterʼs responses, reactions, feelings, thoughts

OverviewofNarration

• Whatarethecomponentpartsofanarrative?• Macrostructure(storygrammar+causalframework)

• Episodestructureandcomplexity• Microstructure

• Languageusedtoconveytheinformationinthestory• Cohesion• SentenceComplexity

Page 3: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 3

Macrostructure

• AgentsCharacter

• Time&PlaceSetting

• TakeOffInitiatingEvent

• FeelingsInternalResponse

• ThoughtsPlan

• ActionsAttempts

• LandingConsequence

Episode structure Some robbers entered the Chicago Bank &

Trust. They demanded money and started shooting their guns in the air. Wonder Woman happened to be driving by when she heard shots coming from the bank. She knew the people inside needed her help. She decided to change into her Wonder Woman suit and get over there. She climbed up the back wall of the bank and went into an upstairs window. Once she got into the bank, the robbers started shooting at her.

She used her magic arm bracelets to knock away the robbersʼ bullets. Then she used her magic lasso to round them up until the police came. The police came and took the robbers to jail. The bank employees were grateful to Wonder Woman for saving their lives. Once again, Wonder Woman saved the day and showed that crime does not pay.

Page 4: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 4

Analysis of Rocket BoyRocket Boy lived in the year 2550 on the planet Zigvol. As he was flying to school one day, his rocket pack started to sputter. Rocket Boy was scared because he didnʼt want to be late for school. He thought, Iʼd better check the pluger hose. He reached back and found the problem - a hole in the pluger hose. Rocket Boy took some gum out of his mouth and covered the hole in the hose. His rocket pack roared to life. Luckily, he made it to school just before the bell rang. Rocket Boy was relieved. He thought, “Thatʼs why my parents always say to check the hoses on my rocket pack every day.” From that day on, he checked his pack carefully before he went anywhere.

Episode ComplexitynDescriptive Sequence – description with no

actionsnAction Sequence - chronology of actionsnIncomplete Episode - IE, A, or C missingnBasic Episode - IE, A, CnComplex Episode - multiple P, IE, or A with a

complicationnEmbedded Episode - multiple episodes

n(often C of 1 is the IE of 2)

Episodic Developmentn3 - 4, Descriptions - Setting/characters n4 - 5, Action Sequence - string of actionsn5, Basic Episode

nInitiating Event, Attempt(s), Consequencen6, Complete Episode

nInternal responses, Plans, Reactions, and Endings begin to appear

n7 - , Complex Episodes and Multiple Episode Stories

Page 5: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 5

Story1

There’salotofkidsandparentsatthebeach.They’retryingtofindseashells.Andthey’regettingwet.

Story2

SoonceuponatimetherewereallofthesemenandtheywereonabikeraceinCacheVilosa.Andsotheydecidedtogetontheirbikesandwaitforthesignaltodropdownforthem.Sothat’swhattheydid.Sotheygotontheirbikesandstartedracingdowntheroadtrack.Andthat’smyverybeststory.

Story3OnceuponatimetherewasaboynamedRylanandhewasinahelicopterintheafternoontime.Hedecidedtojumpinhisbestbluehelicopterandgotinitandflewawayintothesky.Sothat’swhathedid.Hewasgoingtoflyinthehelicopteruntilitwaslunchtime.Butuhoh,hishelicopterfuelgaugesaidthathisgaswasabouttogetdeadsohedecidedtolandatahelicoptergasstation.Sothat’swhathedid.Helandedatahelicoptergasstation.Hegotgasandwentbackintohishelicopter.Hewenttohishouse,atelunch,andhadbiscuitsandgravyforlunch.Hefeltpleasedthathewasfinallyfull.

Page 6: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 6

Microstructureofliteratelanguage

•Complexsentences• Coordinatedconjunctions

• FANBOYS(for,and,nor,but,or,yet,so)• Subordinating(adverbialconjunctions)

• Beforeshestolethecandy,shemadesurenobodywaslooking.• YouhavetosnowboardfastdownthehillifyouwanttocatchAlex.

• Infinitives• Irantolookatthebeautifulsunset.• Iwantyoutocomewithmetoseethatsunset.

Morecomplexsentences

•ClausalComplements• Iknewshewouldn’teatdinnerwithhimaftertheargument.

•RelativeClauses• Subjective:Theboywhotoldonyoudoesn’thavemanyfriends.

• Objective:Isawtheboywhotoldonyou.

Microstructure

• Literatelanguage• Elaboratednounphrases

• Themean,oldgoat…• Adverbs

• When,after,if,since• Metalinguisticverbs

• Said,screamed,yelled,hollered,whispered• Metacognitiveverbs

• Thought,decided,wanted,planned

Page 7: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 7

Microstructure

• Linguisticmarkersfortransitionsandelaboration• Temporal– first,next,before,after,when,while• Causal– because,since,so• Wordsthatindicateacomparison– also,but,however,aswellas• Adverbs– quickly,luckily,warmly

• Linguisticmarkersformentalandcommunicativefunctions• Mental– thought,wondered,worried,hoped• Communicative– said,yelled,whispered,told,

NarrativeComprehension• ConstructionIntegrationModel(Kintsch,2004,2013)

• Constructatextbase – literalrepresentationofwhatthetextsays

• Microstructure- Linksbetweenwordsinsentences• Macrostructure– hierarchical(temporalandcausal)relationshipsbetweenkeyelementsofthestory

• Integration– forasituationmodel(interpretationofwhatthetextmeans)

• Textbase +backgroundknowledgeinLTM• Requiresinferences

• Local– connectionsamongconcepts(temporalandcausal)• Global– linksbetweencharacters,initiatingevents,internalresponses,plans,actions,consequences,andreactions

Narrativeabilitypredicts• Orallanguageskillsingeneral(Bishop&Edmunson,1987;Fazio,Naremore &Connell,1996)

• Languagecomprehension(Bishop&Adams,1992;Gillam,Fargo&Robertson,2009)

• Reading(Cook&O’Brien,2014;Lapp,Flood&Farman,1989;Vandewalle etal.,2012;Zuckeretal.,2013)

• Readingcomprehension(Barton-Husey,Sevcik &Romski,2017)

• Writing(Bain,Bailet &Moats,1991;Koustsoftas &Gray,2012;Montague,1990;Scott&Windsor,2000)

Page 8: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 8

Cultural Differences

nContexts for social usenContent

nMoral codenConcern with authoritynAutonomy and self-determinationnAggressionnEmotional expressiveness

nStructurenSimilar propositions, different arrangement

Children with Language Impairments

nMacrostructure LevelnGreater variability across contextsnIncomplete references to characters and story contexts

(Garnet, 1986)nFewer story grammar propositions related to character

plans, actions, complications, and reactions (Dollaghan & Campbell, 1992; Gillam & Johnston, 1992; Klecan-Aker & Kelty, 1990; Paul et al., 1996)

nLower holistic scores (McFadden & Gillam, 1996)

ChildrenwithLanguageImpairments

nMicrostructureLevelnRestrictedvocabulary(Garnet,1986;Greenhalgh&Strong,2001)

nFewercomplexsentences(Dollaghan&Campbell,1992;Gillam&Johnston,1992;Rileyetal.,2004)

nMoregrammaticalerrors(Gillam&Johnston,1992;Liles,Duffy,Merritt,&Purcell,1995;Scott&Windsor,2000;Rileyetal.,2004)

nFewerorproblematiccohesiveties(Boudreau&Chapman,2000;Pauletal.,1996;Strong&Shaver,1991)

Page 9: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 9

PurposesofAssessmentTOOL PURPOSE

• Norm-referencedTests Determineextenttowhichachilddiffersfromsame-agepeersAssistswithdiagnosis

• DevelopmentalScales• Interviews/Questionnaires• Criterion-ReferencedProcedures

Establishbaselinefunction

• BehaviouralObservations• FunctionalAssessments• Curriculum-BasedAssessments

Identifygoalsforintervention

• ProgressMonitoringTools Proximalmeasuresofchangesthatarerelatedtotreatment

It is not valid to use tools for purposes for which they were not designed (e.g., standardized tests should not be used for monitoring treatment)

Paul, R. (2007). Language disorders from infancy through adolescence. St. Louis, MO: Mosby, Elsevier.

Norm-referencedAssessment

TestofNarrativeLanguage- 2

Norm-referencedtestsofnarration

• RenfrewBusStory(Cowley&Glasgow,1994)• Expression,Reception&RecallofNarrationInstrument(ERRNI–Bishop,2004)

• EdmontonNarrativeNormsInstrument(ENNI– Schneider,Dube’&Howard,2004)

• TestofNarrativeLanguage– 2(TNL-2– Gillam&Pearson,2018)

Page 10: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 10

RenfrewBusStory

• 3;0– 6;11• Retellastoryfromapicturebook• Informationscore(amountofinformationincludedfromoriginalstory)

• MLU(meanof5longestutterances)• Complexityscore(numberofsentencescontainingrelativeorsubordinateclauses)

Expression,Reception&RecallofNarrationInstrument(ERRNI)• 4– adult• TellandthenretelltheFishStoryortheBeachStorywithpicturecues• Measures:

• StoryContentRecalled• GrammaticalComplexity• Comprehension• Forgetting

EdmontonNarrativeNormsInstrumenthttp://www.rehabresearch.ualberta.ca/enni/• 4;0– 9;11• Generatestoriesfrompictures(2storysetsof3storieseach)• Scores

• StoryGrammar• FirstMentions• Languagesamplemeasures:MLCU,SyntacticComplexityIndex,No.ofWords,No.DifferentWords

• Normsfor2measureson377childrenfromEdmonton,Alberta,Canada

• AvailableinFrench

Page 11: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 11

Storyinformation

score

MLCU SentenceComplexity

RBS– ENNI .08 .18 .07

RBS– TNL .34* .03 .19

TNL– ENNI .37* .39* .51**

Test of Narrative Language-2• Ages 4-15• Components of Language -

Language Use• Formats:

• Script-like story• Personal narratives (sequence

pictures)• Fictional narratives (single

picture of a scene)• Modality:

• Comprehension & Production

• No transcription - scored while listening to audiotape

• Test-retest reliability• Comprehension score - 85• Production score - 82• Total Score (combined) - 93

Comprehension: Script-like narratives

• McDonalds Story• examiner tells a script-like

story • unsolved problem at the

end • respond to literal and

inferential questions• solve problem

Page 12: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 12

LiteralQuestions

• Text:OnTuesday,LisaandRaymondraninandthrewtheirbackpacksonthefloor

• Question:Whatdayoftheweekwasit?

InferentialQuestionsLisafinallymadeuphermind.Shetoldtheclerk,“I’llhavechickennuggets,aCoke,andanicecreamcone.”Lisa’smompaidtheclerkandwenttositdown.Whenshelookedup,Lisawasstandingtherewithabigbrownstaindownthefrontofhershirtandskirt.Lisastartedcryingandsaid,“LookwhatIdid!”

Local:WhatdidLisaorder?Elaborative:WhatkindoficecreamconedidLisaorder?Explanatory(causal):WhywasLisacrying?Predictive:HowwillLisa’smomgetthestainout?

Production: Script-like narrative• Child retells the McDonalds Story• Scored for

• Content• Setting, Motivating events, Actions, Consequences

• Temporal and Causal Conjunctions• Grammaticality• Dialogue• Coherence• Complexity

Page 13: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 13

Comprehension: Personal Narrative

• Shipwreck• Sequence of 5 pictures • Examiner reads a story

– 1 episode• Child answers literal

and inferential questions about the story

Production: Personal Narrative

• Late for School• Sequence of 5 pictures about

a boy who is late for school.• Child creates a story.• Elaborative Inference required

Comprehension: Fictional Narrative

• Dragon Story• Fantasy picture (single scene)

about a dinosaur in a cave.• Examiner tells a complex story• Child answers literal and

inferential questions

Page 14: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 14

Production: Fictional Narrative

• Aliens story• Fantasy picture with Alien

family.• Child creates story

Chapter2:Standardizedassessmentofnarrativeproficiency

ReliabilityTable5.6SummaryofTNL-2ReliabilityRelativetoThreeTypesofReliability(Decimals

Omitted)

Typeofreliabilitycoefficient

TNL-2values Coefficientalpha Test-retest Scorer

Comprehension 81 85 99

Production 87 82 99

NarrativeLanguageAbility 90 93 99

SourcesoftesterroraContent

heterogeneityTime

samplingInterscorerdifferences

Note.aThese sourcesoferrorvariancearefromPsychologicalTesting(7thed.,p.101),byA.AnastasiandS.Urbina,1997,UpperSaddleRiver,NJ:PrenticeHall.

ContentValidityComprehension Production

Microstructure Literalquestions GrammaticalityandSentenceComplexity

Macrostructure Gap-fillingInferentialQuestions

StoryElements,Completeness,and

Complexity

Page 15: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 15

ConstructValiditySample

1 2 3 4

Cattsetal(2015) Kim etal.(2014) Pearceetal.(2014)Montgomeryetal.(2015)

LanguageMeasures

GlobalLanguageScore .51(a) .56(b)

SentenceIm itation .53(c) .65(b) .51(c)

SentenceComprehension .57(b) .53(d)

ReceptiveVocabulary .64(e) .51(f) .61(g)

ExpressiveVocabulary .49(h)

ReadingMeasures

LetterNaming .32(i) .40( j)

ReadingComprehension .49(k) .46(d)

TOSREC .46

TOWRESightWordEfficiency .35

CognitiveMeasures

WorkingMemory .54(d) .50(l)NonwordRepetitionTest .319 .38 .305

RAN -.268(m) -.17(m)

Attention .37(n) .37(o)

ConstructValidityCorrelationCoefficientsBetweenTNL-2andMLU,NDW,NSS

(DecimalsOmitted)

TNL-2Values

SALTScores Study NComprehension

SubtestProductionSubtest

NarrativeLanguageAbility

Composite

MeanLengthUtterance 1 142 47* 49* 53*NumberofDifferentWords 1 142 38 57* 54*

NarrativeScoringSystem 1 142 47* 70** 66**

Average 44* 59* 58*

Page 16: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 16

ConstructValidity:ComprehensionProduction

ConstructValidity:DevelopmentalSensitivity

TNL-2Subtest

Agein Comprehension Production

years n M (SD) M (SD)4 125 11 (07) 15 (09)5 147 18 (07) 23 (10)6 125 22 (08) 30 (12)7 138 28 (08) 38 (12)8 119 31 (07) 43 (12)9 112 31 (07) 47 (11)10 132 35 (05) 52 (10)11 135 35 (05) 52 (09)12-13 176 35 (05) 54 (10)14-15 101 36 (05) 54 (10)

Averagecorrelation .69 .71

Construct Validity: Diagnostic AccuracySensitivity - accuracy at diagnosing disorderSpecificity - accuracy at diagnosing non-disorder

+ Disorder - Disorder+ Disorder

- Disorder

Reference Standard

Index

ca b

d

Sensitivity = a/(a+c) = ____Specificity = d/(b+d) = ____

Sensitivity Specificity

Positive Predictive

Page 17: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 17

ConstructValidity:DiagnosticAccuracy

CutoffIndexScore SD

PercentileRank

SensitivityIndex

SpecificityIndex

70 -2 2 .14 1.082 -1.2 12 .43 1.085 -1 16 .55 .9890 -0.7 25 .78 .9592 -0.5 30 .92 .92

SchoolDiagnosis

LLD non-LLD Total

TNL-2

LLD 47a 21b 68

non-LLD 4c 236d 240

Total 51 257 308Note.Sensitivityindex=47/51=.92;specificityindex=236/257=.92;classificationaccuracy=47+236/308=.92

aTruepositive; bFalsepositives; cFalseNegatives; TrueNegatives.

Normreferencedtestsofnarration• Shouldmeasurecomprehensionandproduction• Comprehensionitemsshouldincludebothliteralandinferentialquestions

• Inferences:text-connecting,gap-filling• predictive,explanatory(causal),elaborative(associative)

• Scorebothmacrostructureandmicrostructureelements• Reliability– item,test-retest,scorer• Validity– content,construct(developmental,diagnosticaccuracy)

Page 18: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 18

ClinicalApplications

DelphistudyBishopetal,2016

nConsensus-buildingmethodn59expertsratedstatementsnComparisonofratingsnRe-ratestatements

nPredominantlyUKnAustralia,Canada,Ireland,NewZealand,USA

Page 19: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 19

Diagnosticcriterian Performance<81(-1.25SD)onaGlobalLanguageTest

n CELF-5,CASL,TOLD-P

nObservations– childat-riskfordisvalueinimportantconversational/academicsettings

n Interviews– levelofconcernamongparentsandteachers

EpidemiologicalStudies

Page 20: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 20

nParticipantsn7,019childrennAge5nIowaandWesternIllinois

Epi-SLIMultivariateAssessmentSystem

n2ormoreComposites:-1.25SD(81)nOverallscore:-1.11SD(83)

Receptive Expressive Composite

Vocabulary TOLD:PPictureIdentification

TOLD:POralVocabulary

VocabularyComposite

Grammar TOLD:PGrammaticUnderstanding

TOLD:PGrammaticCompletionandSentenceImitation

GrammarComposite

Narration NarrativeComprehension

NarrativeRecall NarrativeComposite

ReceptiveComposite

ExpressiveComposite

Page 21: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 21

GillamstudiesReceptive Expressive Composite

Vocabulary CREVT- Receptive CREVT- Expressive CREVT– Composite(z-score)

Grammar CASL– SentenceComprehension,GrammaticalityJudgment

CASL– SyntaxConstruction,GrammaticalMorphemes

AverageZscore

Narration TNL -Comprehension

TNL- Production TNL – NarrativeLanguageAbilityIndex

Averagez-score Averagez-score

ZScoreTransformations

n(Score– Mean)/StandardDeviationnDeviationQuotient(M=100,SD=15)

n (83– 100)/15=-17/15=-1.13n (81– 100)/15=-19/15=-1.26

nScaledScore(M=10,SD=3)n (7– 10)/3=-3/3=-1.0n (6– 10)/3=-4/3=-1.33

Summary:LanguageImpairmentPhenotype

nUnusualdifficultiesunderstandingorusinglanguagen Don’tusenonverbalIQasexclusionarycriterianMorelikelytonormalizeinpreschoolersthanschool-agechildrennCo-occurswithotherproblems– motorskills,literacy,mathematics,socialization,behavior,informationprocessingskills

n Ispersistentafterage5(about70%ofchildrendiagnosedatage5stillhavelanguagedeficitsat19)

Page 22: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 22

Atypical Narrative Discourse Abilities

nHome cultures may value different types of narratives than the ones used in school.

nChildren may have less experiences listening to and telling stories or with elaborative language.

nChildren may be poor language learners

Differentiating Language Difference fromLanguage Disorder (Laing & Kamhi, 2003)

nStandardized tests are nearly useless for differential diagnosis when culture or experience is an issue.nLinguistic and Content BiasnRepresentation of Minorities in Normative Samples

nObservation may not readily reveal whether problems are related to lack of experience or lack of language learning abilities.

n“Watch and See?”nProcessing dependent measures (NWR, CLPT) or interactive

assessment procedures such as dynamic assessment may be useful.

Pretest– Posttest

Page 23: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 23

ProgressMonitoring

MonitoringIndicatorsofScholarlyLanguage(MISL)

ComprehensiveProgressMonitoringTool

• MonitoringIndicatorsofScholarlyLanguage(MISL)

• Spontaneouslygeneratednarrativeselicitedfromasinglepicture.

• Designedtomeasuregrowthinmacrostructure(character,setting,initiatingevent,internalresponse,attempts,consequence,internalresponse)andmicrostructure(coordinatingandsubordinatingconjunctions,adverbs,mental/linguisticverbs,elaboratednounphrases).

• Easytoscore

TranscribethestoryintoC-unitsListenforpausesandsentenceendingintonation(risingpitchonquestionsorfallingpitchonstatements)ascluesforutteranceboundaries.

Segmentintoutterancesthatcontainanindependentclauseplusitsmodifiers.

Page 24: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 24

C-unitscanbeincompleteForexample:nThedogaskedhim,“Where'sthecat?”n“Overthere."n"Oh,okay.”

C-unitscanbesingleclauseutterancesForexamplenJohnwantedthatball.nSusanwantedittoo.

CoordinatingConjunctionsnIfasubjectisstatedorrestatedafterthecoordinatingconjunctionsegmenttheutterancejustbeforetheconjunctionnJohnwasyelling.AndSusanwaslaughingathim.

nIfnosubjectisstatedafterthecoordinatingconjunction,donotsegmenttheutterancesuntilthenextmainclauseappearsnSusanwaslaughingandcryingatthesametime.Johncouldnʼtunderstandthat.

AC-unitcancontainanynumberofdependentclauses.

nSubordinatingconjunction- Mydogjumpedonherbecause shewantedto.

nAdverbialclause- Mydogjumpedonherrightaftershecamein.

nClausalcomplement- Shethought,Idon'tevenlikedogs.

nRelativeclause- Thedogthatmybrotherfound jumpeduponmyfriend.

nInfinitives- Mydoglikestojump uponpeople.

Page 25: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 25

TrysegmentingthisstoryintoC-units

WhenJenniferwas8yearsoldhermomhelpedherbuildashipforaschoolprojectwhenJenniferwastakingittoschoolshedroppeditinamudpuddlehershipwasbrokenanddirtyJenniferwasverysadandshestartedcryingthenshesaidtoherselfcryingwonʼthelpIʼmgoingtofixmyshipatschoolwhenshegottoherclassroomshecleanedoffthemudandtapedtheboatbacktogetherwhenherteachercameovershewassurprisedtoseeadirtyhalf-brokenshipthatwasheldtogetherwithtapeJenniferexplainedwhathadhappenedJenniferʼsteachersaidyoudeserveanAfortryingyourbesttosolveaproblemthatmadeJenniferveryhappy.

Story1

There’salotofkidsandparentsatthebeach.They’retryingtofindseashells.Andthey’regettingwet.

Story2

SoonceuponatimetherewereallofthesemenandtheywereonabikeraceinCacheVilosa.Andsotheydecidedtogetontheirbikesandwaitforthesignaltodropdownforthem.Sothat’swhattheydid.Sotheygotontheirbikesandstartedracingdowntheroadtrack.Andthat’smyverybeststory.

Page 26: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 26

Story3OnceuponatimetherewasaboynamedRylan.Andhewasinahelicopterintheafternoontime.Hedecidedtojumpinhisbestbluehelicopterandgotinitandflewawayinto thesky.Sothat’swhathedid.Hewasgoingtoflyinthehelicopteruntilitwaslunchtime.Butuhoh,hishelicopterfuelgaugesaidthathisgaswasabouttogetdead.Sohedecidedtolandatahelicoptergasstation.Sothat’swhathedid.Helandedatahelicoptergasstation.Hegotgasandwentbackintohishelicopter.Hewenttohishouse,atelunch,andhadbiscuitsandgravyforlunch.Hefeltpleasedthathewasfinallyfull.

Page 27: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 27

Wenevergivecreditforelementsthatarenotstatedexplicitly.TheMISListiedtoourintervention.Weteachtheelementsthataretested.

Ifyougivecreditforwhatyouthinkthechildmeant,thereisnowaytodocumentprogresswhenthechildactuallystateswhats/hemeans.

ReliabilityandValidity

• Reliability• InternalConsistency:Cronbach’salpha- .79• Inter-rater:95%

• Validity• ConstructValidity

• MacrostructureandMicrostructuredimensions• ConfirmatoryFactorAnalysis

• overallmodelfit:χ2(df =53)=81.27,p =.008,χ2/df ratio=1.53;CFI=.99;TLI=.98;• RMSEA=.06;andtheaverageWRMR=.82.

Page 28: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 28

Story1

nThere’salotofkidsandparentsatthebeach.nThey’retryingtofindseashells.nAndthey’regettingwet.

Macrostructure0 1 2 3

Character

Setting

InitiatingEvent

InternalResponse

Plan

Attempt

Consequence

Reaction

Microstructure0 1 2 3

Coordinating

Subordinating

Mental

Linguistic

Adverbs

ENP

Page 29: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 29

Story2

SoonceuponatimetherewereallofthesemenandtheywereonabikeraceinCacheVilosa.Andsotheydecidedtogetontheirbikesandwaitforthesignaltodropdownforthem.Sothat’swhattheydid.Sotheygotontheirbikesandstartedracingdowntheroadtrack.Andthat’smyverybeststory.

Macrostructure0 1 2 3

Character

Setting

InitiatingEvent

InternalResponse

Plan

Attempt

Consequence

Reaction

Microstructure0 1 2 3

Coordinating

Subordinating

Mental

Linguistic

Adverbs

ENP

Page 30: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 30

Story3OnceuponatimetherewasaboynamedRylan.Andhewasinahelicopterintheafternoontime.Hedecidedtojumpinhisbestbluehelicopterandgotinitandflewawayintothesky.Sothat’swhathedid.Hewasgoingtoflyinthehelicopteruntilitwaslunchtime.Butuhoh,hishelicopterfuelgaugesaidthathisgaswasabouttogetdead.Sohedecidedtolandatahelicoptergasstation.Sothat’swhathedid.Helandedatahelicoptergasstation.Hegotgasandwentbackintohishelicopter.Hewenttohishouse,atelunch,andhadbiscuitsandgravyforlunch.Hefeltpleasedthathewasfinallyfull.

Macrostructure0 1 2 3

Character

Setting

InitiatingEvent

InternalResponse

Plan

Attempt

Consequence

Reaction

Microstructure0 1 2 3

Coordinating

Subordinating

Mental

Linguistic

Adverbs

ENP

Page 31: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 31

Story4

Thekidiswake/ing up.Hehadtohurryforhegetlate.Hetie/edhisshoe/sbutitbrake.Hehadtogetinschoolbus.Butit/'stoolate.CHerantoit.CButnouse.CWhenhewalk/ed teacherisverymadathim.CThat/'sbecausehegotlate.CYepthebellring/3s.CButhe/'stoolate.CToolate.

Macrostructure0 1 2 3

Character

Setting

InitiatingEvent

InternalResponse

Plan

Attempt

Consequence

Reaction

Microstructure0 1 2 3

Coordinating

Subordinating

Mental

Linguistic

Adverbs

ENP

Page 32: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 32

Story5• CMaxandBobwereatthefoothillsandtheyhavetwohorsesfortheirride.• CAndthenonenightawolf(was)wasnearbyandtheywerescaredbecausethehorsesweretheirride.

• CTheirplanisto(get,Imean)guardthehorses.• CThefirstthingtheydotheyguardthehorses.• CSecondtheylookoutforthewolfthenthewolfwasrunningtothehorsesbutthenbothofthemscaredthewolfandthewolfrantotherockymountains.

• CNowMaxandBobwerecalmbecausenowthewolfisnotgonna gettheirhorsesagain.

• CTheend.

Macrostructure0 1 2 3

Character JoeJoeyObwee

Setting Backyard Duringtheday,Lastnight,Thatday

InitiatingEvent Playinginmud Mombroughtbreakfastanddiscoveredkidshadmudalloverthem

Theydecidedtogotakeabath

InternalResponse Momsurprised(relatestotheirhavingmudonthem)

Plan Thoughtthey’dgofrommuddiesttonotmuddiestPlan,2st IE

Decidedthey’dhitthefaucet(Planin2nd episode)

Attempt Saidtheyhadtotakebath

Waterwasn’trunning/hitthefaucet

Consequence Theydecidedtogotakeabath

Gotclean

Reaction Happytheygotabath

Microstructure0 1 2 3

Coordinating Then,and

then

and but or

Subordinating what Before

Mental

Linguistic said like

Adverbs before so Allofa

sudden

before

ENP Hisfriend

Page 33: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 33

Page 34: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 34

Page 35: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 35

Summary•Keyissuesinmeasuringnarrativecomprehensionandproduction

Comprehension Production

Microstructure

Macrostructure

Page 36: OSHA presentation 1 - Oregon Speech-Language Hearing ......OSHA presentation 1 Ron Gillam, PhD, and Sandi Gillam, PhD Utah State University Assessment of Narratives in School-age Children

RonandSandiGillam 10/2/18

OSHApresentation 36

Norm-referencedvs.ProgressMonitoring

•Purpose•Methods•Reliability•Validity•Clinicaldecisions

Examples•TNL– 2

•Content•Constructs•Psychometrics

•MISL•Content•Constructs•Psychometrics