modules.nceln.fpg.unc.edumodules.nceln.fpg.unc.edu/sites/modules.… · web view · 2014-10-30nc...

Module 1. Overview of the Early Childhood Outcomes System

MODULE 1: Overview of the Child Outcomes Measurement System

Slide 1: Effective Practices Supporting Early Childhood Outcome Measurement

Slide 2: What we will cover

[Trainer note: you may wish to break this training into 6 modules based on the goals listed below, or combine two sections at a time, depending upon time and your professional development calendar for the year]

• Why are we doing this?• What are the three child outcomes?• What is functional performance and how do we measure it?• What is the 7 point rating scale?• What is a developmental trajectory?• What does this mean to me and my program?

Slide 3: What are the expectations of professional who do the COS ratings?We would like to start today by outlining what the expectations of you as a professional are when doing the Child Outcomes Summary Ratings and how they align with your NC Professional Teaching Standards

[Trainer note: the purpose of this activity is to tie the competencies needed to do reliable and valid COS ratings to the teacher’s performance measurement process]

Next we will do a short activity. You will need to take out your handouts titled “North Carolina Professional Teaching Standards.”

Please pair up with one other person and reflect on the 5 bullet points on this slide. Please compare theses bulleted competencies required to do reliable and valid child outcome ratings to the Professional Teaching Standards

Understand the content of the 3 Outcomes

When staff demonstrate their understanding of the content of the 3 child outcomes, what professionalstandards are they demonstrating?

Develop an intentional plan to collect measureable data that will support the ratings

When staff demonstrate an intentional plan for collecting measureable data and can verbally explain to their administrators what collaborative teaming approaches they use to conduct the COS ratings, what professional standards might they demonstrate?

• Know about and use resources to compare a child’s functional behavior to age expected milestones.

When staff demonstrate an understanding of normal child development so as to age anchor their own child observational data, or the data gleaned from interviews with parents and others, what professional standards might they demonstrate?

1Improving the Child Outcomes Summary (COS) Process

NC Early Learning Network, a joint project of NC-DPI and UNC-FPG, 2014


• Know how to use the rating scale and understand the “developmental trajectories”

When staff demonstrate how they can track the progress in the developmental trajectories of the children they serve, what professional standards might they demonstrate?

[Trainer note: allow participants time to come up with the standard they think each competency links to the NC Professional Teaching Standards and report out to the whole group]

Slide 4: Required competencies

[Trainer note: you may wish to break this training into 6 modules based on the goals listed below, or combine two sections at a time, depending upon time and your professional development calendar for the year]

• Why are we doing this?• What are the three child outcomes?• What is functional performance and how do we measure it?• What is the 7 point rating scale?• What is a developmental trajectory?• What does this mean to me and my program?

Slide 5: Chapter section: WHY ARE WE DOING THIS?

Slide 6: Goal of early childhood education

This goal is for children with disabilities, as written by the Early Childhood Outcomes Center. However it is also true for all children.

Slide 7: Child outcome information will be useful to provide:

Data on early childhood outcomes are only collected by the early intervention and preschool special education programs funded by IDEA. The data are reported to congress and are used to justify our continued funding.

We also have interested stakeholder and advocacy groups that are beginning to watch us closely.In addition, programs can use the outcomes data to inform decisions about program improvement.

Slide 8: Accountability to federal government

On the local level a school system’s data are required to be publically reported. To date this reporting has not been very well understood by others, but the state and the technical assistance centers funded by OSEP are working hard to come up with ways to make it more meaningful so that it can be used for school system self-assessment and improvement plans.

As the state is formulating its statewide systemic improvement plan to demonstrate measureable results around a targeted project, most around improving the graduation rates for children with disabilities, the outcomes data are being analyzed along with school aged data around end of grade




assessments for school aged children. We will develop a statewide strategy on the preschool level that supports the ultimate state target and the outcomes data will be used in this process.

Slide 9: Chapter section: What are the three child outcomes?

Slide 10: “Child Outcomes Step by Step”

Please watch this video produced by Larry Edelman as we begin to understand what the three child outcomes actually look like.

Slide 11: Place Holder for the Video

[Trainer note: you will need to embed the video stored on your thumb drive into the PowerPoint. For more information about how to do this, please go to instructions at this link http://nceln.fpg.unc.edu/sites/nceln.fpg.unc.edu/files/resources/howtoembedvideointoppt.doc ]

Slide 12: Chapter section: Functional Outcomes

Slide 13: Functional outcomes

The three child outcomes refer to actions that children need to be able to carry out and knowledge that children need to use in order to function successfully across a variety of settings.

Slide 14: What does it look like?

Functional outcomes are the “why” children do what they do. These outcomes require children to integrate a series of behaviors or skills that allow him or her to achieve things that are meaningful in the context of the child’s everyday life.

Slide 15: Functional Outcomes:

On this slide you will see a specific and objective description of a behavior on the left, with the corresponding purpose on the right.

The outcomes are meant to look at a child’s everyday purposeful functioning rather than how he or she performs on a specific item on an assessment measure. For example, we’re not looking at how a child places a ring on the stacking toy. Rather, can the child find the loop on her coat to hang it on the hook in her cubby. For a younger a child, we’re not just interested in whether or not the child can smile, we want to know whether the child smiles back at mom in interactive social play.

The focus is on the child’s ability to take functionally meaningful actions.

The three outcomes are broad. They are meant to give a snapshot of the whole child in his or her everyday life. The outcomes cross developmental domains in order to see how children integrate skills and behaviors in order to participate in their everyday activities.

The three outcomes can also be viewed as evidence of a young child’s “educational performance” and as such, can be considered when making eligibility determinations. What adverse impact does the child’s



http://nceln.fpg.unc.edu/sites/nceln.fpg.unc.edu/files/resources/howtoembedvideointoppt.doc


delay or disability have on his or her ability to perform the three child outcomes we are required to measure?

Slide 16: Why not just measure developmental domains?

Developmental domains are an artificial construct that adults developed in order for us to categorize and measure children’s behavior. Unfortunately, many assessment tools have items that measure discrete skills that aren’t individually meaningful to the child. The skills are meaningful only when they are integrated with other skills and behaviors so that the child can accomplish something. For example, we are interested in whether children can sort and match objects. We’re not interested in whether a child can sort beads without a purpose, but whether the child can find a matching shoe while getting dressed in the morning. It’s not just whether a child can name 3 objects, but can she use words to identify the toys she wishes to play with.

Slide 17: Valid measurement

This is extremely important for staff to understand. The COS ratings are NOT a measure of how much progress a child has made in your class or therapy sessions. You should not be thinking, “he or she has made so much progress on his IEP goals.”

The COS ratings ARE a comparison of how the child is functioning at a particular point in time, as compared to the age expected behavior.

To be valid and reliable, you need to know what age expected behavior is. You need to know age-expected child development.

Slide 18: Outcome 1 Positive Social Relationships.

Please take out your brochure titled Early Childhood Outcomes System (COS). On page 2, the three child outcomes described in the next few slides are also described here.

The first outcome states that children have positive social relationships. This includes relationships with adults and other children and getting along in groups (for older children).

[Trainer note: have a participant read aloud the information about Child Outcome 1 from the brochure]

Slide 19: Outcome 1 Positive Social Relationships.

Assessment tool items and observations often include looking at: attachment/separation/autonomy, expressing emotions and feelings, learning rules and expectations, and social interactions and play.

Slide 20: Outcome 2 Acquisition of Knowledge and Skills

Outcome two states that children acquire and use knowledge and skills. This involves thinking, reasoning, remembering, problem solving, using symbols and language and understanding physical and social worlds – such as science and social studies.





Slide 21: Outcome 2 Acquisition of Knowledge and Skills

Assessment and observation often include looking at early concepts, expressive language and communication and, for older children, early literacy and numeracy.

Slide 22: Outcome 3 Taking actions to meet needs

Outcome three states that children take appropriate action to meet their needs. This includes self-help and adaptive skills such as taking care of basic needs, getting from place to place, using tools and contributing to their own health and safety. Skills and behaviors might include integrating motor skills to complete tasks, self-help skills and acting in appropriate ways to get what one needs or wants.


The three outcomes are broad. They are meant to give a snapshot of the whole child in his or her everyday life. The outcomes cross developmental domains in order to see how children integrate skills and behaviors for successful participation in their everyday activities.

So, we’ll be asking more functional questions about the whole child—For example, for outcome 1, we might ask questions like -- How does the child show affection? And, How does the child interact with others?

For outcome 2, we might ask questions like -- How does the child solve problems? How does the child show he or she understands concepts or words?

And for outcome 3, we might ask questions like -- How does the child express that he is hungry?

For example, we may observe a child who uses 3-word phrases to engage another child in play during center time. This tells us about how he integrates language with social functioning. We want to know more than the number of words in the child’s vocabulary, we want to know how he puts words and actions together in meaningful, functional ways.

Another example might be a child who can point to juice to indicate that she would like a drink. This tells us about how she uses language to get her needs met. We want to know more than just whether she can point. We want to know whether she can point to communicate a need.

For further information about functional outcomes, refer to the document titled, ‘What is a Functional Outcome?’ at this link -- http://ectacenter.org/~pdfs/eco/Functional_outcomesHO.pdf

Slide 23: Chapter section- What is the process for measurement?



http://ectacenter.org/~pdfs/eco/Functional_outcomesHO.pdf


Slide 24: Initial COS ratings

We recommend that assessment teams who conduct the comprehensive evaluation to determine program eligibility also conduct the initial COS ratings and communicate those ratings to the team who will be responsible for the child’s services. This has proved to be the most reliable process across the state. A consistent team of individuals will then be conducting the COS ratings on a frequent basis using all of the rich information they have gathered in their evaluation process.

Slide: 25 Exit Ratings- For Classroom Teachers

Most classroom teachers are using curriculum-based assessment tools that gather rich information about a child’s behavior and skill level. They also communicate with families on a regular basis. You can use all of these data to support the exit COS ratings. Classroom teachers, itinerant teachers, Speech Pathologists and other related service providers are at a disadvantage when they have to conduct exit ratings if they have not intentionally developed a system for gathering the supporting data on which to base their ratings.

Slide 26: Exit Ratings - Itinerant Teachers

Itinerant teachers serve children in a variety of settings outside of school system-administered classrooms. Some child care teachers or church preschool teachers may not have a curriculum assessment in place. First you need to ask. If they do collect this information, then please access it and interview the teacher to gain more information about the child. Design a system for collecting these data and organizing them for COS rating purposes.

Slide 27: Exit Ratings – Speech and Language Pathologists

It is important for Speech -Language Pathologists to collaboratively team with others to gather child observation information, especially when you have a crowded schedule and you see certain children at particular times of a given day or week. If your schedule will only allow you to observe a child in the same classroom routine each week, you must gather information from the teachers and assistants to get a picture of how that child functions at other times in the day.

Preschool Speech-Language Pathologists (SLPs) and Itinerant Teachers serving children who do not attend (or have limited time/attendance in) a preschool or childcare program with a structured curriculum, or stay at home during the day, are frequently at a disadvantage in collecting supporting data for exit COSF ratings in all the outcome areas.

Slide 28: Service Provider Location

The same challenge that SLPs face when going into preschool classrooms at set times of the day/week is also the case for those providers who serve children in locations in which the family drives them to the location to receive their services. You are limited in your ability to gather direct observational data in the environment in which the child typically functions.




Slide 29: Chapter section: How do you gather reliable data to support the ratings?

Slide 30: Observation: data from you

Next we’re going to address the skills needed when actually writing down your observation notes. This is applicable to classroom and itinerant teachers, speech-language pathologists and others.It is important for Speech -Language Pathologists to collaboratively team with others to gather child observation information for reasons that we’ve already discussed. However, there is another important reason that is related to your training, highly developed listening skills, and your attentional capabilities. SLPs often face particular challenges when trying to “script” child behavior seen during play in such a way that it is easily age-referenced at a later time. They are trained to collect language and phonological samples, so they primarily observe with their ears and they focus on what the child says…not what he/she’s doing. Their area of expertise and focused attention sometimes interfere with the ability to see and document functional behavior which includes motor, adaptive, and other skills pertinent to all three of the child outcomes. Teachers are taught this process as part of ongoing child assessment, however SLPs typically are not. It may be necessary to address these skills in future professional development for SLPs in order for them to successfully obtain child observation data which may be quantified through the age referencing process.

Another challenge people who have been trained to do diagnostic assessments face in conducting observations is the tendency to identify a problem and automatically place a diagnostic descriptor to what they are seeing. For example, “he demonstrated hemiparesis in his right arm.” That is a diagnostic description. You need to use language that describes what the child did so that another person can envision what that behavior actually looked like. Good observations paint a clear picture of behavior. They do not label the behavior. Use the words: “He raised his right arm in an attempt to put his hand through his shirt. He could not complete the motion and gave up trying after a brief effort.”

Slide 31: Scripted Observations

In the book, Observation and Participation in Early Childhood Settings, the authors explain observation as a system of looking at behavior. Teachers and other early childhood professionals observe children to assess their behavior in unobtrusive ways – without interrupting the play or activity (Billman, 2003). Equally important is how we record our observations. Observational data must be measureable.

Slide 32: Interpreting observations

We’re always observing and, as we’re observing, we’re always interpreting what we see. What we see is limited by what we expect to see. To collect objective observational data, we have to train ourselves not to filter observations according to how we think something should look.A photographer once said that photography taught him how to see. Initially, he would look through the lens of the camera and see the finished image. But the finished image was what his mind saw, not what the scene actually looked like through the viewfinder. When he reviewed the printed picture it didn’t look the way he remembered it. He realized he had to clear his mind of interpretation in order to see a scene only the way it appeared through the viewfinder.This is what we do when we observe children: clear our minds of interpretation. We must remove all the extra padding of pre-conceived ideas and just observe. We write down or we “script” what we see and what we hear. Nothing else. Just exactly what we see and hear.




Slide 33: Documenting observations

To record objective observational data, we describe exactly what we observe – not our interpretation or opinion of what we observe. Try not to write down subjective phrases because they can mean different things to different people and are therefore difficult to measure accurately. Language such as “with difficulty” and “excited” express your opinion about what you saw. What would be a more objective way to describe “walking with difficulty”?

[Allow participants to respond. Prompt as needed to say things like, “he walked slowly and stopped to take a deep breath every two to three steps.”]

We also need to steer clear of interpretative words like “persistent” and “determination” which are difficult to measure. What is a more objective and measurable way to describe persistence or determination?

[Allow participants to respond. Prompt as needed to say things like “He worked on the block tower for 4 minutes and rebuilt it when it fell three times.”]

Slide 34: Objective and specific- photo of a smiling boy in a striped shirt

It is our human nature to embellish. “He is a 3 year old with brown hair and brown eyes.” We can observe that. We can measure that. “He was wearing a cute shirt and was happy.” That is harder to measure. What makes his shirt cute? How do you know he is happy? It would be better to say “He was wearing a striped shirt and had a smile on his face.” How we phrase what we see is the key. Here is an example: “The children came inside and were freezing.” What part is measureable? How could it be rephrased so that it is objective?

[Allow participants to respond. Prompt as needed to say things like, “The children came inside with rosy cheeks, wearing coats and mittens.”]

Also keep in mind that observations need to be specific. Give details that are precise. The information we gain from objective and specific observations is used to help us determine where a child is functioning. Being objective and specific is critical.

Slide 35: Observing Nathan- Screen shot of a picture of Nathan

[Trainer note: Make sure that participants have paper and pen sufficient to write observational notes about a child in play. Each table will need to have one set of colored markers (three different colors). Suggestions are yellow, green, and orange. It is suggested that you have your participants sit in small groups at a table.]

Next you will observe a video of a little boy we call “Captain Nathan.” In this clip he is 2 years 11 months of age and is in a play-based assessment. As the clip begins you will see Nathan leaving a small table in the housekeeping area. The woman at the table is his mother. They have just finished a snack. Nathan has what appears to be a patch on his eye, and a scarf on his head. When Nathan and his mom came into the school that morning his mother was given a visitor patch to put on her shirt. When they sat down at the table for snack, Nathan reached over and took the visitor patch off his mom’s shirt and put




it on his eye and said “I’m a pirate.” That led the play facilitator to get an apron in the housekeeping center and fold it up to make a pirate hat for him and put it on his head.As you watch this video we want you to practice writing observational notes. Be specific and objective in your notes. Write only what you see. No interpretations or opinions. Try to write your notes so that you can apply an age-expected behavior measure to them later on.

[Trainer note: play the video]

Now work with your table partners and read the notes you took to one another. Help one another to re-write the notes if you catch yourself not being specific and objective.

[Allow participants time to work in small groups to accomplish this task. While they are talking, put up a chart paper in preparation for the final step in this activity.]

Can some of you tell us about an observational note that you re-constructed with the help of your team members?

[Trainer note: write down what the participants report out on chart paper labeled “From this” “To This”]

Thank you all. Keep these observational notes on hand for a later activity.

Slide 37 – Organizing the observation data into the three outcomes

[Trainer note: Give participants completed observational notes for Nathan titled “Scripted Observation Notes from Captain Nathan Video.” Do not give out till now.]

Now we’re going to do a small group activity looking at the notes taken during the observation of Nathan in this video clip. Please review this information and decide with your table partners which interview notes could go into which of the three outcomes. Code scripted notes that fell into Outcome 1 with a yellow marker, Outcome 2 with a Green marker, and Outcome 3 with an Orange marker.

[Trainer note: Allow participants enough time to discuss and organize. While they are discussing, put up three pieces of chart paper. Title each one for a Child COS Outcome].

Please tell me which interview script fell into Outcome 1: Positive Social Relationships.

[Trainer note: write on chart the interview script they selected to go in this area. It is important for you to acknowledge that some interview scripts will go into multiple outcomes.]

Please tell me which interview script fell into Outcome 2: Acquisition of Knowledge and Skills.


Please tell me which interview script fell into Outcome 3: Getting Needs Met Appropriately.





Slide 38: Collecting data from other sources

We talked about the need for some of you to collect data about children’s functional performance from other people. For classroom teachers you might think about the parents of your children or the related service providers who serve them. For itinerant teachers and SLPs this may include classroom teachers.It’s important to know what data are currently being collected by teachers when the children are in a structured program like the Title I or NC Pre-K classes in the schools and in the community. There is a treasure trove of information out there that you need to tap into.

It is also important that you know how to collect structured interview information from teachers and parents that can be used to support the COS ratings and monitor IEP goals and track a child’s overall progress.

Slide 39: Classroom data to be collected -Screen Shot of the TS-Gold Assessment Portfolio

Teachers using a structured curriculum typically use the corresponding curriculum assessment tool. You need to ask teachers to show you the data they are collecting. Typically, classroom teachers sort their data into the curriculum assessment areas that by and large report behavior by developmental domain. Not to worry, you can use these same data for the purpose of supporting the COS ratings.

Slide 40: Classroom portfolio data – picture of Jennifer in Cabarrus

Teachers frequently have child portfolios with a variety of data. This is an example of a child writing and the teacher taking either a video of her actually writing or a picture of her written product. It’s very important that the teacher label these work samples briefly with an observational note that can help them use these data in a measureable way. For example, she may write “Jessy. 10.20.13. Writing her name.”

Slide 41: Classroom portfolio data – picture of a group of children with a block structure

Teachers can also document child performance about social interactions. In this picture, this little group of friends cooperatively worked together to build a block structure. The teacher’s specific and objective notes about what the children were saying to one another during this activity, and who said what provide very measureable data used with an age-expected anchoring tool.

Slide 42: Classroom portfolio data –picture of two boys cooking together in the house keeping center

In this play sequence, careful notation shed a spotlight on the emotional-social skill level of these two boys by capturing what they were doing, who started this play schema, and how their conversation progressed as they built off of each other’s ideas.

Slide 43: Classroom portfolio data – family input- picture of family pictures

Ask teachers what they collect from families throughout the year that can be used to document child progress. In today’s world of technology many families send videos and pictures of their child doing




various things at home. Make sure you consider all of this information when you gather your supporting documentation to conduct the COS ratings.

Slide 44: How those classroom teachers do it! - Video of Amy Harding from Cabarrus County

Now let’s listen to a classroom teacher explain how she collects data during the day when she is involved in play activities or when she steps out of the action and reflects on what she observed at a later time.

[Trainer note: play the video]

Slide 45: Collecting data through interviews

Data can also be collected through interviews with parents and classroom teachers to find out more in-depth information about a child’s functional performance in the routines of the day. If this data is collected through objective and specific notes when using a standardized interview process, this data can also be sorted into the three outcomes and age-anchored just like good observational data. We have developed another training module (Module 2: Using Parent and Teacher interview Information) which goes into more detail about how to use the Routines-Based Interview Process to gather this information.

Slide 46 – Chapter Section: How do I Measure this Data?

Now that we’ve talked about how to collect data using entry level assessment, curriculum assessment, and the interview process, let’s turn our attention to how to take those data and identify the child’s current age level of functioning.

Slide 47- Age anchoring the data

We need to utilize available tools that can help us identify the most prevalent age levels associated with the data we have collected. We call this “age anchoring” the data. This is the same process used in the Transdisciplinary Play-Based Assessment process. The age tables in those manuals, as well as other developmental milestone tables, can be used once you have organized your data into the three child outcome areas. We have provided you with another handout titled “Age-Anchoring Tool for Use with the Child Outcomes Summary” process. This was adapted from a document North Dakota created in 2012.

It’s important to remember that when we use tools like this to age-anchor a child’s skills and behavior, as captured through observation or interviews, we will see different age levels. The key is to identify the age level in which the data most commonly “cluster.” This is not a perfect science, but it has proven to yield reliable ratings once folks become skilled in the process.

[Trainer note: Handouts: 1) Age Anchoring Tool for Use with the Child Outcomes Summary process and 2) Nathan's Observation Notes Sorted by Outcomes]

Let’s practice age-anchoring Nathan’s scripted observation notes. The handout titled “Scripted Observation Notes from Captain Nathan Video” was developed for us to use as a model, both in terms of the quality of the scripted notes and as a means of consistently organizing the notes into the three




outcome areas. In addition we need you to take out your handout called “Age-Anchoring Tool” to use in this next activity.

For Outcome 1 [Trainer note: ask the participants to identify an age level for each of the notes placed into this outcome area and record the age level beside the note]

For Outcome2 [Trainer note: ask the participants to identify an age level for each of the notes placed into this outcome area and record the age level beside the note]

For Outcome3 [Trainer note: ask the participants to identify an age level for each of the notes placed into this outcome area and record the age level beside the note]

Okay, we will use this information later on.

Slide 48- Chapter Section: Now, How do I determine the rating?

Teams, rather than one individual person, should work together to come up with the ratings. You may do this by meeting face to face, or have one person responsible for collecting data from different team members and assembling it on the child outcomes forms. You can meet to discuss the final rating in a variety of way, e.g., virtually or face to face. Once you organize the data into the three outcome areas and age anchored them the next step in this process is to apply the rating scale to the data. Remember, the ratings themselves are not an assessment. It is a summative process that uses other kinds of formative and summative assessment data.

Slide 49- COS 7 point scale

The rating system is a 7 point Likert scale that compares a child’s functioning to functioning expected for that child’s chronological age. Ratings are conducted at different points in time to assess how well a child has progressed over time. For federal reporting purposes the ratings must be conducted on children who are enrolled in the program for at least 6 months, and be done at program entry and exit. However, these ratings can be done more often to track progress the child makes while in the program and inform the instruction he/she is receiving.

Slide 50- Early Childhood Outcome System brochure

Please take out the brochure titled NC Early Childhood Outcomes System. This is a revised brochure that describes the three child outcomes and the definitions for the Outcome Ratings.

The first concept that we need to emphasize is that child development occurs over time in progressions. The definition of “progression” is “a movement or development toward a destination or a more advanced state, either gradually or in stages.”

On page 3 of this brochure you will see the figure of developmental progressions coded in different colors. Each color coded progression shows the potential growth that a child may make over time while remaining in that same progression. There are seven progressions. As children progress from birth through age 6 the difference between the progressions widens.




The top two progressions are coded in orange and yellow and are given a numerical value of 6 and 7 on the rating scale. These progressions are those of typically developing children. The rate of their development is considered to be within normal limits. Their skills are rated at age level upon program entry. Their development progresses at a normal rate over the time in which the child is enrolled in the program, and they are rated at age level upon exit from the program. The progressions below these top two depict child development that is improving, but below age level functioning. Their rate of development is slower than the typically developing child. The important point here is to note that even children who are functioning well below age expectation do develop new skills even if their development is slower than a typically developing peer. This can be seen by the positive slope of each progression.

When we rate observation or interview or other anecdotal data using the age anchoring process, the reliability of our ratings improve significantly.

Slide 51- Collaborative Teaming

Another practice that increases the reliability of our ratings is to do them as part of a collaborative teaming process. No one person should determine these ratings without gathering information from others through the use of an assessment tool, observation, or interview process.

When teams engage in the rating process together, or if one person determines the rating with information from others, the next step is in applying the decision tree. The Decision Tree for Summative Rating Discussions is on the last page of this brochure.

Let’s practice this process using our data we’ve collected from Nathan.

[Trainer note: point to the Outcome 1 chart with Nathan’s scripted notes and age levels written beside the notes]

The first question in the decision tree asks, “Does this child ever function in ways that would be considered age-appropriate with regard to this outcome?” Let’s look at Outcome 1 for Nathan. How would you answer this question for Nathan for Outcome 1?

[Trainer note: continue this process down the decision tree until you come to the final rating for Outcome 1].

Okay, let’s try Outcome 2.

[Trainer note: point to the Outcome 2 chart with Nathan’s scripted notes and age levels written beside the notes]

Does this child ever function in ways that would be considered age-appropriate with regard to this outcome?”

[Trainer note: continue this process down the decision tree until you come to the final rating for Outcome 2].




Now let’s try Outcome 3. [Trainer note: point to the Outcome 3 chart with Nathan’s scripted notes and age levels written beside the notes] Does this child ever function in ways that would be considered age-appropriate with regard to this outcome?” [Trainer note: continue this process down the decision tree until you come to the final rating for Outcome 3].

Slide 52- Supporting Documentation

Outcomes documentation must be noted on the form and available in each child’s folder and be understandable to others- to the extent that they could use the data and rate the child themselves. If others do a reliability rating check, they team’s rating and the reliability rating check should be within a point +/- of the teams ratings.

[For more information about checking quality on the Child Outcomes Summary Form, please see the information at this link -- http://ectacenter.org/eco/assets/pdfs/Guidance_for_reviewing_COSFs.pdf ]

Slide 53- Child Outcome Summary Form

[Trainer notes: each school system has developed their own child outcome summary form. This slide depicts the generic form that the national organization provided. You will need to have samples of your own outcome summary form available to your participants and review expectations for the use of this form with your participants.]

Slide 54- Chapter Section: So what does this mean?

Our child outcome data are now being used to publically report the extent to which our children make progress in our programs. The aggregated progress data are a measure how effective our services were for those children.

The stakes are getting higher as time goes on.

On the new State Systemic Improvement Plan, our early childhood outcomes data are being used as evidence of how well our early childhood special education programs support children with disabilities in the early years and how that might result in improved graduation rates in the later years.

Slide 55 – Accountability

Our primary assumption is that by providing services to young children with disabilities we help them improve their developmental growth so as to move them to age functioning that is closer to their same aged peers.

Slide 56- Demonstrating Growth

We demonstrate this growth in children by doing at least two child outcomes ratings and charting the difference between status of functioning over time.

Again, it is not about how much progress children make on their IEP goals. It’s about how much progress they make in their growth over time as compared to age level expectations.



http://ectacenter.org/eco/assets/pdfs/Guidance_for_reviewing_COSFs.pdf


Slide 57- Developmental Trajectories- picture of the brochure

Please take out the handout titled “Understanding the Developmental Trajectories for Early Childhood Outcomes.” This handout will describe each of 5 developmental trajectories that we will review in the next few slides.

Slide 58- Plotting a Developmental Trajectory

Each child develops according to their own developmental trajectory. By providing services and supports we are trying to move each child who enters our program below age level to a level closer to age expectation. When we plot a child’s functioning (in each of the three outcomes) at program entry and exit, we can then chart the trajectory that each child achieved. We can see if he/she closed the gap and move closer in functioning to his/her same age peers.

For measurement and reporting purposes, the Early Childhood Outcomes Center developed a profile for each of 5 categories of developmental trajectories called “OSEP progress categories”.


To plot a child’s developmental trajectory, we start by plotting his level of functioning at entry. In this example, the child’s entry rating was at a level 3. Because we will just plot entry and exit, we can indicate the entry rating on the left end of the horizontal axis. We will plot the exit rating at the right end of the horizontal axis.


At the time of exit, this child’s functioning was at a level 4 in the outcome area. We can plot the exit rating toward the right end of the horizontal axis.


Draw a line to show the child’s developmental trajectory. This child came into the program at a level 3 and left at a level 4 in this particular outcome area. The exit rating is below the dotted line, indicating that he did not catch up with his peers and did not leave the program at age level. However, the upward, positive slope indicates that the child made progress.

Slide 62 – Developmental Trajectory A

The first developmental trajectory, called category A, describes a child who did not make progress during the time in our program, AND did not gain any new skills. These are the children who exit services at a lower rating than when they entered and the yes/no progress question indicated that they did NOT acquire even one new skill. Children in category A may have degenerative disease or other significant traumatic brain injuries that prevent them from gaining new skills.




Slide 63- Developmental Trajectory A- picture of the trajectory

In this example of a child’s developmental trajectory, he entered services with a rating of 3 and exited with a rating of 2 in this particular outcome area. The yes/no progress question was answered ‘no.’ Notice the downward slope of the trajectory as plotted on this chart.

Slide 64 - Developmental Trajectory A - picture of the trajectory

This is another example of a developmental trajectory A. The child who enters and exits with a rating of 1 may also be reported in the ‘a’ category – IF the yes/no progress question is answered ‘no.’

Slide 65 – Developmental Trajectory B

The second developmental trajectory – called category B, describes a child who did not make progress BUT did gain new skills. They stayed at the same level of functioning throughout their time in our services and programs.

Slide 66 - Developmental Trajectory B- picture of the trajectory

This is an example of a development trajectory categorized as a B. The child entered services with a rating of 5 and left with a rating of 5. The answer to the yes/no progress question was ‘yes,’ the child had acquired at least one new skill. When you plot the trajectory, note the slope - the line is not flat. The child has improved functioning.

Slide 67 - Developmental Trajectory B- picture of the trajectory

Here is a different example of a category B developmental trajectory. This child entered services with a rating of 3 in a particular outcome. At exit, he was also given a rating of 3. Remember that you always compare the child’s development to age expectations. As the child gets older, more is expected. To receive a rating of 3 at exit means that he still showed immediate foundational skills relative to an older child. Maintaining the same rating actually implies progress. The yes/no question should be answered ‘yes’ and the child reported in the ‘b’ category of progress.

Slide 68- - Developmental Trajectory B- picture of the trajectory

Here is yet another example of a category B developmental trajectory. Even in maintaining a rating of 1, the child may have acquired a new skill and could be reported in the ‘b’ category. Note the slope - this child is showing improved functioning.

Slide 69- Developmental Trajectory C

The third developmental trajectory, categorized as C, depicts children who improved functioning, changed developmental progression to a level closer to age expectation, but did not reach age level functioning by the time they exited our program. They closed the gap.




Slide 70- Developmental Trajectory C- picture of the trajectory

Here is an example with a child who entered with a rating of 3 and left with a rating of 5, showing a very steep growth rate. Note that the child changed from one progression to another that was higher than the starting point. BUT he/she did not reach age level upon exit from the program. The gap was closed.

Slide 71- Developmental Trajectory D

The fourth developmental trajectory, categorized as D, depicts children who improved functioning, changed developmental progression to a level closer to age expectation, and DID reach age level functioning by the time they exited our program. They closed the gap and reached age level functioning

Slide 72- Developmental Trajectory D- picture of the trajectory

This is a picture of a child who fell into the developmental trajectory of category D. The child with this developmental trajectory entered services with a COSF rating of 5 in an outcome area. His exit rating of 6 or 7 indicates the child is now showing age expected functioning. He caught up.

Slide 73- Developmental Trajectory E

The fifth and last developmental trajectory, categorized as E, depicts a child who entered the program at age level functioning and left the program at age level functioning. Remember, there are three outcomes. You may have a child that is within normal limits in some areas of development, but are significantly below age level in other areas of development. We typically would not see a child who is has an IEP with the E trajectories in all three outcomes.

Slide 74- Developmental Trajectory E- picture of the trajectory

Here is an example of a child who has an outcome with a developmental trajectory categorized as an E. He entered services with a rating of 7 in a particular outcome area and exited services with a rating of 7. He maintained age appropriate functioning. Notice the positive, upward slope of the trajectory – which indicates that he made progress in this area. Remember that a child must make progress in order to maintain functioning at any level. This is because the child is older at exit than he was at entry, and we are comparing his functioning to age expectations in order to determine the rating. More skills are expected as a child gets older. So to maintain a level of 7 (or any level) the child would have to have added new skills. If a child looked the very same at exit as he did at entry, the rating would actually go down on the 7-point scale. In other words, a child who was age appropriate when he was 1 year old, but didn’t acquire any new skills by the time he exited, would actually look like a much younger child upon exit – and would receive a much lower rating.


This is another example of a developmental trajectory for a child who is reported in category ‘e.’ Even though he entered services with a lower rating than at exit, he was still in the realm of ‘overall age appropriate’ and therefore ‘maintained functioning at a level comparable to same-aged peers.’





Here is an example of one more developmental trajectory that corresponds with the ‘e’ category. Even though the child’s rating was lower at exit (went from 7 to 6), 6 is still in the realm of overall age appropriate and therefore it is considered that the child “maintained functioning at a level comparable to same-aged peers.” Notice that the trajectory has a slight upward, positive slope – indicating that the child did make some progress. If he had not made any progress at all between entry and exit, he would have looked like a much younger child – and his exit rating would have been lower.

Slide 76- Have you charted the developmental trajectories of children you are currently serving?

We now have an opportunity to chart the developmental trajectories of children we are serving and incorporate that into our progress monitoring process. This might be especially helpful to Speech-Language Pathologist and Itinerant Teachers who serve the same child for more than one year. It is also very important information to share with parents. They would love to see the progress their child made over time, and not just the progress on goals within the IEP.

Think about the parents of children who are really struggling. To see that their child is making a positive increase in their overall development over time could be very uplifting to them.

Now let’s practice and see if we can chart some developmental trajectories. Please take out your handout titled: “Converting COS data to OSEP Progress Categories.”

This is a classroom roster, or a caseload roster, whichever is most applicable to you at the current time. Please compare each child’s entry COS rating to the exit COS rating and determine which OSEP progress category is correct for each child.

[Trainer note: please give participants enough time to complete this assignment. Then debrief with them about the appropriate OSEP category using the “Key: Converting COS Data into OSEP Progress Categories.”

Slide 78- Discussion

[Trainer note: allow participants to form small groups to discuss the two questions on this slide. Post two pieces of chart paper around the room with the two questions. Depending on the size of your group you may want to debrief with the whole group. If you have a large group you may want to have one person at each group write down some of their ideas and post on the chart paper. Then the group may do a gallery walk to review the suggestions].

Slide 78- Chapter Section: How is this information being used?

Slide 79- For the state

We are now moving into a time when our child outcomes data is being looked at nationally and on the state level. This stakes are getting higher. We must improve the quality of our outcomes data so that the numbers we report accurately reflect the extent to which children make progress in our programs.

Slide 80- For each school system




Each school system should be looking at these data to consider how they are providing services and what impact services are having on child outcomes. Data analysis is a very strong tool making decisions about the need for new or improved practices and programs.

Slide 81- For each practitioner

It is now time for each practitioner to consider how these data can inform what you are doing with children. Do the strategies you use in your service really work? How do you know? Look at your children’s individual development across the three outcomes by plotting their developmental trajectories to see if they are making progress. Understanding the trajectories will help you see what kind of progress is being made.



modules.nceln.fpg.unc.edumodules.nceln.fpg.unc.edu/sites/modules.… · web view · 2014-10-30nc...

Documents