Opportunities for Postsecondary Success Project Center for Community Partnerships Department of Occupational Therapy Colorado State University

Opportunities for Postsecondary Success ProjectCenter for Community Partnerships Department of Occupational TherapyColorado State UniversityGoal Attainment ScalingAHEAD 2012 Conference7/11/12Cathy Schelly, M.Ed., OTR Assistant ProfessorDirector, Center for Community PartnershipsPatti Davies, PhD., OTRProfessorResearch DirectorOpportunities for Postsecondary Success (OPS)Implementation site: Center for Community Partnerships (CCP)

OPS Students:CSU & Front Range Community College students with Aspergers/ASD, TBI, IDTransitioning youth with ID from local school district (partnership with Project Search)Young adults with ID seeking community employmentEstablishing measurable goals with participants: Goal Attainment Scaling

Goal Attainment ScalingGoal attainment scaling (GAS): an individualized approach for measuring the achievement of goals (King et al., 1999).Originally developed to assess adults in a community mental health setting, but has since been applied to numerous practice areas, including education, health, and social work (MacKay, Somerville, & Lundie, 1996).

Original goal attainment scaling methodThe original GAS scale uses a 5-point scale, ranging from -2 to +2, with zero representing the expected level of performance after intervention.

LevelsKiresuk, Smith, & Cardillo (1994)- 2Much less than expected outcome- 1Somewhat less than expected outcome 0Projected level of performance+1Somewhat more than expected outcome+2Much more than expected outcomeAlternative methodsLevelsKing et al. (1999)- 2Baseline- 1Less than expected 0Expected level+1Greater than expected +2Much greater than expectedLevelsSteenbeek et al. (2007)+2A lot more+1A little more0As expected-1Partially achieved -2Same as baseline-3WorseLevels1No change2Small improvement3Interest in goal4Takes some action towards goal5Maintains new behavior6Committed to achieving goal7Achieves goalOPS Scale Numerical Level of Goal AttainmentDescriptive Level of Goal AttainmentI lost ground.0Where I am right now. (Baseline)1 Im getting close to my goal. 2 I reached my goal!3I did even better than expected!47OPS Goal Attainment Scaling GOAL: Implement effective study strategies.

Numerical Level of Goal AttainmentDescriptive Level of Goal AttainmentI lost ground.0Not willing or interested in discussing/considering effective study strategies.Where I am right now.1Willing to discuss/consider potential effective study strategies but not implementing any strategies. Im getting close to my goal. 2 Implementing effective study strategies with most difficult course, with cueing and reminders.I reached my goal!3Implementing effective study strategies with all courses, with cueing and reminders.I did even better than expected!4Independently implementing effective study strategies with all courses.8Applying GAS in practiceIdentify the overall objective.Identify the specific problem area(s) that the individual wants/needs to address.Identify the behaviors or events that will indicate improvement in each area.Determine the methodology to collect the desired information.Identify outcomes for scaling.Determine the individuals current status (baseline). Determine how progress will be documented.What is the overall goal?Setting your expected level of outcome or the desired goal:For example, you and the student may agree that developing self-advocacy skills is the overall program goal.Document the students current status (baseline).Establish incremental changes in behavior from baseline to depict improvements or losing ground as student pursues goal.What are the specific problem areas? Prioritize areas of concern and formulate SMART goals.Prioritized goals are given a weight, which is used to convert the scale score into a standard score.

11What are SMART goals?Specific= a specific goal has a much greater chance of being achieved. It is the who, what, where, when, which, and why of the goal.Measurable=establish concrete criteria for measuring progress.Attainable=help client set goals that are meaningful to them and that they want to achieve.Relevant=a goal must represent an objective toward which you are both willing and able to work.Time-specific=a goal should be grounded within a time-frame.

Tips for scaling goalsSelect the expected level of performance.Identify the least favorable outcome and the most favorable outcome.Identify the intermediate levels of performance.Set SMART goals.Develop scale with equally interval levels.Important considerationsWhere is my clients current level of performance (baseline)?When/how often should I follow-up to determine and document clients change in status?

What behaviors and events will indicate improvement?Create a scale that will be useful for measuring changes in performance.For the student working on self-advocacy skills, this may involve creating a scale based on the number of times the student attends his/her professors office hours.For student who is wanting to make friends in the residence hall (RH), this may involve creating a scale based on the number of RH activities he participates in.GAS Goal: Recognize and address issues that result in lack of persistence spring semester. Numerical Level of Goal AttainmentDescriptive Level of Goal AttainmentI lost ground.0Withdraws from all courses spring semester with no warning, no support.Where I am right now. (Baseline)1Planning on withdrawing from challenging courses spring semester seeking support but unwilling to discuss issues that are causing urge to withdraw. Im getting close to my goal. 2 Willing to discuss issues that cause the urge to withdraw from challenging courses.I reached my goal!3Recognizes and begins addressing issues that are causing urge to withdraw from challenging courses.I did even better than expected!4Proactively addresses issues that cause urge to withdraw from challenging courses and persists with all courses.16GAS Goal: Implement study strategies. Numerical Level of Goal AttainmentDescriptive Level of Goal AttainmentI lost ground.0Not willing to develop study strategies.Where I am right now. (Baseline)1Not participating in Academic Advancement Center (AAC) to learn study strategies and not using any study strategies, but willing to consider development of strategies. Im getting close to my goal. 2 Attending AAC study strategy sessions to learn about and develop study strategies.I reached my goal!3Fully implementing study strategies with OPS support during study sessions.I did even better than expected!4Independently implementing study strategies.17GAS Goal: Self-advocacy for learning accommodation needs. Numerical Level of Goal AttainmentDescriptive Level of Goal AttainmentI lost ground.0Not willing to speak with professor.Where I am right now. (Baseline)1States a willingness to speak with professor regarding accommodation needs, but afraid to follow through. Im getting close to my goal. 2 Made list of learning accommodation needs in preparation for meeting with with professor(s).I reached my goal!3With mentor assistance, set up appointment(s) with professor and kept appointment(s), advocating for learning accommodation needs and with mentor attending meeting(s).I did even better than expected!4Independently set up appointment(s) with professors and kept appointments, advocating for learning accommodation needs with behind the scenes encouragement from mentor.18GAS Goal: Completes new tasks when assigned jobs are finished. Numerical Level of Goal AttainmentDescriptive Level of Goal AttainmentI lost ground.0Leaves work area when assigned jobs are finished.Where I am right now. (Baseline)1Sits down and stops working when assigned jobs are finished, even when additional tasks are not completed. Im getting close to my goal. 2 Asks job coach what to do next when assigned jobs are finished and works on additional tasks as directed.I reached my goal!3Asks supervisor and/or natural support staff if there are additional tasks to be done and then works on these tasks as directed.I did even better than expected!4Recognizes additional tasks after assigned jobs are finished and independently completes these tasks.19Process we useDevelop relationship with student.Connect student with mentor.Gather information re: needs, wants of student, and areas where there are challenges (assessment process).Develop semester goal with student review with team to ensure that goal is SMART.Enter goal into GAS database.Begin goal attainment activities, documenting incremental changes.At end of semester, regroup and establish new goal(s) for next semester.CCP Assessment ProcessFunctional, community-referenced processIdentify strengths, challenges, barriers in all domainsIdentify support needsFormulate student/participant driven goal(s)

Challenges/Support NeedsEvaluation

Observed Indicator (actual behavior, action or circumstance)

Fatigue (physical and cognitive)Student describes inability to stay focused and awake in afternoon classes. Student complains that fatigue is unrelenting, even after 8 hours of sleep each night. Student states that mornings are best for being alert and focused.Confusion (short-term memory challenges)Student spends unusually long periods of time studying for exams and completing assignments, but complains of inability to remember content even after reading numerous times.Assessment process is ongoingAssessment is the critical starting point that paves the way to successful planning, educational mentoring, and provision of necessary supports that facilitate success.Functional assessment continues after the student is working towards goals to continually guide planning, solve problems, improve academic performance and, ultimately, persistence, retention and graduation.Same guidelines apply for students pursuing employment/careers.

Canadian Occupational Performance Measure (COPM)Under consideration, to complement GAS.Assesses individuals perceived occupational performance in life areas.Process:Conduct semi-structured interview to identify wants/needs of client.Based on wants/needs, create goals.Supplement interview information through direct observation, environmental assessment, or standardized tests.Benefits of COPMMeasures clients perceived progress in goal attainment between pre-COPM, intervention and post-COPM (designed as outcome measure).Easy to administer ~20 minutes.Demonstrated reliability and validity.Similar to GAS, useful with a wide variety of clients in many different settings.Can weight and prioritize goals (again, consistent with GAS).

Audience DiscussionWhat type of assessment process(es) are you using?What are the advantages of your assessment process?What are the disadvantages, if any, of your assessment process?What population do you serve?Practice session on writing goalsReview GAS worksheets.Develop goals as per worksheet, OR, for student you are working with.Connect goals to domain areas.Develop SMART goals.DiscussionActual goals that we established with our students/participantsHow will data be collected?What process will your group implement to collect and analyze data?

How will information be collected and who will collect it?Role of the Transition CoordinatorRole of the MenteeRole of the MentorOur GAS ResearchExciting results from pilot study fall 2011 and spring 2012Noted progress in following areas:AcademicsSelf-advocacySocializationPersistenceRetentionGraduationEmploymentPreliminary DataParticipants - total 2312 students at CSU5 students at FRCC6 students enrolled in Project Search

Number of goals written total 4315 in Fall semester 2011 28 in Spring semester 2012

Students enrolled in Project Search6 students

Goals written for a 3 month internship

8 goals written 1.3 goals on average per student

83% of the students made steady improvement on their goals during their internships

16% of the students displayed a decline in their GAS score at the end of the internship

100% of students were hired after completing their internships

Students in College (FRCC) and University (CSU)17 students

Goals written for a semester

35 goals written 1.8 goals on average per student

47% of the students made steady improvement on their goals across the semester

24% of the students displayed a decline in their GAS score at the end of the semester

Changes in Goal Attainment Scores Before and After InterventionT ScoreMeanStandard DeviationPre Intervention38.122.23Post Intervention58.907.92

Changes in Goal Attainment Scores Before and After InterventionANOVAStatistic: F(1,20) = 136.17, p < .0005

Effect Size:Partial = .87Converting Goal Attainment Scales Scores to T ScoresSum the GAS score for the goal(s)If a participant only has one goal the sum will be the score for that one goal.If a participant has more than one goal the score for each goal will be added together.Convert the summed score to a T ScoreFormula in Kiresuk & Sherman (1968)More precisionTables in Kiresuk, Smith, & Cardillo (1994)Less precision

Advantages of Using T ScoresHelps to equate scores across participants when they have different number of goals.

Provides variability to pre-intervention scores which is necessary for conducting statistical analyses.

Goals can be Weighted To reflectRelative value of each goal based on intervention outcomesRelative importance of each goal based on participants perspectivesRelative to the difficulty of the goalWeights do not need to sum to a particular fixed totalExamples1, 2, 3 or 2, 4, 6 or 10, 20, 30Equal weighting of goals lose little information

Goal Attainment Scaling as an Intervention ToolWeighting and selecting goals based on clients desires

Setting goals in collaboration with client

Using improvement on GAS scores as motivation for the client to work hard

Can be used in combination with the COPM Determine goal areasWeight goals

Goal Attainment Scaling as an Research ToolThe evaluation and establishment of goals may be conducted by someone different than the person providing the intervention.

Follow-up assessment on goals may be conducted by someone other than the person providing the intervention.

Consider comparing intervention programs or using a control group.

Consider using other evaluation tools in addition to GAS.

Goal: University GraduationSuccess!

Goal: College GraduationSuccess!

Advantages of using GAS with the OPS projectProvides a quantitative measure of outcome.Can be used to compare a students progress over time.Can be used to compare performances across clients in the same program but who have different skills, challenges (Ottenbacher & Cusick, 1989).Allows for a collaborative approach-working with the student to set realistic goals.Provides flexibility for measuring diverse outcomes (Brown, 2009).

Limitations of GASScaling can be a time-consuming process.Biases can occur in goal setting, scaling, and rating.Temptation to modify goals throughout the course of intervention.NOTE: Good goals are really hard to write!!

ReferencesKing, G.A., McDougal, J., Palisano, R.J., Gritzan, J., & Tucker, M.A. (1999). Goal attainment scaling: its use in evaluating pediatric therapy programs. Physical and Occupational Therapy in Pediatrics, p. 31-52.Kiresuk, T. & Sherman, R. (1968). Goal attainment scaling: A general method of evaluating comprehensive mental health programs. Community Mental Health Journal, 4 (6), 443-453. Kiresuk, T.J., Smith, A., & Cardillo, J.E. (1994). Goal Attainment Scaling: Application, Theory, & Measurement. Hilldale, N.J.: Lawrence Erlbaum Associates. Mailloux, Z., May-Benson, T.A., Summers, C.A., Miller, L.J., Brett-Green, B., Burke, J.P., et al. (2007). The Issue Is-Goal Attainment Scaling as a measure of meaningful outcomes for children with sensory integration disorders. American Journal of Occupational Therapy, 61, 254-259. McKay, G., Somerville, W., & Lundie, J. (1996). Reflections on goal attainment scaling (GAS): cautionary notes and proposals for development. Education Research, 38,2, 161-172.Ottenbacher, K.J. & Cusick, A. (1990) Goal attainment scaling as a method of clinical service evaluation. American Journal of Occupational Therapy, 44, 6, 519-525.Steenbeek, D., Ketelaar, M., Galama, K., & Gorter, J.W. (2007). Goal attainment scaling in pediatric rehabilitation: a critical review of the literature. Dev Med Child Neurol, 49, 550-556.Turner-Stokes, L. & Williams, H. (2010). Goal attainment scaling: a direct comparison of alternative rating methods. Journal of Clinical Rehabilitation, 24, 66-73.