2012 qa report
DESCRIPTION
2012 QA reportTRANSCRIPT
Quality
Assurance
Report 2011-2012
Quality Assurance Overview 3
Service Output Statistics 4
File Audits 5
Serious Occurrences and Client Complaints 6
Service Wait Times Community Clinic 7
Referral Source Questionnaire 9
Client Feedback: Community Clinic Consultation Interview 11
Community Clinic Client Satisfaction Blitz 13
Brief Child and Family Phone Interview (BCFPI) 15
Child and Adolescent Functional Assessment Scale (CAFAS) 17
Client Feedback: Community Clinic Groups 18
Client Feedback: Prevention and Early Intervention Groups 20
Client Feedback: Ontario Early Years 21
Client Feedback: Special Needs Resourcing 23
Day Treatment Programs 24
Residential Programs 27
Highfield Community Enrichment Project 29
Family Group Conferencing Project of Toronto 31
Family Group Conferencing Ontario Provincial Resource 33
Toronto Preschool Speech and Language 35
Teaching and Training 37
TABLE OF CONTENTS
Background
The Quality Assurance (QA) process provides a means of reviewing and assessing service delivery and
administrative functions. Quality Assurance is a continuous process involving a cycle of evaluation and
improvement. It is a multi-faceted process emphasizing client-focused elements such as service output,
service access, client satisfaction, client outcomes and resource utilization as well as administrative
functions. This process provides the climate for policy change and service improvement.
Quality Assurance Structure
All Centre Quality Assurance activities are overseen by a Quality Assurance Committee (formed in September 2009).
The Quality Assurance Committee is chaired by the Director of Research. The Committee contains a minimum of four additional members and must include at least one person who has knowledge of the following areas of the Centre: Residential, Day Treatment, Community Clinic, and Central Administration.
The Quality Assurance Committee reports to the Directors’ Group and meets at minimum four times per year. During fiscal 2011-2012, the QA Committee met five times although two of those meetings contained only Clinical Records Subcommittee topics.
Reporting of Quality Assurance Activities
Many Quality Assurance reports are generated monthly (waitlist summaries, intake stats), or quarterly
(total clients served statistics).
A Quality Assurance Report is compiled annually which summarizes the Quality Assurance activities
for the fiscal period. This report is presented to the Management Group, the Directors’ Group and the
Centre’s Board of Directors. It is then distributed to all staff.
In order to facilitate ongoing communication regarding QA activities and the dissemination of QA
findings, QA is now a standing agenda item for all Team Meetings. Minutes of the QA Committee are
circulated to all Managers and Directors. Members of the QA Committee provide critical links to all
staff teams. Program Evaluation and Client Feedback is regularly provided to Teams through reports
and presentations.
Fiscal 2011-2012 Activities
Revised Centre file audit procedures and templates
Revised CYSIS reports to provide more detailed information to Clinic staff and managers
Executed the Community Clinic Client Satisfaction Blitz
Revised intake and clinical records processes and began drafting manuals
Created administrative procedures for new Centre programs and Clinic Group Treatment Programs
Quality Assurance Overview
Page 3
Service Output Statistics 2010-2011
Page 4
Service Output Statistics 2011-2012
Community Clinic Total Served
Total Unduplicated Clinical Clients 808
Total Workshop and Group Participants 890
Family Group Conferencing Total Served
Total Referrals 67
Total Clients Served 112
Toronto Preschool Speech and
Language (GHC Site Only)
Total Referrals Total Client
Contacts
Total Unique
Clients
GHC Site Only 205 2,470 416
Highfield Community Enrichment Project Total Served
Hours of direct service 11,181
Number of children served 1,736
Number of families served 2,020
OEYC Total Served
Number of parent/caregiver visits 9,400
Number of children served 1,150
Number of child visits 10,175
Number of parents/caregivers served 1,300
Special Needs Resourcing (Calendar Year) Total Served
Special Needs Resource Consultant 49
Intensive Resource Support 18
Number of professional/parent workshops 4
Total number of workshop participants 18
Direct Program Consultations 5
Residential Programs Total Served
Libby’s Place 19
Boys House 12
Day Treatment Programs Total Served
School Program 47
Clear Directions 11
Programs that retain clinical client files must have a regular audit process. The Community Clinic, Family
Group Conferencing, Special Needs Resourcing and Toronto Preschool Speech and Language undergo an
audit three times per year. Human Resource files are audited twice yearly. Residential files undergo an
external audit by MCYS as part of their annual licensing review.
Human Resource
The HR department was restructured in July 2011 following the resignation of the HR Specialist. The
administrative responsibilities of the HR Specialist position were combined with the responsibilities of the
Payroll position to create a new position HR/Payroll Administrator. No Human Resource File Audits were
conducted during the 2011-2012 fiscal year. It was determined an HR software program would be beneficial
to the centre, both for file audit purposes and to assist with streamlining the HR/Payroll workload
duplications. The implementation of the new software will begin in Sept 2012 and include an electronic
audit process. As a result of these changes, new Human Resource File Audit procedures will be required.
Community Clinic
The Community Clinic audit involves both closed and open files. The audit includes files that receive
Outpatient services as well as those that terminate at the stage of Consultation Interview and Short Term
Intervention. Community Clinic audits were performed in May 2011, October 2011 and February 2012.
The Community Clinic audit is a peer audit involving all clinicians. In May 2011, 35 files were audited and
14 files had deficiencies. In October 2011, 31 files were reviewed and 10 files had deficiencies. In February
2012, a total of 40 files were reviewed and 15 files had deficiencies. Following each audit, staff were shown
results and deficiencies were corrected where possible. Audit results were also discussed during team
meetings and addressed in supervision where required.
Special Needs Resourcing
Special Needs Resourcing audits involve closed files only and due to the small caseload in this program, all
closed files are audited. Audits took place in May 2011, October 2011 and February 2012. In May 2011,
one file was audited and no deficiencies were found. The October 2011 audit involved 9 files with 4
deficient files (8 deficiencies in total). In February 2012, 5 files were audited and 3 files were deficient (7
deficiencies in total). These deficiencies were addressed with staff and in one case the staff had left the
Centre.
Speech and Language
The Speech and Language audit involves 10 files (active and/or closed) randomly chosen by the program
manager. Speech and Language files were audited in May 2011, October 2011 and February 2012. In
total, 30 files were audited and 1 file had a deficiency that was corrected. The remaining 29 files were found
to be in complete compliance with the standards outlined in the audit forms.
Family Group Conferencing
The FGC audit involves closed files only. Two random files are selected for each coordinator from all the
closed files for the period. FGC audits took place in May 2011, October 2011 and February 2012. In May
2011, a total of 12 files were audited. Two files were deficient (total individual deficiencies two)). In
October 2011, 13 files were audited and deficiencies were noted in 1 file (11 deficiencies total). In February
2012, 8 files were audited and 3 files were deficient (3 total deficiencies). The FGC Team reviewed the
audit results and deficiencies were addressed with staff. In one case, deficiencies were due to a staff health
condition.
File Audits
Page 5
Resolution of Client Complaints
The George Hull Centre has a formal policy for addressing complaints. All complaints are taken seriously and considered a high priority at all levels of the organization. The process of resolving complaints involves full written documentation and follow-up procedures. The goal is to resolve the issue in the spirit of cooperation with the complainant.
Due to the confidential nature of the issues, complaints are not reviewed by the Quality Assurance Committee. All complaint files are reviewed by the Executive Director and the Director of Research and Program Evaluation annually. During 2010-2011, a total of 7 formal complaints were received. In each case, complaints received documentation and thorough follow-up in accordance with the Centre’s policies.
Serious Occurrence Reporting
Serious Occurrences refers to a formal classification of incidents or events which must be reported to the Ministry of Children & Youth Services. Service providers who deliver any direct service to children and youth under the Child and Family Services Act (CFSA), and children under the Day Nurseries Act (DNA) are required to report all serious occurrences to the Ministry within 24 hours. In 2011, there were 38 serious occurrences. Three serious occurrences were reported for the Day Treatment program and the remaining occurrences were for Libby’s Place (25) and the Boy’s House (10). The high number of serious occurrences in part reflects Libby’s Place participation in the Quicker Access program (2 beds). The Quicker Access beds are available to child welfare programs in Toronto so that they have ready and speedy access to children’s mental health beds. These residents tend to have limited family involvement and the staff are not able to pre-screen these clients for appropriate fit or willingness to participate in a treatment program. The mandated nature of many of these admissions has resulted in extreme acting out behavior including running away, self harm and damaging property. In the case of all serious occurrences, an individualized plan was implemented to address the clinical needs of the clients involved. A total of 79% of reports were submitted within the 24 hour reporting period. Late reporting for the remaining occurrences was due to email issues and reporting delays during the Christmas break. The program staff, particularly new staff, have been reminded of the need to make submissions on time.
Serious Occurrences and Client Complaints
Page 6
Background
The Centre is committed to providing the best
possible response time to client requests for service.
Due to limited resources, clients often experience a
wait time for services.
Monitoring wait list times and designing wait list
management strategies are important quality
assurance priorities for the agency.
Reporting on wait times for services is a complex
task because clients may wait at multiple points as
they move through stages of treatment. Wait times
do not only reflect limited service availability and
capacity. Clients may be offered services at an
earlier time but not be able to accommodate these
arrangements into their schedules.
In 2011-2012, Intake Services completed a total of
490 intakes.
Intake Department Response to Call Times
The Centre monitors the response time to intake
calls. For each fiscal year, a total of 4 months are
randomly selected for response time calculations.
In fiscal 2011-2012 only three months were
completed. The fourth month was not done due
to a change in intake worker. Compiled results
are presented in Figure 1.
The percentages in Figure 1 are based on 657
calls. A total of 78% of calls were returned the
same day and 91% of calls were returned within
24 hours (this latter percentage equals that of
2010-2011).
Wait Time for Intake
Every effort is made to provide an intake as
quickly as possible. Figure 2 displays the
aggregate wait times between the initial client
phone call and the completed intake (excludes
Shared Care). Wait-times in 2011-2012 are very
similar to 2010-2011.
Service Wait Times Community Clinic
Page 7
Figure 2 - Client Wait Times for a
Completed Intake
0%
10%
20%
30%
40%
50%
60%
70%
Same day 19% 9%
1-7 days 58% 66%
8-14 days 16% 14%
15-21 days 5% 5%
22-30 days 2% 3%
Over 30 days 1% 3%
2010-2011 2011-2012
Figure 1 - Intake Response Time for
Calls
0%
20%
40%
60%
80%
100%
Same Day 78%
1 13%
2 2%
3 2%
4 1%
Over 4 days 4%
2011-2012
Wait Time for First Service
Figure 3 presents wait-time for first service for Community Clinic clients who received an intake between April 1, 2011 and March 31, 2012 (excluding clients who had residential and day treatment as first service). Of the 458 cases, 66 clients (14%) withdrew with no service and 6 clients are still waiting for services. The remaining 386 cases received services at the Clinic. Wait times are presented in Figure 3. Almost a third were seen within 2 weeks and 67% were served within 30 days (higher than 2010-2011 when this figure was 51%). The higher figure is likely due to new programs in which waitlists do not accumulate internally (Supervised Alternative Learning and School Focused). Waitlist Management
The Centre remains committed to providing timely service to all families. The Centre employs an “active waitlist management strategy” which involves the following activities:
Use of data management systems to support intake and clinical staff to track the presenting needs of clients so that they may be informed of opportunities for treatment
Employing a triage system to assign priority to cases based on high clinical needs after consultation with the multidisciplinary team
Regular reports to assess response times and determine the time it takes us to complete intake assessment and to provide families with service
Weekly Intake Committee meetings where files are reviewed by a team to ensure that the most optimal service is offered to each family.
Distribution of waitlist statistics and snapshot reports to management and senior management levels
Finally, the Intake Service Coordinator and the Manager of the Community Clinic have had numerous opportunities to disseminate information to community partners about the range of services being offered at the Centre and how they can best support families to access these services. Community partners welcomed the access to our Short Term Intervention Program, psychiatric consultations, our Family Wellness Series of groups, as well as trauma assessment and treatment programs.
Service Wait Times Community Clinic
Page 8
Figure 3 - Wait Times For First Service -
Community Clinic
0%
5%
10%
15%
20%
25%
30%
35%
40%
0-14 days 23% 31%
15-30 days 28% 36%
31-60 days 36% 21%
61-90 days 9% 8%
Over 90 days 3% 4%
2010-2011 2011-2012
Note: Figure percentages do not equal 100% due to rounding
Background
As part of our accreditation requirement, the Centre completed a Referral Source Survey in the Spring of
2011. Centre Teams were asked to produce a list of referral sources for their programs, this list was merged
with Referral Source data for an 18 month period extracted from the central data system (CYSIS). A total
of 279 questionnaires were mailed and 58 responses were received (response rate 22%). Below is a
breakdown of respondents by type. The largest referral source category was doctors.
Between 61% and 69% of respondents agreed that they understood the referral process, that the intake
department was responsive to their requests and that the referral process was easy. About 20% of
respondents did not reply to the questions on the intake and referral process (see Figure 4). Those that
George Hull Centre: Referral
Source Questionnaire
Page 9
Organization Total
Received
Percent
Child Welfare 9 15.5%
School/School Board 4 6.9%
Hospital, private practice, walk in clinic,
health centre / organization 36 62.1%
Child Care Agency 3 5.2%
Children's Mental Health Agency 3 5.2%
Community Organization 1 1.7%
Multiple entry 2 3.4%
Total 58 100.0%
0%
10%
20%
30%
40%
50%
60%
Figure 4 - Experience with the referral process
Strongly agree 12% 10% 9%
Agree 52% 59% 52%
Disagree 12% 7% 16%
Strongly disagree 3% 2% 5%
No answ er 21% 22% 19%
I understand the
process for making
the referral
Intake responsive to
my referrals
I found the referral
process easy
95% of respondents said they would refer to the GHC in the future (5% no response).
Referral sources noted many strengths of the Centre including: comprehensive range of services, flexibility,
strong client focus, professionalism, and dedication to child and family needs. Referral sources would like
to see development assessments, more psychiatry, walk-in clinic, services delivered in schools settings, more
services for teens and for those with special needs.
About 10% of referral sources mentioned the need for shorter wait-times. A considerable amount of the
feedback was related to improving communication e.g. stay in contact with practitioners, put things on web
page, more outreach and dissemination of information. Some referral sources lacked an awareness of some
Centre services. Management staff has discussed how to incorporate these changes. The Centre’s new
website design will fill many of these gaps.
George Hull Centre: Referral
Source Questionnaire
Page 10
0%
10%
20%
30%
40%
50%
60%
Figure 5 - How the GHC is doing
Strongly agree 10% 9% 12%
Agree 21% 28% 55%
Disagree 53% 43% 22%
Strongly disagree 10% 16% 7%
No answer 5% 5% 3%
The GHC keeps me
informed of relevant
program developments
I know where to go or how
to be updated on the GHC
programs
I know which type of
clients to refer to the GHC
Background
Clients who are assigned to the Community Clinic for services receive a Consultation Interview
appointment to assess their needs. At the end of this interview, each client is asked to fill out a
questionnaire designed to rate how easy it was for clients to connect with services, get information, and
have their needs assessed.
Response Rates
In 2011-2012, a total of 209 consultation interviews were conducted by the Community Clinic and a total
of 138 completed questionnaires were received. This represents a response rate of 66%. Not all clients
respond to all questions. In this fiscal year many clients left forms unfinished. There was a high rate of
missing responses (13-20%) particularly the for questions on “process” and “progress”.
Ease of Connecting to Services
84% reported that it was “easy” for them to find out about services and how to connect with them. 11% said the process was “not too easy” and only 1% of clients found the process “very difficult” (4% missing).
84% reported that they did not have to keep calling different people to get the information and support they needed and 86% said they did not have to repeat the story of their situation too many times before getting help.
Involvement in the
Process
The majority of clients
said their choices and
preferences were
heard and understood
(79% “yes”, 8%
“somewhat”). One
respondent disagreed
and the rest did not
provide a response.
76% of clients felt
they were well informed about what was going to happen next and 11% reported that they were
“somewhat” informed about what was going to happen next (13% missing).
Client Feedback:
Community Clinic Consultation Interview
Page 11
Yes Somewhat No Missing
2011-2012 67% 17% 2% 14%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Figure 6 - We got advice and recommendations that were useful for us
Support While Waiting
When asked if they were getting the support they needed while waiting for services to be available,
two thirds (57%) of respondents said “yes”, 20% said “somewhat” and 8% said “no” (17% missing).
Perceptions of Progress
83% of clients felt that progress was being made to get their family what they need. Only 2 clients
said “no” (16% missing).
In general, the Consultation Interview Questionnaire shows that, at this initial point of service, clients are
fairly satisfied with the wait times for consultation and ease of connecting to services. The qualitative
statements show that clients felt the staff were caring, helpful professionals who were very easy to talk to.
It was clear that clients felt very good about the initial experience, but several were very concerned with
the long wait time after consultation.
The missing responses and some of the qualitative feedback suggests that the Community Clinic and the
QA Committee should review the Consultation Interview form to determine if all of the questions are
appropriate. Several clients felt it was not possible to report on the questions given that they were just
beginning service.
Client Feedback:
Community Clinic Consultation Interview
Page 12
“It is the beginning of our involvement with the Centre, but I feel hopeful for the
first time in a long time. I know they can help us and I am grateful.”
0%
10%
20%
30%
40%
50%
60%
Figure 7 - In an overall general sense, how pleased are you with
the service you received
Very satisfied 51%
Satisfied 28%
Dissatisfied 1%
Very dissatisfied 1%
Missing 20%
Background The Community Clinic has monitored Client Satisfaction for many years. This has primarily taken the form of a paper/pencil mail survey to closed clients. Historically, response rates for this survey rarely exceeded 20%. Between 2005 and 2009, response rates for this survey began to decline (2009 response rate 14%). Changes in mail survey processes and on-line response options failed to increase the response rates. In early 2011, the Community Clinic and the Research Department discussed options to obtain better feedback. The decision was made to do a Client Satisfaction Questionnaire (CSQ) Blitz during the month of November.
Scope The primary target for the CSQ Blitz was clients receiving long term Outpatient services. Clients who were receiving Outpatient services, Day Treatment services or Psychiatric services during this time period were included in the sample. Clients at consultation and receiving Short Term Intervention and group services were not included as these programs are evaluated in other ways. Intensive Child and Family Services (ICFS) clients were excluded because a quadrant-wide evaluation project was beginning in January 2012.
Results
87 CSQs were returned. This represents 68% of clients who had appointments during the two week period and 35% of the total active clients in service.
All respondents indicated that they were comfortable receiving services in English and no respondents indicated that that needed language support.
Overall, respondents felt that the location of the Centre’s services was convenient and accessible (75%
“yes, definitely”, 20% “yes, somewhat”). Only 5% replied “no, not really”.
Respondents indicated that services were “definitely” (71.3%) or “somewhat” (26.4%) available at times that were convenient for them Less than 3% said “no, not really”.
89% of respondents reported that they “definitely” felt welcome and comfortable and 10% said they “somewhat” felt welcome and comfortable.
A large percentage of
respondents felt that the Centre was helpful in recommending other services (47% “very helpful” and 25% “somewhat helpful”). A total of 20% felt it was too soon to respond and only 5% said we were “not really that helpful” in this respect.
Page 13
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Figure 8 - We were served by knowledgeable and
competent professionals
Yes, definitely 87.4%
Yes, somewhat 11.5%
No, not really 0.0%
No, definitely not 1.1%
Knowledgeable and Competent
Professionals
Community Clinic CSQ Blitz
The majority of respondents reported that “yes, definitely” (79%) or “yes, somewhat” (20%) that the staff recognized the strengths of their family and child.
Overall, a high number of
clients reported that they would come back to our Centre (71% “definitely” and 23% “I think so”). Two clients (2%) reported “no, I don’t think so”.
Qualitative feedback was
overwhelmingly positive,
particularly with respect to
the knowledge and
professionalism of the staff and the useful solutions. A large number of clients felt the staff were caring,
and provided a safe, confidential and non-judgmental environment. Eight clients objected to long wait
times. Other comments included additional services (3) - (e.g. occupational therapist, more psychology),
concerns with progress (3), better appointment times (2), better building and facilities (2), an office in
Toronto (1). One
client mentioned initial
intimidation and
another client was
uncomfortable with the
two-way mirror.
There was a high level
of staff engagement
with this method and a
much larger response
rate was obtained using
this method rather than
the mail survey.
Future CSQ
evaluations will expand
the percentage of the
active client population
and include clients who
have recently exited from service. In January 2012, the George Hull Centre joined a MCYS funded
project to explore best practices for obtaining client satisfaction data. The Community Clinic and the
Research Department will incorporate information from this working group into future evaluation efforts.
Community Clinic Client Satisfaction Blitz
Page 14
Felt involved in
making plans and
goals for treatment
We felt comfortable
asking questions
about the treat-
ment and/or
medication
Our cultural values
and experiences
were considered by
the GHC staff
Yes, definitely 73.6% 79.3% 83.9%
Yes, somewhat 20.7% 13.8% 6.9%
No, not really 2.3% 1.1% 0.0%
No, definitely 0.0% 0.0% 0.0%
No response 3.4% 5.7% 9.2%
0%
20%
40%
60%
80%
Increasing Hope
for Future
Suggesting
Useful Solutions
Understanding
Problems
Helping Your
Child
Figure 9 - How helpful have we been in...
Background
The Brief Child and Family Phone Interview (BCFPI) is a standardized intake interview being used in
children’s mental health centres across Ontario for youth between the ages of 6 and 18 years of age.
Administering this tool involves asking caregivers questions about their children’s behavioural and
emotional adjustment, and overall child and family functioning. The BCFPI is not used with children
under the age of six.
Completion Rates
The Centre began reporting monthly statistics on BCFPI completion rates. In the 2011-2012 fiscal year
there were a total of 435 clients eligible for BCFPI and the completion rate was 87%.
Reasons for non-completion are listed below. These categories were further refined in CYSIS based on data for the fiscal year.
BCFPI Profiles 2011-2012
The BCFPI reports produce a problem prevalence
percentage for a variety of categories. These values reflect
the percentage of clients with t-scores of 70 or above. The t-
scores are standardized to population norms where 98% of
the population would score below 70. A score of 70 or above
puts these children within the top 2% of the
population in terms of difficulties.
Figure 10 (next page) presents the
prevalence percentages.
Almost 60% of clients were in the
clinical range for Global Functioning
and half were in the clinical range for
Social Participation.
Roughly 40% of clients scored in the
clinical range for Managing Moods.
The lowest percentages were found in
the areas of Conduct (14%) and Separation (22%).
Brief Child and Family Phone Interview (BCFPI)
Page 15
BCFPI Completion Rate
Total # eligible 435
Total # completed 380
Percent complete 87%
Reason for Non Completion Total
BCFPI completed < 1 year 6
Child with provider < 6 mos 3
Parent/Informant Refused 9
Language Barrier 15
Client withdrew request for services prior to
completing BCFPI 10
Other (e.g. Clinical decision, completed at
another agency, admin error ) 12
Page 16
Atten
tio
n/
Imp
uls
ivity
C
oo
pera
tive
Co
nduct
Exte
rnaliz
Sep
ara
tio
nM
an
ag
ing
An
xie
tyM
an
ag
ing
Mo
od
sIn
tern
aliz
Ttl 6
MH
D
om
ain
sS
ocia
l P
art
icQ
uality
of
Rela
tio
ns
Sch
ool
Part
ic.
Glo
bal
Fun
ctio
nin
g
2009-2
010
34%
40%
21%
40%
19%
24%
38%
33%
41%
51%
34%
39%
50%
2011-2
012
32%
36%
14%
34%
22%
28%
39%
36%
39%
51%
25%
35%
59%
0%
10%
20%
30%
40%
50%
60%
70%
CAFAS Background
The Child and Adolescent Functional Assessment Scale (CAFAS) is a clinician-rated scale that is being
used in children’s mental health centres across Ontario for youth 6 through 17 years of age. The scale
assesses 8 areas of child functioning (role performance in school, home, and community, behaviour
towards others, substance use, mood, self-harm, and thinking), and 2 areas of caregiver functioning
(material needs and social support). Behaviour in these areas is rated as severe, moderate, mild, or
showing no impairment. Lower scores on the CAFAS indicate higher levels of functioning.
CAFAS for 2011-2012
Treatment Outcomes
In 2011-2012 a total of 86 Outpatient clients received an exit CAFAS evaluation. Full pre-post treatment
scores were available for all of these clients. Decreases in CAFAS scores represent improved functioning.
For the 2011-2012 data, three clients withdrew from treatment, 20 clients were classified as “treatment
interrupted” and 63 clients were classified as “treatment accomplished”. Comparisons of CAFAS change
are presented in Figure 9.
79% of clients in the treatment accomplished category had lower scores at exit. This figure is 55% for
clients who treatment was interrupted.
62% of the cases where
treatment was
accomplished show a
CAFAS improvement of
20 pts or more.
The percentage of clients in
the treatment interrupted
category who had higher
CAFAS scores at exit is
three times greater than
cases in the treatment
accomplished category
(30% versus 10%).
Child and Adolescent Functional
Assessment Scale (CAFAS)
Page 17
Treatment Interrupted Treatment Accomplished
Higher Score at Exit 30% 10%
No Change 15% 11%
Lower Score at Exit 55% 79%
Lower Score at Exit by
20pts or more30% 62%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Figure 9 - CAFAS Outcomes by Closed Reason
Help I have Teens!!
Help I have Teens!! is a group designed for parents to learn how to improve their relationships with their
teenage children. The group teaches parents how to set reasonable limits, how to communicate effectively,
and how to prepare their teen for independence. The group was held for six sessions between January 31,
2012 and March 6, 2012. There were a total of 11 participants and 7 completed feedback questionnaires
(response rate 64%).
71% of the respondents rated the overall quality of the group as “excellent” (29% rated it as “good”).
The majority of respondents said that since being in the group, the way they view their teen’s behavior had
changed (86% “a lot”).
Participants were asked what they liked best about the group. Four respondents liked learning that they were
not alone and one parent felt relief from sharing. Respondents also liked the role plays, homework, learning
about the developmental stage, and how they need to change. Respondents requested handouts, more role
plays, and scenarios about out-of-control behaviour. As well, respondents suggested a follow-up group to
discuss results and an interactive email site for discussions.
Turning Down the Heat
“Turning Down the Heat” Art Therapy Group works with children between 7-12 years old who are having
difficulty coping with their anger. The group was held three times during the fiscal year and group feedback
was collected for all sessions. In addition to client satisfaction, an MSW student worked with the group leaders
to revise the logic model, compile program materials and create a binder of standardized measures that might
be useful in program evaluation for the upcoming year. A total of 18 participants completed a Client
Satisfaction Questionnaire, representing a response rate of 82%.
All child respondents said the group leaders knew a lot about anger and how to help them (78% of participants
said “yes, definitely”). When asked if they were happy with what they learned in the group, 72% of
respondents answered “yes, definitely” and 22% answered “yes, somewhat”.
Many child respondents (67%) found the concept of “when anger comes to visit” helpful. One participant said
it was helpful because it helped them to talk about their problems instead of yelling them and one participant
said the ‘recorder trick’ was helpful in avoiding fights.
83% of respondents said they liked learning about the tricks to deal with anger.
89% of respondents said they could use the tricks they learned in the group when anger comes to visit in
the future.
All parents felt the group helped them to understand anger triggers and gave them useful strategies to help
manage their own anger. The majority of respondents (64%) said since being in the group the way they viewed
their child’s behavior has changed “a lot” and 36% said it had changed a “a little”
79% said the group was helpful in improving their child’s behavior and 58% said the group had helped
improve the behavior of other children in the family.
The majority of suggestions for program improvement were related to increased program time and more one-
on-one support for families.
Page 18
Client Feedback: Community Clinic Groups
Cognitive Behaviour Therapy Anxiety Group
Cognitive Behaviour Therapy (CBT) is a brief evidence-based intervention for treating anxiety in children. The Centre runs a CBT group for children experiencing issues with anxious/nervous feelings, thoughts or physical sensations. This group is divided into two sections – one group for the parents and one group for the children. Both parent and child must attend the group. Evaluation of the intervention includes a parent and child feedback questionnaire. Results of standardized measures of anxiety administered to group participants are not presented in the report due to difficulty in obtaining complete and reliable data. The group had two sessions ending December 2011 and May 2012. A total of 19 children and 19 parents participated in these groups and 15 children (79% response rate) and 18 parents (95% response rate) completed feedback questionnaires. Children rated the group as friendly and welcoming (67% “yes definitely” and 33% “yes somewhat”).
93% of child respondents agreed that they learned how to calm themselves when anxious and also how
to reward themselves when they have coped with an anxious situation.
73% of child respondents said that as a result of the group they were able to cope with their anxiety a
“little bit” better and 27% said they were able to cope with their anxiety “a lot” better.
All children agreed that being in a group made them feel supported. They all liked the use of art in the
group and the majority found the art helped them to understand what they were learning, release their
stress and help them talk about their feelings. Several children said they learned that they were not alone
with anxiety. Suggestions for improvement were more art (5), longer group (2) and more parent-child
activities (1). Parent feedback is provided in Figure 10.
Page 19
Client Feedback: Community Clinic Groups
0%
10%
20%
30%
40%
50%
60%
70%
80%
Figure 10 - How Helpful was the CBT Parent's Group in:
Very helpful 72% 61% 39%
Helpful 28% 28% 56%
Somewhat helpful 0% 11% 6%
Not helpful 0% 0% 0%
Teaching strategies for
encouraging your child to face
their fears & worries
Prov iding relaxation
strategies to your child to use
in anxious situations
Help family dev elop parenting
approach for dealing with
your child's anxiety
Kindergarten Readiness
The Kindergarten Readiness Program is designed to meet the needs of families who feel their children
(between the ages of 3.5 to 5 years) require extra supports before entering the school system. The group
consists of up to 10 children who may be
presenting with developmental delays and/or
social or emotional struggles that could
potentially interfere with school adjustment. This
program enables children to develop and practice
the necessary school readiness skills in a
structured, safe, warm and nurturing group
environment with a low staff to child ratio (1:3).
Kindergarten Readiness held five sessions in
2011-2012. Four sessions received client
satisfaction questionnaires (39 participants).
Eighteen completed questionnaires were received
which represents a response rate of 46%.
A high majority of caregivers agreed that their
child is less anxious about starting
kindergarten because of this program, and
that they were less anxious as well (see Figure
11).
All respondents said that their child was more comfortable being away from them (72% “strongly agree”).
A total of 94% said their child was better prepared for school as a result of the program. All respondents
felt their children had learned new skills (83% saying “strongly agree”, 17% “agree”).
Nursery School Program
The Nursery School Program offers a structured, caring and warm group experience to young children
between the ages of 18 months to 3 years. These children may be displaying developmental delays, anxiety
separating from parents, and/or struggles in large group structures. Through staff support, the children
develop social skills such as turn taking, language skills, self help skills, and cognitive and motor skills. They
also develop an increased sense of self which leads to a healthy independence from parents.
The Nursery School Program held nine sessions in 2011-2012. Seven sessions received client satisfaction
questionnaires (68 participants). Thirty-four completed questionnaires were received (response rate 50%).
All respondents agreed or strongly agreed that the program was friendly and welcoming (91% “strongly
agree”). All of the caregivers felt their child had learned new skills in the program (71% “strongly agree”
and 29% “agree”).
All respondents said that they would recommend the program to a friend (91% “strongly agree”). Most
respondents agreed that their child is more comfortable being away from them as a result of the program
(79% “strongly agree”, 15% “agree”, 6% did not answer this question).
Parents observed many positive changes including improved confidence, language skills, social skills,
interest in activities.
Parents suggested both programs could be improved by increasing the number of days it is held per week.
Page 20
Client Feedback:
Prevention & Early Intervention Groups
Figure 11 - Kindergarten Readiness Anxiety
Rating
0%
20%
40%
60%
80%
100%
Strongy agree 72% 78%
Agree 22% 17%
Disagree 6% 6%
Strongly disagree 0% 0%
Child Less AnxiousParent Less
Anxious
Background
The George Hull Ontario Early Years Centre offers free universal programming to the community for children from birth to age six and their caregivers. Programs range from structured parent/caregiver education sessions to drop-in programs. Programs are offered six days a week, during traditional and non-traditional hours.
Evaluation Structure
Prior evaluations have been done on a monthly schedule. In fiscal 2011-2012 the OEYC had an opportunity to participate in an evaluation process designed by the Family Resource Programs of Canada. Participants had the opportunity to complete feedback forms during a three-week period in November 2011.
Respondents rated their Centre experiences highly (see Figure 12). Almost all respondents felt welcomed and accepted when they came to the centre (89% “strong agreement”). Respondents said they were treated with respect by staff members (92% “strong agreement” 6% “moderate agreement”), and the centre does its best to be welcoming to the diverse groups of people who live in the community (81% “strong agreement” and 17% “moderate agreement”). The majority of respondents said that since coming to the Centre they have become more aware of the services and resources available in their community (50% “strong agreement” and 18% “moderate agreement”).
Areas for Growth
The distribution of responses also highlighted a few places where there is opportunity for growth. These included the following:
The creation of opportunities to involve parents and caregivers in decision making about the programming and operations of the Centre.
To help parents and caregivers deal more effectively with the day to day challenges encountered as a family.
Page 21
Client Feedback: Ontario Early Years
Figure 12 - Participant Experience
0%
20%
40%
60%
80%
100%
Strong agreement 69% 83%
Moderate agreement 22% 17%
A little agreement 3% 0%
No agreement 0% 0%
Cannot say/ n/a 3% 0%
Missing 3% 0%
It is possible for me to
participate in programs
and activities
Staff & services are
available when I need
them
Respondents
reported positive
impacts on their
parenting as a result
of participating in
the Centre’s
programs (see
Figure 13).
Respondents
reported being more
aware of activities
that are appropriate
for their
child/children (72%
“strong agreement”
and 22% “moderate
agreement”) and
using activities at
home that were
learned at the
Centre (61%
“strong agreement”
and 25% “moderate
agreement”).
Participant Comments
Participants were asked “how has this program made a difference for you or your family?” The answers given
most frequently by respondents were that the program provided opportunities for social interaction for the
children. Respondents said that this interaction helped their child to learn to share, to play, and to improve
their interactions with other children. Respondents also said that the program helped them to meet, connect,
and share with other parents and caregivers. Respondents reported that the program helped them to develop
a routine, schedule, and provided structure to the day.
Participants were asked “what would you like us to do differently?” No patterns emerged from the many
suggestions provided. There were comments about the schedule of activities, such as to schedule programs
around typical routine times for infants and toddlers, to open earlier, and to start snack time earlier. However,
most comments were unique and varied. Respondents suggested offering coffee and tea during the morning
programs, to keep the website up-to-date, to have a larger space, more bathrooms, to provide more
educational toys, and to have a program for pre-school aged children where they can learn letters and
numbers.
Page 22
Client Feedback: Ontario Early Years
Figure 13 - Since coming to the centre:
0%
10%
20%
30%
40%
50%
60%
70%
Strong agreement 44% 58% 47%
Moderate agreement 28% 33% 33%
A little agreement 6% 6% 11%
No agreement 11% 0% 6%
Cannot say/ n/a 8% 3% 3%
Missing 3% 0% 0%
Feel more confident
as a
parent/caregiver
More aware of what
to expect child to do
at their age
Use strategies for
guiding child's
behaviour
“The children look forward to coming each day and have learned to interact and share with other children.”
Background
The Special Needs Resource Consultant works with children who attend licensed group care settings and home child care. Consultation includes activities such as creating program plans for individual children; consultation on programming, customized workshops for providers and parents or information sessions. The Intensive Resource Support Worker is available to licensed child care programs to support the inclusion of children with complex and/or intensive developmental, social, emotional or behavioural needs.
Child Care Centre Feedback
Each year questionnaires are mailed to child care centres that receive services from both these programs. In
February 2012 questionnaires were mailed to 18 centres. A total of 15 completed forms were received (83%
response rate) and several contained missing data. Seven centres had used Special Needs Resourcing in the
past year. All respondents who had used services found them to be “helpful” or “very helpful.” Child
Specific Referrals and Program Development were rated highly, although several centres did not respond to
these questions (see Figure 12). All the centres who had used the services “definitely” had their needs met
and said that they would “definitely” use the services in the future.
Suggestions for program improvement included providing suggestions and feedback, strategies on how to
help children, and to offer workshops in the evening. One centre suggested offering an informal session at the
beginning of the school year to increase parent’s awareness of services and roadblocks to watch out for so
that parents may be more open to accepting outside help.
Intensive Resource Support A total of 3 child care centre questionnaires were received.. In all cases, workers “strongly agreed” that the
IRS staff shared information in a clear, direct and supportive manner and that IRS staff helped everyone
involved to openly discuss concerns. Two centres “strongly agreed” that as a result of services, child care
staff are better able to meet the child’s needs and
have learned new skills and ideas to use with
other children. One centre did not answer these
questions. One centre suggested the service
could better meet their needs by extending the
service to a longer period.
Program Review
The City of Toronto, Children's Services is
undertaking a review of the "Every Child
Belongs Model" that has been in place for the
past 7-8 years. This model focuses on the
Special Needs Resource Consultation and the
Intensive Resource Support service. The model
review is aligned with maintaining the key
priorities of City Council (customer service and
accountable and transparent service) and talks to
the 2010-2014 Service Plan goal to reconfigure
and adjust services for children with special
needs as required during the introduction of the
Full Day Early Learning Program.
Page 23
Special Needs Resourcing
Figure 14 - Yearly Child Care Feedback
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Very H elpful 14% 29% 86% 14%
H elpful 43% 29% 14%
So mewhat H elpful 14%
N o t H elpful
D id N o t Use/ B lank 43% 43% 72%
T raining P ro g D evC hild
R eferralsIR S
School Program The School Program offers academic programming for up to 24 youth through three Section 23 classrooms. The youth who attend the school exhibit acute or chronic behavioral, emotional, learning or developmental difficulties. The data below reflects the evaluation completed for the 2011-2012 academic year. In the 2011-2012 academic year, 35 students entered the School Program and two students had two admissions during the academic year, for a total of 37 admissions to the School Program.
CAFAS Ratings Table 2 displays CAFAS change for males and females in the School Program. The CAFAS tool assesses 8 areas of child functioning (role performance in school, home, and community, behaviour towards others, substance use, mood, self-harm, and thinking). Total scores reflect overall functioning and lower ratings represent improvement. Data presented is for clients in the program not less than 60 days and with complete data (28 clients).
Additional Findings School anxiety in the School Program increased slightly by the end of the year. Pre-post data shows
that the percentage of students who were “not at all anxious” decreased from 14% to 0%. The
number of students who were “extremely anxious” increased from 3% to 14%.
69% of students earned academic credits during the school year and 49% of students earned 3 or more
credits. 11 students (31%) did not earn any credits during the school year. 10 of these students were
enrolled for 47% or less of the total academic year
Student Feedback
At the end of the 2011-2012 academic year, a survey was conducted to obtain feedback from the students.
A total of 35 participants were eligible for follow up at the end of the School Program. Twelve participants
provided feedback, representing a response rate of 34%.
The majority of respondents felt that the School Program was a welcoming and comfortable place
(58% “yes, definitely” and 33% “yes, I think so”), that the program staff listened to them (67% “almost
always), and treated them in a polite and friendly way (67% “yes, very much” and 33% “yes, kind of”).
The majority of respondents said the program helped them to better manage the problems that
brought them to the program (33% “yes, it helped a lot” and 42% “yes, it helped a little”).
Page 24
Day Treatment Programs
Table 2: School Program - Total CAFAS Score Change Indicators By Gender
Females Males
Percentage with Higher CAFAS scores at exit. 38% 40%
Percentage with No Change in CAFAS total score at exit. 8% 7%
Percentage with Lower CAFAS score at exit. 54% 53%
Percentage with Lower CAFAS score at exit >= 20pts. 46% 53%
Percentage with Lower CAFAS scores at exit >=40pts. 31% 13%
92% of respondents were satisfied with the amount of help they received from the program.
67% of respondents said that the program helped them to meet their personal goals and 25% said the
program did not help them to meet their personal goals. 42% of respondents said the program helped
them to meet their academic goals and 50% felt it had not helped them in this area. 42% of
respondents said that the program helped them to understand their family differently and 42% felt it
had not helped them to understand the family differently.
42% of respondents said that
the daily program at school
was “excellent”, 50% said it
was “good” and 8% said it
was “fair”.
75% of respondents would
recommend the program to a
friend in need of similar help
and 25% would not
recommend the program.
92% of respondents were
happy with the program
(42% “very happy”).
The majority of respondents
said that the staff support,
academics, discussion
groups, physical activities,
and peer support were
helpful (see Figure 15).
Positive Aspects
of the Program
Participants were asked what they liked best about the School Program and what was good about the
program. They mentioned the supportive and empathetic staff and the support system. Other positive
comments were the opportunity to learn and not to be left out of everything, the exercise program, and the
students.
Suggestions for Improvement
Participants were also asked what they didn’t like about the School Program and what could be improved.
Respondents mentioned placing a greater emphasis on academics, improving the math program, fewer
interruptions to the physical education program, increased respect for students’ private and personal space,
and improving peer interactions. A few respondents said that the program did not need improvement.
Page 25
Day Treatment Programs
Figure 15 - How helpful were the following:
0%
10%
20%
30%
40%
50%
60%
70%
80%
Very helpful 75% 25% 33% 50% 33%
Helpful 8% 33% 25% 25% 33%
Somewhat helpful 17% 42% 33% 17% 17%
Not helpful 0% 0% 0% 8% 17%
Missing 0% 0% 8% 0% 0%
Staff
Support
Academi
cs
Discussi
on
groups
Physical
activities
Peer
Support
Clear Directions The Clear Directions program is designed to assist youth, 18 years and under, who are struggling with
serious substance abuse, family difficulties, and mental health issues. The data below reflects the evaluation
completed for the 2011-2012 academic year. In the 2011-2012 academic year, 9 students were enrolled in
Clear Directions.
CAFAS Ratings Table 3 displays CAFAS change for males and females in Clear Directions. The CAFAS tool assesses 8 areas of child functioning (role performance in school, home, and community, behaviour towards others, substance use, mood, self-harm, and thinking). Total scores reflect overall functioning and lower ratings represent improvement. Data presented is for clients in the program not less than 60 days and with complete data (9 clients).
Additional Findings School anxiety ratings were available for 7 of the 9 students. School anxiety increased for Clear
Directions clients overall. 22% of clients were rated as “fairly anxious” or “very anxious” at the start of the year. This percentage increased to 67% (56% “fairly anxious” and 11% “extremely anxious”) by the end and the number of students who were not anxious at all decreased from 22% to 0%.
56% of students achieved 3 or more academic credits and one student achieved 6 credits during the
program year. 2 students (22%) did not earn any academic credits. These students were enrolled in the program for
32% or less of the total academic year. Student Feedback Only one client feedback form was received from Clear Direction in this fiscal year. This person indicated some positive changes as a result of the program, including academic achievement and decrease in substance use. No suggestions for program improvement were offered by this client.
Page 26
Day Treatment Programs
Table 3: Clear Directions - Total CAFAS Score Change Indicators – By Gender
Females Males
Percentage with Higher CAFAS scores at exit. 33% 17%
Percentage with No Change in CAFAS total score at exit. 33% 17%
Percentage with Lower CAFAS score at exit. 33% 67%
Percentage with Lower CAFAS score at exit >= 20pts. 33% 67%
Percentage with Lower CAFAS scores at exit >=40pts. 33% 33%
The main Quality Assurance activity for the Centre’s Residential Programs is the formal licensing program
conducted by MCYS on an annual basis. This rigorous process involves determining compliance with
regulations through multiple sources: staff and resident interviews, file audits, and physical inspections.
Changes that came out of the licensing process were addition of a school reports section in all clinical case
files, staff review of the policy regarding dispensing of medication, staff review of the enhanced
serious occurrence policy, and a change in the procedures for providing medications to clients during home
visits. In June 2012, the Centre was found to be in compliance with all licensing requirements and was granted
a regular license with no term and conditions, expiring July 2, 2013.
Program evaluation for the Residential Programs continues to be a challenge due to the small sample and
complex nature of the population and the intervention.
Boys House
In the 2011-2012 fiscal year, four boys were discharged from the Boys House. One of the boys was discharged
to his own apartment with child welfare support. This young man had completed high school and was going
on to college. Two of the boys were discharged to their family homes and the fourth left the program to live
with family friends. Of the four discharges, two were planned and two were not. The unplanned discharges
came about due to their behaviour in the program. Despite this, they were both planning to move from the
residence within a few months, and had made many gains within the program. The boys who left in a planned
way had done extremely well in the program. Some of the outcomes present in this group were:
Significant increase in school attendance.
Resolution of difficult family issues that allowed for better connection to family and in some cases ability
to move home.
Increased skills in managing severe anxiety symptoms and a dramatic decrease in overall mental health
symptoms allowing for wider engagement in everyday and challenging activities,
Scores on the Child and Adolescent Functional Assessment Scale (CAFAS) showed clinically significant
improvements in overall functioning for 3 of the 4 clients.
Client Feedback
Client satisfaction forms are made available to clients for feedback at the plan of care meetings every 6 months.
Only three completed forms were received during the 2011-2012 fiscal year. Based on the form dates, it was
determined that these forms represented the views of three distinct clients.
The forms represented mixed views. All three agreed that family were encouraged to visit the program. All
three said they did not have a clear discharge plan. Two felt involved in discharge planning and the third did
not. Two boys felt the staff “usually listened” and one said “sometimes”. There was a range of responses in
terms of rating the helpfulness of various parts of the program with some saying ‘very helpful”, another saying
“a little” and one saying “not at all”. Two said the program had helped to meet their academic goals and one
said this was not applicable. All agreed that the Boys House was helping to improve their social relations with
peers.
Residential Programs
Page 27
Libby’s Place
At Libby’s Place, 16 girls were discharged during the fiscal year 2011-2012. Of these 16 girls, 8 of them were admitted through the Quicker Access Program. The Quicker Access Program accepts clients from the child welfare system who child welfare believe are in urgent need of a mental health bed. This isn’t intended to be a crisis bed program, but the assessment process is forgone and admission generally happens within 24 hours of referral, often within hours. The Quicker Access clients at Libby’s Place have had a higher turnover than many of our clients. We believe this is due to the mandated nature of the treatment for these clients. We have participated in the program for a year and plan to move one of the Quicker Access beds out of Libby’s Place to the Boys House so each program will have one Quicker Access bed. The hope is that the Quicker Access client will adapt to the existing culture within the program and better engage in the treatment. All 8 Quicker Access clients who were discharged in the reporting period were in the program less than 4
months, and 50% of them were in the program less than 1 month. Two of these clients showed an improved
CAFAS score at exit. Program turnover was higher for regular clients, partly due to the instability generated by
the Quicker Access protocol. Of the 8 non-Quicker Access clients, four clients had planned discharges; two
returned to their family homes and two moved on to semi-independent programming. The two girls who
moved home continued to attend the School Program for additional support. Four of the eight clients
showed an improved CAFAS score (see table below).
CSQ
Only two client satisfaction questionnaires were received during the fiscal year. One reason for this is that
many clients did not remain in the program long enough to receive these forms which are given every six
months at plan of care meetings. The program is currently reviewing their process for obtaining feedback
from clients.
Residential Programs
Page 28
CAFAS Change Category Quicker Access
Clients
Non Quicker Access
Clients
# with CAFAS scores showing a decline in
functioning
1 2
# with CAFAS showing an improvement in
functioning
2
4
# with CAFAS that showed no change in
functioning
1 1
# with no reliable CAFAS due to stay in
program less than one month
4 1
CAFAS Results—Libby’s Place
Background
The Highfield Project is a Better Beginnings, Better Futures site based at Highfield Junior School in Rexdale. The area is a high-need, low-income neighbourhood in Etobicoke that is populated by struggling newcomers. The Project offers multiple programs to children and families.
Highfield Summer Enrichment Program (SEP)
The Highfield Summer Enrichment Program (SEP) is a kindergarten readiness program that has run yearly since the summer of 1995. During the 2011 summer session, approximately 80 children took part and 55 caregivers completed a feedback from at the end of the program.
All respondents found the program friendly and welcoming, said that their child looked forward to coming to the program, and that they would recommend this program to a friend.
Respondents also said that they read or sing more with their child because of this program (52% “strongly agree” and 42% “agree”).
Almost all respondents supported the
idea that their child was better
prepared for school as a result of the
program (86% “strongly agree”, 14%
“agree”).
88% of caregivers reported that their
children were less anxious/nervous
about starting kindergarten because
of the Highfield program.
90% of caregivers reported that they
were less anxious about their children
starting school as a result of the program.
95% of caregivers said they made new friends because of this program and 97% of respondents said
their child had made new friends because of this program.
Program Strengths and Improvements Caregivers said that the strengths of the program were the teachers, that their children made friends, learned things such as shapes, counting, and how to recognize their name, and that their children are now prepared for kindergarten.
Page 29
Highfield Community Enrichment Project
“I am confident she is ready to be alone in the class”
Figure 16 - My child is better prepared for
school as a result of this program
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Strongly agree 81%
Agree 19%
Kindergarten Readiness and Over a Cup of Tea
The Kindergarten Readiness program at Highfield provides preschool children the opportunity to experience attending school in a non-intimidating way to facilitate their transition from home to school life. Children attend the program while caregivers attend their own program, Over a Cup of Tea, that provides a socialization and learning opportunity for parents.
A total of 12 feedback forms (response rate of 71%) were received for Kindergarten Readiness. As shown in Figure 17, all respondents indicated that their child learned new things, had improved social skills and became more independent.
Respondents suggested that the Kindergarten Readiness Program could be improved by increasing the number of days it is held per week, by teaching table manners, and by having the children remove their hats and coats by themselves.
Almost all respondents reported positive benefits for caregivers for participating in the Over a Cup of Tea program.
92% agreed or strongly agreed that the program taught them new and important information and taught them parenting skills.
92% said the program helped them to make new friends and feel less isolated.
The caregivers in the program participated in a series of information sessions including—healthy eating,
child discipline, internet safety, job search, and preparing children for school. Qualitative feedback from
the participants indicated that they found these sessions to be very useful.
Page 30
Highfield Community Enrichment Project
0%
10%
20%
30%
40%
50%
60%
Figure 17 - Kindergarten Readiness Program Outcomes:
Strongly agree 50% 50% 58%
Agree 50% 50% 42%
Disagree 0% 0% 0%
Strongly disgree 0% 0% 0%
No answer 0% 0% 0%
My child
learned new
things
My child's
social skills
improved
My child is
more
independent
Background Family Group Conferencing (FGC) is an alternative approach to working with and engaging families. The main objective of FGC is to give the extended family group (i.e., nuclear family, extended family, and friends) a meaningful voice in the decision-making process to ensure the safety and well-being of children who are at significant risk of or in need of protection from abuse and neglect.
Plan Development
The Family Group Conferencing Project of Toronto received 67 referrals in 2011-2012. A total of 43 conferences were held during the fiscal year and an agreed upon plan was produced for 43 conferences (100%). In total, 68 children were involved in these plans and for 64 of them (94%) the plan was for the children to remain with or be returned to kin.
Client Feedback
The project offers all family members and professionals the opportunity to fill out a Client Satisfaction Questionnaire at the end of the conference day. A total of 462 questionnaires were received during the fiscal year - 312 (67%) of the forms were received from kin participants and 114 (25%) forms were filled in by professionals. The remaining 36 (8%) forms either did not specify or were of an indeterminate
Family Group Conferencing Project of Toronto
Page 31
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Figure 18 - Family Satisfaction with FGC Program
Yes, definitely 92.9% 76.7% 67.3% 86.9% 82.9% 91.2% 95.1%
Yes, somew hat 6.1% 19.1% 21.0% 11.8% 13.5% 6.5% 3.9%
No, not really 0.6% 3.6% 9.0% 1.0% 2.3% 1.3% 0.7%
No, definitely not 0.3% 0.6% 2.7% 0.3% 1.3% 1.0% 0.3%
Clear
Purpo
se
Adeq
Info
Every
one
there
Freed
om to
Expre
Freed
om to
Disagr
Cultur
al
Respe
Safety
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Figure 19 - Professional Satisfaction with FGC Program
Yes, definitely 97.4% 83.2% 48.7% 92.0% 93.8% 93.5% 94.6%
Yes, somewhat 2.6% 15.0% 37.2% 8.0% 6.2% 5.6% 5.4%
No, not really 0.0% 1.8% 8.0% 0.0% 0.0% 0.0% 0.0%
No, definitely not 0.0% 0.0% 6.2% 0.0% 0.0% 0.9% 0.0%
Clear
Purpo
se
Adeq
Info
Every
one
there
Freed
om to
Expre
Freed
om to
Disag
Cultur
al
Resp
Safety
The questionnaire addressed a variety of aspects of the FGC process (clear purpose, adequate information,
family members present, freedom to share opinions, freedom to disagree, cultural responsiveness, and
safety). (see previous page—Figure 18 for family responses and Figure 19 for professional responses).
There is a slight difference between families and professionals in their agreement levels for many areas.
Professionals were more likely than family members to select the higher level of satisfaction for all
dimensions except for “everyone who needed to be there was” and “I felt safe during the conference.”
Very few respondents expressed high disagreement with any area of the conference process. For all
questions except “everyone who needed to be there was”, the percentage of “no, definitely not”
responses was less than 5%.
Levels of satisfaction with the FGC coordinator
were high. Over 90% of family members and
professionals said the coordinator was definitely
“knowledgeable”, “well organized”, “respectful and
courteous” and able to “keep the group focused on
the purpose of the conference”.
Outcomes of the Process
Participants were asked about the outcomes of the
FGC process (see Table 4 for results).
The results show very high levels of satisfaction along
all dimensions.
There are not large differences between families
and professionals in their agreement levels. For the
most part, very few respondents expressed high
disagreement with any area of outcomes for the
conference process. For the majority of questions,
the percentage of “no, definitely not” and “no, not
really” responses was less than 5%.
All professionals and 99% of family members felt
the plan protects the child’s safety.
99.1% of professionals and 93.8% of family
members felt the FGC helped the family to get
along.
All professionals and 97% of family members said
they would definitely recommend FGC to others.
Family Group Conferencing Project of Toronto
Page 32
Family Prof
Family made decisions
Yes, definitely 90.3% 91.0%
Yes, somewhat 8.7% 9.0%
No, not really 0.3% 0.0%
No, definitely not 0.6% 0.0%
FGC helped family get along
Yes, definitely 79.2% 81.8%
Yes, somewhat 14.7% 17.3%
No, not really 5.9% 0.9%
No, definitely not 0.3% 0.0%
FGC helped family and
professionals get along
Yes, definitely 82.2% 77.7%
Yes, somewhat 14.8% 20.5%
No, not really 3.0% 1.8%
No, definitely not 0.0% 0.0%
Decisions were respected
Yes, definitely 90.3% 93.7%
Yes, somewhat 9.4% 6.3%
No, not really 0.3% 0.0%
No, definitely not 0.0% 0.0%
The plan protects the child's safety
Yes, definitely 95.1% 88.2%
Yes, somewhat 3.9% 11.8%
No, not really 0.3% 0.0%
No, definitely not 0.7% 0.0%
The plan explains what is to be done
Yes, definitely 89.3% 82.0%
Yes, somewhat 9.4% 18.0%
No, not really 1.3% 0.0%
No, definitely not 0.0% 0.0%
Table 4 - FGC Outcome Ratings
Background
At the provincial level, The George Hull Centre is mandated to lead the FGC Provincial Resource (PRG),
providing training, mentoring and consultation across the province. The Provincial Resource is
responsible for developing and managing a provincial roster of coordinators and mentors, a training
manual, and ongoing professional development activities.
This project ensures the integrity of the FGC service model through quality and consistency in training
and mentorship of new service providers.
Key Activities and Accomplishments
Completion of the revised FGC/Family Group Decision Making (FGDM) Coordinator Manual for
Ontario
Revision to the Basic and Advanced training slides
Participation in the organizing and delivering of the 1st Canadian Conference on FGC held in
September 2011 in collaboration with Ontario Association of Children’s Aid Societies (OACAS) and
the American Humane Association (AHA)
The program delivered a total of 977.9 hours of mentorship. As at March 31, 2012 the provincial roster
had 73 coordinators, 20 trainers and 19 mentors.
Youth in Transition Research Project
In March 2008, the Family Group Conferencing (FGC) Ontario Provincial Resource Advisory Group
identified the need to gain more knowledge about the use of FGC with youth in the Province of Ontario.
The review of outcome literature for youth involved in child welfare suggests that this group is at risk for
many negative outcomes. The literature also points to social support as a key variable in ameliorating
negative outcomes. Family Group Conferencing provides a promising intervention for youth because it
engages youth in a planning process for their future, encourages decision-making and youth voice, and
focuses on repairing relationships and expanding supports. Between 2008 and 2012, interview data was
collected from FGC coordinators and from youth at 6 months post-conference.
Although based on a small number of cases, almost all youth reported positive changes that emerged from
their experience with FGC. The conference process for the youth produced many changes including
affirmation and clarity on social support, improved contact and communication with family, increased
self-esteem, and various forms of practical support.
The coordinator data provides a rich source of information about working with youth in the FGC context
and generated many helpful suggestions that can be incorporated into FGC coordinator training materials.
Interim results of this study were presented to the Ontario FGC Provincial Resource Steering Committee
on March 5, 2012 and during the April 18, 2012 semi-annual FGC Ontario Provincial Resource
professional development day. The Provincial Resource has made a commitment to continue ongoing
training in this area by having workshops and training materials on how to provide FGC services to youth.
Family Group Conferencing
Ontario Provincial Resource
Page 33
Basic Training
CSQs were collected at 5 of the 6 Basic Training Sessions held in fiscal 2011-2012. In total, 75 forms were
received (response rate of 93%)
The participants rated the trainers’ knowledge very highly. A total of 85% of the respondents said the
trainers’ knowledge was "Excellent," 13% said it was "Good," and 1% said it was “Fair”. A total of
88% said the trainer was "definitely" effective in helping them to learn more about FGC.
All participants said they would recommend FGC training to a colleague (71% “yes definitely” and
29% “yes, I think so”).
59% of participants were “very
satisfied” with the amount of
information they received in the
training and overall training
experience and 37% were “mostly
satisfied”.
Advanced Training
CSQs were collected at 5 of the 6
Advanced Training sessions held in
fiscal 2011-2012. A total of 71 CSQs
were collected, representing a response
rate of 86%.
78% of respondents said the
trainers’ knowledge was “excellent”
and 18% felt it was “good”.
Almost all respondents rated the
quality of training as “excellent”
(69%) or “good” (30%).
Most respondents (97%) indicated
they would be interested in
receiving additional training. Almost all participants said they would recommend training to a
colleague (79% “yes, definitely”, 17% “yes, I think so”). (see Figure 20, Column 2).
Suggestions for improvement included having longer training sessions, using more videos and more
role plays.
Orientation
14 CSQs were collected during the FGC Orientation Day held in fiscal 2011-2012. All respondents
said that the trainer was effective in helping them to learn more about FGC (36% “yes, definitely” and
64% “yes, somewhat”). 71% said they would consider using FGC in their work. Respondents said
they would use their experience from the orientation by keeping the service in mind when working
with families.
Family Group Conferencing
Ontario Provincial Resource
Page 34
Figure 20 - After taking part in FGC Training
would you consider using the process in
your work?
0%
20%
40%
60%
80%
100%
No, definitely not 0% 3%
No, not really 1% 0%
Yes, I think so 21% 17%
Yes, definitely 75% 79%
No answer 3% 1%
Basic -
Use FGC in Work
Advanced -
Recommend FGC
Ministry Deliverables
Motor Speech Research Study The Toronto Speech and Language program will be involved in a large scale multi-centre study led by Dr.
Aravind Namasivayam, Ph.D. from the Dept of Speech-Language Pathology, U of T and The Speech &
Stuttering Institute. The study is aimed at (a) estimating the magnitude of treatment effects, (b) treatment
efficacy and (c) identifying the key factors that contribute to treatment effectiveness of motor speech
intervention for children presenting with speech-motor difficulties. The study is organized by the Ministry
of Child and Youth Services (MCYS) and approximately 52 specially trained clinicians from 32
Preschool Speech and Language (PSL) programs in Ontario, Canada will be participating. It is expected
that about 250 children (about 5 per clinician) with motor speech issues will be participating in this study.
Training for the TPSLS clinician who will participate in this study began in Feb 2011. Recruitment of
clients for this project will begin in the next fiscal year
Client Feedback Questionnaires
During 2009-2010 the Toronto Speech and Language team worked to design a feedback form for clients
exiting service. The feedback forms were administered to clients exiting service in the Spring of 2011
and results will be released in September 2011
More Than Words
The More Than Words Program is a parent training program for children who have Autism Spectrum
Disorder (ASD) and social communication delays. The More Than Words program ran from November
2, 2010 to February 8th 2011 and involved 9 children and their caregivers. Final evaluations were
completed by 89% of participants. All caregivers felt the child's communication had improved as a result of taking the program (50% "a lot", 50 "a little"). At the close of the program, respondents were asked how satisfied they were—5 respondents described it as said they were "very satisfied" and people said "excellent". On a five point scale—75% of respondents rated the classes as either “4” or “5” and 75% rated the visits as “5”. Some of the suggestions for improvement were as follows: more home visits, monthly interaction with kids, sessions not so late, more involvement of children in program, toys/books/games in video tapings, and getting kids together once a month.
Toronto Preschool Speech and Language Services
Page 35
Ministry Deliverables
Motor Speech Research Study
The Toronto Speech and Language program will be involved in a large scale multi-centre study led by Dr.
Aravind Namasivayam, Ph.D. from the Dept of Speech-Language Pathology, U of T and The Speech &
Stuttering Institute. The study is aimed at (a) estimating the magnitude of treatment effects, (b) treatment
efficacy and (c) identifying the key factors that contribute to treatment effectiveness of motor speech
intervention for children presenting with speech-motor difficulties. The study is organized by the Ministry
of Child and Youth Services (MCYS) and approximately 52 specially trained clinicians from 32
Preschool Speech and Language (PSL) programs in Ontario, Canada will be participating. It is expected
that about 250 children (about 5 per clinician) with motor speech issues will be participating in this study.
Training for the TPSLS clinician who will participate in this study began in Feb 2011. Recruitment of
clients for this project will begin in the next fiscal year
Client Feedback Questionnaires
During 2009-2010 the Toronto Speech and Language team worked to design a feedback form for clients
exiting service. The feedback forms were administered to clients exiting service in the Spring of 2011
and results will be released in September 2011
More Than Words
The More Than Words Program is a parent training program for children who have Autism Spectrum
Disorder (ASD) and social communication delays. The More Than Words program ran from November
2, 2010 to February 8th 2011 and involved 9 children and their caregivers. Final evaluations were
completed by 89% of participants. All caregivers felt the child's communication had improved as a result of
taking the program (50% "a lot", 50 "a little"). At the close of the program, respondents were asked how
satisfied they were—5 respondents described it as said they were "very satisfied" and people said "excellent".
On a five point scale—75% of respondents rated the classes as either “4” or “5” and 75% rated the visits as
“5”. Some of the suggestions for improvement were as follows: more home visits, monthly interaction with
kids, sessions not so late, more involvement of children in program, toys/books/games in video tapings, and
getting kids together once a month.
Toronto Preschool Speech and Language Services
Page 35
Background
The George Hull Centre is the West Quadrant Local Coordinating Agency for Toronto Preschool Speech and
Language Services (TPSLS), overseeing the services in seven different sites. TPSLS at the George Hull site
are provided in close collaboration with the Community Clinic multidisciplinary team and the Ontario Early
Years Centre (OEYC).
The program is collaboratively managed in partnership with the City of Toronto, providing community-based
speech and language services to children and their families before the children are eligible for senior
kindergarten. The range of services includes assessment, consultation and support to parents and staff in
child care settings, training for parents and professionals, and group or individual therapy. Speech and
Language Pathologists, Communications Disorder Assistants and Early Childhood Educators provide the
services of the TPSLS program.
* TPSLS provided a full day workshop to clinicians that focused on group interventions. Clinical quadrant discussions
continue to support group interventions as an effective intervention for certain communication disorders.
Speech and Language Outcome Measures
Effective December 2011, the Severity Rating Scales (SRS) that have been in use by Preschool Speech and
Language (PSL) clinicians since 2009 are no longer required. A workgroup of provincial PSL clinicians and
coordinators reported several persistent challenges
with the scales, including perception that the SRSs:
Do not capture changes in children with more
severe disorders;
Lack appropriate levels of regional and provincial
inter-rater reliability for staff using the scales;
Would require the development of additional
scales in order to capture change in certain
communication domains (e.g. fluency, emergent
literacy, dysphagia);
Do not capture changes in parent and caregiver
skills, despite this being a core mandate of the PSL
Program.
MCYS will replace the existing PSL Functional
Communication Questions (FCQs) with the
FOCUS©, beginning in the summer of 2012. The
FOCUS©, a tool developed collaboratively by a team
of researchers at the Bloorview Research Institute (led
by Prof. Nancy Thomas-Stonell), CanChild Centre for
Childhood Disability (led by Dr. Peter Rosenbaum) and Laurentian University (led by Dr. Bruce Oddson) has
been demonstrated to be a reliable and valid measure of communication participation for preschool children.
Program Deliverable Actual
45% of all initial assessments will be provided to children by 30 months of age 42%
100% of children who are discharged for the reasons "attending JK", "attending SK" and "attending
Grade 1" will receive transition plans
92%
Individual treatment with a speech language pathologist and individual treatment with a mediator together
will comprise no more than 35% of the total interventions provided *
23%
0 %
2 0 %
4 0 %
6 0 %
8 0 %
Figure 21 - How helpful have
we been in...
Strongly Agree 69%
Agree 25%
Somewhat
Disagree
6%
Strongly
Disagree
0%
I now understand my
child's speech and
language difficulties.
Ministry Deliverables
Motor Speech Research Study The Toronto Speech and Language program will be involved in a large scale multi-centre study led by Dr.
Aravind Namasivayam, Ph.D. from the Dept of Speech-Language Pathology, U of T and The Speech &
Stuttering Institute. The study is aimed at (a) estimating the magnitude of treatment effects, (b) treatment
efficacy and (c) identifying the key factors that contribute to treatment effectiveness of motor speech
intervention for children presenting with speech-motor difficulties. The study is organized by the Ministry
of Child and Youth Services (MCYS) and approximately 52 specially trained clinicians from 32
Preschool Speech and Language (PSL) programs in Ontario, Canada will be participating. It is expected
that about 250 children (about 5 per clinician) with motor speech issues will be participating in this study.
Training for the TPSLS clinician who will participate in this study began in Feb 2011. Recruitment of
clients for this project will begin in the next fiscal year
Client Feedback Questionnaires
During 2009-2010 the Toronto Speech and Language team worked to design a feedback form for clients
exiting service. The feedback forms were administered to clients exiting service in the Spring of 2011
and results will be released in September 2011
More Than Words
The More Than Words Program is a parent training program for children who have Autism Spectrum
Disorder (ASD) and social communication delays. The More Than Words program ran from November
2, 2010 to February 8th 2011 and involved 9 children and their caregivers. Final evaluations were
completed by 89% of participants. All caregivers felt the child's communication had improved as a result of taking the program (50% "a lot", 50 "a little"). At the close of the program, respondents were asked how satisfied they were—5 respondents described it as said they were "very satisfied" and people said "excellent". On a five point scale—75% of respondents rated the classes as either “4” or “5” and 75% rated the visits as “5”. Some of the suggestions for improvement were as follows: more home visits, monthly interaction with kids, sessions not so late, more involvement of children in program, toys/books/games in video tapings, and getting kids together once a month.
Toronto Preschool Speech and Language Services
Page 36
Ministry Deliverables
Motor Speech Research Study
The Toronto Speech and Language program will be involved in a large scale multi-centre study led by Dr.
Aravind Namasivayam, Ph.D. from the Dept of Speech-Language Pathology, U of T and The Speech &
Stuttering Institute. The study is aimed at (a) estimating the magnitude of treatment effects, (b) treatment
efficacy and (c) identifying the key factors that contribute to treatment effectiveness of motor speech
intervention for children presenting with speech-motor difficulties. The study is organized by the Ministry
of Child and Youth Services (MCYS) and approximately 52 specially trained clinicians from 32
Preschool Speech and Language (PSL) programs in Ontario, Canada will be participating. It is expected
that about 250 children (about 5 per clinician) with motor speech issues will be participating in this study.
Training for the TPSLS clinician who will participate in this study began in Feb 2011. Recruitment of
clients for this project will begin in the next fiscal year
Client Feedback Questionnaires
During 2009-2010 the Toronto Speech and Language team worked to design a feedback form for clients
exiting service. The feedback forms were administered to clients exiting service in the Spring of 2011
and results will be released in September 2011
More Than Words
The More Than Words Program is a parent training program for children who have Autism Spectrum
Disorder (ASD) and social communication delays. The More Than Words program ran from November
2, 2010 to February 8th 2011 and involved 9 children and their caregivers. Final evaluations were
completed by 89% of participants. All caregivers felt the child's communication had improved as a result of
taking the program (50% "a lot", 50 "a little"). At the close of the program, respondents were asked how
satisfied they were—5 respondents described it as said they were "very satisfied" and people said "excellent".
On a five point scale—75% of respondents rated the classes as either “4” or “5” and 75% rated the visits as
“5”. Some of the suggestions for improvement were as follows: more home visits, monthly interaction with
kids, sessions not so late, more involvement of children in program, toys/books/games in video tapings, and
getting kids together once a month.
Toronto Preschool Speech and Language Services
Page 36
Client Feedback Survey
The Toronto Preschool Speech and Language Services (TPSLS) program conducted phone interviews with
caregivers of children completing a treatment cycle within the program from April 2011 to September 2011. A
total of 65 families participated in the program during this time and were eligible for the evaluation. Thirty six
caregivers provided feedback, representing a response rate of 55%.
Survey Findings
All respondents said they “almost always” or “usually” felt welcome and comfortable
94% said they would recommend the program to a friend
89% agreed that they were given accurate information about service wait times.
83% indicated their questions were almost always answered thoroughly (14% said “usually”)
81% felt the suggestions were almost always useful and 17% felt the suggestions were usually useful
Qualitative feedback from respondents showed a high appreciation for the staff. Staff were seen as caring,
professional and flexible. Caregivers liked the individual attention, and that programs were all located in the
same centre. Many respondents saw changes in their child’s skills and felt that they as caregivers learned as
well. Suggestions for program improvement included:
More notice prior to starting service
Shorter wait times for services
More sessions
Increasing upper age limit
Adjusting group size
“Approachable, flexible, down-to-earth and really focusing
on where the child is at. Very good service”
0%
20%
40%
60%
80%
Figure 22 - Outcomes
Strongly Agree 64% 80%
Agree 30% 11%
Somewhat Disagree 3% 3%
Strongly Disagree 3% 3%
No answer 0% 3%
My child's speech
and language has
I now have skills that
I can use to help my
Masters of Social Work Student Feedback
There were six MSW students at the GHC during 2011-2012 and all six students completed the evaluation All “agreed” or “strongly agreed” that they were provided with timely and appropriate constructive feedback and that staff were available to them for questions. Field instructors were seen as role models who facilitated achievement of the objectives of their respective learning plans. Five students “strongly agreed “ and one “agreed” that the placement provided the learning experience required to develop competency in this area of practice. The education seminars provided were highly regarded by the students. Areas of improvement were identified as more orientation to safety procedures and clinical recordings. Students also suggested the placement have additional opportunities to follow clients throughout the year, more contact with the short term cases, more opportunities to observe the work of other clinicians and additional group learning.
Child and Youth Work Student Feedback
A total of 5 CYW evaluation forms were received (response rate—71%). The feedback received was very
positive. All students “totally agreed” that they were valued as a team member. 80% of the students “totally
agreed” and 20% “somewhat agreed” that their experience increased their understanding of children and their
families . The students felt they were respected and able to share ideas with the team. 60% of the respondents
“totally agreed” that the program challenged their existing skills and allowed them to develop new ones (40%
somewhat agreed). All students “totally agreed” that their experience helped them to define future career
goals. All students “totally agreed” that they had an increased level of clinical knowledge of the clients and
their families. All agreed that they received regular supervision which was conducive to open communication.
Student suggestions for placement improvement included additional placement time, holding the student
support group meeting more frequently, having the supervisor directly in the classroom, and getting feedback
from other staff early in the program.
Psychiatry Rotation Feedback
The Centre receives a summary of the feedback from the University of Toronto on the residents’ experience
with their placements. Feedback was received in November 2011 and May 2012 from 2 of the 4 residents.
Areas of Strength
Good teaching – staff approachable, flexible, plenty of feedback and good multi-disciplinary rounds
Good breadth of cases and no difficulty getting cases
Good psychopharmacology
No difficulty getting psychotherapy patients
Good family therapy seminar/supervision and CBT
Community atmosphere
Focus on education (good balance between learning and service)
Flexible, e.g. off-site supervision
Supervisor that does 1 consult per week with young children (e.g. PDD)
Areas for Growth
No dynamic supervisors, difficult to find a case, need off-site dynamic supervision
Antiquated health records system
Improve scheduling, especially considering distance from core teaching
More physician-led rounds to focus on medical aspects of cases
Teaching and Training
Page 37