The Creative Therapy Consultants Homemaking Assessment: Administration Guide

Version 1.0 July 27, 2016

Prepared by:

Nicole Matichuk, BKIN1, Liv Brekke, MPA1, Hilary Drummond, OT(C) 2, & Susan Forwell, PhD, OT(C) 1 1Department of Occupational Science & Occupational Therapy, University of British Columbia

2Creative Therapy Consultants

1

Table of Contents

BACKGROUND ......................................................................................................................... 2

ASSESSMENT FEATURES ....................................................................................................... 3

MEASUREMENT PROPERTIES ............................................................................................... 4

Population .............................................................................................................................. 4

Ecological Validity .................................................................................................................. 4

Face Validity and Acceptability ............................................................................................... 5

Content Validity ...................................................................................................................... 5

Internal Consistency ............................................................................................................... 6

Interrater Reliability................................................................................................................. 6

ADMINISTRATION AND SCORING .......................................................................................... 7

Scoring Individual Tasks ......................................................................................................... 7

Light Task Example – Folding Clothing................................................................................... 9

Medium Task Example - Cleaning the Toilet..........................................................................10

Heavy Task Example – Vacuuming .......................................................................................11

Calculating Subscale and Total Weighted Scores .................................................................12

ADDITIONAL CONSIDERATIONS ............................................................................................13

REFERENCES .........................................................................................................................14

APPENDIX A - EXAMPLE OF SCORED ASSESSMENT .........................................................15

2

BACKGROUND

The Creative Therapy Consultants (CTC) Homemaking Assessment is a comprehensive, in-depth occupational

therapy assessment tool used to determine the percentage of housework tasks an individual can perform. It

was developed by Creative Therapy Consultants and has been in use for over 10 years.

Demand for the CTC Homemaking Assessment arose primarily from the need to provide a fair, objective, and

consistent approach to assessing clients who might require assistance with housekeeping tasks due to

accident or disability. While other assessments of instrumental activities of daily living (IADLs) existed at the

time, there was no tool that focused exclusively on homemaking tasks and which could provide insurance

companies, lawyers, and medical professionals with a quantifiable percentage of homemaking tasks an

individual could complete.

Individuals who have been injured in a motor vehicle accident comprise one of the primary populations with

which the CTC Homemaking Assessment is used. Results of the assessment have been used to recommend

to the insurer (often ICBC) whether or not an individual should receive additional homemaking support at

home as a result of their injuries/disability. The percentage score is particularly valuable because the

primary motor vehicle insurance agency, ICBC, provides homemaking supports only for those individuals who

can no longer perform the majority (i.e. 50 percent) of their previous homemaking tasks. The assessment has

also been used with other insurers or payers to support the credibility of clinical recommendations, and

results have been used in medical-legal assessments and discussed in court proceedings.

Key strengths of the CTC Homemaking Assessment include:

● Ecologically valid, client-centered, and occupation-based

● Tested in a broad population of ages and injuries/disabilities

● Diversity of items assessed in terms of the potential physical and cognitive demands of

homemaking tasks

● Weighting system which helps to capture the relative differences in difficulty between tasks

● Psychometric data showing good internal consistency

● Well-accepted by clinicians, fee payers, and the legal system

3

ASSESSMENT FEATURES

The following table provide a brief overview of the administration, applications, and content of the CTC

Homemaking Assessment. An example assessment can also be found in Appendix A.

Procedures Client-therapist interview supplemented by In-home observation of task performance

Setting

Chronic or acute

Populations

Physical, cognitive, mental health issues

Content 29 tasks divided into 3 categories: a. Light: 8 tasks (e.g. dusting, chopping food, folding clothes) b. Medium: 12 tasks (e.g. ironing, washing dishes, cleaning toilet) c. Heavy: 9 tasks (e.g. vacuuming, cleaning bathtub, grocery shopping) Permits addition or deletion of items based on client’s occupational profile

Scoring Simple weighting and scoring system that captures differences in task difficulty (see Scoring section of this guide for further details)

Time to Administer 1-2 hours

4

MEASUREMENT PROPERTIES

The following synopsis of measurement properties is based on a study conducted in 2015-2016 by a student,

clinician, and faculty research team with the Department of Occupational Science and Occupational Therapy

at the University of British Columbia. The study examined aspects of the validity and reliability of the

Assessment using a retrospective chart review and a small prospective sample of clients. Validity is the

degree to which an assessment measures what it is intended to measure (Law & Letts, 1989). Reliability

refers to the stability, consistency, and dependability of an instrument’s measurements (Law, 1987).

Specific results of the UBC study and their implications for administering the CTC Homemaking Assessment

are described below, along with additional information which can help inform clinical use of the Assessment.

Population

The CTC Homemaking Assessment has been effectively used with a broad range of individuals. In the sample

client population (n=113) from the UBC study, the mean age was 50.6 ± 15.2 years, and clients ranged in age

from 19 to 92. Two-thirds of these individuals were female; thus some caution is warranted in terms of

generalizing the reliability and validity of the assessment to men.

To date, the Assessment has been used extensively with clients who have experienced a motor vehicle

accident (MVA). It is therefore particularly applicable and reliable for use with this population. The

assessment has also been used with clients referred for falls, life care planning, and medical-legal

assessment, and to a lesser extent for functional capacity evaluations and return to work planning.

The assessment has also been used with clients with a wide range of diagnoses. Some of the most common

of these have been pain (including headaches), fracture and/or joint issues, soft tissue injuries (including

whiplash), and mental health diagnoses (anxiety, depression, PTSD, and adjustment disorder). The

Assessment has also been used with individuals with sensory changes (hearing and vision loss, peripheral

sensation changes), head injuries and/or cognitive issues (hemorrhage, TBI, concussion), and chronic disease.

Most clients who have undergone the CTC Homemaking Assessment have had multiple (2+) diagnoses.

Ecological Validity

Individuals participate in tasks or activities within their own unique physical, social, institutional, cultural and

temporal context. Therefore, the assessment environment will have an influence on a person’s performance

(Law, Baum, & Dunn, 2005). Research suggests that assessments of everyday activities should occur in the

environment in which the client usually completes them (Rogers et al., 2013; Law, Baum, & Dunn, 2005). The

CTC Homemaking Assessment is administered in the client’s own home; it is therefore grounded in the

client’s own context and can be said to have good ecological validity.

Similarly, the CTC Homemaking Assessment is also client-centered, and can account for differences in the

patterns of homemaking tasks each individual completes. The Assessment does this by allowing clinicians to

5

add tasks if necessary, and to score existing items as “not applicable” if it is a task not typically done by the

client (e.g. handwashing clothes) or if the client’s unique environment does not allow them to complete this

task (e.g. they do not have a dishwasher).

Face Validity and Acceptability

Clinicians who have used the CTC Homemaking Assessment to date have consistently provided positive

feedback about it. It provides a clear framework for assessment, more evidence upon which to base clinical

recommendations, and also assists with treatment planning. In addition, the assessment has been well-

accepted by insurers and lawyers. Because it has been reviewed by experts and determined to be credible,

the assessment can be said to have good face validity and clinical acceptability.

Content Validity

Content validity is the extent to which an instrument’s items comprehensively represent all the

characteristics of a construct or concept it is designed to measure (Mokkink et al., 2010; Yuen & Austin,

2014). One way of establishing content validity is through expert opinion (Mokkink et al., 2010. If clinicians

judge that the assessment tool comprehensively measures the thing it is intended to measure, then the tool

can be said to have content validity.

The CTC Homemaking Assessment is designed to measure the construct of homemaking ability, and the

development process of the Assessment supports its content validity. It was initially developed through

collaboration between six CTC occupational therapists. The team of therapists began by establishing a list of

homemaking tasks that were common to the clients they were seeing in practice, particularly those tasks

with which clients had the most difficulty. The team then classified the tasks as light, medium, or heavy to

reflect the relative demands of the different homemaking activities. Therapists then began to trial the

assessment and would reconvene as a group to discuss how it had worked in practice and consider

modifications that could be made. The assessment was regularly evolving for the first couple of years, and

the list of tasks on the assessment has continued to be adjusted throughout its years of use, as has their

categorization.

Content validity can also be established by using statistics (Mokkink et al., 2010). Correlational analyses

conducted during the UBC study demonstrated that, as expected, the entire assessment measures a single

construct overall, i.e. homemaking. All three subscales (light, medium, and heavy) demonstrated a moderate

to strong positive relationship with each other. Individual assessment items also showed consistently

positive relationships with each other, suggesting that the scale overall is generally unidimensional (i.e.

assesses homemaking and nothing else).

Results from the UBC study also demonstrated the validity of the subscales themselves. The correlations

between subscale scores were stronger between light and medium scores, and between medium and heavy

scores, than between light and heavy scores. This suggests that there is indeed a gradient in task difficulty,

6

as the assessment intended. Moreover, over two-thirds of clients with whom the assessment has been

administered have demonstrated decreasing performance as task difficulty increased, thus reinforcing the

distinction between the subscales.

Not only does the assessment include homemaking tasks with a range of difficulty levels, but also tasks

which place different kinds of performance demands on individuals. Factor analysis results from the UBC

study identified several additional sub-constructs across assessment items. These constructs, or factors,

suggested more specific physical and cognitive requirements involved in completing a task. Identified

categories of task demands included: bending tasks, upper extremity tasks, manipulating loads, and planning

tasks. The implications of these additional constructs are discussed in the Administration and Scoring section

of this guide.

Internal Consistency

Internal consistency refers to the sameness of items or scores within an instrument (Law & MacDermid,

2014) and is a type of reliability. Research into the CTC Homemaking Assessment shows that all three

subscales have demonstrated adequate to good internal consistency. In a sample of 113 clients, Cronbach’s

alpha values were as follows: 0.78 for light tasks; 0.77 for medium tasks; and 0.80 for heavy tasks. An alpha

value above 0.70 is considered adequate, 0.8-0.9 good, and above 0.9 excellent (Vroman & Stewart, 2014).

Therefore, clinicians can say that the assessment’s subscales consistently and reliably measure homemaking

tasks of light, medium, and heavy difficulty.

Interrater Reliability

Interrater reliability assesses how consistently an assessment performs when administered by different

practitioners (Vroman & Stewart, 2014). As part of the UBC study, researchers examined interrater

agreement in a small sample (n=9) of clients. Subscale scores assigned to the same client by two different

raters, a student and a clinician, were compared. The smallest mean difference in subscale scores between

the two raters was 6.3 ± 6.3 percent, and the largest mean difference was 8.1± 11.5 percent. This

preliminary information is promising, but further research is required in order to confidently quantify the

strength of the assessment’s interrater reliability.

7

ADMINISTRATION AND SCORING

As with any other measure, administration of the CTC Homemaking Assessment should begin with explaining

the purpose and implications to the client, responding to any questions, and obtaining and documenting

consent.

The clinician typically begins by working through the items on the assessment via interview with the client,

documenting scores and qualitative information as they go. Clients are then asked to demonstrate only

some of the tasks in the assessment. For this performance-based component, it is recommended that the

clinician choose tasks that the client expresses more difficulty in completing. This likely means observing

more medium and heavy tasks than light ones. Scores should be recorded as the clinician observes client

performance of the task. Any other pertinent information, such as reports of pain or use of adaptive aids,

should also be noted in the space provided on the assessment form. The number of tasks observed (versus

relying on self-report) is at the discretion of the clinician.

Based on results of the UBC study, clinicians should consider not only the task difficulty, but also the

underlying type of task demand when selecting tasks to observe. Clinicians should observe homemaking

tasks that place varying types of physical demands (bending, upper extremity work, and managing loads) and

cognitive demands (planning/organizing) on clients in order to obtain the most comprehensive picture of the

client’s overall homemaking ability. For example, if a client struggles to complete a medium bending task

(e.g. cleaning the toilet), the clinician administering the assessment may wish to also observe a medium

upper extremity-based task (e.g. washing dishes) because the two tasks may reveal different performance

abilities.

Scoring Individual Tasks

Each task is scored out of 1, where 1 represents full, complete, independent performance of the

homemaking task. A score of 0.5 generally represents incomplete or inadequate performance on the task,

while a score of 0 means the client is unable to complete the task at all.

If a client does not perform a task (e.g. they do not have a dishwasher, or they never clean the windows),

then the task should be marked “not applicable” or “N/A”. These tasks will not be included in the

assessment totals.

In addition, questions about pain, fluctuating performance, compensations, and safety should be scored as

follows:

● The client can complete the task but is in pain

If the client can complete the whole task, but is in pain, their score should be either a 1 or 0.5. The

score depends on how long it took the client to complete the task, how safe the client was, and the

8

degree of pain. The longer the completion time, the less safe the performance of the task, and the

higher the degree of pain, the lower the score should be.

● The client could complete the task once, but performance decreased or fluctuated over the course

of the assessment

A score of 0.5 is suggested.

● The client can complete the task using compensatory strategies or aids

A score of 1 is suggested. Note the clinician should record the aids/methods the clients used directly

on the CTC Homemaking Assessment form.

● The client can complete the task but your clinical judgment suggests that it is unsafe for them to

continue to do so.

A score of 0.5 is suggested. Clinicians should record that “the client is not safe and should not do this

task” on the assessment form. Safety recommendations should be communicated directly to the

client and further documented by the clinician.

Clinicians should thoroughly document the occurrence of the above listed events and/or limiting factors on

the assessment form in order to support their score.

To further support clinicians in consistently scoring assessment items, the following pages provide brief case

studies of how to score example tasks from each of the subscales.

9

Light Task Example – Folding Clothing

Case Study 1 (brain injury):

Suzy has had a brain injury. Her symptoms are fatigue,

dizziness, and executive functioning difficulties such as

organization, problem solving and prioritizing. She

cannot get herself organized to perform her household

tasks and fatigues quickly when doing much at one time.

She is also experiencing right-sided weakness.

Case Study 2 (musculoskeletal injury):

Fred has been in a motor vehicle accident. He has a

fractured right hip and spine. His symptoms are pain in his back, restricted movement, and limited mobility

while in his wheelchair.

Suzy Fred

0 Physical: Unable to fold any clothes at all as

her right hand cannot hold onto the fabric

OR

Cognitive: unable to problem solve a

technique for folding clothing in the same

manner

Unable to access laundry basket of

clothes and unable to reach down to

pull clothes out of basket.

0.5 Physical: Able to fold smaller items, such as

clothing, but cannot fold sheets and towels

due to restricted right arm function.

Cognitive: Folds clothes into impractical

shapes for putting away.

OR

Can figure out how to fold socks and

underwear but not shirts and pants.

Able to fold smaller items when laundry

is moved right beside him.

1 Able to fold all items without difficulty. Able to fold all items without difficulty.

10

Medium Task Example - Cleaning the Toilet

Case Study 1 (brain Injury):

Suzy has had a brain injury. Her symptoms are fatigue,

dizziness, and executive functioning difficulties such as

organization, problem solving and prioritizing. She cannot get

herself organized to perform her household tasks and fatigues

quickly when doing much at one time. She is also experiencing

right-sided weakness.

Case Study 2 (musculoskeletal injury):

Fred has been in a motor vehicle accident. He has a fractured

right hip and spine. His symptoms are pain in his back,

restricted movement, and limited mobility while in his

wheelchair.

Suzy Fred

0 Physical: Unable to perform task at all as

cannot lean over the toilet without

getting dizzy and falling- safety risk

Unable to perform task at all as cannot

reach area or bend over to clean.

0.5 Physical: Able to clean top part of toilet,

but cannot reach down into toilet or

clean bottom area,

Cognitive: Able to wipe down the toilet

but doesn’t remember to clean the bowl,

or requires cueing to wipe down all parts

and clean bowl

Able to wipe down top of toilet, unable

to reach lower areas.

1 Able to clean all of toilet without difficulty

Able to clean all of toilet without

difficulty

11

Heavy Task Example – Vacuuming

Case Study 1 (brain Injury):

Suzy has had a brain injury. Her symptoms are fatigue, dizziness,

and executive functioning difficulties such as organization,

problem solving and prioritizing. She cannot get herself

organized to perform her household tasks and fatigues

quickly when doing much at one time. She is also

experiencing right-sided weakness.

Case Study 2 (musculoskeletal injury):

Fred has been in a motor vehicle accident. He has a

fractured right hip and spine. His symptoms are pain in

his back, restricted movement, and limited mobility while in

his wheelchair.

Suzy Fred

0 Physical: Unable to do any vacuuming;

cannot move vacuum.

Cognitive: Cannot operate vacuum

cleaner controls, or sequence properly

(plug in, turn on, and operate).

OR

Does not initiate task at all.

Unable to vacuum at all.

0.5 Physical: Able to vacuum smaller flat

areas. Cannot vacuum stairs and/or

cannot move furniture to vacuum

underneath it.

Cognitive: Vacuums obvious areas but

does not move furniture.

OR

Perseverates and goes over the same

areas many times.

Able to vacuum small areas with

lightweight vacuum, or cannot move

vacuum very far.

1 Able to vacuum all areas independently. Able to vacuum all areas in his home. He

has no stairs.

12

Calculating Subscale and Total Weighted Scores

1. Establish the total number of tasks that were applicable to a client in each subscale. If a client does not

do a task, and did not do this task before their injury, that task is marked N/A and not included in the

total. For example, the light subscale includes 8 possible tasks. However, one is not applicable to the

client (e.g. because the husband has always swept the floor). Therefore the client’s light task score would

be divided by 7, not by 8.

2. Add up the scores (i.e. 0, 0.5, or 1) assigned to each of the tasks. Complete this for each subscale: light,

medium, and heavy. For example, the client’s scores in the light subscale could be 0, 0.5, N/A, 1, 1, 1,

0.5, 1 = 5

3. Calculate the percentage of tasks the client was able to complete for each subscale. For example, if the

client scored 5/7 for light tasks, this would be 71.43%. Do this for light, medium and heavy tasks:

Light tasks 5/7 = 71.43%

Medium tasks 6/10 = 60%

Heavy tasks 5/9 = 55.56%

4. Complete the weighting multiplication for each of the subscales. The light task percentage score is

multiplied by 1, the medium task percentage score by 2, and the heavy task percentage score by 3.

Light tasks 5/7 = 71.43% x 1= 71.43

Medium tasks 6/10 = 60% x 2 = 120

Heavy tasks 5/9= 55.56% x 3= 166.67

5. Add up the total weighted scores from each subscale. For the example listed above, this would equal:

71.43 + 120 + 166.67= 358.1

6. To calculate the final weighted percentage, divide by 6 (because the scores were multiplied by 1, 2, and

3 earlier). From the example listed above, 358.1/6 = 59.68%

59.68% is the final weighted percentage of homemaking tasks that the client can perform.

13

ADDITIONAL CONSIDERATIONS

The CTC Homemaking Assessment is intended to support, but not replace, clinical judgment and should not

form the sole basis for decisions about recommending in-home supports. Administering the tool effectively

requires the clinician to critically observe the consistency of the findings, as there is potential for the client to

try to skew the results. For example, a clinician might question a client further if the client indicates that

they can do no light or medium tasks, but can, in fact, complete several heavy tasks. Similarly, a clinician

might ask additional questions if a client reports they can perform a task, but the clinician observes them

having difficulty with much easier tasks.

14

REFERENCES

Law, M. (1987). Measurement in occupational therapy: Scientific criteria for evaluation. Canadian Journal of

Occupational Therapy, 54(3), 133-138. doi:10.1177/000841748705400308

Law, M. C., Baum, C. M., & Dunn, W. (2005). Measuring occupational performance: Supporting best practice

in occupational therapy. Thorofare, NJ: SLACK Inc.

Law, M., & Letts, L. (1989). A critical-review of scales of activities of daily living. American Journal of

Occupational Therapy, 43(8), 522-528.

Law, M. C., & MacDermid, J. (2014). Evidence-based rehabilitation: A guide to practice. Thorofare, NJ: Slack.

Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., . . . de Vet, H. C. W. (2010).

The COSMIN checklist for assessing the methodological quality of studies on measurement properties

of health status measurement instruments: An international delphi study. Quality of Life Research,

19(4), 539-549. doi:10.1007/s11136-010-9606-8

Rogers, J., Holm, M., Beach, S., Schulz, R., Cipriani, J., Fox, A., & Starz, T. (2003). Concordance of four

methods of disability assessment using performance in the home as the criterion method.

Arthritis & Rheumatism-Arthritis Care & Research, 49(5), 640-647. doi:10.1002/art.11379

Vroman, K., & Stewart, E. (2014). Occupational therapy evaluation for adults: A pocket guide. Baltimore:

Lippincott Williams & Wilkins

Yuen, H. K., & Austin, S. L. (2014). Systematic review of studies on measurement properties of instruments

for adults published in the American Journal of Occupational Therapy, 2009-2013. The American Journal

of Occupational Therapy: Official Publication of the American Occupational Therapy Association, 68(3),

e97. doi:10.5014/ajot.2014.011171

15

APPENDIX A - EXAMPLE OF SCORED ASSESSMENT

16

17