presenter: kristyn j. smith, ba research assistant case family medicine research division...

Presenter: Kristyn J. Smith, BA Research Assistant CASE Family Medicine Research Division

Authors: Sonja Harris-Haywood, MD, MSAssistant Professor,Research Division, Dept. Family MedicineCase Western Reserve School of Medicine

Tawara Goode, MSAssistant Professor and Director of the National Center for Cultural Competence (NCCC)Georgetown University Medical Center

Suzanne Bronheim, PhDAssociate Research Professor of PediatricsGeorgetown University Center for Child and Human Development

Psychometrics of the CCHPA7th National Conference on Quality Health Care for Culturally Diverse Populations (DiversityRX)Baltimore, MD

October 20,2010

National Center

for Cultural Competence

Presentation Created By: Kristyn Smith, BA

Psychometrics of the CCHPA• Background

▫ How was the CCHPA developed?• Methods

▫ Who participated in the study ?• Analyses

▫ What analyses was performed?• Results

▫ Is the CCHPA valid and reliable? ▫ What items are on the CCHPA?▫ What are the respondents scores on the CCHPA?

• Conclusion▫ Implications▫ Future studies

©Case Western Reserve University – Family Medicine Research Division & National Center of Cultural Competence – Georgetown University Center for Child and Human Development

Background

Cultural and linguistic competency is hypothesized to be a mechanism by which health care providers can reduce health disparities and provide quality and equitable care to racially, ethnically, culturally, and linguistically diverse populations.

Effective measures of CLC are needed to test this hypothesis.


Gap in Knowledge

Few valid, reliable tools exist to accurately assess the cultural and linguistic competence (CLC) of health care providers

Most tools that exist have not had any validation and reliability analyses performed, meaning they have little data grounding their tools as an actual assessment of CLC


Objective

To examine the validity and reliability of the Cultural Competence Health Practitioner Assessment (CCHPA)


The Cultural Competence Health Practitioner Assessment (CCHPA)129 item self-assessment of health care provider

cultural and linguistic competence developed by the National Center of Cultural

Competence(NCCC) ,Georgetown University Medical Center

Contained the following 6 content area subscales:▫ Values and Belief Systems▫ Cultural Aspects of Epidemiology▫ Clinical Decision-Making▫ Life Cycle Events▫ Cross-cultural Communication▫ Empowerment and Health Management


Cultural and Linguistic CompetencyThe CCHPA is based on 2 definitions:

Cultural Competency is “a set of values and principles, that are demonstrated through behaviors, attitudes, policies, and structures that enable effective operations across cultures”. (Cross 1989)

Linguistic competency is “the capacity to communicate effectively and convey information that is easily understood by diverse audiences, including persons of limited English proficiency and low literacy skills” (NCCC 2000)


Development of the CCHPA (2000-2002)


Psychometrics of the CCHPA Outline• Background


▫ Who participated in this study?• Analyses

▫ What analysis was performed?• Results

▫ Is the CCHPA reliable and valid? ▫ What are some items on the CCHPA?▫ What are the healthcare providers’ scores on the

CCHPA?• Conclusion

▫ Implications▫ Future studies


MethodsCCHPA was posted online from January

2005 to May 2008

8,600 respondents voluntarily completed the CCHPA▫2,504 self identified as MD/DO

(physicians), PA/NP (clinicians), or RN/LPN/BSN (nurses)


Study SampleCharacteristics of Respondents n* %

Gender Female Male

2176 325

86.9 13.0

Age<3031-4041-5051-60>60

72663166541358

29.1 25.3 26.716.62.3

Race White Black Asian Missing+

1816213145330

72.58.55.8

13.2Profession Nurse (RN, LPN, BSN) Clinician (P.A./N.P.) Physician (M.D./D.O.)

1864341299

74.4 13.7 11.9

*Due to missing data not all responses sum to the total of N=2504+Due to the small sample size of n = 40 respondents who identified as Pacific Islander, Native American/Alaskan Native, Latino, and Other were categorized as missing






▫ Is the CCHPA reliable and valid? ▫ What are some items on the CCHPA?▫ What are the respondents scores on the CCHPA?



Psychometric Analysis of the CCHPA

•Conducted using a 3-step process:

1)Factor Analysis

2)Item Response Theory (IRT)

3)Differential Item Function (DIF)



1)Factor Analysis • determines how items group together to

measure smaller ideas within a larger construct.

Example: Math Test

Addition Multiplication



2)Item Response Theory(IRT)• uses mathematical models (eg. Rasch) to provide a framework for evaluating the reliability and validity of a questionnaire. • IRT is based on the idea that the probability of answering an item correctly is a mathematical function of the person’s ability and the item difficulty.

Example: Math TestEasy question: 1+1 = 2More difficult question: (53 + 28) -18 = 135



3)Differential Item Function(DIF)• determines whether item difficulty differs by

groups(eg. gender, race, and profession).• the presence of DIF indicates an item in a

measure may be biased toward a particular group.

Example: Math Test1. If the number of yards on a football field were reduced by half, how long would the new field be?






▫ Is the CCHPA reliable and valid? ▫ What are some items on the CCHPA?▫ What are the respondents scores on the CCHPA?




1)Factor Analysis • determines how items group together to measure smaller domains with

in the larger construct. 9

2)Item Response Theory (IRT)• uses mathematical models(e.g. Rasch) to provide a framework for

evaluating the reliability and validity of a questionnaire.

• IRT is based on the idea that the probability of a correct response to an item, is a mathematical function of the person’s ability and the item difficulty.

25

3)Differential Item Functioning (DIF)• determines whether item difficulty differs by groups. (eg. gender, race,

and profession). The presence of DIF indicates an item in a measure may be biased toward a particular group.

28 62 of the original 129 items were removed

67 valid and reliable items underwent further analysis


Results• 3 factors responsible for 46% of the variance+.

Factor I : Knowledge of cultural and linguistic competence Factor II :Implementation of culturally and linguistically

competent careFactor III: Community Engagement

Factors Items Separation Index

Cronbach’s α

Factor I 24 13.82 0.92

Factor II* 21 11.59 0.88

Factor III* 23 22.64 0.92

* Includes 2 questions that loaded on factors 2 & 3 (so 2 questions counted twice)+Factors contributing to at least 10% of the variance are desirable


Factor 1: Knowledge of Cultural and Linguistic Competence

Item Difficulty Calibration

Logit Easiest Item

6. I know the impact of family dynamics on health care decisions (e.g., high value placed on decisions of elders, differing gender roles, role of extended family etc.).

-1.01

Most Difficult Item

1.I know the following values and belief systems for culturally diverse groups in my service area related to: g. traditional healing practices

1.11

Scored on a 4 point Likert scale of knowledge with 1 being Not at all, 2 barely, 3 fairly well, and 4 very well


Factor 2: Implementation of Culturally and Linguistically Competent Care


Logit Easiest Item

30. I modify treatment and discharge plans when serving individuals with disabilities and their families.

-0.7

Most Difficult Item

17. I integrate knowledge of diverse values and belief systems related to healing and wellness in treatment/discharge planning.

0.52Scored on a 4 point Likert scale of frequency with 1 being Never, 2 Seldom, 3 Sometimes, and 4 Regularly


Factor 3: Community Engagement


Logit Easiest Item

21. I keep abreast of current literature on research findings and evidence-based practices that report the complex array of biologic, economic, environmental, social and cultural factors that are known to contribute to health disparities among racial and ethnic groups.

-1.05

Most Difficult Item22.I have developed affiliations with traditional healers (e.g., curanderas, santeras, medicine men or women, espiritistas) to enhance health care access and utilization within the communities I serve.

1.38

Scored on a 4-point Likert scale of frequency and knowledge with 1 meaning “Never or Not at all” to 4 “Regularly or Very Well”, respectively


Scoring of the CCHPAEach factor is scored individually

Scores from LEAST competent to MOST competentFactor 1 (Knowledge) score range: 24 - 96Factor 2 (Implementation) score range: 21 – 84Factor 3 (Community Engagement) score range:

23 - 92


Factor 1 and 3 : Knowledge & Community Engagement Wright Item Person Map

Items

Respondents


F2: Implementation Wright Item Person Map

Majority of People

Majority of Items



Wright Item Person maps show that:

Factor 1 & 3 need additional items to better distinguish people with high and low ability

Factor 2 needs additional items in the difficult range


Relationship between Factors Scores:

Pearson Correlations**

FACTORSFactor 1

Knowledge

Factor 2Implementatio

n

Factor 3 Community Engagement

Factor 1

1 .568* .584*

Factor 2

.5 68 * 1 .665*

Factor 3

.584 * .665* 1

*p=.001**Pearson correlation >0.5 is a moderate association


Factor 1: Knowledge Scores by Respondent Demographics

Group N MeanF1 Score+

(24-96)

Std. Deviation

Effect Size

Race/Ethnicity*

White 1816 63.8 12.5 .42

Black 213 69.1 12.5

Asian 145 65.0 13.8 .31*Effect size ≥ 0.5 indicates moderate clinical effect+Mean scores may be slightly inflated due to the homogeneity of the sample


Factor 2: Implementation Scores by Respondent Demographics

Group NMean

F2 Score(21 -84)+

Std. Deviation

Effect Size*

Gender*Male 262 64.7 14.2 .30

Female 1614 68.1 12.4

*Effect size ≥ 0.5 indicates moderate clinical effect+Mean scores may be slightly inflated due to the homogeneity of the sample


Factor 3: Community Engagement Scores by Respondent Demographics

Group N Factor 3 MeanScore

(23-92)+

Std. Deviation

Effect Size*

AgeUnder 30 years 413 57.3 15.7 small31 – 40 years 58 58.6 17.341 – 50 years 726 53.3 15.0 .3151 – 60 years 631 51.7 14.7 .43Over 60 years 665 53.6 15.7 .30Profession

Physicians (MD/DO) 299 58.5 15.3 smallClinicians (PA/NP) 341 59.1 14.3Nurses (RN/LPN) 1864 52.1 15.2 .47

*Effect size ≥ 0.5 indicates moderate clinical effect+Mean scores may be slightly inflated due to the homogeneity of the sample


Factor Scores & Employer Sponsored Training – Variable Relationship ValidityEmployer Sponsored Training+

N F1 Score**

Effect Size#

F2 Score*

Effect

Size#

F3 Score*

Effect Size#

No Training 307 63 small 64 .32 47 .48

Training 2152

65 68 55

*Statistically significant at the .001 level**Statistically significant at the .002 level+45 respondents did not answer the training question#Effect size ≥ 0.5 indicates moderate clinical effect






▫ Is the CCHPA reliable and valid? ▫ What items are on the CCHPA?▫ What are the respondents scores on the CCHPA?



Implications: Cultural & Linguistic Competence as a Continuum

CCHPA contains 3 factors representing 3 domains▫Knowledge▫Implementation▫Community Engagement

Builds on Cross’ theory of cultural competency as a continuum


Implications: Cultural & Linguistic Competency

3 Level Continuum


Conclusion

The Cultural Competence Health Practitioner Assessment (CCHPA) is a criteria-related valid and reliable self-assessment of healthcare provider cultural & linguistic competence


Current & Future Research CCHPA is currently being used to:

assess health care provider CLC in multiple settings and disciplines (including cancer and emergency care settings), this data will be used to validate the new items being added to the tool

test its association with the newly developed Patient Assessment of Cultural and Linguistic Competence (PACLC)

In the future the CCHPA will be used to:test the association of cultural and linguistic

competency to patient outcomes to reduce disparities in medication adherence and patient trust


Funding InformationNational Institutes of Health

KL2 Multidisciplinary Clinical Research Training Program

National Cancer Institute RO1 Appraisal and Diagnostic Delay of Colon

Cancer

University Hospitals of Cleveland’s Case Medical Center Minority Faculty Development Fund Award


AcknowledgementsStephen Zyzanski, PhD

Psychometrician , Classical Theory ProfessorResearch Division Dept Family MedicineCase Western Reserve School of Medicine

Yong Gao, PhDPsychometrician, Item Response Theory & Differential Item Functioning Assistant ProfessorDept. KinesiologyBoise State University


Acknowledgements

Panel of Content Experts Joseph Betancourt, MD, MPHAlexander Green, MDDenice Cora-Bramble, MDMelanie Tervalon, MD, MPHRobert Like, MD, MSElena Rios, MD


AcknowledgementsSenior Investigators:

Sonja Harris-Haywood, MD, MSAssistant Professor Dept. Family Medicine, Research DivisionCase Western Reserve University, School of Medicine

Tawara Goode, MAAssistant ProfessorDirector, National Center of Cultural Competence Georgetown University Medical Center

Suzanne Bronheim, PhDAssociate Professor Dept. Pediatrics, Georgetown University Center of Child & Human DevelopmentNational Center of Cultural Competence, Georgetown University Medical Center


THANK YOU FOR ATTENDING!

To reach us email [email protected]


presenter: kristyn j. smith, ba research assistant case family medicine research division...

Documents

quality health care

health disparities

health ma

cchpa7th national conference

human developmentgap

human developmentpsychometrics

human developmentobjectiveto

equitable care