Gargi Roysircar

Nicole C. Spanakis, Catherine M. Iacuzzi, Connie L. Bindewald, Susan G. Danielson, Josefina Irigoyen, Michael Brodeur,

Mary Quinn, Michiko Ishibashi, Sarah Smith, and Marcelle Abela

Antioch New England Graduate School

Paper Presented at the 113th Annual Conference of the American Psychological AssociationFriday, August 19, 2005, 12.00-12.50 p.m., Washington, DC

Cultural Diversity Observer Rating Scale: Construct, Convergent, and

Discriminant Validity

Twelve trained raters analyzed 480 process notes from 48 doctoral clinical psychology trainees (10 session notes per trainee) who offered one-to-one conversation and mentoring services to culturally diverse individuals in their respective local communities. Raters identified the presence of cultural awareness themes and the use of multicultural interventions in these process notes. Interrater reliability was very high, Cohen’s Kappa .95. The themes and interventions identified by the raters were proposed as the item contents of a measure, the Cultural Diversity Observer Rating Scale (CDORS). Exploratory Factor Analysis using principal components with varimax rotation indicated four factors: Counselor Helping Behaviors; Counselor Awareness of Client Contexts, Counselor Perceived Barriers and Frustrations, and Diunital Reasoning. The four factors accounted for 37% percent of the variance in the CDORS. Trainees also provided self-report data on the Interpersonal Reactivity Scale, The Hope Scale, Racial Ideology, and the Attribution Questionnaire. Pearson correlations among the CDORS factors and these measures provided evidence of convergent and discriminant validity. Frequency of factors across 10 sessions, as well as in early sessions (1, 2, and 3) versus later sessions (8, 9, and 19) indicated significant differences on two CDORS factors, Counselor Awareness of Client Contexts and Diunital Reasoning. Trainees expressed more Connection thoughts for their clients than Disconnection thoughts across sessions.

Abstract

The Multicultural Interactions Project is an experiential component of a course, called Human Diversity and the Clinical Enterprise, which is taught in a doctoral clinical psychology program in the Northeast by the first author. Trainees are asked to either interview an individual or engage in a service-learning project with an individual who is culturally different from them. This activity is conducted within one’s local community. Trainees’ interviewees are culturally different from them in sociodemographic characteristics such as age, sex, class, culture, race, ethnicity, national origin, sexual identity, religion, and/or ability status. Diversity in multiple areas characterize most of the interviewees. The meetings are a mutual learning opportunity through conversations and friendly support and could have therapeutic effects for the interviewees (called “clients” henceforth). The purpose of these meetings is for trainees to gain awareness of (a) their own assumptions, values, and biases; to be responsive to (b) the worldviews of culturally different individuals; and to develop culturally appropriate intervention skills (Sue, Arredondo, & McDavis, 1993).

Description of a Community Activity in Multicultural Training

Components of the Multicultural Interactions Project

●10 meetings (approximately 60 minutes each) in one semester with an individual who is culturally diverse from the trainee

● Weekly process notes submitted to the professor based on these meetings

● Trainee self-reports on measures of personality: Empathy, Hope, Racial Ideology, and Causal Attribution

●Trainees are asked to submit weekly process notes to the instructor

●These process notes are meant to be an opportunity for the trainee to examine his or her experience of meeting with a culturally diverse interviewee

●Trainees are asked to focus on process issues (dyadic interactions and trainee intrapersonal reactions) within a meeting session

● Trainees write self-reflections in the process notes.

● For more information on the writing, goals, and outcome of the use of process notes, see:

Roysircar, G. (2004). Cultural self-awareness assessment: Practice examples from psychology training. Professional Psychology: Research and Practice, 35(6), 658-666. Roysircar, Gard, Hubbell, & Ortega (2005). Development of counseling trainees’

multicultural awareness through mentoring ESL students. Journal of Multicultural Counseling and Development, 33(1), 17-36

Process Notes

MethodSampleForty-eight trainees voluntarily and anonymously contributed the above-mentioned course materials for research. Each trainee contributed 10 process notes for 10 sessions. A total of 480 process notes were analyzed. Forty-eight completed questionnaires were received for each self-report personality measure, Interpersonal Reactivity Index (IRI, Davis, 1980), The State Hope Scale (Snyder, Sympson, Ybasco, Borders, Babyak, & Higgins, 1996), Racial Ideology (adapted from Multidimensional Internal-External Control Scale, Gurin, Gurin, Lao, and Beattie, 1969), and The Attribution Style Questionnaire (ASQ, Peterson, Semmel, von Baeyer, Abramson, Metalsky, & Seligman, 1982). The ASQ has not yet been analyzed.

● 12 coders held one-hour meetings from January 2004 through July of 2004, from September 2004 through November 2004, and from

February 2005 through April 2005 for a total of 50 hours of training. They received training in counselor connection and disconnection themes developed from the qualitative analysis of process notes from 6 counselor trainees in another study (Roysircar et al., 2005). The training involved the 12 raters in understanding and identifying the themes in process notes from trainees utilized for the current study. The meetings also provided an opportunity for peer consultation on the themes and the coding of sample process notes. Some consultations were also held on an individual basis outside of the meetings.

● See Table 2, Themes for Connection, Disconnection, and Interventions and their Examples. The themes provided the item contents for the

proposed measure, Cultural Diversity Observer Rating Scale (CDORS).

Training of Coders

● The coders kept a frequency count of each theme for each process note to determine the presence of the themes and to gain proficiency in recognizing the themes. Coders also tracked client responses, as reported in a process note, which were deemed relevant to a particular theme that was identified in a session process note.

● A rating system was developed to convert frequency counts to a 5-point Likert scale. The scoring reflected the presence or absence of a theme, as well as any response by the client. The scale used was as follows: 1 = theme not present; 2 = one theme with no client response; 3 = the same theme twice without a

client response or 1 theme with a client response; 4 = 3 or more times the same theme without a client response or the same theme twice with a client response; and 5 = 3 or more times of the same theme with a client response.

● A separate Likert scale was developed to rate the strength or degree of emphasis placed on interventions that the trainee reported in a process note. The scale

developed is as follows: 1 = weak; 2 = mild; 3 = moderate; 4 = strong; 5 = very strong.

● See the Appendix for the CDORS Rating Scale.

Rating of Themes

Two raters independently coded each case of 10 process notes. Subsequently, for each case the pair met to discuss their identification of themes and the Likert ratings given to the identified themes. They discussed their respective similar and dissimilar ratings and often through debate reached consensus. The 48 cases were coded by 12 coders, with each coder coding an average of four cases of 40 process notes. The average interrater reliability across the 12 raters (6 pairs) was: Cohen’s Kappa .95.

Interrater Reliability

● An EFA using principal components with a varimax rotation was conducted on 24 items, which were themes generated from trainee process notes for the Cultural

Diversity Observer Rating Scale. A criterion of .30 and above item loading was applied as the cut- off for item retention. A four-factor solution emerged according to the eigenvalue greater- than-one rule and the scree plot (36.96% of the total variance accounted for). Inspection of the pattern matrix revealed that all 24 items had the absolute loading of .30 or higher on one, and only one factor (See table 1 for the 24 loaded items). Thus all 24 items were retained.

● The variance contributed by each factor was: 12.84% (Factor 1); 8.43% (Factor 2); 8.03% (Factor 3), and 7.67% (Factor 4). Factor 1 (9 items) was called Counselor Helping Behaviors, which included interventions and counselor communication practices. The second factor (6 items) was called Counselor Awareness of Client Contexts, which included the counselor’s observations of the client’s cultural background, customs, and practices. The third factor was called Counselor’s Perceived Barriers and Frustrations (3 items), which tapped counselor negativity. The fourth factor was called Diunital Reasoning (6 items), which is a process skill in recognizing the reality inherent in two competing, even exclusionary worldviews, that frees the counselor to accept the culturally specific views of the client without having to abandon the counselor’s own worldview.

● See Table 1 for the CDORS item contents and factor loadings

Exploratory Factor Analysis

Empathy. Perspective taking was of interest to the study because we wanted to know whether psychology trainees could make empathy-related responses to clients’ references to cultural beliefs and behaviors. Perspective taking is the tendency to spontaneously adopt the viewpoint of another person. In a multifaceted measure of empathy, the Interpersonal Reactivity Index (IRI, Davis, 1980), Perspective Taking is one of four aspects of global empathy. Another subscale of the IRI is Empathic Concern, a dimension of empathy that represents other-oriented feelings of sympathy and concern for the difficulties of others. We expected three factors of the CDORS, Counselor Helping Behaviors, Counselor Awareness of Client Contexts, and Diunital Reasoning to be correlated strongly with both Perspective Taking and Empathic Concern. Personal Distress, the third subscale of the IRI, was expected to have strong negative correlations with the three previously mentioned CDORS factors because it taps into feelings of personal anxiety and unease in tense interpersonal situations. The fourth IRI subscale, Fantasy, which assesses the ability to empathize with fictitious characters was not of interest for the purpose of the current study. All three IRI subscales (7 items each) had moderate Cronbach’s alphas: .78 (Perspective Taking), .73 (Empathic Concern), and .77 (Personal Distress).

Convergent and Discriminant Validity of the CDORS

Hope. Hope represents expectation for goal attainment. Individuals with high hope approach goals with a positive emotional state and a sense of challenge; these individuals focus on success rather than failure. Low-hope persons experience negative emotional states and a sense of ambivalence; these individuals focus on failure rather than success. The State Hope Scale (12 items) (Snyder, Sympson, Ybasco, Borders, Babyak, & Higgins, 1996) operationally defines hope as goal-directed behavior. It is composed of two factors: agency and pathways. Agency is defined as an individual’s perceived capacity for starting and maintaining the actions necessary to succeed in reaching goals. The pathways factor taps into the individual’s ability to generate routes to those goals. The two subscales were collapsed for the current study and used as a full scale. The Cronbach’s alpha for the full scale was .81. We expected three factors of the CDORS, Counselor Helping Behaviors, Counselor Awareness of Client Contexts, and Diunital Reasoning to be strongly correlated with The State Hope Scale.

Racial Ideology. Race Ideology is a subscale of the Multidimensional Internal-External Control Scale (Multi IE Scale, Gurin, Gurin, Lao, & Beattie, 1969). In recognizing that external, sociopolitical forces operate against individuals who occupy minority status and who face racial, sexual, and economic discrimination, Gurin et al. hypothesized that minority groups may differ in their control beliefs compared to White American men who do not have to contend with such barriers. Race Ideology consists of 13 race-related items. For the purpose of the current study, only 13 external items were used (that is, 13 internal items were excluded) to assess trainees’ beliefs regarding the efficacy of exercising external forces in dealing with racism and discrimination experienced by African Americans (Sodowsky, Kuo-Jackson, Frey, & Corey, 1998). This adapted measure with a Likert scale was called Racial Ideology. Higher scores indicate higher externality. The Cronbach’s alpha for Racial Ideology was .87. We expected the three factors of the CDORS, Counselor Helping Behaviors, Counselor Awareness of Client Contexts, and Diunital Reasoning to be correlated strongly with Racial Ideology.

●We expected the subscale Counselor Perceived Barriers and Frustrations of the CDORS to have low to nonsignificant correlations with Empathy, Hope, and

Racial Ideology, and the three other subscales of the CDORS.

●See Table 3 for the Pearson correlations among all measures and the means and standard deviations of the full sample on these measures.

● See Figure 1 for the mean frequency of the CDORS factors in the process notes across 10 sessions. Diunital Reasoning and Counselor Awareness of Client Contexts were more frequently referred to in the process notes than Counselor Helping Behaviors and

Counselor Perceived Barriers and Frustrations.

● See Figure 2 for Connection and Disconnection thoughts expressed in process notes across 10 sessions. Trainees made significantly more Connection thoughts than Disconnection thoughts in their process notes across sessions.

● See Table 4 for ANOVAS on frequency of factors between early sessions (Sessions 1, 2, and 3) and later session (Sessions 8, 9, and 10). Counselor Awareness of Client Contexts showed significant difference at the .01 level, with trainees being more concerned about knowing the client’s cultural practices in the early sessions than in the later sessions. Diunital Reasoning showed significant difference at the .05 level, with trainees practicing more diunital reasoning in the later sessions than in the earlier sessions.

● See Table 5 for means and standard deviations for frequency of factors in early and later sessions.

Between Session Analyses of Process Notes

Another set of process notes contributed voluntarily and anonymously by trainees will be similarly analyzed in order to perform a confirmatory factor analysis on the factor structure derived from the EFA. This will be Study 2. Convergent and discriminant validity will be further investigated in Study 2 by studying the relationships of the CDORS with the Multicultural Counseling Inventory (Sodowsky, Taffe, Gutkin, & Wise, 1994), the Multicultural Social Desirability Scale (Sodowsky et al., 1998), and the Working Alliance (Tracey & Kokotovic, 1989).

Future Analyses

The Cultural Diversity Observer Rating Scale (CDORS) may serve the need expressed in the empirical literature for measures of observed multicultural competencies in counselors and trainees. It could serve as a complement to several self-report measures of counselor multicultural counseling competencies that are currently available. Clinical supervisors could use the CDORS to provide evaluation and supervisory feedback, while counselors could answer the self-report measures for their personal assessment and understanding. Because the CDORS is a short measure (24 items), it serves as a useful outcome measure in studies of counselor effectiveness. The CDORS appears to be psychometrically sound with acceptable reliability and validity. We request supervisors, community clinics, university counseling centers, and instructors of multicultural courses to use the CDORS and test its clinical utility. The first author will be happy to share the CDORS and its codebook upon the completion of the instrument’s development. You may contact the first author at: Gargi Roysircar, Professor of Clinical Psychology, Director, Multicultural Center for Research and Practice, Department of Clinical Psychology, Antioch New England Graduate School, 40 Avon Street, Keene, NH 03431-3552, Ph. (603) 357-3122 ext. 342, FAX (603) 357-1679; g_roysircar-sodowsky@antiochne.edu, http://www.multiculturalcenter.org.

Conclusion

Table 1.Exploratory Factor Analysis of the Themes Found in the

Process Notes and Cronbach’s Alphas for Factors

Item # Item Content Counselor Helping Behavior

Counselor Awareness of

Client Contexts

Counselor Perceived

Barriers and Frustrations

Diunital Reasoning

_____________________________________________________________________________________________

Int 1. ESL Tutoring.67 -.01 .05 -.06

Int 2. Sharing objects (food, pictures) .40 .12 .01 .16

Int 5. Client giving gifts .73 .11 .03 -.06

Int 6. Joking/humor to facilitate interaction .47 .13 -.12 .10

Int 7. Advocacy/Social Justice action .40 .01 -.16 .03

Int 8. Power Differential and equalizing power .32 -.07 .21 .21

Int 13. Taking chance/risk .43 .10 .15 .13

Int 15. Assuring confidentiality .74 .05 -.00 -.03

Int 16. Setting goals .69 .17 -.02 -.10

Factor 1 (9 items) Cronbach’s Alpha .68

Factor Loading _____________________________________________________________________________________________________________

_____________________________________________________________________________________________________________

C2_C3. Cultural Empathy: Cognitive and Affective .10 .35 .26 .29

Int3. Information exchange on customs and values .06 .52 .11 .21

Int 4. Talking about family members, behaviors, practices .08 .57 .05 .33

Int 9. Acculturation of client .01 .67 -.04 -.15

Int 10. Racism experience of client .16 .53 -.10 .06

Int 17. Client tells a story .12 .57 .09 -.14

Factor 2 (6 items) Cronbach’s Alpha .62

D1. Environmental barriers .03 -.03 .72 -.11

D2. Frustration with Environmental Barriers -.11 .06 .68 -.10

D3. Unintegrated Differences -.02 .01 .61 .14

Factor 3 (3 items) Cronbach’s Alpha .70

C1. Differences Integrated .01 -.02 -.02 .68

C4. Counselor Self-Disclosure -.14 .07 -.10 .57

C5. Counselor Self-Reflection .33 -.18 .41 .41

D4_D5. Stereotypes and Overgeneralization .03 .17 .22 .33

Int11. Worldviews of the client and counselor .07 .33 .37 .42

Int12. Discussion of comfort level .25 -.01 -.05 .41

Factor 4 (6 items) Cronbach’s Alpha .53

Note. N = 468 process notes (listwise deletion of missing values). Loadings in bold designate to which factor an item was assigned for subsequent analyses. Int = Intervention; C = Connection; D = Disconnection. Chronbach’s alphas were calculated using ratings across sessions for each factor. Since sessions were situation specific and there was a different case for each session, Cronbach’s alphas may have been lowered with changing data.

Number Theme Example

C1 Differences Integrated I have not experienced political oppression, so I can only imagine and ask questions. If I don’t understand, I ask him to help me learn what it was like.

C2 Cultural Empathy Cognitive

It was a reminder to me that it is important to never take anything for granted when working with a client because there are an infinite number of factors contributing to the client’s worldview.

C3 Affective Connection With Client’s Culture

I enjoyed hearing about her family of origin because it gave me insight into who she was and the value she placed on the family.We found lots of articles concerning Armenia and Armenian culture. I printed out lots of articles and gave a copy to S. She was so content and thanked me for it.

C4 Counselor Self-Disclosure

I shared with P about how my family moved a lot when I was a kid. It became increasingly difficult to keep ties to the last town along with the current social needs of the new town.

C5 Counselor Self-Reflection

I definitely felt ignorant asking her questions that probably reflect things I should have learned in a history book in middle or high school, but at the same time, it is possible that our history books don’t present the truth with regards to Arab nations. I realized through that process that one of the reasons I may have asked her about her bias was to make me feel more comfortable about mine.

Table 2.Themes of Connection, Disconnection, Interventions,

and Their Examples

D1 Environmental Barriers I had not known this before; I actually had not given it a second thought given that R has such a good vocabulary, but R translates everything he hears in English into Portuguese so he can understand it.

D2 Frustration with Environmental Barriers

I tried to relate that she had learned a great deal of English and could communicate quite well. However, she said to me that it is very honorable that I felt that way, but that it didn't help her in her written work or when she had to speak in class.

D3 Unintegrated Differences

In some ways it was difficult for me to relate to her feelings of awkwardness or embarrassment at being bilingual.She explained that if they were to have children, she would be interested in teaching them to speak Chinese. However, since B has really shown no desire to learn, she said that she probably would not teach her children. My own feelings about this were very strong; I felt that she should teach her kids regardless of whether or not he chose to learn.

D4 Overgeneralizations of Similarities

I consider myself a culturally sensitive person; however I think I still have the tendency to overlook differences among people in favor of universal traits which we all share.Although we both look very different physically we have so much in common.

D5 Stereotypes I know my family is confrontational (at least the Italian side!).As a result of this statement, I made a defensive comment that I couldn’t believe that women actually put up with that role and that his family was obviously very traditional.

Examples of InterventionsNumber Theme Example

Int1 ESL Tutoring My interviewee brought a paper she had written to this meeting. She asked me to critique her work and make corrections. She is still working on her English, both verbal and written, and wanted to make sure her work was correct.”

Int2 Sharing objects This week my interviewee invited a mutual friend and me for a sushi dinner at her apartment

Int3 Information Exchange on Customs and values

My interviewee’s education in Kenya was actually quite structured, and school was something he stated was much more valued there than in United States culture.

Int4 Talking about family members, behaviors,

practices

●We spent part of this session talking about gender roles. We compared and contrasted expectations for both men and women in Korea and here in the United States. I was pleased to learn many new interesting details. I also got the chance to hear more about her family life back in Korea. This gave me the opportunity to understand my interviewee more in terms of her family and cultural dynamics.

Int5 Client giving gifts My interviewee invited me to Japan this summer!

Int6 Joking/Humor to facilitate interaction

We did have a good chuckle about the irony of the United States monitoring other countries' elections when we can't even figure out how to do an accurate recount in our own.

Int7 Advocacy/Social Justice actions

This interaction caused me to consider more proactive ways that I can react to this discrimination, including advocating for cultural diversity awareness in my home town.

Int8 Power Differential and equalizing power

I am having a hard time with this because I am unsure of when I should enter a conversation because I feel like he has to have control. There is a lack of balance in our conversations.

Int9 Acculturation of client Having been here seven years now, my interviewee feels that he is used to life in the US. On a recent visit home to Greece, he realized that, in some ways, he is more accustomed to life here than he is to some customs in his own country.

Int10 Racism experience of client

My interviewee further described his views on racism and how he maintains an optimistic view on life in order to deal with everyday struggles.”

Int11 Worldviews of client and counselor

I acknowledged that my comment was perhaps related to my Western, individualistic worldview. My interviewee was kind of direct in her agreement, indicating that in her culture there is a good deal of support for individual development.”

Int12 Discussion of comfort level

I made my interviewee aware that if anytime I offend him that he should tell me.

Int13 Taking a chance/risk I had mentioned in an earlier process note that he was always apologizing to me as if he was insulting me personally when discussing negative aspects of White culture. I finally told him that it was bothering me that he did that throughout all of these sessions.

Int14 Importance of asking questions/hypothesis-

testing

I not only learned more about my interviewee's level of acculturation today, I also learned about the importance of asking questions and testing assumed knowledge. Had I not asked, I would have missed out on very important information.

Int15 Assuring confidentiality I explained the issue of confidentiality and made it clear he was in no way obligated to take part in this project and that he could withdraw his participation at anytime if he chose to.”

Int16 Setting goals My first meeting with my interviewee encompassed conversation about what this project represents for each of us.

Int17 Client tells a story He stated that one day walking home from school he saw a lion, and it scared him so much that he ran all the way home”

Perspective Taking 24.94 5.09 -- . 97** -.96** .87** .87** .64** .85** .25 .83** Empathic Concern 28.53 3.93 -- -.96** .87** .90** .67** .87** .21 .82**

Personal Distress 18.39 4.73 -- -.81** -.87** -.59** -.83** -.20 - .82**

Hope 12.31 1.41 -- .84** .71** . 71** .24 .72**

Racial Ideology 44.39 10.98 -- .61** .83** .17 .74

Factor 1: 2.34 1.90 -- .46** .09 .36* Counselor Helping Behaviors Factor 2: 7.66 3.89 -- .10 .58** Awareness of Client Contexts Factor 3: 3.84 0.96 -- .39** Perceived Barriers, Frustrations Factor 4: 5.54 2.59 -- Diunital Reasoning ______________________________________________________________________________________________________________________________

M SD Perspective Empathic Personal Hope Racial Factor 1 Factor 2 Factor 3 Factor 4 Taking Concern Distress Ideology Counselor Awareness Perceived Diunital

Helping of Client Barriers, Reasoning Behaviors Contexts Frustrations

Table 3.

Pearson Correlations of the CDOR with Empathy, Hope, and Racial Ideology: Convergent and Discriminant Validity

______________________________________________________________________________________________________________________________

___________________________________________________________________________________________________________________

Note. * p < .05; ** p < .012-Tailed

Sums of Squares

DfMean

SquareF Sig.

Factor 1Counselor Helping Behavior

Between Groups 4.50 1 4.50 .43 .51

Within Groups 2862.31 271 10.56

Total 2866.81 272

Factor 2Counselor Awareness of Client Contexts

Between Groups 271.25 1 271.25 9.2 .003**

Within Groups 7989.13 271 29.48

Total 8260.38 272

Factor 3Counselor Perceived Barriers and

FrustrationsBetween Groups .59 1 .59 .30 .58

Within Groups 529.97 271 1.96

Total 530.56 272

Factor 4Diunital Reasoning

Between Groups 66.41 1 66.41 4.40 .037*

Within Groups 4091.36 271 15.10

Total 4157.77 272

ANOVAS on Frequency of Factors Between Early Sessions (Sessions 1, 2, and 3) and Later Sessions

(Sessions 8, 9, and 10)

Note. *p <.05; **p <.01

Table 4.

Factor 1Counselor Helping

Behavior

Factor 2Counselor

Awareness of Client Contexts

Factor 3Counselor Perceived

Barriers and Frustrations

Factor 4Diunital

Reasoning

Early Sessions(1 through 3)

MeanSD

2.193.38

8.565.16

3.801.42

9.233.74

Later Sessions(8 through 10)

MeanSD

1.933.09

6.575.71

3.711.38

10.224.04

Table 5.Means and Standard Deviations of Frequency

of Factors for Early and Later Sessions

Note. Trainees referred less to the cultural practices of clients in later sessions than in the early sessions when they were becoming acquainted with the cultural background of clients (Factor 2; also graphed in Figure 1). There was an increase in trainees’ acceptance of client and counselor competing worldviews in later sessions than in earlier sessions (Factor 4; also graphed in Figure 1).

Figure 1. Mean Frequency of Factors in Process Notes Across Sessions

0

2

4

6

8

10

12

Sessi

on 1

Sessi

on 2

Sessi

on 3

Sessi

on 4

Sessi

on 5

Sessi

on 6

Sessi

on 7

Sessi

on 8

Sessi

on 9

Sessi

on 10

Diunital Reasoning

Counselor Awareness of Client Context

Counselor Perceived Barriers and Frustrations

Counselor Helping Behavior

FACTORS

Figure 2. Connection and Disconnection in

Process Notes Across Sessions

0123456789

10

Sessi

on 1

Sessi

on 2

Sessi

on 3

Sessi

on 4

Sessi

on 5

Sessi

on 6

Sessi

on 7

Sessi

on 8

Sessi

on 9

Sessi

on 10

Connection

Disconnection