ORIGINAL ARTICLE

A Comprehensive Model for Developing and EvaluatingStudy Abroad Programs in Counselor Education

Syntia Dinora Santos

# Springer Science+Business Media New York 2014

Abstract This paper introduces a model to guide the process of designing andevaluating study abroad programs, addressing particular stages and influential factors.The main purpose of the model is to serve as a basic structure for those who want todevelop their own program or evaluate previous cultural immersion experiences. Themodel is based on the existing literature, actual experience in designing short-termprograms, and on the theoretical evaluation principles of Appreciative Inquiry (Dunlap 2008)and Logic Models (Renger and Titcomb 2002).

Keywords Studyabroad . International counseling .Cultural competence . Training .Counseloreducation

Introduction

The variety of settings and diverse populations, and the increased mobility of people around theworld requires mental health professionals who can effectively serve clients from cultural back-grounds different from their own. Training institutions have the responsibility and challenge ofleading future generations of counselors through a path of self-discovery and development towardscultural competence. Sue and Sue (2003) describe culturally competent professionals as those whoare actively involved in the process of becoming aware of themselves, aware of other’sworldviews, and effective in choosing appropriate interventions, strategies and techniques.

In order to develop awareness of others, exposure seems to be a fundamental first step togain knowledge and understanding. Giving special attention to the process of exposure andreflection during training may offer an important support to the overall cultural competence ofcounselors in training. One of the teaching methods used to support cultural competence byhelping connect theory to practice is experiential learning (Arthur and Achenback 2002; Jaoko2010). Experiential learning can be used to raise awareness, challenge personal frameworks,and help develop cultural empathy (Arthur and Achenback 2002). It has been described byJaoko (2010) as a tool for developing appreciation of diversity. Furthermore, he specificallymentioned that study abroad programs, when well designed and combined with academic

Int J Adv CounsellingDOI 10.1007/s10447-014-9210-7

S. D. Santos (*)North Carolina State University, Raleigh, NC, USAe-mail: sdsantos@ncsu.edu

learning, field activities and reflections, represent an important experiential learning strategyfor developing cultural competence.

Study abroad experiences in conjunction with teaching, research, and service have beenwidely used to expand the learning experience and promote mutual understanding andcollaboration among nations (Brockington et al. 2005; Leung et al. 2009; Santos Figueroa2014). Study abroad programs may also play an important role in increasing commitment tosocial justice. They can provide opportunities to connect with the reality of others, their needsand their strengths, and open opportunities for future collaborations and mutual support(Cordero and Negroni Rodríguez 2009).

Study abroad programs should be carefully designed and evaluated to ensure quality,sustainability, and impact. This article introduces a comprehensive model for designing andevaluating study abroad programs. The main purpose of the model is to serve as a basicstructure for those who want to develop their own program or evaluate previous endeavours.This comprehensive program evaluation framework was created by analyzing the existingstudy abroad literature in the helping professions and actual experiences of conducting a short-term study abroad program to Honduras. Descriptions of how the model was used, lessonslearned, and recommendations for others are presented.

Study Abroad Program to Honduras

Education abroad is based on the conviction that students who have experienced a foreignculture will benefit greatly academically and culturally in ways that cannot be accomplished intheir home country, and that as a result of that experience, students will be better prepared for thechallenges of a globalized world (Brockington et al. 2005). As a result of this conviction, staff ata graduate-level counselor education program in the southeastern United States designed astudy abroad program to support the cultural competence development of their students.

Although the students in the counselor education program had successfully engaged inpersonal immersion experiences organized by themselves and their academic advisors asindependent studies, or attended other existing study abroad programs, the university involvedhad not previously offered a structured opportunity to participate in a study abroad programthat was tailored to the particularities of graduate students in the counseling field. The idea ofcreating such an opportunity emerged organically but also intentionally as a consequence ofthe presence of an international student in the counselor education program who was also acounselor educator in Honduras. The subsequent study abroad program emerged with theprimary idea of making connections and providing an opportunity to establish a relationshipbetween countries with the common goal of supporting the cultural competence developmentof counseling students both in the U.S. and in Honduras.

The process of creating this faculty-led short-term study abroad program was initiated in2011 and implemented in 2012. The program was designed with the express purpose of:

Providing learning opportunities that will prepare counseling professionals for their roles asmulti-culturally competent counselors, educators, leaders, and advocates for social justice.The program constitutes a comprehensive cultural immersion experience that promotes self-awareness, mutual understanding and reflection beyond traditional pedagogical methodscurrently employed in the Counselor Education Program. It provides amulti-lens view of thecounseling field in terms of diversity, cultural and social differences, knowledge, andpractice. Hence, the objectives of the program spotlight the three major components ofhigher education: education, research, and service. (Santos Figueroa 2014, p. 195)

Int J Adv Counselling

The program was designed after a careful review of the available literature and by followingthe guidelines of the participating University’s study abroad office. The implementation of theprogram took place in May 2012, involving a 13-days visit to the destination countryHonduras. Eleven master’s-degree-level students participated in the program. Eight of theparticipants were in a counselor education program, two were alumni from such a program,and one was enrolled in an education master’s program. Program participation was openednationwide, and the participants were not only from the lead institution but also fromuniversities in North Carolina, Philadelphia, and Kansas. The program included pre-departure orientation through on-line communications and one face-to-face session, 13-daysin the destination country Honduras, and one face-to-face debriefing meeting in a local LatinAmerican restaurant upon return to the U.S.

Time in Honduras was structured across five basic components (a) academic activities, (b)site visits, (c) research, (d) service, and (e) reflection. Academic activities included lectures anddiscussions regarding topics such as history, culture, education and counseling in Honduras.Site visits were organized to acquire a more holistic view of Honduras and to observe diversitywithin the country. Participants were able to visit different schools (private and publicinstitutions), college campuses in different cities, and different regions of the country includingthe capital city and some tourist and cultural sites. The research component of the programinvolved participation in a counseling conference in Honduras, where students were requiredto attend and present using local translators. Regarding service, students were invited to jointheir Honduran peers in a mental health campaign at the host institution, where they were ableto interact with the university community. Reflections were extremely important throughoutthe process; students were asked to maintain journals and participate in daily group meetings toreflect on activities and events. Evaluation was considered fundamental in the process, and itwas carried out throughout every stage of the program.

Considering that this was the first study abroad program in counselor education organizedby the leading institution, and that the evaluation process was self-conducted, the process ofreviewing available literature and selecting a theoretical framework that fitted the needs of thestudy abroad experience was instrumental in the development of the program. AppreciativeInquiry (Dunlap 2008) and Logic Models (Renger and Titcomb 2002) of evaluation wereselected as the theoretical foundations to systematize the experience and to create a model thatguided the process of designing and evaluating the study abroad program. AppreciativeInquiry centers on identifying the program’s strengths and providing an analysis of the stateof the program and the improvements needed to achieve its goals and the stakeholders’ vision(Coghlan et al. 2003). Logic models support the construction of a graphic representation of theprogram and the process of design and evaluation (Renger and Titcomb 2002). This theoreticalframework, acquired experiences, and a comprehensive review of the literature were thefoundations for the model that was developed.

Literature Review

A review of published and unpublished study abroad program literature reflected the range ofstructures available in counseling and related fields. Programs varied in location, duration,purpose, and teaching and evaluation strategies. Some programs declared their emphasis asbeing on cultural competence in general (e.g., Anderson et al. 2006; Jaoko 2010), others inadvocacy and social justice (e.g., Cordero and Negroni Rodríguez 2009; Fairchild et al. 2012);some had a service learning focus (e.g., Tomlinson-Clarke and Clarke 2010; West-Olatunjiet al. 2011), a few were specifically designed for counseling (e.g., Jurgens and McAuliffe

Int J Adv Counselling

2004; McDowell et al. 2012), and there were others centered on practicum or internshipexperiences abroad in which students were able to have hands-on counseling experiencesthrough counseling hours in the host country (e.g., Alexander et al. 2005).

Programs included several teaching and evaluation strategies to achieve proposed objec-tives. There were three activities that were frequently mentioned in the reviewed programs: (i)lectures (Cordero and Negroni Rodríguez 2009; McDowell et al. 2012), (ii) site visits (Corderoand Negroni Rodríguez 2009; Fairchild et al. 2012; Jaoko 2010; McDowell et al. 2012; West-Olatunji et al. 2011), and (iii) journal reflections (Alexander et al. 2005; Jaoko 2010; Jurgensand McAuliffe 2004; West-Olatunji et al. 2011). All programs included orientation anddebriefing sessions that varied in length and content. In addition, some of the programsincluded standardized normative measures, such as the Intercultural Development Inventory(Anderson et al. 2006) and the Multicultural Awareness/Knowledge/Skills Survey(Fairchild et al. 2012), to assess cultural competence development.

Short-term programs tend to be more attractive for their duration and cost, allowingstudents to fit them more easily into their curriculum and even perhaps participate in morethan one such experience. Short-term programs are typically one to eight weeks in length,usually faculty-led, and sponsored by a university. Programs usually are held in one specificcity, one country, or at multiple sites (Brockington et al. 2005).

The latest standards of good practice, specifically applicable to short-term educationabroad programs, were published by the Forum on Education Abroad (2009) as acomplement to their more general standards of practice. The standards were created toensure essential elements of quality that would maximize student learning and develop-ment, while assuring safety and wellbeing. The Forum on Education Abroad is a “non-profit association recognized by the U.S. Department of Justice and the Federal TradeCommission as the Standards Development Organization for the field of educationabroad” (http://www.forumea.org/). The creation of the Forum was initiated in 2000,their first annual meeting was held on 2002, and their first annual conference was on2004. The latest strategic plan of the association was approved in 2012 and is intended toguide their activities through 2015 (http://www.forumea.org/Strategicplan2012.cfm).

The Forum on Education Abroad (2009) established nine basic standards of quality forshort-term education abroad programs:

(a) Mission, objectives, and purpose; the program relates to the education abroad institu-tional mission and has well-defined objectives.

(b) Student learning and development; the program is reviewed according to its statedpurpose and outcomes.

(c) Academic framework; the program follows the institutional academic framework andpolicies for education abroad.

(d) Extra-academic framework; the program includes clear policies for non-academic aspectsof the experience abroad.

(e) Preparation for the learning environment abroad; the program provides comprehensivepreparation and support for the experience abroad.

(f) Student selection and code of conduct; the program maintains fair and appropriatepolicies for student selection and conduct.

(g) Organizational and program resources; the program defines adequate financial andpersonnel resources.

(h) Health, safety, and security; the program has established policies, procedures and trainingto ensure health, safety, security, and risk management.

(i) Ethics and integrity; the program follows ethical principles and practice.

Int J Adv Counselling

Theoretical Evaluation Framework

Appreciative Inquiry (AI) (Dunlap 2008) evaluation emphasizes strengths. It in-volves a systematic and participatory discovery of the process and expectations ofany program and lays out the state of the program at the time of the evaluation, aswell as its desired goals for the future. While Logic Models (LM) (Renger andTitcomb 2002) constitute a visual representation of the processes and expectationsfor any program, they specifically help program managers and evaluators to have aclearer understanding of the rationale and functioning of the program. Both of theseideas contributed to the creation of a comprehensive, systematic, flexible, and user-friendly model.

Appreciative Inquiry

Instead of focusing on issues to be solved, AI first centers on discovering what isworking well. The process then shifts to envisioning what it may be like if thosestrengths were enhanced and occurred more often in the daily life of the organiza-tion, institution, or program. The processes of discovering strengths and ofenvisioning the future leads to the implementation of desired changes. Hence, themost powerful strengths of the AI approach are the level of engagement of theparticipants and its focus on their positive experience. It is important to note that AIdoes not deny problems but, instead, it looks at them from a more constructiveperspective (Coghlan et al. 2003).

The process of conducting AI has been captured in what is called the 4-D cycle: (a)Discovery, appreciating and valuing the best of what the program is; (b) Dream, envisioningwhat the program may be; (c) Design, co-constructing the future of what the program shouldbe; and (d) Destiny, learning, empowering, and sustaining the future of what the program willbe (Cooperrider et al. 2003).

For the study abroad program to Honduras, using the AI foundations involved theparticipation of all stakeholders, fluid communications among all components and,more importantly, a strong motivation to continue the development of the relationshipsand improve the program for the future. Focusing on the positives allows participantsto remain motivated and ready to work on challenges as they are presented, withoutbecoming discouraged. The foundations of AI align well also with the internationalperspective of the program. The positive, participatory and constructive nature of AIallows the identification of the value of the program for all stakeholders, and therecognition of the strengths that every stakeholder brings to the program. Thesefoundations may be instrumental in establishing more equalitarian and positive inter-national relationships.

Appreciative Inquiry constitutes the search for the best in people, their organiza-tions, and their programs. AI core principles include: (a) the constructionist principle,suggesting that social knowledge and interactions create the future, (b) the principle ofsimultaneity, referring to the concurrent processes of inquiry and change, whichshould not be separated, (c) the anticipatory principle, that imagining the future willguide the steps to take in the present, (d) the positive principle, acknowledging thatchange requires positive affect and social bonding, and (e) the wholeness principle,referring to the importance of a participatory process that includes all stakeholders(Dunlap 2008).

Int J Adv Counselling

In addition to these core principles, Hammond (1996, p. 91) presented eight AI assumptionsthat guide the process of using AI, changing as they do the usual perspectives of the meaningand purpose of evaluation:

1. In every society, organization or group, something works.2. What we focus on becomes our reality.3. Reality is created in the moment, and there are multiple realities.4. The act of asking questions of an organization or group influences the group in some way.5. People have more confidence and comfort to journey to the future (the unknown) when

they carry forward parts of the past (the known).6. If we carry parts of the past forward, they should involve what is best about the past.7. It is important to value differences.8. The language we use creates our reality.

Logic Models

Incorporating the theory of Logic Models with the foundations of AI enables a betterrepresentation of the process followed in developing and evaluating a study abroad program.The result of this combination has been captured in the graphic representation of the presentprogram design and evaluation model (see Fig. 1).

In addition, Logic Models utilize terms that are more common in the evaluation arena and,therefore, are clearer to the readers. Constructing a Logic Model creates a framework tovisualize the overall picture of the program by incorporating its basic elements. Renger andTitcomb (2002) referred to the importance of not trying to include everything in a logic model.Creating a complex image with a large number of boxes may cause lack of clarity about theunderlying rationale of the program and the incorporated elements of evaluation.

Logic models are a fundamental step in program evaluation. The main features of logicmodels are the visual representation of the program rationale and the relationship between theelements of evaluation and the rationale (Renger and Titcomb 2002). Logic Models are asystemic approach that portrays the path towards a desired reality. They are useful to thoseaiming to plan, manage, or evaluate the connection between the resources needed for aprogram and what it seeks to accomplish. Logic models are used to understand what is neededto achieve the desired outcomes; they help to identify factors that could impact the programoutcomes and partners that may be influential for the success of the program. Logic models arevaluable complements to formative and summative evaluations, helping to identify interme-diate results that will contribute to the achievement of ultimate goals. They are also essentialfor visualizing the process and sequence of events, identifying alternative strategies toaccomplish desired end-results, and serving as a tool to facilitate communication betweenstakeholders (Millar et al. 2001).

Building a traditional logic model involves five major stages: (a) collecting relevantinformation, (b) describing the problem and the context, (c) defining the elements of the logicmodel in a Table, (d) constructing the logic model, and (e) verifying the model (McLauglin andJordan 1999). The process and elements were not used in their traditional form in the developedmodel for study abroad programs. They were used as a foundation to build the model. Both AIand Logic Models together constituted a basic structure for program development and evalu-ation, offering at the same time the flexibility required to add new components and perspectivesneeded to evaluate the program as it evolved to more advanced stages. For the study abroad

Int J Adv Counselling

program design and evaluation model these frameworks merged to create a theoretical foun-dation, to determine a new perspective of approaching evaluation, and to create a visualrepresentation of the process of designing and evaluating a study abroad program.

Program Design and Evaluation Model

The design and evaluation process of creating an effective study abroad program entails anintentional reflection process in a series of stages. The overall process has been summarized inFig. 1, which was created by the author to serve as a guideline for future counseling studyabroad programs. The center of the model is the interaction and relationship building amongcultures, followed by five stages: (a) needs assessment, (b) design, (c) implementation, (d)outcomes assessment, and (e) implications. Fig. 2 presents an example of how the model wasused in the study abroad program to Honduras. Although the stages appear to have a logicalorder of presentation, they are all interconnected and are continuously influenced and changing

Fig. 1 presents the necessary elements required to implement the program evaluation model. The middle circlesrepresent the stages of designing a study abroad program. Every stage is followed by the components of theevaluation process: objectives, evaluation question, outcome indicators and performance indicators. Additionally,external factors that may influence the process are placed outside the model

Int J Adv Counselling

according to the process, acquired knowledge, and experience. Other external influencingfactors, such as environment and context, organizational factors, socio-political factors, previ-ous experience and knowledge, external threats or events, available resources and economicfactors, may also influence the stages.

Several factors impact the process making it a dynamic rather than linear force. There isactive movement throughout the planning and evaluation of the program and there is alwaysthe possibility of expansion, change, and improvement. The cultures of the participants and theimportance of the relationship between the visitors and the representatives of the host countryare highlighted at the center of the model. It is also important to acknowledge that the designand evaluation process are greatly influenced by the stakeholders’ needs and interests, the timeframe and resources available, and the experience and knowledge of the organizers.

Relationships at the Center

The interest and commitment of both the visitors and the representatives of the host country arethe foundation for success in any international program. Building a growth fostering

Fig. 2 constitutes a graphic example of the program evaluation model applied to a study abroad program incounselor education conducted in Honduras in 2012

Int J Adv Counselling

relationship where both parties contribute, grow, and make the relationship a priority isessential for the program’s success. Relational Cultural Theory (Duffey and Somody 2011)explains that, as a result of such relationships, each part feels a greater sense of “zest” (vitality,energy) and is more capable to act. Growth fostering relationships will help each one of theparticipants to develop a more accurate picture of themselves and of others, a greater sense ofworth, a stronger connection with each other, and the motivation to develop connections withothers outside that relationship.

Specific to the program in Honduras, there were two essential factors that supported thestrength of the relationship between the visitors and the representatives of the host country.Firstly, having a person from the host country as one of the organizers offering knowledge andmediating cultural differences in every stage was very significant. Secondly, choosing a partneruniversity that had a counseling program and was willing to establish a collaborative relation-ship allowed the experience to be more significant and impactful. Opportunities, such as peerinteractions, conference participation and counseling discussions, transformed the plannedprogram into a meaningful life experience and one involving lasting relationships.

Ng and Noonan (2012) included in their definition of internationalization of counseling theneed for a professional collaborative multidimensional movement, developed through equalpartnerships, and with culturally respectful attitudes. Cross-cultural relationships are indeedmultidimensional. Sue (2001), in his Multidimentional Model of Cultural Competence(MDCC), included three dimensions that help to understand the complexity of acquiringcultural competence, and also the complexity of relationships: (a) racial and cultural attributes,(b) components of cultural development, and (c) foci of cultural development. In buildinginternational relationships these dimensions should also be considered. The culturalcharacteristics, similarities, differences, interest, and readiness of both parties to develop aprofessional relationship are indispensable for success.

The cultural competence of the leaders of the programs in both contexts will have animportant impact on the development of the relationship, the design and implementation of theprogram, and the outcomes of the experience. The attitudes, knowledge, and skills of theleaders will be reflected in the program and will make the process move more smoothly, or not.They will also be key elements in the design of the program because the main goal is tosupport the cultural competence development of the participants. Teaching and evaluationstrategies should align with this goal.

In addition, the foci of cultural development is also relevant to the process. Sue (2001)included individual, professional, organizational, and societal levels of foci. Every stakeholderis important in the process, from the participants to the leaders and the administrators. Eachone plays an important role in building the relationship between the leaders of the visitor groupand the representatives of the host country, as well as in achieving the program objectives andensuring the sustainability of the program. In addition, the vision, mission, regulations andresources of each institution involved, the socio-economic and political contexts, and theavailable opportunities also impact the design and implementation of the program.

Stage I: Needs Assessment

The interest and readiness of both institutions, along with the feasibility of conducting thestudy abroad program, should be assessed as a foundation for program development. There arethree levels at which to conduct the needs assessments: institutional, departmental, and inregard to potential participants. The program leaders are challenged to have the motivation todevelop the program and assess their resources and capabilities to successfully implement it.The institution should allow an international agenda and have the resources and interest to

Int J Adv Counselling

support the program. And, the potential participants should show interest in participating in theprogram, especially regarding location, cost, duration, and potential activities.

Additionally, the available professional literature may serve as resource for ideas aboutassessing feasibility. Reviewing similar programs and their characteristics, such as in regard tocost, number of credits offered, and number of days abroad, may be helpful in aligning theprogram with other successful experiences in the field. It is also important to gather informa-tion about the administrative requirements and procedures from both countries to develop theprogram, including the need to have documentation translated to the language of the hostculture. Another critical aspect of the program will be establishing a contact person from thehost culture that can support the interactions and processes.

When the relationship is initiated between the visitors and the representatives of the hostcountry, a description of inputs should be included in the action plan. It is necessary to have acomprehensive description of the available resources that can be used for the program. Theneeds assessment stage will be strengthened if both parties can identify the value of theprogram, their potential contribution, and their available resources to support the program.This process may also help in foreseeing any potential challenges and needed resources forprogram design, implementation, and evaluation. The evaluation results of an implementedprogram may serve as part of the needs assessment for the following experiences. Datacollected, such as marketing responses, number of applicants, number of participants, andother evaluation results, will be significant indicators for program planning and sustainability.

The program to Honduras was considered relevant and feasible for all stakeholders. Theneeds assessment was based on an interest survey for potential participants, communicationwith administrators and other stakeholders, and a literature review. Assessing interest andfeasibility from both institutions was the foundation for commitment, relationship building,and the development of a successful program. One of the most important lessons learned at thisstage was the value of taking the time to talk with all stakeholders to assess the interests of thevarious members of both parties, describe expectations, identify available resources, andclarify administrative procedures.

Stage II: Design

The design is a comprehensive plan of action that will function as a working documentthroughout the process. An international experience requires careful preparation to ensuresafety and success. The leaders should also be flexible enough to encounter unplannedsituations and adapt to them. Preparation is the key for overcoming or adapting to the changes.Leaders should obtain all possible information in advance, gather complete onsite contactinformation, and prepare the participants for the organic nature of the experience. Ideally, oneof the leaders should be from the host country and be familiar with the culture and the site.When that is not an option, it will be important for the leaders to visit the site in advance tobecome familiar with the context, strengthen the relationship, and familiarize themselves withlocations prior to traveling with a group of students. Even allowing a couple of days forpreparation prior to the arrival of the travel abroad group may be significant to enhancing theadaptation process.

The design document should clearly state the purpose and objectives of the program,describe the intended audience, present the proposed itinerary and activities, indicate therequirements for the participants, present a comprehensive budget for the program implemen-tation and evaluation, and document the administrative details of the program, such as cost,location, travel dates, and application process. The description of every stage in the processshould be presented in the action plan as well as a time line to help with the organization and

Int J Adv Counselling

preparation of the program. Details on marketing, recruitment, selection of participants,orientation session(s), onsite activities, debriefings session(s), and evaluation strategies shouldbe defined. In addition, it is imperative to include information about safety, preventionstrategies, and a plan of action in case of emergencies.

Planning eased the process and more importantly ensured having safety measures in placewhen traveling to Honduras. The program design was aligned with recommendations found inthe literature, followed the administrative procedures, and included the stakeholders’ goals andexpectations. This interactive process helped not only to create a feasible design, but also tobuild relationships among the stakeholders and to maintain fluid communication. Taking theextra time to elaborate a chronogram for time management and recording the steps followed indesigning the program was extremely important for organization, monitoring and futureprogram planning. Mediation and cultural sensitivity between the parties was extremelyrelevant for navigating administrative differences, such as time management, paymentmethods, invoices, and reservations, among others.

For both the design and implementation stages cultural sensitivity is critical in order toestablish an equal partnership. The action plan and the strategies implemented should reflectthe cooperation, mutual understanding and growth-fostering nature of the relationship. Thevisitors and host community should share time and interpersonal interactions with one another,learn from the experience, and contribute to the improvement of the program. Together, theyshould analyze the evaluation data and make relevant decisions to ensure future impact,effectiveness, and sustainability of a program that benefits the host country and the visitors.

Stage III: Implementation

Creating a comprehensive design will support the implementation and minimize complicationsduring the process. The implementation stage involves the preparation and realization of allactivities prior, during, and after the trip. It should include thoughtful well-developed teachingand evaluation strategies aligned with the purpose and objectives of the program, especiallyrelated to the overall goal of contributing to the enhancement of the cultural competencedevelopment of the participants. The information and evidence collected in this stage willserve greatly for enhancing future experiences.

During the implementation time it is important for the leaders to (a) pursue the achievementof the program objectives, (b) encourage interactions among program participants and betweenparticipants and hosts, (c) facilitate student engagement and reflection by dedicating inten-tional time to explore feelings and thoughts about the experience, (d) contextualize theexperience for the participants in order to reflect on the applicability to life in their homecontexts and their roles as counselors, (e) be flexible and prepare for changes and unplannedevents, and (f) document the experience by collecting information that can subsequently beused in the evaluation and decision-making processes.

During the time spent on site, it is essential to strengthen the relationship between thevisitors and the representatives of the host country. The leaders should dedicate time to discussthe expectations about the program and the collaboration being undertaken, as well as futureexperiences and changes needed to improve the program. The relationship should develop as amutual collaboration where both parties are contributing to and benefiting from the experience.Developing a growth-fostering relationship requires interaction, communication, real engage-ment, and ensuring sufficient time for interactions and to develop trust. The sustainability andcontinuity of the program will affect and be affected by the type of relationship established.The program should be part of an international collaboration agenda and not be only a one-sided interest (Comstock et al. 2008).

Int J Adv Counselling

While in Honduras, the program evolved mostly as planned, and the required changes madeon site were managed effectively and did not affect the overall implementation of the program.The participants were actively engaged, relationship building was strongly encouraged andnoticeable, and conversations about future projects among the program leaders were initiated.Being flexible as leaders in order to adjust and make decisions on site was extremely importantin this stage. Preparing the students for the reality of unplanned activities, changes andadjustments was also very important to support their preparation for the trip and their timein Honduras. Continuous and open interactions and relationship building were essential forprogram success.

Stage IV: Outcomes

An outcomes assessment, also called an impact assessment, constitutes the analysis of theimplemented program and its impact on the participants. Data should be collected throughoutthe process in a formative manner in order to obtain accurate and significant information thatwill support future improvements. Outcomes can be defined as indicators of the state of theparticipants in relation to the program purpose and objectives, and they represent the assess-ment of the participants’ learning and development through the experience (Rossi et al. 2004).

The ultimate goal of study abroad programs from an international counseling perspectiveshould be to contribute to the cultural competence development of the participants through atransformative experience abroad. Kottler (2003) stated that true transformative travel happenswhen the participants are required to solve problems in a different way, are confronted by anew environment, new relationships are developed, and they gain new perspectives on theirlives through the eyes of someone different. The evaluation of the program should assess theimpact of the services delivered. More importantly, the benefits and impact of the program inthe participants’ development should also be assessed.

The leaders of the program also serve as the evaluators. They are challenged to providesufficient time and undertake needed consultation in order to select their assessment methodsto evaluate the program appropriately. This is especially relevant when considering an impactassessment. It is important to consider the purpose of the evaluation, the intended outcomes,and factors both related to the program and those unrelated that might account for the achievedoutcomes. In addition, the evaluation process should also lead to reporting the unintendedoutcomes, positive or negative, that were brought about by processes outside the program.Even though unintended outcomes are unplanned, the evaluators should prepare for them bybeing involved in consultation and exploring the available literature (Rossi et al. 2004).

Acknowledging that not all outcomes have the same relevance, the evaluators may alsoconsider a process for reviewing importance, feasibility and the cost of measuring andanalyzing specific outcomes. Lastly, the evaluators should carefully select strategies andmeasures to be used for program evaluation considering the nature of the program, the purposeand objectives of the program, and the evaluation process. Hence, if quantitative measures areselected, they should be carefully analyzed, reviewing their properties, feasibility, and appro-priateness (Rossi et al. 2004).

The impact assessment for the program in Honduras focused on the participants’ experi-ences and their cultural competence development through their participation in the program.The process highlighted the importance of planning in advance, the need for identifyingresources for implementation and evaluation, and the value of seeking guidance from expe-rienced evaluators to conduct an impact assessment as the program evolved. Outcomeevaluations may be expanded, deepened or tailored to assess the program from differentperspectives or for different purposes and stakeholders.

Int J Adv Counselling

Stage V: Implications

The final stage of the process may also be considered the first step for future programdevelopment. The discussion of the four previous stages and the learning process willcomplete the presentation of the model. Analyzing the experience and discussingimprovements will help stakeholders in assessing the current state of the program andenvisioning future developments. Aligned with the foundations of Appreciative Inquiry,this stage will serve to guide reflections about the program experience, the actual stateof the program, and improvements needed by contrasting the actual program with thedesired program (Dunlap 2008).

In addition to analyzing the outcomes of the program, it is necessary to include cost-effectiveness and sustainability analyses in order to have a realistic view of the programimplementation and make more accurate decisions for the future. This stage will serve as a keypoint for evaluation and decision-making; therefore, it should consider the opinions andparticipation of all stakeholders. Participation of the stakeholders throughout the process willensure more accurate and useful information, and will result in a stronger commitment to theprogram and to a collaborative relationship between the participating countries.

The decision-making process and the incorporation of changes to the program will require acareful revision of the outcomes and processes of every stage. This stage becomes essential inconnecting the results of all the stages in order to analyze the experience as a whole. Theevaluators will be able to identify the strengths, accomplishments, and needed improvementsof the program overall, and shed light on the characteristics, requirements, and improvementsfor future programs. For example, in the present study abroad program the data gatheredthroughout the process were utilized to identify needed improvements, such as adding moretime for orientation and spending less time on the road to contribute to the quality of theexperience. Evaluation meetings with all stakeholders portrayed their overall satisfaction withthe program and support for implementing it in the future. Additionally, the financial reportssupported the cost of the program and the distribution and administration of the availablefunds. It was important for all the leaders to discuss impressions about the program andsuggestions for improvements on site and upon return. It was important to identify strategies tocontinue the relationship building among the stakeholders and to improve the overall programfor sustainability, quality, and impact.

Complementary Factors

In order for the aforementioned stages to transcend from being a mere written plan to asystematic design and a formative evaluation model, the program should include: plannedpurposeful evaluation strategies, data collection, reflection, and monitoring strategies through-out the process. In Fig. 1, every stage of the model is complemented by four additionalevaluation factors: objectives, evaluation questions, outcome indicators, and performanceindicators. Fig. 2 exemplifies the inclusion of the factors in the study abroad program toHonduras. These factors are intrinsically related to the purpose, design, and goals of theprogram and mirror the simultaneous nature of the design and evaluation processes, whichshould not be separated.

The evaluators should state clear achievable evaluation objectives to guide every stage andindicate the intended results. The evaluation questions are a set of questions developed by theevaluator and other stakeholders to delineate what the evaluation will aim to investigate. Thequestions should be useful for the stakeholders and be able to be answered utilizing available and

Int J Adv Counselling

affordable methods. The objectives, in conjunction with the evaluation questions, will help toidentify the most appropriate research methods to use in assessing the effectiveness of theprogram. Methods could vary greatly depending on the characteristics of the program, thestakeholders’ interests, and the evaluators’ knowledge and experience. Consultation, discussion,and investigation are important elements to commit to in utilising specific methods of evaluation(Rossi et al. 2004).

Selecting indicators will also support the evaluation process and serve as a source ofevidence of achievement. They should be related to the objectives and evaluation questions.Outcome indicators should present the effect of the program on the participants. Indicators area tool to monitor outcomes; therefore, the program should be able to impact the selectedoutcome indicators, and the evaluators should be able to analyze their effects. Outcomemonitoring within this model refers to continual measuring and reporting of indicators thataccount for the cultural development of the participants. The careful selection of the indicatorsand the accurate interpretation of the resulting data will determine the effectiveness of theoutcome monitoring. The data collected based on the outcome and performance indicators willcontribute to the analysis and decision-making process conducted in stages IV and V of themodel (Rossi et al. 2004).

The performance indicators refer to the program itself. While outcome indicators refer tothe participants, performance indicators refer to the program. They complement one another,describing the services and activities that produce the program outcomes. Assessing theprogram performance is relevant for sustainability and decision-making. The evaluators shouldincorporate a process of program monitoring by analyzing the information collected throughthe set indicators for assessing program performance. The purpose of the process is to estimatewhether the program is functioning as intended and following the established quality stan-dards. Key elements to assess include: whether the program is reaching the target population, ifthe service delivered and the support functions respond to the program design and qualitystandards, and what resources have been used to conduct the program. The outcome andperformance evaluation complete the evaluation cycle, providing the stakeholders with a clearunderstanding of the program strengths, achievements, impact and needs (Rossi et al. 2004).

Discussion

This self-evaluation model is intended to guide the development of study abroad programs andto purposefully incorporate a formative evaluation process to ensure usefulness, impactfulness,and sustainability of the program. The model highlights the relevance of study abroadprograms in developing cultural competence, building international relationships andsupporting the process of internationalization of the counseling profession. The modelwas created as a self-evaluation following the basic foundations of Appreciative Inquiryto create a more positive and participatory evaluation process. Self-evaluations have theadvantage of being conducted by program leaders who have knowledge about theprogram and an interest in improving it. The process of self-reflecting and contributingto the program enhancement motivates a sense of belonging and commitment of thestakeholders to the program (Rossi et al. 2004).

This level of participation, collaboration, and commitment that emulates the concept ofempowerment evaluation should result in skills development in program design and evalua-tion, a sense of control over the program from all the participating stakeholders, and anincreased capability for advocacy. Social justice and advocacy become evident not only inrelation to believing in and promoting the program, but more importantly when the

Int J Adv Counselling

relationships involved provoke changes and support the process of mutual understandingamong nations (Rossi et al. 2004).

That is not to say that as the program develops, the stakeholders cannot chose other types ofevaluation procedures, which may include hiring an external evaluator. The evolution of theprogram may require the use of different evaluation methods; the model can be developedfrom a basic design and evaluation process to more complex procedures. Several factors willdetermine the complexity of the model, including: (a) the number of times a program has beenconducted; (b) the knowledge and experience of the stakeholders; (c) the evaluation methodschosen, based on the evaluation objectives, questions and indicators; (d) the strength andcultural characteristics of the international relationship; and (e) the influence of externalfactors, such as socio-politics or economics.

The purpose of the study abroad program and the quality of the international relationshipwill be determining factors for the program success. Building growth-fostering relationshipsthat allow interaction, mutual collaboration, cultural understanding, and equal partnerships willcontribute to the process of internationalization and attest for the significance and value of theprogram itself. Furthermore, the relevance of the relationship between the visitors and therepresentatives of the host country indicate the importance of flexibility in the process and theneed to contextualize the model and the experience. This evaluation model should beconsidered a foundation for program development and evaluation; rather than as a stringentfit-all model. Stages may be altered, re-organized, transformed and overlap differently accord-ing to the program specifications, location and changes over time.

The model was created to serve as a foundation for program development and evaluation. Itis important to consider that the complexity of study abroad programs, the flexibility requiredwhen working in the international arena, and the variety of cultural characteristics to beconsidered, make any attempt to generalize program design and evaluation extremely compli-cated and at times inappropriate. Therefore, the model should be used with caution, afterdiscussion of appropriateness and contextualization to the purpose and particularities of theprogram under construction or evaluation. The model may be used as a tool to guide theprocess of design and evaluation of study abroad programs and may serve to share studyabroad experiences with others.

It is also relevant to consider that this model was developed through personal experienceand an analysis of the existing literature, a fact that denotes the influence of the author’sperspectives, biases, knowledge, and personal interpretations. The model was created in anattempt to contribute to the body of knowledge about study abroad programs, the importanceof cultural competence development in the internationalization process, and to initiate discus-sion about the development and evaluation of study abroad programs in the counseling field.Further experiences, discussions, publications and research are necessary to support the qualityand impact of study abroad programs and to ensure the promotion of growth-fosteringinternational relationships.

References

Alexander, C. M., Krucsek, T., & Ponterotto, J. (2005). Building multicultural competencies in school counselortrainees: an international immersion experience. Counselor Education and Supervision, 44, 255–266. doi:10.1002/j.1556-6978.2005.tb01754.x.

Anderson, P. H., Lawton, L., Rexeisen, R. J., & Hubbard, A. C. (2006). Short-term study abroad and interculturalsensitivity: a pilot study. International Journal of Intercultural Relations, 30(4), 457–469.

Arthur, N., & Achenback, K. (2002). Developing multicultural counseling competencies through experientiallearning. Counselor Education and Supervision, 42, 2–14. doi:10.1002/j.1556-6978.2002.tb01299.x.

Int J Adv Counselling

Brockington, J. L., Hoffa, W. W., & Marin, P. C. (2005). NAFSA’s guide to education abroad for advisers andadministrators (3rd ed). Washington: NAFSA: Association of International Educators.

Coghlan, A. T., Preskill, H., & Catsambas, T. T. (2003). An overview of appreciative inquiry in evaluation. NewDirections for Evaluation, 2003, 5–22. doi:10.1002/ev.96.

Comstock, D. L., Hammer, T. R., Strentzsch, J., Cannon, K., Parso, J., & Salazar II, G. (2008). Journal ofCounseling & Development, 86, 279–287. doi:10.1002/j.1556-6678.2008.tb00510.x.

Cooperrider, D. L., Whitney, D., & Stavros, J. M. (2003). Appreciative inquiry handbook: The first in a series ofAI workbooks for leaders of change. Bedford Heights: Lakeshore Communications Inc.

Cordero, A., & Negroni Rodríguez, L. (2009). Fostering cross-cultural learning and advocacy for social justicethrough an immersion experience in Puerto Rico. Journal of Teaching in Social Work, 29, 134–152. doi:10.1080/08841230802238195.

Duffey, T., & Somody, C. (2011). The role of relational-cultural theory in mental health counseling. Journal ofMental Health Counseling, 33(3), 223–242.

Dunlap, C. A. (2008). Effective evaluation through appreciative inquiry. Performance Improvement, 47, 23–29.doi:10.1002/pfi.181.

Fairchild, S. R., Pillai, V. K., & Noble, C. (2012). The impact of social work study abroad program to Australiaon multicultural learning. International Social Work, 49, 390–401. doi:10.1177/0020872806063413..

Hammond, S. A. (1996). Thin book on appreciative inquiry. Plano: CSS Publishing Co.Jaoko, J. (2010) Study abroad: Enhanced learning experience in cultural diversity. College Quarterly, 13(4).

Retrieved from http://www.collegequarterly.ca/2010-vol13-num04-fall/jaoko.htmlJurgens, J. C., & McAuliffe, G. (2004). Short-term study-abroad experience in Ireland: an exercise in cross-

cultural counseling. International Journal for the Advancement of Counseling, 26, 147–161. doi:10.1023/B:ADCO.0000027427.76422.1f.

Kottler, J. A. (2003). Transformative travel: international counselling in action. International Journal for theAdvancement of Counseling, 24, 207–210. doi:10.1023/A:1023364628260.

Leung, S.-M. A., Clawson, T., Norsworthy, K. L., Tena, A., Szilagyi, A., & Rogers, J. (2009). Internationalizationof professional counseling: an indigenous perspective. In L. H. Gerstein, P. P. Heppner, S. Aegisdóttir, S.-M.A. Leung, & K. L. Norsworthy (Eds.), International handbook of cross-cultural counseling: Culturalassumptions and practices worldwide (pp. 111–123). Thousand Oaks: Sage Publications.

McDowell, T., Goessling, K., & Melendez, T. (2012). Transformative learning through international immersion:building multicultural competence in family therapy and counseling. Journal of Marital and FamilyTherapy, 38, 365–379. doi:10.1111/j.1752-0606.2010.00209.x.

McLauglin, J. A., & Jordan, G. B. (1999). Logic models: a tool for telling your program’s performance story.Evaluation and Program Planning, 22(1), 65–72.

Millar, A., Simone, R. S., & Carnevale, J. T. (2001). Logic Models: A systems tool for performance management.Evaluation and Program Planning, 24(1), 73–81.

Ng, K., & Noonan, B. M. (2012). Internationalization of the counseling profession: Meaning, scope andconcerns. International Journal for the Advancement of Counselling, 34, 5–18. doi:10.1007/s10447-011-9144-2.

Renger, R., & Titcomb, A. (2002). A three-step approach to teaching logic models. American Journal ofEvaluation, 23, 493–503. doi:10.1177/109821400202300409.

Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7th ed.). CA: SagePublications, Inc.

Santos Figueroa, S. D. (2014). Cultural competence development through study abroad initiatives: Case studyHonduras (Doctoral dissertation). Retrieved from http://www.lib.ncsu.edu/resolver/1840.16/9287

Sue, D. W. (2001). Multidimensional facets of cultural competence. The Counseling Psychologist, 10, 45–52.doi:10.1177/0011000082102008.

Sue, D. W., & Sue, D. (2003). Counseling the culturally diverse: Theory and practice (4th ed.). NY: Wiley.Tomlinson-Clarke, S. M., & Clarke, D. (2010). Culturally focused community-centered service learning: an

international cultural immersion experience. Journal of Multicultural Counseling and Development, 38,166–175. doi:10.1002/j.2161-1912.2010.tb00124.x.

West-Olatunji, C., Goodman, R. D., Mehta, S., & Templeton, L. (2011). Creating cultural competence: anoutreach immersion experience in southern Africa. International Journal for the Advancement ofCounselling, 33, 335–346. doi:10.1007/s10447-011-9138-0.

Int J Adv Counselling