monitoring and evaluating volunteer tourism: a review and analytical framework
TRANSCRIPT
1
Monitoring and Evaluating Volunteer Tourism: A Review and Analytical Framework
Jessica Taplina*, Dianne Dredgeb & Pascal Scherrera
Preprint version, published as:
Taplin, J., Dredge, D. & Scherrer, P. (2014). Monitoring and Evaluation of Volunteer Tourism: A Review and Analytical Framework. Journal of Sustainable Tourism.
DOI: 10.1080/09669582.2013.871022
Abstract
The rapid expansion and commercialisation of the volunteer tourism sector and the potential for negative impacts on host communities has put the sector under increasing scrutiny. Monitoring and evaluation are key aspects of sustainable tourism planning and management and play important roles in the project planning and implementation cycles of volunteer tourism organisations and destination managements. However, they can be both value-laden and politically charged, making an understanding of context, purpose and various approaches to monitoring and evaluation important. Drawing from evaluation and critical management studies, this conceptual paper reviews the literature, presenting an analytical framework aimed at improving the quality of monitoring and evaluation. The paper is positioned within the adaptancy platform and focuses on qualitative, critical approaches to evaluation. The framework highlights the important influence of context (the issue the volunteer tourism programme is addressing, the nature of the intervention, the setting, the evaluation context, and the decision-making context), and identifies four dimensions of volunteer tourism (stakeholders, organisations, markets and programmes) which may influence monitoring and evaluation practices. The analytical framework presented is useful for practitioners developing monitoring and evaluation processes, and for researchers interested in empirical studies which seek to evaluate monitoring and evaluation practices. 198 Keywords: volunteer tourism organisations; monitoring; evaluation; critical management studies (CMS) ________________________ a Southern Cross University Australia b Department of Culture and Global Studies, Aalborg University, Denmark
* Lead author
2
Introduction Monitoring and evaluation are key aspects of sustainable tourism planning and management. Incorporating monitoring and evaluation within a holistic sustainable tourism management approach can assist in determining if volunteer tourism programmes are benefitting local communities and environments; and are meeting the needs of key stakeholders. They serve to highlight what, if any, changes need to be made. Monitoring and evaluation also provide insights into the progress, impacts and outcomes of a programme (e.g. Stufflebeam & Shinkfield, 2007). Thus findings can inform evidence-based planning and policy for sustainable development (Meyer, 2012).
This conceptual paper responds to a gap in the literature about how volunteer tourism organisations monitor and evaluate their programmes within the complex organisational landscape that they are situated within. We deconstruct this landscape and identify various dimensions which contribute to the context surrounding a volunteer tourism programme (e.g. type of organisation offering the programme), and consider their implications for the monitoring and evaluation of volunteer tourism programmes. An analytical framework aimed at improving the quality of monitoring and evaluation is outlined. The paper draws on the fields of evaluation studies and critical management studies (CMS). In addressing these objectives, this paper first provides a brief overview of current issues in volunteer tourism related to monitoring and evaluation and critically discusses how definitions of volunteer tourism compound the problem of developing generic guidance for monitoring and evaluation. A review of current approaches and issues in the monitoring and evaluation of volunteer tourism programmes follows. Building on these insights, a conceptual framework for examining the monitoring and evaluation of volunteer tourism programmes is presented. Current issues in volunteer tourism The practices of volunteer tourism organisations and volunteer tourists are coming under increasing scrutiny due to concerns about the rapid expansion and commercialisation of the sector (e.g. Brown & Hall, 2008; Butcher, 2006, 2011; Butcher & Smith, 2010; Guttentag, 2009, 2011; Raymond, 2011; Raymond & Hall, 2008; Sin, 2010; Tomazos & Cooper, 2012; Vodopivec & Jaffe, 2011). Wearing (2001), one of the original proponents of volunteer tourism as a form of sustainable development, has recently highlighted the increasing commoditisation of volunteer tourism and contends that this shift could undermine the sustainability of the communities and environments that these projects are seeking to help instead of benefiting them (Lyons & Wearing, 2008b). In this issue of this Journal, Font & Smith (2014) raise a new area of scrutiny, by analysing the marketing practices used by organizations seeking to attract volunteer tourists. Despite the growth of the volunteer tourism industry and growing concern about its impacts on host communities, there remains a lack of research on what the impacts of volunteer tourism actually are or how volunteer tourism organisations monitor and evaluate their programmes and potential impacts. In addition, the ‘blurry’ and ‘ambiguous’ nature of volunteer tourism exacerbates the challenge of providing guidance for responsible practice in the industry (Lyons & Wearing, 2008a, 2012). Wearing & McGehee (2013) reflect that much of the existing literature on volunteer tourism has focused on the volunteers, examining their motivations and experiences. In their
3
review of the volunteer tourism literature and industry to date, Wearing and McGehee (2013) utilise Jafari's (2001) four platforms of tourism research (advocacy, cautionary, adaptancy, and scientific) to contextualise the development of the field. After an initial advocacy phase, which positioned volunteer tourism as a wholly positive and altruistic activity, the literature took on a cautionary stance raising concerns about the potential for negative impacts on local communities (Wearing & McGehee, 2013). They observe that there is now a growing literature that takes an adaptancy platform. The adaptancy platform explores and suggests ways for organisations to better manage volunteer tourism and its associated activities in order to promote positive over negative outcomes (Wearing & McGehee, 2013). This paper is positioned within the adaptancy platform. Volunteer tourism: Ambiguity and challenges for evaluation As knowledge and understanding of volunteer tourism expands, the phenomenon continues to demonstrate its complex, often ‘blurry’ and ‘ambiguous’ nature (Lyons & Wearing, 2008a, 2012). Consequently, an overarching definition of volunteer tourism has failed to emerge and new definitions continue to surface (Benson, 2011b). The first and most widely cited definition of volunteer tourism is that by Wearing (2001, p. 1) who defined it as encompassing “those tourists, who for various reasons, volunteer in an organised way to undertake holidays that might involve aiding or alleviating the material poverty of some groups in society, the restoration of certain environments or research into aspects of society or environment”. A more recent definition, describes volunteer tourism as “a combination of development work, education and tourism” (Keese, 2011, p.258). As the types of volunteer tourism experiences on offer, and the types of organisations offering them, have rapidly multiplied, the industry has become “increasingly ambiguous in definition and context” (Callanan & Thomas, 2005, p. 195). Thus, Callanan and Thomas (2005, pp. 184-186) describe volunteer tourism as an “eclectic” tourism product with a “multi-dimensional nature”. They propose that volunteer projects can be placed at various positions along a continuum ranging from “Shallow” volunteer tourism projects, which are driven by the demands and interests of volunteers, to “Deep” volunteer tourism projects, which are primarily focused on having a positive impact on the local environment and/ or community (Callanan & Thomas, 2005, pp.183-200). Lyons and Wearing (2008a, 2012) further argue that the notions of service learning, cultural exchange and charity fundraising challenge what could also be considered volunteer tourism. They maintain that “volunteer tourism cuts across typologies of volunteering and tourism” and cannot be neatly placed into tidy, narrow boundaries (Lyons & Wearing, 2012, p.89). Difficulties in defining what volunteer tourism actually is make it challenging to consider how volunteer programmes might or should be monitored and evaluated. In the context of current concerns about industry impacts and practices, firstly, a clearer understanding of the nature of volunteer tourism organisations and their activities are fundamental to addressing the concerns about impacts of current practice and commoditisation of the sector. Secondly, the question of how to frame evaluation and monitoring processes is important in addressing concerns about current practice. This is also recognised in industry attempts to encourage responsible practice in volunteer tourism which
4
have been gaining momentum in recent years (e.g. Fee & Mdee, 2010; Mdee & Emmott, 2008; The International Ecotourism Society, 2012). Purpose of monitoring and evaluation The processes of monitoring and evaluation play important roles in the wider project planning and implementation cycle of an organisation. In this paper, monitoring is considered to be the purposeful checking of how a programme’s activities are progressing and the gathering of information on the various activities taking place within a programme (Bartle, 2007). Monitoring activities often feed into evaluation. Evaluation is the process of determining the merit and worth (value) of a programme, serving as a basis for determining if and how a programme needs to be improved or even terminated (Stufflebeam & Shinkfield, 2007). Monitoring and evaluation can serve as valuable learning tools for organisations (Preskill & Torres, 1999). To ensure that their services or programmes are meeting the needs of their clients, organisations need to “continually obtain pertinent evaluative feedback” on their programmes and services (Stufflebeam & Shinkfield, 2007, p. 29), an issue recognised in both the volunteer tourism and the wider evaluation literature (e.g. Raymond, 2011).
In addition, evaluation can serve different functions within society. Meyer and Stockman (in Meyer, 2012) propose that evaluation uses can be classified into four overlapping functions: insight, control, development and legitimation. For the purposes of volunteer tourism programme monitoring and evaluation, conducting evaluation for the purposes of producing new insight by producing new knowledge about the programme which can then be used to learn from experience and aid the development of the programme is proposed as the priority approach to mitigate against negative impacts on host communities and dissatisfaction of volunteers. The evaluation functions of control and legitimation are also important. For organisations engaged in funder-fundee relationships (often found in volunteer tourism partnerships), evaluation functions as a control mechanism to verify that a programme is meeting agreed goals (Meyer, 2012). Evaluation that functions to legitimate organisations’ activities allows organisations to prove their worth to current and potential supporters (Meyer, 2012). Supporters may be customers and/ or donors, both of which are essential to the sustainability of volunteer tourism.
Monitoring and evaluation in volunteer tourism Responsible practice in volunteer tourism is currently a key issue receiving increasing attention from the tourism industry, the media, and academia (e.g. Birrell, 2010; Boffey, 2011; Mdee & Emmott, 2008; Ong, Pearlman, Lockstone-Binney & King, 2013; The International Ecotourism Society, 2012). The monitoring and evaluation of volunteer tourism programmes is identified as an important example of responsible practice in volunteer tourism (Comhlámh, 2011; Lamoureux, 2011; PEPYTours, n.d; Power, 2007; Raymond, 2007, 2011; The International Ecotourism Society, 2012). However, understandings of the practical realities of the complex landscape of volunteer tourism need strengthening to better support and inform efforts to promote responsible practice. For example, volunteer tourism in a not-for-profit context that is predominantly focused on volunteering and includes some
5
tourism aspects is a different context to volunteer tourism in a commercial context that is mostly focused on tourism and some volunteering. Thus approaches to monitoring and evaluation will likely vary between contexts.
Highlighting the potential for a variety of negative impacts on host communities, Guttentag (2009) argues that potential risks of volunteer tourism include a neglect of locals’ interests; a hindering of work progress and completion of unsatisfactory work; a disruption of local economies; a decrease in local labour demand which reinforces dependency; reinforcing conceptualisations of the ‘Other’; rationalisation of the ‘Other’ and rationalisation of poverty and the instigation of cultural change. To date, concerns about the negative impacts of volunteer tourism have mostly been centred on volunteer tourism programmes that take place in developing countries (e.g. Guttentag, 2009, Palacios, 2010, Sin, 2010). Accordingly, this paper focuses on the monitoring and evaluation practices of volunteer tourism organisations based in developed countries which send volunteer tourists to developing countries. Engagement with monitoring and evaluation of volunteer tourism organisations’ activities and services is essential to determine whether or not communities really benefit from such initiatives (Raymond, 2011; Simpson, 2008; Stufflebeam & Shinkfield, 2007). This is particularly so for organisations which aim (and/or openly claim) to deliver benefits to communities and are engaged in cross sector partnerships (Simpson, 2008). Empirical case studies have investigated how volunteer tourism works in practice and highlight the need for ongoing monitoring and evaluation processes to ensure that volunteer tourism projects remain relevant to host communities and are implemented in ways that satisfy the needs of both volunteer tourists and host communities (Barbieri, Santos, & Katsube, 2012; McGehee & Andereck, 2008; Nelson, 2010; Raymond, 2007, 2011; Sin, 2010). For example, Barbieri et al. (2012) suggest that, in order to improve the volunteer tourism experience from the perspective of volunteer tourists and host communities, volunteer tourism organisations should consider implementing a system in which the volunteers can evaluate their volunteer experiences, so that the information gained can be used to improve future services offered to volunteers; and likewise, that local community members should be given the opportunity to evaluate their project experiences and identify where and how volunteer tourists dedicate their time and efforts.
Nelson (2010, p.78) argues that seeking community perspectives is essential for evaluating the success of programs, asserting that the long term outcomes of volunteer tourism could be significantly improved if evaluation measures enabled “honest conversations” between the community, the volunteers and the volunteer tourism organisations. However, Guttentag (2011) raises the issue of potentially biased project evaluations, arguing that locals may be wary of criticising projects for fear that projects and their funding will be withdrawn. A similar issue was raised by Sin (2010), whose research also found that local NGO workers and members of local host communities were wary of offending funding organisations despite having suggestions for ways that the volunteer tourism experience could be improved. Such sentiments emphasize unequal power relationships and dependencies between host communities and volunteer tourism organisations. The growing number of guidelines, tools, campaigns and academic literature promoting responsible practice in volunteer tourism also endorse the importance of volunteer
6
tourism organisations engaging in the ongoing monitoring and evaluation of volunteer tourism projects and cite the need to monitor and evaluate the experiences of the volunteers, host partner organisations and host communities (Comhlámh, 2011; Lamoureux, 2011; Power, 2007; The International Ecotourism Society, 2012). Yet, despite advocating its implementation, there is a lack of understanding of how, why (and indeed, if) volunteer tourism organisations actually conduct the monitoring and evaluation of volunteer tourism projects in practice. Nor has it been considered how monitoring and evaluation can effectively take place within such a complex and ambiguous landscape. Approaches to monitoring and evaluation Traditionally, the field of evaluation has been dominated by the assumptions of the positivist paradigm which seeks to measure the social world, promoting objective and value-neutral inquiry (Mertens & Wilson, 2012). In this context, the historical, social and political contexts surrounding a program are viewed as variables to be controlled (e.g. Campbell, 1969). Evaluators aligned with this school of thought have proposed the use of quantitative methods such as quasi-experimental methods which adapt scientific methods for use within the social world (Campbell, 1969; Mertens & Wilson, 2012). However, a broad spectrum of approaches to monitoring and evaluation now exist. These approaches can be positioned along a continuum. At one end of the continuum are methods which are influenced by rational scientific approaches and the positivist paradigm, and which seek to collect quantifiable data (Campbell, 1969; Mertens & Wilson, 2012). At the other end of the continuum are qualitative, critical approaches which recognise the inherently ethical, political, social and contextual factors involved in evaluation processes and recognise that the way in which people make sense of their situations are heavily influenced by their values (Guba & Lincoln, 1989). Thus from this perspective, engagement with peoples’ values, particularly the values of key stakeholders, is seen as a fundamental part of evaluation (Guba & Lincoln, 1989; Greene, 2005, 2012). Sitting between these two ends of the spectrum are mixed method approaches to evaluation, which utilise both qualitative and quantitative methods (Bamberger, 2012; Greene, 2002; Mertens, 2010).
It is now recognised within evaluation studies that there is no single best method or model for carrying out evaluations. A multitude of evaluation approaches, underlying concepts and models exist (Meyer, 2012). Choice of methods will differ according to the context of the particular programme being evaluated (Bamberger, 2012; Rog, 2012). As discussed earlier, the nature of volunteer tourism programmes varies greatly and so it is unlikely that a 'one size fits all' approach to programme monitoring and evaluation is appropriate. Evaluations can be conducted internally or externally (Mertens & Wilson, 2012), formally or informally (Stufflebeam & Shinkfield, 2007) and on a large or small scale (Robson, 2000).
Recognising that there are various political, social and contextual factors played out in volunteer tourism, such as the potential for unequal power relationships and dependencies outlined earlier, this paper focuses on qualitative, critical approaches to evaluation. However, we acknowledge that quantitative approaches can still be of value within the volunteer tourism context, particularly when quantifiable data is required. Further, mixed method
7
approaches to evaluation will be appropriate in many contexts where there is a need for both quantitative and qualitative evaluative data (Bamberger, 2012). Drawn from the literature, a summary of qualitative approaches to evaluation is outlined in Table 1. The value of Table 1 is the identification of different ways in which qualitative evaluation can be undertaken, contributing to a more informed choice of evaluation methods. Approaches include Empowerment Evaluation (Fetteman 1994, 1996 in Jackson & Kassam, 1998), Participatory Evaluation (Cousins & Whitmore, 1998; Cullen & Coryn, 2011; Jackson & Kassam, 1998), Values Engaged Evaluation (Greene, 2005, 2012; Greene, DeStefano, Burgon, & Hall, 2006), Responsive Evaluation (Stake 1975 in Guba & Lincoln, 1989) and Evaluation for Social Justice (Mertens, 2012). A key theme running throughout each of these approaches is the notion that evaluation is an essential tool for improving situations and services/programmes for the people that those services/programmes are meant to serve.
Table 1: Qualitative Approaches to Evaluation
Evaluation Approach
Summary References
Empowerment Evaluation
Empowerment evaluation “is the use of evaluation concepts, techniques and findings to foster improvement among citizens and the programs intended to serve them”.
Fetteman (1994, 1996) in Jackson and Kassam, (1998, p.10).
Participatory Evaluation (PE)
Involves evaluators collaborating with stakeholders of the entity being evaluated, streams within PE include practical PE and transformative PE. PE is often used by development organisations, in such a context the focus of PE is on sharing knowledge through the evaluation process. Advocates that together citizens and professionals can generate analysis and new knowledge that will increase the capacity of interventions to produce improved and lasting results. However, no universal agreement as to what constitutes PE, a continuum of participation exists.
Cousins & Whitmore (1998). Jackson & Kassam (1998). Cullen & Coryn (2011).
Values Engaged Evaluation
Emphasises evaluation as an avenue for democratising dialogue about critical social and educational issues –includes more stakeholder perspectives.
Greene (2005); Greene (2012).
Responsive Evaluation
Responsive evaluation actively seeks out different stakeholders views. Evaluation parameters are guided by an interactive and negotiated process with stakeholders.
Stake (1975 in Guba & Lincoln, p.45).
Evaluation for Social Justice
Incorporates fundamental attributes from human rights. Mertens (2012).
Programme beneficiaries and monitoring and evaluation
For monitoring and evaluation to respond to the needs of programme beneficiaries, these people should be included in monitoring and evaluation processes (e.g. Guba & Lincoln 1989). It is important to highlight here that the intended beneficiaries or ‘communities’ hosting volunteer tourism activities are not homogenous. ‘Host’ or ‘local’ communities often comprise individuals and groups with both shared and conflicting social, cultural and
8
economic values and interests. Volunteer tourism organisations, therefore, need to take this into account and consider appropriate monitoring and evaluation methods to gather information relevant to stakeholders and the intended beneficiaries of a volunteer programme (Mascarenhas, Coelho, Subtil & Ramos, 2010). At its best, evaluation can improve services and programmes, promote the accountability of organisations and empower stakeholders (Guba & Lincoln, 1989; Jackson & Kassam, 1998; Mayhew, 2011; Stufflebeam & Shinkfield, 2007). At its worst, however, evaluation processes can disempower and marginalise stakeholders, such as when they are not consulted appropriately (Guba & Lincoln, 1989). Whether or not evaluation works to empower or disempower is dependent on the way in which evaluations are (or are not) carried out, how the findings are disseminated and whether or not findings are acted upon responsibly by key agents such as policy makers, regulatory bodies and service providers (Guba & Lincoln, 1989; Stufflebeam & Shinkfield, 2007). Therefore, understanding processes of evaluation is essential to improving evaluation findings (Guba & Lincoln, 1989), and requires further theoretical and practical attention within volunteer tourism.
Monitoring and evaluation through a critical lens In this paper, critical management studies (CMS) is adopted as an appropriate theoretical underpinning to examine a volunteer tourism organisation’s monitoring and evaluation practices. This critical approach is distinguished from supposedly objective, value-free quantitative, positivistic approaches to monitoring and evaluation. Rather, this critical approach seeks to incorporate different types of knowledge from various stakeholders in an effort to unmask unquestioned relations of control and domination within and between organisations and in organisation’s relations with wider society (Alvesson & Ashcraft, 2009; Alvesson & Deetz, 2005; Buchanan & Bryman, 2009; Deetz, 2009). In the social sciences, critical research approaches seek to question established landscapes of power and to examine taken for granted assumptions that tend to reproduce the status quo. In doing so, critical research seeks to uncover practices and ideas that have led to oppression and marginalisation (Ateljevic, Pritchard, & Morgan, 2007). Critical researchers hope that such research can lead to new practices and ways of seeing (Kincheloe & McLaren, 2005). CMS is a subset of critical theory and applies critical research approaches to management and organisations in an effort to interrogate issues including social justice, inequality and ethics (Alvesson & Ashcraft, 2009). In essence, CMS holds that prevailing forms of business management routinely reproduce a range of effects including inequality, oppression, gender bias and so on (Alvesson & Ashcraft, 2009; Alvesson & Deetz, 2005; Buchanan & Bryman, 2009; Deetz, 2009). However, CMS is unique from other forms of critical research in that it is careful not to ‘demonize’ all who are in positions of power unnecessarily; rather, CMS scholarship is generally appreciative of the conditions and constraints that arise from organisational contexts and is responsive to the practical and ethical dilemmas that organisations face (Alvesson & Ashcraft, 2009). For example, a lack of resources may mean that organisations must decide between using available funding to fund more projects at the expense of monitoring and evaluating existing ones. As critical tourism researchers we take the position that monitoring and evaluation should be tools for empowering key stakeholders, in particular host communities. Although we recognise the importance of other key stakeholders, such as the volunteers, existing research has demonstrated the potential for negative impacts on host communities including unequal power relationships and dependencies (Guttentag, 2009; Sin, 2010). We are therefore particularly interested in how organisations engage with host communities with respect to monitoring and evaluating their programmes.
9
The volunteer tourism literature has highlighted that many issues and negative and positive impacts can potentially eventuate from the interaction between volunteer tourists, host communities and volunteer tourism organisations (Guttentag, 2009). However, if evaluation processes are to empower stakeholders, it is important that key stakeholders, such as volunteer tourists and host communities, have the opportunity to voice concerns over any issues they are not satisfied with or to confirm aspects that are working well (Barbieri, et al, 2012; Guba & Lincoln, 1989; Jackson & Kassam, 1998; Mayhew, 2011; Nelson, 2010). Further, it is recognised that monitoring and evaluation alone is not sufficient to ensure that programmes are meeting the needs of all stakeholders. In order for evaluation processes to improve programmes, findings must be obtained, followed up, and acted upon responsibly (Stufflebeam & Shinkfield, 2007). An approach for examining monitoring and evaluation in volunteer tourism Appreciating context Literature from within evaluation studies recognises that evaluations are highly contextual in nature (Conner, Fitzpatrick, & Rog, 2012; Fitzpatrick, Christie, & Mark, 2009; Rog, 2012), arguing the need to “foreground the background” and make the often implicit acknowledgement of context explicit (Rog, 2012, p. 25). The various contexts surrounding a programme will influence how evaluation is approached, for what purpose it is carried out and how the findings are utilised and reported. Consequently, context impacts upon the outcomes of programme evaluation (Rog, 2012). Despite this, context does not usually take a leading role in evaluation practice and is often “more of an afterthought when the work does not go as planned or findings emerge that are difficult to interpret” (Rog, 2012, p.26). Rog (2012) argues that “context matters” and that “context-sensitive evaluation” can help evaluators determine the appropriate way for them to approach and carry out an evaluation, improving evaluation practice. In a similar vein, the authors of this paper argue that a context-sensitive analysis of evaluation practice can help with understanding the programme monitoring and evaluation practices of volunteer travel organisations.
Traditionally evaluations are driven by a “methods first approach” and an ongoing discussion within the field of evaluation has been “how best to match designs and methods to particular programme and policy contexts to produce the most useful and actionable evidence” (Rog, 2012, p. 267). Stemming from these discussions, and drawing from Rog and others’ experiences as programme evaluation practitioners in a variety of settings (including interventions for homelessness, environmental interventions and educational programmes), Rog (2012) argues that it is inappropriate to select a particular evaluation method without first understanding the particular context of a programme and the (potentially competing) needs of stakeholders. Rather, Rog’s (2012) Context Assessment Framework takes a 'context first approach', following which evaluators can then decide which methods, such as the examples summarised in Table 1, might be most appropriate.
The framework proposes five key areas of context that affect evaluation practice from design of the evaluation right through to how findings are reported. These five key areas include the phenomenon and the problem (e.g. the issue that a volunteer travel programme is trying to address), the nature of the intervention (i.e. the actual programme to be evaluated, e.g. the volunteer programme itself), the broader environment/setting (e.g. market drivers, stakeholders involved and organisations offering the programme), the evaluation context (e.g. the parameters of the actual evaluation which does/ does not take place), the decision-making context (e.g. who decides what is monitored and evaluated and how, e.g. volunteer travel organisations, host communities, host partner organisations, donor agencies). With this is
10
mind, how (and indeed if) volunteer tourism organisations engage with the monitoring and evaluation of their programmes will be influenced by the “myriad of variables at play” in the complex volunteer tourism landscape (Benson, 2011, p.43). Drawing from the literature, the various blurry and ambiguous dimensions of volunteer tourism can be loosely grouped into four broad intersecting areas:
(1) Stakeholders (i.e. the various stakeholders and their objectives and motivations for engaging with volunteer tourism).
(2) Organisations (i.e. the various types of volunteer tourism organisations and their organisational contexts).
(3) Markets (i.e. the various markets and their characteristics which drive the demand for volunteer tourism).
(4) Programmes (i.e. the various types of programmes offered and the nature of their engagement with volunteer tourism).
Volunteer travel programmes are situated within varying contexts within these broad
dimensions which make up the volunteer tourism landscape. Critical interrogation of these dimensions is required in order to try and understand their influence on how an organisation monitors and evaluates volunteer travel programmes and for what purpose. For example, the successful application of Rog’s (2012) framework in different areas including environmental and indigenous community settings has found that depending on the setting, different dimensions of the framework can have varying (and sometimes conflicting) levels of importance for stakeholders. In addition, the dimensions can interact with one another in different ways (Conner et al., 2012). The following section examines the broad dimensions of volunteer tourism and considers their potential implications for the monitoring and evaluation of volunteer tourism programmes. Developing a framework for examining programme monitoring and evaluation In order to develop a framework for examining programme monitoring and evaluation in volunteer tourism it is necessary to organise the current literature and bring together the various dimensions which contribute to the complex landscape. Based on the four broad areas identified above, and drawing on Rog’s (2012) Context Assessment Framework, Figure 1 demonstrates how the various dimensions and contexts of the volunteer travel landscape influence the programme monitoring and evaluation practices of volunteer travel organisations.
11
Figure 1. Areas of context that affect volunteer travel programme (VTP) evaluation practice (adapted from Rog, 2012, p. 28)
Figure 1 highlights three broad and overlapping areas of context: the broader environment that volunteer programmes and volunteer tourism organisations are situated within, the programmes themselves (intervention context) and the nature of the evaluation that is carried out (evaluation context). Each of these areas is influenced by broad dimensions identified within the volunteer tourism landscape: stakeholders, organisations, markets and programmes, which in turn have further sub-dimensions which also require attention through a qualitative, critical lens. We now simultaneously outline the complex volunteer tourism landscape and its dimensions whilst integrating the literature on volunteer tourism and the literature on monitoring and evaluation. In doing so, we highlight issues for critical consideration in empirical studies examining an organisation's monitoring and evaluation practices.
Dimension 1: Stakeholders Stakeholders are defined as “Any person or group that is able to make a claim on an organization’s attention, resources or output, or who may be affected by the organization” (Lewis, 2001). In order to ascertain whether or not a programme is benefitting those it is meant to serve, it is essential that the relevant stakeholders and their agendas and interests are identified and their feedback sought in monitoring and evaluation processes (e.g. Guba & Lincoln, 1989 & Stufflebeam & Shinkfield, 2007). Further, it is generally agreed that the
Broader Environment
Intervention Context
(Actual VTP)
Evaluation Context
Dimensions • Stakeholders • Organisations • Markets • Programmes
Programme revisions based on information gained via monitoring and
evaluation
12
inclusion of a wide range of stakeholder perspectives is essential for achieving sustainable tourism development (Timothy, 2002). Different types of stakeholders have been identified, each with varying interests, values, motivations and objectives for engaging in volunteer tourism (ATLAS/TRAM, 2008). Stakeholders can include funders such as volunteer tourism organisations, governments and volunteer tourists (who can also be beneficiaries), and beneficiaries such as host communities (ATLAS/TRAM, 2008; Lewis, 2001). Thus volunteer tourism organisations have to manage multiple stakeholder relationships whilst being stakeholders themselves (Coghlan and Noakes, 2012). In addition, it can be difficult to identify all potential stakeholders (De Araujo & Bramwell, 1999), and it should be noted that volunteer tourism stakeholders may not be limited to those outlined in Table 2. Table 2: Stakeholders and their importance in monitoring and evaluation processes
Type of Stakeholder Why are they important? References Volunteer tourism organisations (see Table 3 for further information about the different types of volunteer tourism organisations)
• Recruit and/or receive volunteers • Organise programmes • Organise and/or host volunteers • Oversee volunteer activities • Intermediary between volunteer
and host community • May fund programmes • May profit from programmes
(ATLAS/TRAM, 2008; Lamoureux, 2011; Raymond, 2007, 2008, 2011).
Host communities • Host volunteers • May or may not be satisfied with
programmes/ and or volunteers
(Benson, 2011a; Gray & Campbell, 2007; Guttentag, 2009, 2011; Higgins-Desboilles, 2003; McGehee & Andereck, 2009; Sin, 2010; Wearing & Darcy, 2011).
Volunteer tourists (see Table 4 for further information about the different types of volunteer tourism markets)
• Pay and/or volunteer to participate in programme
(ATLAS/TRAM, 2008; Benson, 2011a; Wearing, 2001).
Governments and government agencies
• May fund programmes • May impose regulations on
volunteer tourism activities
(ATLAS/TRAM, 2008; Benson, 2011a; Garland, 2012).
Incorporating a CMS lens, it would be of particular interest here to examine which stakeholder’s feedback is deemed important to a volunteer travel organisation and why. For example, are members of the host community involved in monitoring and evaluation of volunteer travel programmes or is feedback only sought from the volunteers themselves? Further, although feedback from members of the host community may be sought, it will be important to take into consideration the complex power relations outlined earlier by Sin (2010) and Guttentag (2009) and recognise that some local stakeholders may be wary of criticising projects. Dimension 2: Organisations There are a number of sub-dimensions within volunteer tourism that fall under the broad dimension ‘organisations’ (Table 3). A key sub-dimension identified includes the type of volunteer tourism organisation offering a volunteer tourism programme. A plethora of
13
different types of organisations offer volunteer tourism programmes including, non-government organisations (NGOs), not-for-profit organisations, for-profit organisations, social enterprises, academic institutions and religious organisations (e.g. ATLAS/TRAM, 2008; Coghlan & Noakes, 2012; Tomazos & Cooper, 2012), plus a range of situations where different types of organization work in partnership (see below and see Font & Smith, 2014). Table 3: Organisations
Dimension References Types of volunteer tourism organisations
Non-government organisations (NGOs)
(ATLAS/TRAM, 2008; Coghlan & Noakes, 2012 ; Keese, 2011; Tomazos & Butler, 2009a; Tomazos & Cooper, 2012; Wearing, 2001, 2004).
Not-for-profit (including charities and some NGOs)
(Brown & Hall, 2008; Coghlan & Noakes, 2012 ; Raymond & Hall, 2008; Tomazos & Butler, 2009a, 2009b; Tomazos & Cooper, 2012).
For Profit (Brown & Hall, 2008; Coghlan & Noakes, 2012 ; Raymond & Hall, 2008; Tomazos & Butler, 2009a, 2009b; Tomazos & Cooper, 2012).
Social enterprises (Benson & Henderson, 2011; Mdee & Emmott, 2008). Academic organisations (ATLAS/TRAM, 2008; Palacios, 2010; Sin, 2009, 2010). Religious groups (ATLAS/TRAM, 2008). Operational arrangements Sending organisation (may be any of the above types of organisation and partnered with multiple host organisations)
(Raymond, 2008, 2011).
Host organisation (may be any of the above types of organisation and based in the destination where volunteer tourism takes place and partnered with sending organisation/s)
(Raymond, 2008, 2011).
Organisation both ‘sends’ and ‘hosts’ volunteers (ATLAS/TRAM, 2008). Main focus of organisation
Volunteer travel (ATLAS/TRAM, 2008). Niche travel (ATLAS/TRAM, 2008). Community development/service (ATLAS/TRAM, 2008). Conservation of wildlife and/or environment (ATLAS/TRAM, 2008). Religious activities (ATLAS/TRAM, 2008). Organisation motives for engaging in volunteer tourism Altruistic (Ong, Pearlman, & Lockstone-Binney, 2011; Raymond
& Hall, 2008; Scheyvens, 2002; Stoddart & Rogerson, 2004; Tomazos & Butler, 2009a, 2009b).
Commercial (e.g. to make financial profit) (ATLAS/TRAM, 2008; Brown, 2005; Coren & Gray, 2012; Cousins, Evan, & Sadler, 2009; Raymond & Hall, 2008; Tomazos & Butler, 2009a, 2009b).
Religious convictions (ATLAS/TRAM, 2008). Political objectives (ATLAS/TRAM, 2008). See also ‘Philanthropic-commercial continuum’ and ‘Spectrum between commercialisation and service’
(Coghlan & Noakes, 2012 ) (Tomazos & Cooper, 2012).
The type of volunteer tourism organisation will influence the way that it engages with volunteer tourism, the values it brings to its engagement and the motivations and objectives for engaging. Thus, this can influence what an organisation sees as important to monitor and
14
evaluate, how they do it and how the information gained from such processes is used. In addition, the type of organisation can impact on the resources available for monitoring and evaluation and an organisation’s capacity to implement recommended changes. Further, volunteer tourism organisations can have varying operational arrangements. Although the umbrella term ‘volunteer tourism organisation’ is commonly used, a ‘sending organisation’ based in a developed country will often partner with multiple ‘host organisations’. Host organisations are based in the destination country where the volunteer tourism project takes place. Host organisations may then be engaged in further partnerships with other local organisations and/or host communities which receive the volunteer tourists (Raymond, 2008, 2011). However, the complexity of these partnerships and operational arrangements have not been explored in any detail, nor has consideration been given to the effect that they have on the monitoring and evaluation processes of volunteer tourism projects. Depending on the partnerships which take place between sending organisations and host organisations, it may influence which party is deemed responsible for monitoring and evaluating various aspects (if any) of a programme. Partnerships between organisations may also effect whose goals, values and interests are represented in monitoring and evaluation processes. In partnerships which exhibit funder–fundee relationships, evaluation might be used primarily as a control mechanism to ensure that the programme is “on track” to meet the goals agreed to between the funder and fundee (Meyer, 2012), and therefore what is monitored and evaluated is largely dictated by the interests of the funder. The main focus of organisations can vary enormously. For example organisations may be focused solely on volunteer tourism, or volunteer tourism may be just one activity that the organisation is engaged in, with other activities such as religion or community development being their main focus (ATLAS/TRAM, 2008). Again, this will influence the nature of an organisation’s engagement with volunteer tourism (including nature of engagement with volunteers, host communities and host partner organisations) and the values driving monitoring and evaluation processes. It will come as no surprise then to find that an organisation’s motives for engaging with volunteer tourism can vary greatly too. Current motivations include altruism, commercial profit, religious convictions and political objectives (e.g. ATLAS/TRAM, 2008; Coghlan & Noakes, 2012; Ong, Pearlman, & Lockstone-Binney, 2011). These different motivations influence the types of programmes which are offered, the types of volunteers recruited and an organisations attitude towards responsible practice, and thus monitoring and evaluation. Evaluation can also function to legitimate the various activities of an organisation to third-parties (Meyer, 2012). The different motivations of organisations will therefore influence which activities and goals the organisation seeks to legitimate, determining which activities and goals are evaluated and the dissemination of evaluation findings. Dimension 3: Markets The literature reveals a number of sub-dimensions within volunteer tourism which fall under the broad dimension ‘markets’ (Table 4). A key sub-dimension identified includes the flow of volunteers. At present the most common flow is for volunteers from developed countries to travel to volunteer in developing countries (ATLAS/TRAM, 2008). However it is becoming increasingly popular for volunteers from developed countries to travel within their own or to other developed countries, and for volunteers from developed countries to volunteer within both developed countries and developing countries (Mwaruta, 2008, 2012). Depending on the flow of volunteers, it will change the context of the programme and the interaction between organisations, volunteers and host communities. For example, Raymond and Hall (2008) argue that programmes which involve western volunteers being sent to developing
15
countries must be wary of reinforcing negative cultural stereotypes and cross-cultural misunderstanding. In addition, Guttentag (2009) warns that volunteer tourism may promote dependency between developed and developing countries. These are important issues that organisations operating in this context need to be aware of, requiring on-going monitoring and evaluation. Another sub-dimension is the age of volunteers. This can vary from high school age to volunteers who have retired. Different age groups can potentially have different priorities and thus organisations may seek to evaluate different criteria according to age groups (e.g. Callanan & Thomas, 2005). The skills set required from volunteers also varies widely from programme to programme ranging from no relevant skills required through to specific skills and experience being a necessity (Callanan &Thomas, 2005). For example, younger or unskilled volunteers may prioritise learning new skills, whilst older or skilled volunteers may prioritise sharing their skills. Organisations may wish to evaluate the extent to which each of these varying priorities has been met from the volunteer perspective and also whether or not the volunteers have been able to fulfil the role they have been given from a programme perspective. A further sub-dimension is that volunteer tourists can volunteer individually, as a family, as a couple or with an academic or religious group (e.g. ATLAS/TRAM, 2008; Clemmons, 2009). This will influence the type of volunteer tourism experience being sought. In addition, volunteer tourists can have a diverse array of motivations ranging from altruism to egotism, which can influence how the volunteer approaches their chosen programme and the type of experience they are seeking (e.g. Lo & Lee, 2011). Egoistic volunteers may be primarily focused on their own personal development (e.g. Lo & Lee, 2011) or increasing social capital (e.g. Simpson, 2004), whereas altruistic volunteers may be more concerned about “giving back” (Brown, 2005, p.488). Organisations will need to take these different motivations and contexts into account when reviewing feedback from volunteers.
Table 4: Markets
Dimension References Flow of volunteers *North-‘developed countries’, South-‘developing countries’ North-South (ATLAS/TRAM, 2008; Gray & Campbell, 2007; Palacios,
2010; Scheyvens, 2002, 2011; Simpson, 2004; Tomazos & Cooper, 2012; Wearing, 2001).
North-North (ATLAS/TRAM, 2008; Ellis, 2003; Erdely, 2011; Higgins-Desboilles, 2003; McIntosh & Zahra, 2007).
South-South (Mwaruta, 2008, 2012).
South-North (Mwaruta, 2008, 2012).
Volunteer age group
Secondary/high school (ATLAS/TRAM, 2008).
‘Gap Year’ (18-24) (Callanan & Thomas, 2005).
‘Young’ (18-30) (Callanan & Thomas, 2005)
‘Middle-aged’ (31-54) (Callanan & Thomas, 2005).
Retired (55+) (Callanan & Thomas, 2005).
Volunteer/s
Travelling alone (ATLAS/TRAM, 2008).
16
School or university travelling as a group (Palacios, 2010).
Family volunteering (Clemmons, 2009).
Honeymooners/couples (De Castella, 2011; Nash, 2010).
Religious group volunteering (ATLAS/TRAM, 2008).
Volunteer motivations
Egotistic (e.g. personal/self-development (Lo & Lee, 2011; Sin, 2009; Wearing, 2001, 2002) and/or to increase social capital (Simpson, 2004; Sin, 2009; Snöderman & Snead, 2008)).
Altruistic (e.g. desire to ‘give something back’ ( Brown, 2005; Lo & Lee, 2011) and/or to conserve wildlife or the environment (Broad, 2003; Lyons, 2003; Wearing, 2002; Wearing & Neil, 2001)).
Religious involvement (ATLAS/TRAM, 2008; Lo & Lee, 2011; McGehee & Andereck, 2009).
‘Pluralistic motivations’ (both altruistic and egotistic)
(McGehee & Andereck, 2009).
Shallow Intermediate Deep continuum (Callanan & Thomas, 2005).
Volunteer skills set required
No relevant/limited skills (Callanan & Thomas, 2005).
Limited to moderate skills (Callanan & Thomas, 2005).
Skills, experience and/or qualifications related specifically to work being undertaken
(Callanan & Thomas, 2005)(See also Raymond (2007) on ‘the match-up process’).
Dimension 4: Programmes Finally, a number of sub-dimensions which fall under the broad dimension ‘programmes’ are identified (Table 5). These range from community development and conservation to more ambiguous volunteer tourism phenomena such as service learning, cultural exchange and charity fundraisers (e.g. Sin, 2010; Broad, 2003; Lyons &Wearing, 2012). The focus of the programme will likely influence what is monitored and evaluated and which stakeholder’s feedback is sought. As a result, overarching prescriptions about what questions to ask should be treated with caution, with more nuanced context and programme specific questions needed. In addition, the duration of a volunteer tourism experience can range from a couple of hours if the trip is an ‘add on’ activity to an existing holiday, to twelve months and longer (Raymond, 2007. This may influence the level of engagement with stakeholders and the perceived need for monitoring and evaluation processes. Table 5: Programmes
Dimension References
Focus Community development (can include but not limited to: construction and refurbishment of
(Ingram, 2010; Palacios, 2010; Sin, 2010; Stoddart & Rogerson, 2004; Vodopivec & Jaffe, 2011).
17
buildings, social work, orphanage work, education activities) Conservation (animal and environmental preservation) (e.g. Broad,
2003; Campbell & Smith, 2006; Coghlan, 2007a, 2007b; Cousins, 2007; Gray & Campbell, 2007; Wearing, 2001) and/or research (Clifton & Benson, 2006; Ellis, 2005; Ellis, 2003).
Service Learning (Lyons, 2003; Lyons & Wearing, 2008a, 2012). Cultural Exchange (Lyons & Wearing, 2008a, 2012). Medical
(Martiniuk & Negin, 2012).
Fundraising adventures
(Lyons & Wearing, 2008a, 2012).
Type of programme One off single ‘project’ (See discussion by Papi 2010; Raymond, 2007, 2011). Project contributes to a wider programme which the volunteer tourism organisation has a longer term commitment to
(PEPY Tours n.d; Fair Trade Volunteering n.d.).
Individual placement with an organisation (Raymond, 2007). Multi-project (‘volunteers spend 1 or 2 days on a variety of projects’)
(Raymond, 2007, p.5). Duration of volunteer experience Micro/day trip
(e.g. 2 hour ‘Reading Road Trip’ with Sandals Resorts International (2011) and ‘Crystal Voluntourism Adventures with Crystal Cruises (n.d.)).
Short term -up to 3 months (ATLAS/TRAM, 2008). Medium term -typically less than 6 months (Callanan & Thomas, 2005). Long term – up to 12 months or longer (ATLAS/TRAM, 2008). Amount of time spent volunteering versus leisure/ holidaying Almost all the time (Volunteering is the main focus of the programme and may include a day or two leisure experience)
(Lamoureux, 2011).
Some of the time (volunteering takes up a minimal amount of the overall programme)
(Lamoureux, 2011).
However, many variations in between these two ends of the spectrum “depending on the desire of the volunteer and the needs of the receiving organisation”
(Lamoureux, 2011).
Depth/Integrity of Programme Shallow -Intermediate -Deep continuum (See Callanan and Thomas (2005, p.198) conceptual
framework for volunteer tourism projects). Further niches in Volunteer Tourism Surf Voluntourism
(Aabo & Ponting, 2011).
Dive Voluntourism
(www.divevoluntourism.com; Walsh & Hampton, 2011).
Archaeological Voluntourism (Kaminski, Arnold, & Benson, 2011). Honeyteering (De Castella, 2011; Nash, 2010).
Volunteer tourism programmes can also differ in the amount of time that is spent volunteering and actually in service versus the amount of time that is spent taking part in leisure and other recreational activities (Lamoureux, 2011). Again this may influence the level of importance placed on monitoring and evaluation (i.e. organisation may not see value in evaluating a mini-trip given its short duration). There are also different types of
18
programmes (e.g. ATLAS/TRAM, 2008; Callanan & Thomas, 2005). Whether or not a volunteer tourism programme is a one-off project, part of a wider ongoing programme or a placement with a partner organisation will also influence what information is sought in monitoring and evaluation processes and which parties are deemed responsible for the monitoring and evaluation of the programme. Finally, the depth/integrity of a programme may also influence the level of engagement with monitoring and evaluation processes. For example, organisations operating “shallow” programmes may not be concerned with evaluating their impacts on the local community and focus on the volunteer experience. On the other hand, those operating “deep” programmes may prioritise the needs and feedback of the local community (Callanan & Thomas, 2005). A CMS lens encourages questioning the motives of organisations that prioritise the needs of volunteers above the needs of the host community. An analytical framework for volunteer tourism programme monitoring and evaluation
We have demonstrated that the context in which a volunteer tourism programme is embedded varies greatly. Consequently, to make sweeping prescriptions of specific methods for carrying out monitoring and evaluation would be inappropriate. However, we do think that a more applied, rather than prescriptive stimulus tool, in the form of an analytical framework would be of value to both those working in the realm of volunteer tourism programme monitoring and evaluation and those interested in examining an organisation’s current programme monitoring and evaluation practices. Building upon Figure 1 which outlines the context for monitoring and evaluation of volunteer travel programmes, Figure 2 aims to promote discussion and deeper consideration of the issues that have been outlined in this paper. In doing so it has the potential to provide a deeper and more critical approach to monitoring and evaluation that, if findings are acted upon, may improve the positive impacts of volunteer tourism on local communities. In line with the discussion throughout this paper, the hybrid framework shown in Figure 2 incorporates both consideration of context and the various dimensions that may influence volunteer tourism programme monitoring and evaluation. In addition, some basic, generic monitoring and evaluation processes are outlined, whilst recognising that specific processes and methods will likely differ by case. Each component includes CMS inspired guiding considerations that can be utilised by both organisations and those seeking to examine an organisation's monitoring and evaluation practices.
19
Figure 2. An analytical framework for volunteer tourism programme monitoring and evaluation, including contextual influences and monitoring and evaluation processes.
Conclusion
This paper reviewed the literature on monitoring and evaluation of volunteer tourism and presents an analytical framework aimed at improving the quality of monitoring and evaluation in the volunteer tourism sector. Within the current climate of growing concerns about the increasing commercialisation of the volunteer tourism industry and the potential for negative impacts to host communities, the monitoring and evaluation of volunteer tourism programmes is identified as essential for sustainable tourism management. Through an analysis and synthesis of the literature, this paper has demonstrated that monitoring and evaluation are value-laden processes, embedded within uneven power relations, agendas and interests. Claims that programmes are being monitored and evaluated can be little more than
What are the interests & values driving organisation in relation to M&E? Which stakeholders does monitoring and evaluation aim to represent ? Who is/ not included?
Who are beneficiaries of the programme - in what ways and to what extent are they included in M&E? Are a range of stakeholders represented?
What available resources and expertise does organisation have for M&E? Are quantitative, qualitative or mixed methods appropriate? Does gathering information from different stakeholders require different approaches? Which stakeholders are/ should be included in making above decisions?
What kinds of data to gather? Who and what does this data represent? Are there any external or internal influences which may bias feedback from stakeholders?
How will findings be utilised? Who will findings be shared with?
Does organisation have capacity to implement potential changes/ developments? Is termination of programme necessary?
Actual programme to be evaluated (First, understand
dimensions of context that influence VTP)
Determine purpose for monitoring
and evaluation
Identify programme beneficiaries
Collect data
through monitoring
Evaluate
Review findings
Disseminate
Determine appropriate approach & methods
Programme revisions based on information gained via monitoring & evaluation
Markets Programmes Organisations Stakeholders
Contextual dimensions Each embedded within further social, cultural, historical, economic & political contexts
20
rhetoric: there is a need for closer examination by asking critical questions including: Evaluated by whom? To what end? What interests are embedded in the evaluation process? It is through answering these questions that greater transparency and accountability of volunteer tourism programmes can be achieved and this type of tourism can genuinely make a contribution to sustainable development. To this end, this paper has argued for qualitative, critical approaches to monitoring and evaluation in particular, because they can promote dialogue between a range of key stakeholders about how volunteer tourism programmes can be improved and determine if a programme has achieved the desired goals.
The contribution of this paper is in providing a foundation for critically examining how volunteer tourism organisations currently engage in monitoring and evaluating their programmes, acknowledging that monitoring and evaluation are not neutral processes. The qualitative, critical framework goes beyond taken for granted and uncritical approaches to evaluation and such a framework contributes theoretically to the existing literature and debates on volunteer tourism. In addition, the paper provides a stimulus tool in the form of an analytical framework (Figure 2) to assist those involved in managing volunteer tourism to identify which context they place their volunteer tourism activities in, promoting situated discussions of how monitoring and evaluation might take place within their particular context and which methods could be appropriate. Given the contextual nature of volunteer tourism, it would be inappropriate to provide prescriptive methodological guidance on how to monitor and evaluate volunteer travel programmes. Rather, drawing from Rog’s (2012) “context first approach”, once evaluators have considered the implications of the particular context/s a programme is situated within, it can then be decided which evaluation methods are most appropriate within that space. Further research which empirically interrogates the practical implications of the landscape through a critical lens, such as that advocated by CMS, would provide enlightening perspectives, promoting new knowledge on the subject. New knowledge and understandings of how organisations monitor and evaluate their programmes within the complex landscape in which they are located could serve to support and inform practical guidance for the monitoring and evaluation of volunteering tourism programmes within a variety of contexts, fulfilling an essential part of an organisation’s duty of care to both volunteer tourists and host communities. References Aabo, D, & Ponting, J. (2011). Surf VolunTourism: Go Surf, Stay to Serve. VolunTourist
Newsletter, 7(2). Alexander, Z, & Bakir, A. (2011). Understanding voluntourism. A Glaserian grounded theory
study. In A. Benson (Ed.), Volunteer Tourism. Theory Framework to Practical Applications (pp. 9-29). Oxon, UK: Routledge.
Alvesson, M., & Ashcraft, K L. (2009). Critical Methodology in Management and Organization Research. In D. Buchanan & A. Bryman (Eds.), The SAGE Handbook of Organizational Research Methods. London: SAGE Publications Ltd.
Alvesson, M., & Deetz, S. (2005). Critical Theory and Postmodernism: Approaches to Organizational Studies. In C. Grey & H. Willmott (Eds.), Critical Management Studies. Oxford: Oxford University Press.
21
Ateljevic, I, Pritchard, N, & Morgan, N. (2007). The critical turn in tourism studies : innovative research methodologies. Oxford, UK: Elsevier.
ATLAS/TRAM. (2008). Volunteer tourism: A global analysis. Barcelona: ATLAS. Bamberger, M. (2013). The Mixed Methods Approach to Evaluation. Social Impact, 1(June),
1-18. Barbieri, C., Santos, C.A., & Katsube, Y. (2012). Volunteer tourism: On-the-ground
observations from Rwanda. Tourism Management, 33(3), 509-516. Bartle, P. (2007). The Nature of Monitoring and Evaluation: Definition and Purpose.
Retrieved 4 July 2012 from: http://cec.vcn.bc.ca/cmp/modules/mon-wht.htm. Benson, A. (2011a). Volunteer tourism: Structuring the research agenda. In A. Benson (Ed.),
Volunteer Tourism: Theory framework to practical applications. Oxon, UK: Routledge.
Benson, A (Ed.). (2011b). Volunteer tourism: Theory framework to practical applications. London: Routledge.
Benson, A., & Henderson, S. (2011). A strategic analysis of volunteer tourism organisations. Service Industries Journal, 31(3), 405-424.
Birrell, I. (2010). Before you pay to volunteer abroad, think of the harm you might do, The Observer. Retrieved 15 January 2011 from: http://www.guardian.co.uk/commentisfree/2010/nov/14/orphans-cambodia-aids-holidays-madonna.
Boffey, D. (2011). Students given tips to stop gap year travel being 'a new colonialism'. The Guardian (online). Retrieved 31 July 2011 from: http://www.guardian.co.uk/education/2011/jul/30/gap-year-volunteers-demos.
Broad, S. (2003). Living the Thai life- A case study of volunteer tourism at the Gibbon Rehabilitation Project, Thailand. Tourism Recreation Research, 28(3), 63-72.
Brown, F., & Hall, D. (2008). Tourism and Development in the Global South: The issues. Third World Quarterly, 29(5), 839-849.
Brown, S. (2005). Travelling with a Purpose: Understanding the Motives and Benefits of Volunteer Vacationers. Current Issues in Tourism, 8(6), 479-496.
Buchanan, D., & Bryman, A. (2009). The Organizational Research Context: Properties and Implications. In D. Buchanan & A. Bryman (Eds.), The SAGE Hanbook of Organizational Research Methods. London: SAGE Publications Ltd.
Butcher, J. (2006). A Response to Building a Decommodified Research Paradigm in Tourism: The Contribution of NGOs by Stephen Wearing, Matthew McDonald and Jess Ponting. Journal of Sustainable Tourism, Vol. 13, No. 5, 2005: 424-455. Journal of Sustainable Tourism, 14(3), 307-310.
Butcher, J. (2011). Volunteer Tourism May Not be as Good as it Seems. Tourism Recreation Research, 36(1), 75-76.
Butcher, J., & Smith, P., (2010). 'Making a Difference': Volunteer Tourism and Development. Tourism Recreation research, 35(1), 27-36.
Callanan, M., & Thomas, S. (2005). Volunteer Tourism. Deconstructing volunteer activities within a dynamic environment. In M. Novelli (Ed.), Niche tourism. Contemporary issues, trends and cases. Oxford: Elsevier Butterworth-Heinemann.
Campbell, D. T. (1969). Reforms as Experiments. American Psychologist, 24: 409-429. Campbell, L., & Smith, C. (2006). What makes them pay? Values of volunteer tourists
working for sea turtle conservation. Environmental Management, 38(1), 89-98. Clemmons, D. (2009). Multi-Generational & Family VolunTourism. Voluntourism.org.
Retrieved 25 June 2012 from: http://blog.voluntourism.org/?p=100.
22
Clifton, J., & Benson, A. (2006). Planning for Sustainable Ecotourism: The case for research eco-tourism in developing country destinations. Journal of Sustainable Tourism, 14(3), 238-254.
Coghlan, A., (2007a). Towards and integrated image-based typology of volunteer tourism organisations. Journal of Sustainable Tourism, 15(3), 267-287.
Coghlan, A., (2007b). Volunteer tourism. A better understanding of the volunteer experience Tourism Research Report 4. Australia: James Cook University.
Coghlan, A., & Noakes, S. (2012 ). Towards a philanthropic continuum for volunteer tourism: A consideration of organisational drivers. Tourism Recreation Research, 37(2), 123-131.
Comhlámh. (2011). Code of Good Practice for Sending Organisations: 11 principles of good practice in volunteering for global development Dublin, Ireland.
Conner, R.F., Fitzpatrick, J.L., & Rog, D.J. (2012). A First Step Forward: Context Assessment. New Directions For Evaluation(135), 89-105.
Coren, N., & Gray, T. (2012). Commodification of Volunteer Tourism: a Comparative Study of Volunteer Tourists in Vietnam and in Thailand. International Journal of Tourism Research, 14(3), 222-234.
Cousins, J. A., (2007). The role of UK-based conservation tourism operators. Tourism Management, 28, 1020-1030.
Cousins, J.A., Evan, J, & Sadler, J. (2009). Selling Conservation? Scientific Legitimacy and the Commodification of Conservation Tourism. Ecology and Society, 14(1), 32-49.
Cousins, J.B., & Whitmore, E. (1998). Framing Participatory Evaluation. New Directions for Evaluation, 80, 87-105.
Cullen, A., & Coryn, C. (2011). Forms and Functions of Participatory Evalution in International Development: A Review of the Empirical and Theoretical Literature. Journal of MultiDisciplinary Evaluation, 7(16), 47.
De Araujo, L.M. and Bramwell, B. (1999) Stakeholder assessment and collaborative tourism planning: The case of Brazil’s Costa Dourada project. Journal of Sustainable Tourism, 7 (3–4), 356–378.
De Castella, T., (2011). Would you do voluntary work on your honeymoon? BBC News Magazine (online). Retrieved 2 July 2011from: http://www.bbc.co.uk/news/magazine-12381477.
Deetz, S. (2009). Organisational Research as Alternative Ways of Attending to and Talking About Structures and Activities. In D. Buchanan & A. Bryman (Eds.), The SAGE Handbook of Organizational Research Methods. London: SAGE Publications Ltd.
Ellis, C. (2005). Tourism, Volunteers and Environmental Researchers. An Analysis of Participatory Environmental Research Tourism. (Doctoral Dissertation), University of Tasmania, Tasmania.
Ellis, C. (2003). Participatory Environmental Research Tourism - A Global View. Tourism Recreation Research, 28(3), 45-55.
Erdely, J.L. (2011). When The Saints Go Marching In: An Ethnography of Volunteer Tourism in Post-Hurricane Katrina New Orleans (Doctor of Philosophy PhD Thesis), Louisiana State University and Agricultural and Mechanical College.
Fee, L., & Mdee, A. (2010). How does it make a difference? Towards 'accreditation' of the development impact of volunteer tourism. In A. Benson (Ed.), Volunteer tourism: Theory framework to practical applications. London: Routledge.
Fitzpatrick, J., Christie, C., & Mark, M. (2009). Evaluation in Action. Interviews with Expert Evaluators. Thousand Oaks, California: SAGE Publications.
Font, X. & Smith, V.L. (2014) Journal of Sustainable Tourism
23
Garland, E. (2012). How should anthropologists be thinking about tourism? Practicing Anthropology, 34(3), 5-9.
Gray, N. J., & Campbell, L.M. (2007). A Decommodified Experience? Exploring Aesthetic, Economic and Ethical Values for Volunteer Ecotourism in Costa Rica. Journal of Sustainable Tourism, 15(5), 463-482.
Greene, J., (2002). Mixed-method evaluation: A way of democratically engaging with difference. Evaluation Journal of Australiasia, 2 (2), 23-29.
Greene, J., (2005). A Value-Engaged Approach for Evaluating the Bunche-Da Vinci Learning Academy. New Directions in Evaluation, 106, 27-45.
Greene, J., (2012). Values-Engaged Evaluation (Webinar). My M&E: Retrieved 1 September 2012 from: http://www.mymande.org/index.php?q=content/values-engaged-evaluation.
Guba, E., & Lincoln, Yvonna S. (1989). Fourth Generation Evaluation. Newbury Park, California: SAGE Publications.
Guttentag, D.A. (2009). The Possible Negative Impacts of Volunteer Tourism. International Journal of Tourism Research, 11, 537-551.
Guttentag, D.A. (2011). Volunteer Tourism: As Good as It Seems? Tourism Recreation Research, 36(1), 69-74.
Higgins-Desbiolles, F. (2003). Reconciliation tourism: Tourism healing divided societies! Tourism Recreation Research, 28(3), 35-44.
Ingram, J. (2010). Volunteer tourism: how do we know it is 'making a difference'? In A. Benson (Ed.), Volunteer tourism: Theory framework to practical applications. London: Routledge.
Jackson, E., & Kassam, Y (Eds.). (1998). Knowledge Shared: Participatory Evaluation in Development Cooperation. Connecticut, USA: Kumarian Press.
Jafari, J. (2001). The scientification of tourism. In V. L. Smith & M. Brent (Eds.), Hosts and guests revisited: Issues of the 21st century (pp. 28-41). Elmsford, New York: Cognizant Communications Corporation.
Kaminski, J., Arnold, D., & Benson, A. (2011). Volunteer archaelogical tourism: an overview. In A. Benson (Ed.), Volunteer Tourism: Theory Framework to Practical Applications. Oxon, UK: Routledge.
Keese, J. (2011). The Geography of Volunteer Tourism: Place Matters. Tourism Geographies, 13(2), 257-279.
Kincheloe, J., & McLaren, P. (2005). Rethinking critical theory and qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The SAGE Handbook of Qualitative Research. Thousand Oaks, Califorina: SAGE Publications.
Lamoureux, K. (2011). Chapter 3: Voluntourism. In M. Honey (Ed.), Travelers’ Philanthropy Handbook. Washington, DC: Center for Responsible Travel.
Lewis, D. (2001). The management of non-‐governmental development organizations: An introduction. London, UK: Routledge.
Lo, S., & Lee, C. (2011). Motivations and perceived value of volunteer tourists from Hong Kong. Tourism Management, 32(2), 326-334. doi: 10.1016/j.tourman.2010.03.002
Lyons, K.D. (2003). Ambiguities in Volunteer Tourism: A Case Study of Australians Participating in a J-1 Visitor Exchange Programme. Tourism Recreation Research, 28(3), 5-13.
Lyons, K.D., & Wearing, S. (2008a). All for a Good Cause? The Blurred Boundaries of Volunteering and Tourism. In K. Lyons & S. Wearing (Eds.), Journeys of Discovery in Volunteer Tourism: International Case Study Perspectives. Cambridge: CABI Publishing.
24
Lyons, K.D., & Wearing, S. (2008b). Volunteer tourism as alternative tourism: Journeys beyond otherness. In K. Lyons & S. Wearing (Eds.), Journeys of Discovery in Volunteer Tourism: International Case Study Perspectives (pp. 3-11). Cambridge: CABI Publishing.
Lyons, K.D., & Wearing, S. (2012). Reflections on the Ambiguous Intersections between Volunteering and Tourism. Leisure Sciences, 34(1), 88-93.
Martiniuk, A., & Negin, J. (2012). Voluntourism: the downsides of medical missions. ABC News (online). Retrieved 1 September 2012 from: http://www.abc.net.au/unleashed/4064550.html.
Mascarenhas, A., Coelho, P., Subtil, E., Ramos, T.B. (2010). The role of common local indicators in regional sustainability assessment. Ecological Indicators,10 (3), 646-656.
Mayhew, F. (2011). Integrating the Funder-Fundee Relationship into a Model of Evaluation Utilization. Journal of MultiDisciplinary Evaluation, 7(16), 1-19.
McGehee, N., & Andereck, K. (2008). 'Pettin' the Critters': Exploring the Complex Relationship Between Volunteers and the Voluntoured in McDowell County, West Virginia, USA and Tijuana, Mexico. In K. Lyons & M. Wearing (Eds.), Journeys of Discovery in Volunteer Tourism. International Case Study Perspectives (pp. 12-24). Oxfordshire: CABI.
McGehee, N., & Andereck, K. (2009). Volunteer tourism and the "voluntoured": the case of Tijuana, Mexico. Journal of Sustainable Tourism, 17(1), 39-51.
McIntosh, A., & Zahra, A. (2007). A Cultural Encounter through Volunteer Tourism: Towards the Ideals of Sustainable Tourism? Journal of Sustainable Tourism, 15(5), 541-556.
Mdee, A., & Emmott, R. (2008). Social enterprise with international impact: the case for Fair Trade certification of volunteer tourism. Education, Knowledge & Economy, 2(3), 191-201.
Mertens, D. M. (2010). Transformative Mixed Methods Research. Qualitative Inquiry, 16(6), 469-474.
Mertens, D.M. (2012). Evaluation for Social Justice (Webinar). My M&E: Retrieved 1 September 2012 from: http://mymande.org/index.php?q=content/evaluation-social-justice.
Mertens, D. M., & Wilson, A.T. (2012). Program Evaluation Theory and Practice. New York, The Guildford Press.
Meyer, W. (2012). Should Evaluation be Revisted for Sustainable Development. In A. von Raggamby & F. Rubik (Eds.), Sustainable Development, Evaluation and Policy-Making: Theory, Practice and Quality Assurance. Cheltenham, UK: Edward Elgar Publishing Limited.
Mwaruta, R. (2008). Volunteer Tourism: Yet another Form of “Othering”? (MSc Leisure, Tourism and the Environment Masters thesis), Wageningen University.
Mwaruta, R. (2012). Volunteer Tourism Yet: Another Form of "Othering"?: International Voluntary Service a new form of missionaries? . Germany: LAP LAMBERT Academic Publishing
Nash, M. (2010). Helpful honeymooners: Why newlyweds volunteer abroad. CNN Travel. Retrieved 1 September 2012 from: http://articles.cnn.com/2010-08-20/travel/helpful.honeymooners.abroad_1_volunteer-honeymoon-peace-corps?_s=PM:TRAVEL.
Nelson, E. (2010). A Community Perspective on Volunteer Tourism and Development in South Africa. (Master of Arts Masters Thesis), Miami University, Oxford, Ohio.
25
Ong, F., Pearlman, M., & Lockstone-Binney, L. (2011). An examination of not-forprofit volunteer tourism sending organisations' guiding considerations that influence volunteer tourism programmes. World Leisure Journal, 53(4), 296-311.
Ong, F., Pearlman, M., Lockstone-Binney, L., & King, B. (2013). Virtuous volunteer tourism: Towards a uniform code of conduct. Annals of Leisure Research, 16(1), 72-86.
Palacios, C. (2010). Volunteer tourism, development and education in a postcolonial world: conceiving global connections beyond aid. Journal of Sustainable Tourism, 18(7), 861-878.
PEPYTours. (n.d). Voluntourism 101: Operator self check guide. Retrieved 18 July 2012 from: http://voluntourism101.com/voluntourism101_self-check_tool.pdf.
Power, S. (2007). Gaps in Development. An analysis of the UK international volunteering sector. from http://www.tourismconcern.org.uk.
Raymond, E.M. (2007). Volunteer Tourism in New Zealand: The role of sending organisations in ensuring that volunteer tourism programmes ‘make a difference’.
Raymond, E.M. (2008). 'Make a Difference!': the Role of Sending Organizations in Volunteer Tourism. In K. Lyons & S. Wearing (Eds.), Journeys of Discovery in Volunteer Tourism: International Case Study Perspectives Cambridge: CABI Publishing.
Raymond, E.M. (2011). Volunteer Tourism: Looking Forward. Tourism Recreation Research, 36(1), 77-79.
Raymond, E.M., & Hall, C.M. (2008). The Development of Cross-Cultural (Mis)Understanding Through Volunteer Tourism. Journal of Sustainable Tourism, 16(5), 530-543.
Robson, C. (2000). Small-Scale Evaluation. London: SAGE Publications Limited. Rog, D.J. (2012). When Background Becomes Foreground: Towards Context-Sensitive
Evaluation Practice. New Directions for Evaluation(135), 25-40. Scheyvens, R. (2002). Tourism for Development: Empowering Communities. Essex, England:
Pearson Education Limited. Scheyvens, R. (2011). Tourism & Poverty. Oxon, UK: Routledge. Simpson, K. (2004). 'Doing development': the gap year, volunteer-tourists and a popular
practice. Journal of International Development, 16(5), 681-692. Simpson, M. (2008). Community Benefit Tourism Initiatives-A conceptual oxymoron?
Tourism Management, 29, 1-18. Sin, HL. (2009). Volunteer tourism-"Involve me and I will learn"? Annals of Tourism
Research, 36(3), 480-501. Sin, HL. (2010). Who are we responsible to? Locals’ tales of volunteer tourism. Geoforum,
41(6), 983-992. Snöderman, N., & Snead, S. (2008). Opening the gap: The motivation of gap year travellers
to volunteer in Latin America. In K. D. Lyons & S. Wearing (Eds.), Journeys of discovery in volunteer tourism: International case study perspectives (pp. 118–129). Wallingford, UK: CABI.
Stoddart, H., & Rogerson, C.M. (2004). Volunteer tourism: The case of Habitat for Humanity South Africa. GeoJournal, 60, 311-318.
Stufflebeam, D., & Shinkfield, A. (2007). Evaluation, Theory, Models, & Applications. San Francisco: Jossey-Bass.
Timothy, D. J. (2002). Tourism and Community Development Issues. In R. Sharpley & D. J. Telfer (Eds.), Tourism and Development: Concepts and Issues (pp. 149-164). Clevedon, UK: Channel View Publications.
26
The International Ecotourism Society (2012). International Voluntourism Guidelines for Commercial Tour Operators. Retrieved 10 May 2013 at: http://www.ecotourism.org/voluntourism-guidelines.
Tomazos, K., & Butler, R. (2009a). Volunteer Tourism: The New Ecotourism? Anatolia: An International Journal of Tourism of Tourism and Hospitality Research, 20(1), 196-211.
Tomazos, K., & Butler, R. (2009b). Volunteer Tourism: Working on holiday or playing at work? Tourismos, 4(4), 331-349.
Tomazos, K., & Cooper, W. (2012). Volunteer tourism: at the crossroads of commercialization and service? Current Issues in Tourism, 15(5), 405-423.
Vodopivec, B., & Jaffe, R. (2011). Save the World in a Week: Volunteer Tourism, Development and Difference. European Journal of Development Research, 23(1), 111-128.
Wearing, S. (2001). Volunteer Tourism. Experiences that make a difference. Oxon, UK: CABI Publishing.
Wearing, S. (2002). Re-centering the self in volunteer tourism. In G. S. Dann (Ed.), The Tourist as Metaphor of the Social World (pp. 237-262). Oxon, UK: CABI
Wearing, S. (2004). Examining best practice in volunteer tourism. In R. Stebbins & M. Graham (Eds.), Volunteering as leisure/ leisure as volunteering: an international assessment. Cambridge MA: CABI.
Wearing, S., & Darcy, S. (2011). Inclusion of the "Othered" in Tourism. Cosmopolitan Civil Societies Journal, 3(2), 18-34.
Wearing, S., & Neil, J. (2001). Refiguring self and identity through volunteer tourism. Loisir & Societe, 23(2), 387-417.
Wearing, S., & McGehee, N. (2013). Volunteer tourism: A review. Tourism Management, 38, pp.120-130.