1

[CPTS FINAL EVALUATION] Alice Schmidt

The Child Protection Trainee Scheme (CPTS) Final Evaluation Alice Schmidt December 2011 office@aliceschmidt.at +4369910606673 www.aliceschmidt.at

2

Table of Content Glossary 3

Executive Summary 4

Introduction 6

Methodology 7 Analytical framework 7 Desk research 7 Empirical research 7 Constraints 10

Findings and Analysis 11 Overall success of the scheme 11 Quality of the scheme 13 Recruitment and diversity 18 Blended learning approach 23 Coaching 27 Career paths 29 Tools 31 Networks 31 Outlook 32

Conclusions and Recommendations 34

Annexes 37 Annex 1 – list of individuals who responded to the online survey 37 Annex 2 – list of individuals interviewed individually and in focus group discussions 38 Annex 3 – CPTS graduates and their current positions 39 Annex 4 – questionnaire used for semi-structured interviews and focus group discussions 41 Annex 5 – CPTS logical framework 46

3

Glossary CBHA Consortium of British Humanitarian Agencies CPIE Child Protection in Emergencies CPTS Child Protection Trainee Scheme CPWG Child Protection Working Group DFID Department for International Development EHVP European Humanitarian Volunteer Programme ELRHA Enhancing Learning & Research for Humanitarian Assistance HLDP Humanitarian Leadership Development Programme HSDP Humanitarian Skills Development Programme IASC Inter-Agency Standing Committee (on humanitarian assistance) NOHA Network of Humanitarian Assistance ORCE Observe, Record, Classify and Evaluate R&R Rest and Relaxation

4

Executive Summary This final evaluation of the Child Protection Trainee Scheme (CPTS) was conducted based on a review of key background documents, analysis of raw data collected by the scheme’s managers during the course of the scheme and, most importantly, an empirical exercise consisting of an online survey, semi-structured interviews with key stakeholders and focus group discussions with trainees from different intakes. In total, responses were received from 60 individuals, most of whom completed the online survey and many of whom additionally participated in interviews or focus group discussions. The analytical framework applied is a combination of the CPTS project logical framework (as designed and agreed with DFID, the project donor) with nine interrelated themes that have emerged as particularly important over the duration of the scheme. These are as follows: Overall success of the scheme Quality of the scheme and its graduates Recruitment and diversity of CPTS trainees Blended learning approach Career paths of CPTS graduates Coaching quality Tools: Coaching Toolkit and Competency Framework Professional networks Outlook on CPIE capacity building

Overall, the CPTS has been very successful in strengthening organisational capacity, improving interagency collaboration, ensuring major humanitarian responses have a child protection component and in increasing speed of deployment. CPTS graduates, most of whom have gone on to work in the sector of child protection in emergencies and did not take long to find a job, are seen as reliable resources. In other words, with very few exceptions, CPTS graduates are seen as excellent child protection professionals. The focused nature of the scheme and its blended learning model have been particularly appreciated. Combining invaluable in-country placements with training and networking opportunities at central level, as well as other systematically structured learning opportunities, has proven to be a successful approach in maximising learning outcomes. The fact that the CPTS has provided a systematic framework for coaching is also positively recognised. However, trainees’ coaching experiences have been very diverse and, while in most cases providing good opportunities for learning no matter what the experience, a number of CPTS graduates feel that better coaching would have improved their experience overall. While coaching quality is more difficult to control, particularly from central level, the fact that the CPTS has not managed to recruit a diverse range of trainees – despite efforts to do so – is a key limitation that future schemes should aim to overcome if diversity is maintained as goal. Since it is widely recognised that diversity is essential, not least in regard to ensuring sustainability of the investments made in human resource capacity, more and more rigorous efforts at improving diversity may be necessary. The tools and networks created by the CPTS are widely appreciated and, alongside agency-specific tools, used for recruitment, self-assessment, information sharing and standard setting. The Competency Framework, based on the core capabilities framework for child protection workers developed by Save the Children for the CPTS in 2007, stands out as an important, comprehensive tool. It nevertheless may need to be refined to better reflect intercultural realities and to become a key tool for use also within individual agencies.

5

Calls for a continuation of the Child Protection Trainee Scheme are loud and clear. Respondents agree that a discontinuation would leave a gap strongly felt by the sector, particularly as graduates gravitate towards headquarter-bound jobs after having worked in unstable settings for a number of years. Nevertheless, there is also significant support for a new scheme, such as the Advanced Diploma currently being developed under a CPWG umbrella that targets mid-level professionals rather than entry-level individuals. In other words, many individuals working in the sector of child protection in emergencies feel that there is room for two schemes. They also believe that, while the scheme’s costs cannot be neglected and sustainability is important, it has provided good value for money. Respondents were asked to comment on the necessity of focusing on capacity building in emergency settings. Overall, there is wide agreement that capacity building needs to continue to emphasise child protection in humanitarian emergencies and fragile settings, regardless of the exact nature of the scheme. There is a fear that, if opened up towards development, the emergency element might gradually be lost altogether. It is therefore recommended that the CPTS, in a slightly refined and streamlined format, is continued and linked with a second scheme, such as the Advanced Diploma, which may be developed jointly with an academic institution and target mid-level child protection workers, particularly from non-Western countries. Since sustainability is key, refined ways of working will be important, including more comprehensive interagency collaboration; a slightly revised financing model; and a stronger focus on facilitating participation in all aspects of the recruitment and selection process by individuals from developing countries.

6

Introduction The Child Protection Trainee Scheme The Child Protection Trainee Scheme (CPTS; also referred to as ‘the scheme’ hereafter) is a programme that aims to enhance the capacity of the international humanitarian system to protect children in emergencies. Launched and financed through a pilot year by Save the Children in 2005, the CPTS received a five-year grant from the UK Department for International Development (DFID) in 2006. Since then, 53 trainees have graduated from the scheme and the sector has benefited from increased support for child protection in emergency settings. As part of the training scheme up to 10 trainees were recruited each year to undertake two five to six-month placements in different emergency or fragile state settings. The scheme applied a ‘Blended Learning’ approach that combined learning-by-doing with support from child protection professionals in the field; structured workshops held at central level at the beginning, middle and end of the year; self-learning and, importantly, ongoing guidance from trained coaches. In-country placements were hosted by both Save the Children country programmes as well as other humanitarian agencies specialising in child protection, such as the International Rescue Committee, UNHCR, UNICEF, Terre des Hommes and War Child Holland. Many graduates from the scheme have continued to work in the field of child protection in emergencies for one or several of these organisations.1 The evaluation Preceding this final evaluation, two evaluation exercises had been conducted. A Save the Children evaluation of the CPTS was completed in May 2008 and a mid-point evaluation was completed in August 2010. Besides presenting a view on the scheme from beginning to end, the present evaluation aims to complement these earlier exercises by offering not only qualitative insights but also a quantitative overview of key parameters of the scheme as well as issues that have emerged during its lifetime. Moreover, synthesis and analysis is emphasised over a more descriptive approach in order to provide useful insights, conclusions and action points for DFID, Save the Children and other players in the sector, particularly those currently engaged in developing and leading capacity building schemes related to child protection and humanitarian emergencies. Structure of the report After summarising the research methodology and survey design, findings are presented and analysed. Following this, conclusions are presented and recommendations made with a view towards informing the continuation of capacity building for child protection in humanitarian settings. In order to give a voice to respondents, selected quotes from the online survey, interviews and focus group discussions are marked in bold, grey italics

1 For more information on the Child Protection Trainee Scheme see the project document, logical framework, donor reports as well as various tools generated under the scheme, such as the trainee handbook or coaching toolkit.

7

Methodology

Analytical framework The overall goal of the CPTS is to enhance the capacity of the international humanitarian system to protect children in emergencies. More specific sub-goals are outlined in the logical framework which represents a key part of the original project documents.2 The goals relate to 1, the deployment of a cadre of highly-skilled protection personnel throughout the humanitarian system; 2, the development of a Coaching Toolkit; and 3, the establishment of a practitioners’ network. While largely following this framework, methodology design also took into account issues and themes that had emerged over the course of the scheme as well as resulting priority shifts mutually agreed with the funding organisation. Thus, nine key themes covering the essence of, but putting different emphasis on, the three goals outlined in the logical framework were identified and used throughout the evaluation process. They are: Overall success of the scheme Quality of the scheme and its graduates Recruitment and diversity of CPTS trainees Blended learning approach Career paths of CPTS graduates Coaching quality Tools: Coaching Toolkit and Competency Framework Professional networks Outlook on CPIE capacity building These themes, with some slight modification, are also reflected in the structure of the present report.

Desk research All key project documents, such as the original project proposal, logical framework, original and revised budgets, donor reports, training evaluation summaries, trainee handbook, coaching toolkit, competency framework, CPTS newsletters, etc. were reviewed prior to evaluation design. In-depth discussions were held with the scheme’s current manager. The two previous evaluations were also reviewed as well as raw data relating to ethnic and gender diversity in the CPTS recruitment and selection process.

Empirical research Online survey Using the web tool SurveyMonkey3, an online questionnaire was sent to 93 individuals, giving them a two-week deadline to respond. A reminder email was sent to those who had not yet responded three days before the deadline. The questionnaire contained 30 questions distributed over 10 thematic pages. 60 individuals replied to the survey. This indicates a response rate of 65 percent. 82 percent of respondents completed the survey. Of those that did not, most are based in places with poor

2 The CPTS logical framework is in Annex 5. 3 See http://www.surveymonkey.net

8

internet connection and some, during telephone interviews, mentioned that connectivity problems prevented them from completing the survey. Except for two responses which only contained information about the respondent but no answers to the actual survey questions, these were therefore factored into the analysis nevertheless. The remaining set of 58 responses is the basis for quantitative analysis in this evaluation report4. Online survey respondents Of the 58 valid responses, 555 percent come from current or former trainees; 31 percent are from current or former coaches; and 12 percent are from current or former line managers of the scheme. The remainder of respondents work in child protection and/or humanitarian capacity building either at Save the Children, with other organisations active in the sector or as consultants, and some have assisted the scheme in various functions, such as assessment during the selection process and development of capacity building for coaches. A quarter of trainees who responded have also been CPTS coaches at some point, and several now work at Save the Children or elsewhere on child protection or humanitarian affairs. In other words, they fit more than one response category. A graphical representation of the distribution of respondents is below:

78 percent of respondents are female and 22 percent are male. 13 of the 58 participants have a non-Western background. In other words, as much as 78 percent of respondents are of European or other Western origin. British respondents account for more than a third, followed by respondents from other European countries with 26 percent and from North America with 16 percent. Less than a quarter are from non-Western countries: 12 percent are African, seven

4 A list of respondents to the online survey is in Annex 1. 5 To facilitate readability and understanding in the text, all percentages are rounded to whole numbers. In the charts one decimal place is provided.

9

percent are Asian and four percent are from the Middle East. Six participants also mentioned a second nationality, which, with one exception, in all cases is Western. The distribution of first nationalities among respondents is depicted below.6

Question design Questions were designed to fit with the respective theme and ranged from open to closed questions, rankings, ratings and multiple-choice questions. Only selected questions were mandatory and thus required an answer before the respondent was able to move to the next survey page. Mandatory questions were not marked to avoid encouraging respondents to reply to mandatory questions only. Each closed question provided an “I can’t say” option in order not to force respondents to provide an answer where they could or did not want to. Before being sent out, the online survey was tested by three CPTS experts and adapted in accordance with their suggestions. Individual interviews 12 interviews were conducted over telephone and skype, using a semi-structured approach7. The questionnaire was developed jointly by the consultant and the current CPTS manager who was also counted as a respondent8. On average, interviews took close to one hour. In cases where the interviewee had responded to the online survey prior to the interview, which was the majority, their answers were analysed beforehand to facilitate in-depth questioning and maximise discussion outcomes. In the smaller number of cases where the interviewee had not responded to the survey beforehand, they were encouraged to do so following the conversation. All except one of the interviewees also completed the online survey. Focus group discussions

6 This is not to be confused with ethnic diversity of CPTS trainees, which is discussed later. Nevertheless, this diversity distribution does roughly reflect ethnic diversity among trainees. 7 A list of individual interviewees as well as focus group participants is in Annex 1. 8 The questionnaire is in Annex 4.

10

Two focus group discussions were conducted using teleconferencing technology. The same questionnaire was used as for individual interviews, albeit with a slightly different emphasis across themes. One focus group consisted of trainees in the last phase of training and thus about to graduate. The other group consisted of trainees who graduated from the scheme several years ago. The discussions took about 1.5 hours each and were moderated focusing on the themes outlined above. Similarly to individual interviews, participants’ online survey responses were taken into account to shape questions and moderate the discussion. All focus group participants also completed the online survey.

Constraints Despite thorough planning of focus group discussions several participants who had agreed to join were eventually not able to do so, mostly due to technical difficulties with internet and phone connections. Therefore, the focus group discussions only had a few participants each, some of whom had to leave early due to other commitments. Moreover, several individuals contacted for in-depth interviews by email did not respond.

11

Findings and Analysis

Overall success of the scheme The Child Protection Trainee Scheme has been successful in achieving its overall goal of enhancing the capacity of the international humanitarian system to protect children in emergencies. The success of the scheme was measured using the following indicators: inclusion of a child protection component in major humanitarian emergencies; speed of deployment; interagency collaboration; and organisational capacity. Overall, respondents feel that the CPTS has been successful on all four accounts. As outlined in the chart below, agreement is strongest on success in terms of increased organisational capacity (62 percent agreeing ‘strongly’, 29 percent agreeing ‘somewhat’), followed by interagency collaboration (with 50 percent agreeing ‘strongly’ and 24 percent agree ‘somewhat’); inclusion of a child protection component and speed of deployment have the same level of ‘strong’ agreement (45 percent), but, when counting those that agree ‘somewhat’, the former have much stronger overall agreement than the latter9. As one respondent sums it up: ”The CPTS takes entry-level people and uses the year to turn them into attractive, employable child protection professionals.”

9 The full statements respondent were asked to rate their agreement with read as follows: “The CPTS has helped ensure that child protection is a component of all responses to major humanitarian emergencies”; “The CPTS has improved the speed of deployment of child protection in emergencies”; “The CPTS as improved inter-agency collaboration in child protection in emergencies”; “The CPTS has increased the capacity of Save the Children and partners to protect children in emergencies”.

12

The emergency response to the earthquake in Haiti is cited by several respondents as a key example of CPTS graduate involvement that not only demonstrated the CPTS’ success in facilitating the inclusion of a strong child protection component but strongly benefited from the availability of rapidly deployable personnel generated, thus improving speed of deployment. Several CPTS graduates were deployed to Haiti within less than a week from the earthquake. In fact, some were contacted on the day of the earthquake and were ready to be deployed but due to the national airport being closed could not start their in-country engagement until a few days later. The floods in Pakistan as well as the recent humanitarian crises in Libya and Côte d’Ivoire were also mentioned as examples of child protection being a major component of the response and former CPTS trainees playing an important role. Former trainees are also part of several agencies’ rapid response teams. The table below shows the number of CPTS graduates recently deployed as part of the response to the following major emergencies of 2010 and 2011: Emergency response

Number of CPTS graduates deployed

Côte d’Ivoire/Liberia (2010/2011) 6 Haiti (2010/2011) 7 Libya (2011) 5 Pakistan (2010/2011) 4

This table shows that almost half of those that had graduated from the CPTS by 2010/2011 were involved in major recent emergency responses10. The CPTS systematically involves other agencies working on child protection in emergencies, particularly in London-based trainings, through in-country placements and during the selection weekend. Therefore, it facilitates interagency collaboration and exchange among staff of different organisations by design. As a result, most CPTS graduates, particularly those from more recent intakes, were placed with Save the Children as well as a partner agency during their training. This encourages and facilitates cross-agency thinking and understanding, as well as a solid overview of the sector. The process to develop the Competency Framework11 may be credited with having been a particular driver of interagency collaboration. Nevertheless, it may be argued that the networks thus created sometimes lead to collaboration more on an individual level than on a more formal, organisational level and that collaboration may not always translate into joint efforts on the ground. Moreover, interagency involvement in the design and management of the scheme has been lacking, with Save the Children “really having to drive the scheme along.” It is interesting to note that some respondents perceive the CPTS as an interagency more than as a ‘Save the Children’ scheme, which points to successful collaboration between agencies. Save the Children’s efforts in encouraging other organisations to join the scheme and actively contribute are widely recognised. Nevertheless, more interagency collaboration from the outset, e.g. in the programme design phase, would be needed in order to turn the CPTS into a true collaborative effort driven by joined ownership. Capacity of Save the Children and its partner organisations to respond to emergencies has improved considerably through the CPTS. While the number of available personnel thus has increased, it may be argued that some CPTS graduates, when coming out of the scheme, do not yet possess the level of experience that would be desirable for independently leading an emergency response, which may often be required. Attribution

10 Considering the fact that four CPTS graduates mentioned in this list participated in two responses, the total number of different CPTS graduates deployed in these four emergencies is 18 (22 minus 4). This equals close to 50 percent of the 44 trainees that had graduated by 2010/11. 11 See more in the section on tools below.

13

In qualitative interviews with key experts it became clear that not all improvements in the sector in regard to ensuring and strengthening child protection in major humanitarian emergencies can be attributed to the CPTS and that, in fact, attribution and measurement of such success is difficult. While there is widespread agreement that the CPTS has helped improve all the indicators mentioned above, investments in CPIE by other actors, particularly through the Child Protection Working Group (CPWG), have also been key. In other words, the creation of the CPTS coincided with, or formed a part of, other important developments in the sector, including a process that made the CPWG stronger, more coherent and thus more effective. The CPTS may thus be seen not only as a contributing cause for improvements in the sector but also as an effect of a trend that had begun earlier with key players starting to define the concept of child protection in emergencies and setting standards as part of the humanitarian reform process. Nevertheless, the success of the CPTS should not be underestimated and is obvious in view of the overwhelming majority of respondents who feel that the scheme has been successful. Emulation Given the CPTS’ success, it is not surprising that several other capacity building schemes now follow similar structures or incorporate elements of it. At the same time, these examples of emulation are markers for success in themselves. Examples of other capacity building schemes using similar approaches include the following Save the Children and interagency programmes: European Humanitarian Volunteer Programme (EHVP). Funded by the European Union, the

EHVP is managed by Save the Children UK in coordination with Bioforce and NOHA (Network of Humanitarian Assistance) International Association of Universities, and with input from Save the Children Denmark, Johanniter Germany and Caritas Czech Republic.

Humanitarian Leadership Development Programme (HLDP). The HLDP is funded by DFID and implemented by Save the Children with involvement from the Consortium of British Humanitarian Agencies (CBHA)12. The programme runs in the UK as well as in the Horn of Africa.

Humanitarian Technical Trainee Scheme (HTTS) implemented by Save the Children. Logistics Skills Development Programme (LSDP) implemented by Save the Children.

Quality of the scheme Overall, the scheme receives very high ratings, both by current and former trainees as well as by others working in the sector. As shown in the chart below, two-thirds of respondents have recommended the scheme to people they know; many of them did so more than once. In addition, 28 percent indicate they would recommend the scheme if they knew the right person. This is a clear sign for appreciation of, and trust in, the quality of the scheme.

12 The CBHA consists of 15 UK-based NGOs. See http://www.thecbha.org/ for details.

14

Strengths The CPTS has a great variety of strengths, some of which are unique to the scheme; these are widely recognised. The fact that the scheme gives trainees exposure to field-based work, thus providing an important career entry point towards becoming CPIE professionals for people who otherwise might struggle to build the necessary experience that would get them a job in the field, is key. Moreover, the scheme’s structure, which systematically complements in-country placements with other elements of learning, including London-based trainings, coaching and self-study, is also seen as a key strength. The scheme’s focus on child protection in emergencies is also mentioned as a strength. Generally speaking, administrative and technical support provided to trainees by the scheme’s managers and their headquarters-based advisers is highly appreciated, as is the rigorous recruitment process, which allows for selection of high-quality candidates.13 This in turn has created an excellent reputation and ensured that the CPTS ‘brand’ has become a valuable signal to potential employers. One partner organisation representative stressed the scheme’s function as an effective “one-stop-shop” for high-quality child protection professionals and also referred to the “high return on investment” for CPTS partner organisations when taking on trainees: since Save the Children takes responsibility for organising the placements, only minimal efforts are required from partner organisations in this respect. Last but not least, the CPTS is credited with having strengthened information sharing between agencies, as well as between HQ and field offices, and for having created a platform for agencies to share ideas about training and recruitment. One respondent summed up their impression as follows: “I am a big fan of the scheme”. Weaknesses

13 See the section on recruitment and diversity below for more detail.

15

The scheme’s inability to attract and select a diverse group of trainees, despite continued efforts to do so, is a key weakness of the scheme. Another weakness perceived by respondents has been the scheme’s inability to ensure consistency across country programme and particularly coaching quality, causing individual trainees’ learning experiences to be very diverse. Beyond those weaknesses which are further discussed below, there are no major weaknesses respondents agree on. A few respondents feel that the lack of an assessment mechanism, such as a pass and fail system moderated by the scheme’s managers in London, makes employing graduates more difficult than necessary since, while seen as excellent overall, a few graduates are not yet ready to be deployed as professionals in emergencies on graduation. Others, however, argue that anything more than a “formalised process for systematic feedback” would change the nature of the scheme and provide additional pressure that may distract trainees from achieving their learning objectives. The fact that the CPTS is a “high-maintenance” and cost-intensive programme has been mentioned by some as a weakness. However, there are at least equally as many who feel that it is a worthwhile investment that fares well in terms of cost effectiveness when compared with other investments in human and organisational capacity. This is also further discussed below. Trainee knowledge, experience and skills Trainees themselves predominantly rate their experience with the CPTS as ‘good’ (46 percent) or ‘very good’ (39 percent), stressing its hands-on nature and focus on complementing theoretical learning with exposure to real life challenges in the two placements. Not a single trainee rates their experience as ‘not satisfactory’ and, likewise, no respondent rates trainees and graduates they have come across in this way either. When it comes to the ‘quality’ of CPTS trainees and graduates, it is important to distinguish between knowledge, experience and skills. While 83 percent of respondents rate trainees’ knowledge as good or very good, 74 percent do so in regard to skills and 64 percent in regard to experience. Given that the CPTS is a scheme targeting professionals relatively early in their career, the fact that trainees are rated better in terms of knowledge and skills than in experience is not surprising. At the same time, their high rating, particularly in regard to knowledge, is considerable. The rating of trainees’ knowledge, experience and skills is shown in the chart below:

16

Coaches, who support, mentor and in most cases manage trainees during their in-country placements, and hosts, who manage country offices and programmes that hosted CPTS trainees, are in a good position to share insights since they have had ample opportunity for engagement with trainees in real-life settings. They also give trainees a very high rating: coaches and hosts predominantly rate trainees as very good (53 percent) or good (37 percent). Specific qualities mentioned positively are enthusiasm, respectfulness and willingness to learn. Lack of experience and exaggerated confidence were mentioned as negative characteristics. The rating is shown in the chart below:

17

In summary it can be said that overall trainee motivation, knowledge and skills are excellent, with their experience being adequate to good. An additional indicator for the ‘quality’ of graduates is their employability and their ability to find ‘good’ jobs, which is further discussed below. While pointing to a very positive impression overall, there is some variation nevertheless. One respondent put it as follows: “The majority of trainees are excellent, but there are a few outliers”. Cost effectiveness The cost effectiveness of the scheme has been subject to debate. While the investment made in training 53 CPIE professionals has been considerable at around 1.4 million GBP for the six-year period,14 its return and impact has been substantial as well. Compared to other human resource investments, such as annual salaries of mid-career professionals working for UN agencies or international NGOs, “the CPTS can be considered good value for money”. At roughly 26,000 GBP invested per person, the scheme’s cost appear reasonable, particularly as in “child protection what we need is people, so we need to train people, and we need to invest in training them”. Moreover, there are wider cost benefits than ‘only’ the number of trainees the scheme has turned out: in emergencies, where rapid deployment is critical, it can be difficult and costly for organisations to identify the right individuals. Having a pool of CPTS graduates easily accessible drastically reduces the related costs and difficulties for the sector. Cost scrutiny is warranted nevertheless. At around a third of the overall budget, trainee allowances stand out as the single largest budget item. Besides staff costs for a programme manager, the cost of the London-based training events is also substantial at close to 200,000

14 The bulk of this amount was financed by DFID, with additional contributions by Save the Children. The first year was a pilot year funded entirely by Save the Children.

18

GBP in total (covering trainees air tickets and accommodation). Given the blended learning approach and the focus on strong management and support from central level, the latter cost items are easily justified. As for the allowances of about 11,000 GBP per annum per trainee, there is a question to what extent these are necessary and facilitate equal opportunity or whether a different system, for example one only supporting in-need individuals with scholarships, may be more effective in improving ethnic diversity. After all, at the current scale of the programme, cutting the allowance would significantly lower the investment per trainee to around 15,000 GBP. Nevertheless, it should be stressed that, to the extent that trainees provide professional and valuable support to country programmes, it is appropriate and justified to provide financial compensation for their work. Currently, host programmes cover trainees’ in-country accommodation and travel as well as R&R if applicable. Having them contribute to an allowance is an option to be considered. When assessing cost effectiveness it is also important to consider the length of time CPTS graduates stay in the sector. When a CPTS trainee leaves the sector immediately or shortly after graduation from the scheme, the investment can be considered lost to the sector. Hence, this aspect of cost effectiveness may be improved by lowering drop-out rates from the sector. Ensuring that those selected as CPTS trainees fully understand the implications of working in CPIE, particularly in regard to their work-life balance, is therefore key and needs to be emphasised in the selection process. This may reduce the numbers of CPTS graduates who move into different sectors soon after completing the scheme, thus increasing cost effectiveness and sustainability of the investment made.

Recruitment and diversity The CPTS recruitment process The CPTS recruitment process consists of four key steps: 1, candidates submit a written application; 2, around a quarter are selected for phone interview by a number of different assessors; 3, assessors conduct phone interviews and select about 25 to 30 candidates for participation in a UK-based selection weekend; 4, candidates undergo a three-day assessment run in collaboration with Ground Truth, an agency specialising in training and assessment for clients who send their staff to challenging and hostile environments. The selection weekend consists of face-to-face interviews, tests (psychometric, written, numerical, budget) as well as a simulation exercise that includes a number of tasks testing candidates’ competencies considered key for CPIE, ranging from self-awareness to effective communication in dangerous settings. Assessors use the ORCE (Observe, Record, Classify and Evaluate) model to rank candidates and complement this with behavioural insights gained through structured observation as well as in discussion with other assessors.15 Diversity A majority of 80 percent of respondents agree that for a scheme such as the CPTS, diversity – both of trainees and of coaches – is important. In addition, close to 15 percent agree ‘somewhat’ to this question. In other words, almost all respondents find diversity important. A key reason cited for this is sustainability and the fact that, while many individuals from industrialised countries tend to gravitate towards more stable headquarter-based positions, individuals from other countries may be more likely to stay ‘in the field’. Moreover, ensuring a variety of perspectives on child protection in emergencies, particularly from those countries where emergencies happen, is key.

15 For example, in the 2010 recruitment process, more than 600 candidates applied, just over 150 were interviewed over the phone, 24 participated in the selection weekend and nine ultimately got selected for participation in the scheme.

19

For the purposes of this evaluation diversity can be defined as a balance of both sexes among CPTS trainees as well as participation in equal shares by individuals from developing (‘non-Western’) countries in Africa, Asia, Latin America and the Middle East alongside individuals from industrialised (‘Western’) countries. Notwithstanding the importance of diversity, only a quarter of respondents agree that the CPTS has managed to put in place a recruitment and selection process that provides equal opportunities to all applicants; nevertheless, an additional 30 percent agree ‘somewhat’. While this means that more than half of respondents evaluate diversity in the CPTS fairly positively, almost half do not. This discrepancy between desired and actual diversity is depicted and analysed in more detail below:

These results are corroborated when actual placement data is considered. The chart below shows that while ethnic diversity16 has improved somewhat over the duration of the scheme, this improvement has not led to a balance of trainees from Western and non-Western backgrounds. In all years, trainees from Western backgrounds (British, other European and North American) accounted for at least 70 percent, with some years having an even higher share of up to 100 percent.

16 As measured by the diversity of trainee nationalities.

20

For example, candidates from Latin America and francophone Africa have hardly been present at all. Males, too, have been significantly underrepresented. The chart below shows that, while in one year (2008) the share of male CPTS trainees reached 33 percent, in all other years it ranged from zero to just over ten percent only.17

Despite these figures, there is a strong recognition of the fact that the scheme’s managers have been very mindful of diversity and made several adjustments to the recruitment process with the aim of improving diversity. Besides starting to collect data disaggregated by nationality and sex, other adjustments made over the course of the scheme include the incorporation of a selection weekend in collaboration with an external agency (Ground Truth) and the dropping of a second international foreign language as a requirement for application to the scheme. It is likely due to these adjustments that ethnic diversity has improved somewhat over the duration of the scheme.

17 An analysis of the gender balance over different parts of the recruitment process is further below.

21

It should also be mentioned that some respondents feel that equal opportunity is more important than diversity, referring to positive discrimination as a negative side-effect of efforts to improve diversity. They argue that, in some instances, male trainees and trainees from non-Western backgrounds were selected into the scheme at the expense of – better suited – candidates that did not fulfil these diversity criteria. In other words, they feel that while diversity is important, it should not override the goal of ensuring equal opportunity. Analysing the recruitment process When analysing diversity in relation to the CPTS recruitment and selection process it is important to understand at what point or points diversity of candidates increases or reduces. In other words, an analysis of whether the lack of diversity among successful trainees is due to an insufficient number of candidates from each sex and part of the world being aware of the scheme and applying; getting selected for phone interview; being invited to the selection weekend; getting selected into the scheme; or all of the above. The chart below shows data on ethnic diversity for the 2009 intake, which is used as an example.18 It demonstrates that, while the share of candidates from Africa stayed relatively constant throughout the 2009 recruitment and selection process at around 20 percent, the already high share of Europeans further increased over the course of the process, while the share of candidates from North America plus Australia and New Zealand decreased, and Asian as well as Latin American candidates disappeared entirely.

While it is impossible to identify trends and draw conclusions from one year alone, it is interesting to note that, in 2009, for candidates from Northern America the telephone interview was the step in the process where their share started to decrease, while the opposite was true for candidates from Asia. During the selection weekend, however, this trend reversed with the share of North Americans remaining constant thereafter while no Asians were eventually selected for participation in the CPTS. In regard to gender diversity it should be noted that the share of males, which was around 30 percent at the beginning of the 2009 recruitment process, from the telephone interview onwards gradually declined to around ten percent when it came to selection. This is depicted in the chart below. 18 2009 is the only year for which data is available that allows for a sufficiently detailed analysis.

22

When analysing diversity of trainees, analysing the ethnic background of assessors is also relevant. The chart below provides an overview of assessors’ ethnic backgrounds in the last four intakes. It shows that with the exception of 2007, only a sixth of assessors had a non-European background:

Implications for recruitment Following their analysis and observations, respondents proposed a number of actions and practical changes to the recruitment process that may improve diversity: Ensuring ethnic and gender diversity among assessors Advertising the scheme more widely in Southern fora, potentially through collaboration with Southern-based universities and other institutions Proactively targeting individuals and groups that appear suited to the programme, for example by tapping into existing networks Advertising the scheme in languages other than English, for example French and Spanish Focusing less on language skills as a condition for applying to the scheme Holding the selection event in countries that have less stringent visa requirements than the UK

23

Creating regional centers for CPTS recruitment and training Considering the creation of stipends to enable non-Westerners to take time off their regular jobs in order to participate in the scheme Placing additional emphasis on checking references and recommendations from individuals who have managed or otherwise worked closely with the candidate and are able to assess how candidates conduct themselves under pressure Setting regional criteria for participation (i.e. a predefined share of the group of trainees coming from each region)19

Some of these approaches are already being implemented by other humanitarian capacity building initiatives managed by Save the Children. For example, the Humanitarian Leadership Development Scheme conducts recruitment in, and for, the Horn of Africa region. While some of the above suggestions may increase cost effectiveness, others may have considerable resource implications. Therefore, decisions on adjustments to the recruitment process would have to follow a cost-benefit analysis. 20 It should be noted that a number of respondents feel that the lack of diversity among trainees is due to a “Westernised” selection and recruitment process guided by “narrowly defined profiles” of the ideal candidate that make it difficult for non-Western candidates to succeed. One respondent goes so far as to argue that “attributes that are deemed essential in emergency workers and the CPTS are attributes which directly conflict with the way some people from different cultures behave and conduct themselves”. In other words, some respondents question whether the recruitment framework currently used is appropriate for selecting an ethnically diverse range of candidates. In this context it may be argued that in order to ensure that CPTS trainees are internationally deployable, maintaining certain ‘Western’ standards may be necessary. An analysis of the recruitment framework - which was developed in an elaborate process and has arguably contributed to the CPTS recruitment process being widely recognised as thorough and rigorous - is outside the scope of this evaluation. Nevertheless, there is an indication that the recruitment framework might benefit from a review through a ‘diversity lens’. It should also be mentioned that while providing equal opportunities to all candidates is an explicit indicator mentioned under Output 1 in the project logical framework21, ensuring that successful candidates are ‘internationally deployable’ is another key goal of the CPTS. Therefore, a balancing of goals, which at times appear to be conflicting, may be necessary. Moreover, more research is needed to establish why the CPTS struggled to ensure participation of a more diverse range of trainees, and particularly how each part of the recruitment and selection process affects diversity vis-à-vis other parts of the process.

Blended learning approach The Blended Learning approach applied by the CPTS consists of London-based trainings, coaching, in-country placements, the five-stage development process22 and self-study. Of these, in-country placements clearly stand out as being the most highly appreciated learning element of

19 If regional criteria were set this, by definition, would prevent equal opportunity. Nevertheless, diversity would increase. 20 Assessing potential cost implications for each of these recommendations and conducting a cost-benefit analysis is outside the scope of this evaluation. 21 Output 1, indicator 1.2: “The application process enables all candidates to have equal opportunities of success in order to promote a gender and ethnic mix.” 22 The five-stage development process consists of the following: 1, Self-assessment against core competencies; 2, Writing of learning and development plan; 3, Identification and planning of an assignment; 4, Submission of reflective report; 5, Completion of mid-placement/end of placement performance review

24

the scheme. They receive the best ranking by far with more than 80 percent of respondents ranking this practical element of the training first. Coaching and mentoring is ranked second, followed by the London-based trainings. Competency-based learning (using the Competency Framework to set learning and development goals and monitor their achievement), the five-stage development process and self-study follow behind. The ranking is shown in the chart below.23

In-country placements The opportunity for on-the-job training and learning by doing provided by the CPTS is greatly valued by trainees, many of whom would not be able to get other opportunities to acquire international work experience and thus build a necessary foundation for their career. This is particularly true for individuals from non-Western countries who are dependent on such opportunities to get exposure to different countries and become ‘international staff’. It is widely agreed that the experience gained during the placements is indispensable, even if circumstances are not ideal or the coaching situation is difficult.24 By the same token, “placements are the make or break section of the scheme” and while a good, well-matched placement often means great learning, some placements may have been less well conceived or supported and thus did not help the trainee meet their competency based learning goals as much. However, it is important to note that the majority of trainees who had challenging in-country experiences stressed that these had nevertheless provided them with excellent learning opportunities. Interagency participation is a key element of the scheme. The fact that the CPTS has been successful in drawing in other agencies is demonstrated by the growing number of trainee placements in agencies other than Save the Children over the course of the scheme. The chart below shows that, while of the first two intakes all trainees were placed with Save the Children

23 Besides order, this chart also demonstrates the ‘strength’ of each rank. The greater the difference in length between bars the more useful the learning element depicted in the shorter bar is considered. 24 More on coaching is below.

25

country programmes, in the last two intakes approximately half of all placements were with partner organisations. In other words, almost all trainees in these intakes got the opportunity to work both within a Save the Children programme as well as with a partner organisation.

London-based trainings Three training events are held in London over the duration of the scheme. The 10-day induction workshop at the beginning also includes four days of extreme risk security and first aid training. The classroom-based part of the training comprises technical and operational modules aiming to provide a foundation in key child protection and humanitarian programming competency areas.25The 8-day “Annual Learning Event” in between the trainees’ two country placements includes individual debriefs, reviews of the trainees’ first placement and group discussions on key issues26. The 8-day end of cycle-workshop after the second placement also includes debriefs and discussions on lessons learned27. The first of the three London-based training events, i.e. the induction including security training, on average ranks first. As depicted in the chart below, this is followed closely by the Annual Learning event held between trainees’ two placements. The end-of-cycle workshop ranks third.

25The modules cover child rights programming, communications in emergencies, media in emergencies, gender-based violence, working with separated children, community-based mechanisms, fundraising and proposal writing in emergencies, child protection monitoring and evaluation, working with children affected by armed conflict, trends in child protection, introduction to emergencies, wellbeing in the field. 26 Additional modules include humanitarian operations, monitoring and reporting of grave child rights violations, psychosocial programming, case management, children's participation and considerations on ‘life after the CPTS’. 27 Sessions include additional modules on child protection or humanitarian operations at a more advanced level than in previous workshops. Emphasis is also placed on career development and networking.

26

This ranking may also reflect the different purposes of the respective learning events: while induction is vital, not least from a security point of view, debriefing is important to provide space for reflection and learning. The Annual Learning Event, on the other hand, has a strong focus on networking and knowledge sharing. Overall, the London-based trainings are seen as essential in providing a theoretical foundation and framework to the other learning elements. Importantly, they are also seen as a key opportunity for networking and exchange rather than as mere learning opportunities. Several respondents point to learning being most effective after the first placement, when trainees have a concrete experience to apply their newly gained knowledge to and feel more confident about contributing to discussions. The security training is also mentioned as key. On the whole, the quality of the three learning events is seen as high, albeit with some variation. Many participants would have liked more in-depth sessions rather than too many different topics being addressed. Some respondents argue that the events, particularly the Annual Learning Event, should be extended by a few days in order to allow for more learning and exchange at a time that many trainees consider crucial. While the interagency approach to training is appreciated, particularly for providing an opportunity to learn from and network with different organisations, some respondents feel that the variety of speakers and topics is distracting. Some argue that the training approach is “too piecemeal” und “would benefit from one moderator to pull it all together”. The training events may be moderated by a few – ideally professional – facilitators, thus allowing more focus and better framing which would arguably facilitate learning. Another consideration mentioned in terms of format is holding the learning events away from the Save the Children office with its busy-ness, distractions and meeting rooms that do not let in daylight. One respondent argued that they had learned more during the selection weekend, part of which is held outside the Save the Children office in Farringdon, than during the London-based trainings held over the course of the scheme.

27

While there is clear acknowledgement of the fact that it is impossible for a one-year capacity building scheme to cover all relevant areas of learning of development, there are a number of areas, such as financial, human resource and project management as well as proposal writing and logframe development that are often mentioned as areas that should be part of the CPTS training. Moreover, some trainees would like to get better prepared for the challenges of interagency coordination and collaboration. It is also suggested that the induction include regional briefings to ensure trainees have an understanding of typical patterns and protection concerns in different parts of the world. This would also facilitate their understanding of the country and local context they are eventually placed in. It is important to stress that the value of the London-based training events does not lie merely, or even predominantly, in the ‘delivery’ of knowledge but particularly in the fact that they provide a platform for networking, sharing and exchange – both between peers and with experts. Interagency participation is therefore important. Last but not least, particularly during inception, the London-based trainings “provide trainees with faces to contact in case of need” during the programme. Other learning elements Competency-based learning and reflective reports on practical in-country assignments submitted by trainees as part of the five-stage development process are seen to be important for framing the learning and development process and give it structure. Trainees appreciate the materials and tools provided for self-study and a few even consider it their main learning method. However, many feel that during the programme there is too little time for self-study. Instead, some CPTS participants use the materials for professional development after graduation when they have more time available.

Coaching Coaching issues Coaching is an important element of the CPTS learning and development process. A coaching toolkit has been developed as part of the scheme (see section below) and, besides training provided for coaches in London, there is systematic support from headquarters as well as coaching consultants at a distance through structured phone calls as well as ad hoc support as necessary. However, while clearly recognised as a key factor in learning and development of the scheme, coaching also appears to be its weakest link. Differing quality of coaches is the issue participants most consistently mention as a weakness of the scheme. Coaching significantly affects trainees’ experiences of their participation in the scheme. Nevertheless, on average there was improvement from the first to the second placement. While this relatively small difference should not be over-interpreted, it may indicate an increased ability of trainees to ‘manage’ their coaches and co-create and positively influence the relationship and situation. The great variety of coaching experiences is reflected in the trainees’ rating: while the second placement is rated slightly better than the second, overall the number of trainees considering their placements ‘very good’, ‘good’, ‘adequate’ or ‘not satisfactory’ is near equal, leading to a distribution of approximately a quarter in each group. This is depicted in the chart below:

28

When thinking of defining qualities, CPTS trainees feel that a good coach is accessible both time-wise and geographically; able and willing to delegate work; provides honest and constructive feedback; ideally is experienced in dealing with child protection in emergencies in several organisational settings; and is “experienced in their role while not yet being on their way out”. Those who had less satisfactory coaching experiences predominantly point to coaches’ lack of willingness or ability to spend time and engage with the trainee by providing learning opportunities and constructive feedback; geographical distance and a lack of child protection and management skills are also stressed as negative examples of experiences with CPTS coaches. One reason for the great variety of coaching quality may be related to training. A quarter of coaches who participated in the online survey did not receive training. This has been due to a combination of administrative and timing issues. Moreover, while the majority of those who did participate considered the training useful or very useful, some felt it was not useful and several pointed to potential for improvement, such as more in-depth training. Other potential reasons for the diverse quality of coaches are an insufficiently structured selection process as well as a lack of wider host organisation support for true engagement with, and investment in, CPTS trainees as professionals. Practical recommendations to improve coaching are to provide a strengthened framework to ensure that processes are implemented as per their design; more detailed briefings to coaches before trainees arrive, to “ensure they are not seen as just a spare pair of hands”; country programme commitment about time coaches spend on coaching; and the provision of more resources for long-distance coaching, for example through a part-time adviser hired for this purpose. Despite the issues outlined above, the fact that a coaching mechanism was set up as a core element of learning and has been receiving systematic support is a key strength of the scheme, and this has been recognised. Moreover, facing difficult management situations is a common real life experience. Therefore, having to deal with a coach who, for whatever reason, is not fulfilling

29

certain expectations may be seen as an excellent opportunity for personal and professional learning and development. Matching process Related to the trainees’ experience in their in-country placements, particularly in regard to coaching, is the matching process. While the process is appreciated as being well-structured and designed to ensure trainees can achieve their learning objectives, there are several areas of criticism. The one coming out most strongly is trainees being placed in chronic emergencies or fragile settings rather than first phase emergencies, which is where they were hoping to get experience. In this context it should be mentioned that the scheme cannot guarantee placements in a first-phase emergency response, not only because emergencies are in most cases unpredictable but also because the skills of the trainee to contribute to a particular response will need to matched as well. Moreover, the bulk of humanitarian work at present takes place in fragile state settings rather than in rapid onset emergencies. A few respondents also point to negative coaching experiences that could have been prevented as previous trainees had alerted the scheme’s managers to issues with the coaches in question. In another instance, the host organisation apparently had not been aware of, or in agreement with, arrival of the trainee, leading to questions about security and financial compensation and forcing the trainee to move to a different country. Some trainees feel that there was no ‘space’ for them in the host programme, thus making it difficult to add value.

Career paths With almost 60 percent of trainees finding paid employment in the area of child protection in emergencies within three months of graduation, the scheme appears to be an invaluable springboard for a career in the sector. As outlined in the chart below, almost a third of trainees found work immediately after graduating from the CPTS. Another 27 percent found paid employment one to three months after their graduation. While a further 19 percent found work before six months had passed, 12 percent had to continue their job search beyond this timeframe. Another 12 percent had not found work in the area when taking part in the survey since they were only about to graduate or had chosen to work in a different field.

30

Between the online survey and the finalisation of this report, several recent graduates also found work in the sector. In other words, they found work almost immediately after graduation, thus demonstrating the value of participating in the CPTS in regard to career development.28 It should be noted that several CPTS graduates get hired by the offices that had hosted them during the traineeship, thus providing a direct link between the scheme and graduates’ careers. At the same time, those who could not easily find a paid position after graduation may have had strong preferences in regard to location and salary level, which might have prolonged their job search. Overall, former trainees feel that the CPTS prepares them well for a career in CPIE and many think that they could not have entered the sector without having participated in the scheme. A few respondents suggest that the CPTS provide ongoing support to graduates in finding a job. However, it can be argued that the Good Practice Network29 is an adequate instrument established by the CPTS to facilitate the process of finding a job in CPIE. Given that they are professionals, even at an entry-level, it may be argued that it is graduates’ responsibility to create the right networks and obtain information on vacancies and recruitment. The CPTS’ high degree of recognition and good reputation in the sector further facilitates this process. Nevertheless, there is a clear recognition of the fact that, on graduation, some participants are still relatively inexperienced and therefore do not qualify for a number of the positions advertised. This is partly due to some placements not exposing trainees to first phase emergency settings, which is where human resources are often required. 69 percent of the current and former trainees who responded to the survey are planning to continue to work in the field of CPIE for the next five to ten years. 14 percent cannot say, and 17 percent are planning to move to a different field. This demonstrates that the sector’s staff retention

28 An updated list of CPTS graduates and their current positions is in Annex 3. 29 See more on the network below.

31

rate must be considered when planning CPIE capacity building initiatives. For those who do not plan to continue working in the sector, or who are unsure, family and other work-life balance considerations are key.

Tools Coaching Toolkit Close to 57 percent of respondents (and 73 percent of coaches) are aware of the Coaching Toolkit for Child Protection in Emergencies, with many able to name key elements of its content. However, few have actually used it and a number of respondents point out that they have not seen a final version and are unsure about where to find it. This brings up questions about dissemination and distribution. The Toolkit, in an adapted format, is also used by the following capacity building programmes managed by Save the Children and its partners: Humanitarian Skills Development Programme (HSDP) Humanitarian Leadership Development Programme (HLDP) European Humanitarian Volunteer Programme (EHVP) The fact that other capacity building initiatives are using the toolkit is an indicator for quality and demonstrates additional returns to investment in its creation. Competency Framework The Child Protection in Emergencies Competency Framework was developed and agreed by the Child Protection Working Group (CPWG) in 2010. This process was managed by Save the Children, and was based on the core capabilities for child protection workers framework developed by Save the Children for the CPTS in 2007. The CPIE Competency Framework is a key tool used within the CPTS, primarily in the five-stage development process mentioned earlier and for trainees to track their own learning and development. 82 percent of respondents are aware of the Child Protection in Emergencies Competency Framework, with most able to cite its key aspects. Among current and former trainees, awareness is even higher at 87 percent. The Framework is seen as particularly useful for learning and self-assessment, with almost three quarters of respondents considering it ‘useful’ or ‘very useful’ in this regard. Close to 60 percent consider the Framework ‘useful’ or ‘very useful’ for recruitment, for example in developing job descriptions. Performance management as well as (interagency) standard setting and conceptualisation of child protection in emergencies are seen as other potential or actual uses of the Framework. While the Framework is thus widely known and seen as useful by many, those who regularly use it are fewer. Reasons cited for this are the fact that the Competency Framework sits alongside other, more agency-specific tools. Moreover, while it is given considerable credit for being comprehensive and systematic and thus useful in setting a framework for CPIE, it is also criticised for being extensive, complex and lacking in cultural sensitivity, thereby discouraging widespread use.

Networks Good Practice Network 53 percent of respondents are active users of the Good Practice Network, a Google group bringing together current and former trainees as well as other professionals working in the sector of child protection in emergencies. An additional six percent are members of the group but admit

32

to not being active users. Among current and former CPTS trainees, as much as 81 percent are active users with an additional three percent being non-active members. The group is used mostly to post as well as obtain information on vacancies, including recruitment information not posted elsewhere, and to share tools and other resources relating to CPIE. Occasionally, members use it to ask and discuss questions. The group is particularly appreciated for its focus on child protection in emergencies and for the fact that it allows fast and efficient interaction with members. Other global networks Besides networks that are specific to a region, country or organisation, trainees also belong to a good number of global networks in the area of child protection and/or emergencies. These include the Child Protection Working Group (CPWG), the Better Care Network, the Protection Cluster, the Interagency Working Group on Separated and Unaccompanied Children, the Inter-Agency Standing Committee (IASC) on humanitarian assistance, the Humanitarian Law and Policy Forum, and others. Other respondents also are members of the Keeping Children Safe Coalition, the Inter-Agency Network for Education in Emergencies, Child Protection in Crisis, Enhancing Learning & Research for Humanitarian Assistance (ELRHA), and others. Informal networks Besides formal networks such as the Good Practice Network that plays a key role in connecting trainees with each other as well as with other professionals in the sector, the informal network generated by the CPTS also goes a long way in connecting individuals and facilitating employment, sharing information and resources and ultimately improving interagency coordination and collaboration. Some trainees prefer to rely on one-to-one networking much more than on interaction through means of the Google group, for example. Several respondents also point out that, particularly during their CPTS training, the network helps trainees not only professionally but also allows for support on a personal level.

Outlook Focus A majority of 61 percent agree that it is necessary to run a scheme focusing specifically on child protection in humanitarian emergencies, with another 35 percent agreeing ‘somewhat’. This overwhelming majority of close to 100 percent is confronted by a nevertheless important number of people who feel that it would be preferable to run a general child protection trainee scheme, focusing on both emergencies and development contexts. 29 percent agree with this statement and 33 percent agree ‘somewhat’ while 12 percent disagree and 22 percent disagree ‘somewhat’. The chart below demonstrates that while, on the whole, people feel that the focus on child protection in emergencies must be maintained there is also considerable support for the idea of bringing in a development focus.

33

This apparent ambiguity of strong agreement with the emergency focus option which is not entirely matched by disagreement with the development focus option may be explained by a number of factors: in reality, the lines between emergency and development are often blurred. Moreover, the CPTS did not always place trainees in first-phase or rapid onset emergencies. Therefore, it may be argued that in essence the CPTS has focused on both development and emergency contexts anyway. Moreover, with the systems approach being promoted, “both disciplines are becoming more closely aligned”. Importantly, the response also points to a need for more capacity building in child protection for development which does not contradict the fact that continued capacity building focusing on child protection in emergencies is necessary. While the issues affecting children are very similar in both first phase and chronic emergencies, as well as in development settings, the competencies needed to respond effectively in emergencies are quite specific, particularly in regard to building a response under great time-pressure and with limited information. Furthermore, a more general scheme may attract and train individuals who might move towards more senior, headquarters-based positions shortly after graduation, thus reducing deployability of personnel for emergencies and accelerating the gravitational trend in the sector. In sum, it is widely agreed that a focused scheme is necessary. Moreover, there clearly is room for an additional capacity building scheme on child protection in development settings.

34

Conclusions and Recommendations The CPTS has been a success Overall, the CPTS has been very successful in enhancing the capacity of the humanitarian system to protect children in emergencies as per its overarching goal. It has helped increase the number of skilled child protection staff deployable in humanitarian emergencies, thereby improving the speed of deployment and contributing to ensuring that child protection is an explicit component of major humanitarian responses. At the same time, the CPTS has strengthened the capacity of Save the Children and its partner organisations. Moreover, by ensuring that almost all trainees are placed with a partner agency as well as in a Save the Children programme in recent years, the scheme has made a great leap forward compared to the early years in regard to interagency collaboration. In addition, a high-quality coaching toolkit, including a competency framework, was developed and made available to a number of individuals and organisations. Last but not least, besides an invaluable informal network, a Good Practice Network of child protection practitioners has been established that links professionals in the sector across organisations via a Google group. The CPTS has exceeded expectations With 53 successful trainees, the CPTS has exceeded its objective of training at least 46 highly skilled child protection staff. In addition, the CPTS has contributed to a ‘professionalisation’ of the sector through standard setting, improved interagency collaboration and by helping to raise the profile of child protection in emergencies more generally. The fact that most of these successes were achieved jointly with other institutions, such as the Child Protection Working Group, and that direct attribution is difficult, does not lessen the positive impact the scheme has made. Moreover, such joint efforts are in themselves an indicator for improved interagency collaboration. CPTS graduates have gone on to being successful professionals in the field of child protection in emergencies, in many cases finding employment immediately after graduation from the scheme. This is another indicator of the scheme’s quality. Thereby, besides reinforcing the strong reputation of the scheme, former trainees directly impact on the lives of children. Key strengths and weaknesses The focused nature of the scheme and the fact that it provides trainees with field exposure, complemented by extensive training and networking opportunities at central level, are key strengths that other capacity building initiatives have learned from and should continue to learn from in the future. This is also true for the scheme’s systematic framework and strong management. However, the scheme has not managed to provide equal conditions and learning opportunities for all trainees, making it more difficult to assess and potentially certify trainees’ ‘successful’ completion of the scheme. While this is largely due to the nature of the scheme and can therefore be justified, the lack of diversity among trainees has been an important issue the scheme’s managers have been grappling with from the start but have not been able to solve. Key area for improvement: diversity While diversity did improve over the course of the scheme, particularly if compared with the pilot year, diversity among CPTS trainees, if maintained as an important goal, is a key area for improvement. From the available evidence it cannot be said with certainty whether the recruitment and selection process has provided equal opportunity for all candidates or not, or indeed whether there might have been positive discrimination. However, it is widely acknowledged that CPTS managers have made considerable efforts to ensure equal opportunity and thus improve gender and ethnic diversity, It is noticeable that a large number of trainees come from the UK, North America, African countries that have English as an official language as well as Western Europe, where English is common as a second language. This indicates that language may be a key factor in recruitment and selection. If the recruitment and selection process was at least partly run in alternative languages this pattern may be addressed relatively easily. Similarly, ensuring diversity among assessors and/or conducting the selection weekend in regional centers, as is already done by the CPTS’ sister scheme HLDP, may also go a long way in improving diversity. While some of

35

the recommendations made (e.g. ensuring a gender and ethnic balance of assessors throughout the recruitment process) may have substantial cost implications, particularly if the scheme continued to be headquartered in the UK, others (e.g. proactively targeting potential applicants from non-Western countries) may not necessitate substantial additional financing. A more focused and in-depth analysis on diversity is necessary to understand what exactly prevented the scheme from being able to recruit a more diverse group of trainees each year. For this purpose, gathering more data to understand the ‘triggers’ that caused gender and ethnic diversity to decrease or increase at each junction in the recruitment process will be worthwhile. Last but not least, it must be kept in mind that the distribution of sex and nationality in the scheme is an approximate reflection of human resource diversity in the sector, which is generally dominated by Western females. While it may be argued that a scheme like the CPTS could see improving diversity in the sector as an end in itself, it cannot be held accountable for reflecting diversity of the sector unless improving ethnic and gender diversity in the sector was an explicit goal of the scheme. Key issue: coaching Coaching, too, is an area that provides some room for improvement. It is commendable that the CPTS includes a strong focus on coaching at all, manifested by the toolkit, availability of training as well as coaching support from headquarters. Moreover, it is unlikely – and not necessarily desirable – that any scheme entirely controls the quality of coaching in the field, no matter how good the coaching model. However, it has been argued that a capacity building scheme such as the CPTS that enjoys widespread recognition and a strong reputation must ensure to consistently apply its own rules and standards when selecting, training, motivating and supporting coaches, as well as in optimising communication, information sharing and knowledge management prior to trainees’ arrival in their respective postings. In other words, continued efforts are necessary to ensure country programmes understand the role of, and are supported in identifying the right individuals as, coaches; coaches obtain training; and trainees’ concerns about individual coaches are treated seriously and followed-up swiftly, particularly to avoid disappointments of consecutive trainees with the same coach. It must nevertheless be considered that the goal of a scheme like the CPTS is partly to provide exposure to real-life settings, which are indeed often challenging, not least in regard to management and support. In other words, even less satisfactory coaching experiences provide valuable learning opportunities for trainees. Key issue: cost effectiveness The CPTS provides good value for money and a good return on investment. Considering not only the number of highly-skilled graduates but also the important cost savings that occur due to there being a trusted source for CPIE personnel reinforce this fact. Moreover, CPTS graduates tend to stay in the sector for a number of years. Nevertheless, in order to increase cost effectiveness and thereby sustainability, looking for savings is a sensible option. For example, cutting trainees’ allowances, which currently make up about one third of the overall investment, should be considered. Scholarships should nevertheless be provided to ensure equal opportunity for candidates who could not otherwise participate in the scheme. Other areas for improvement Other suggestions for improvement are: to strengthen the collaborative nature of the scheme by involving partner organisations from the outset; systematically engaging them in the recruitment, selection, tool design, management and training process as equal partners; and thereby increasing sustainability through improved ownership, multiplied networks and enhanced funding opportunities, even if such strengthened involvement comes with increased management complexities. An additional, easy-to-implement solution would be to more actively and systematically share tools, progress reports and other key documents with partners both as a means for improving interagency collaboration and, importantly, also as an end in itself. The tools, particularly the Competency Framework, may also benefit from a review through a ‘diversity lens’ as well as wider consultations in the global South, inviting feedback from non-Western CPIE professionals in regard to cultural appropriateness and sensitivity. Key considerations in moving forward The sector needs continued support and investment. Moreover, it is widely agreed that capacity building needs to continue to emphasise child protection in humanitarian emergencies and fragile

36

settings. It is arguable that, given the level of funding available for emergencies vis-à-vis development, this is also beneficial from a funding and political support point of view. Moreover, if opened up towards development, there is a risk that the emergency element would be lost. Therefore, calls for a continuation of the Child Protection Trainee Scheme are loud and clear and it is certain that the CPTS would leave a gap if it was discontinued. One respondent summed this feeling up by admitting that they are “worried for the sector if the CPTS doesn’t continue”. Taking a two-pronged approach A new and different capacity building scheme, for example one targeting mid-career professionals such as the Advanced Diploma currently being discussed, would fill part but not all of the gap left by the CPTS. Therefore, a two-pronged approach, continuing the CPTS in a slightly modified format alongside setting up a new capacity building scheme for mid-level CPIE managers should be considered as this would address both key areas of capacity needed in the field. In order to ensure cost effectiveness, both schemes may be jointly managed, thus increasing scale, reducing cost per person, improving synergies and enlarging the network. This would further strengthen interagency participation and thereby, potentially, also improve funding opportunities. Similarly, it is arguable that a continued focus on child protection in humanitarian emergencies vis-à-vis development would be beneficial in regard to funding. Any recruitment process, particularly for an entry-level scheme, will need to ensure that those selected fully understand, and are willing to accept, the challenges in balancing work and life when engaging in CPIE. This will likely improve sustainability by reducing drop-out rates from the sector. More research and analysis The CPTS provides ample learning opportunities about humanitarian capacity building. Nevertheless, continued efforts are needed to monitor, evaluate and, if necessary, adjust capacity building programmes. Besides more research into diversity as outlined above, the long-term impact of the CPTS, for example in terms of follow-up on graduates’ careers, should be captured regardless of whether the scheme is continued or not. To conclude with the words of one respondent: “There are lots of keen individuals who want to get into the sector - give them the chance!”

37

Annexes

Annex 1 – list of individuals who responded to the online survey30 Achieng Kokonya Adam Clark Alyson Eynon Ann Hall Aye Aye Tun Camilla Jones Catherine Barnett Catherine Cunningham Claire Whiting Claudine Inamahoro Diana Keller Elena Giannini Esther Lieu Esther Obdam Fiona Bukirwa Gareth Owen Géraldine Liénhart Hanna-Tina Fischer Hannah Thompson Hellen Nyangoya Iris Knuppel Janis Risdel Jennifer Moorehead Jennifer Morgan Joanne Doyle Jorge Valles Joy Cheung Kate O'Brien Katie Bisaro Laura Boone Laura Miller Laura Payne Lotte Claessens Lucy Batchelor Luis Sfeir-Younis Maria Makayonok Mazen Haber Monica Nyatega Mubarak Maman Muna Basnyat Mustafa Hassan Nicolas Martin-Achard Paul Murphy Rachel Pounds Rebecca Musgrove

30 As outlined in the methodology section, these respondents represent a mix of individuals associated with the CPTS. In order to ensure anonymity and avoid comments made to be associated with specific individuals, the nature of their involvement with the CPTS is withheld.

38

Rinske Ellermeijer Rivka van Deijk Salam Kanaan Sara Mabger Shona Bezanson Sian Kelly Tony Dynes Victoria Whitaker Wajahat Farooqi William Harper William Lynch Ylva Sperling Yvonne Agengo

Annex 2 – list of individuals interviewed individually and in focus group discussions Achieng Kokonya Alyson Eynon Ann Hall Catherine Barnett Catherine Cunningham Hannah Thompson Janis Risdel Katie Bisaro Laura Boone Laura Miller Luis Sfeir-Younis Mazen Haber Mubarak Maman Rinske Ellermeijer Sabine Rakotomalala Sara Mabger William Harper

39

Annex 3 – CPTS graduates and their current positions31 2010 Intake Yvonne Agengo Emergency Response Personnel, Save the Children William Harper Emergency Education Manager, Save the Children Tunisia Crystal Stewart Child Protection Consultant, WarChild Holland OPT Joy Cheung Child Protection Coordinator, Terre des Hommes Sri Lanka Rinske Ellermeijer, Child Protection Programme Manager, Terre des Hommes

Dadaab, Kenya Tessie Semo Child Protection Consultant, Kenya Lotte Claessens, Currently looking for work in child protection Elisa Calpona Currently looking for work in child protection Catherine Cunningham Child Protection in Emergencies Specialist, Plan International,

Liberia 2009 Intake Samuel Asanga Child Protection Adviser, UNICEF, Central African Republic Marie Dahl Humanitarian Coordinator, Save the Children Denmark Elena Giannini Emergency Response Personnel, Save the Children Nidhi Kapur Emergency Response Personnel, Save the Children Achieng Kokonya Child Protection and Education Programme Officer, Save the

Children, Kenya Emergency Reponse Programme Iris Knuppel Child Protection Consultant Maria Makayonok Protection Program Officer, IRC Pakistan Laura Payne Child Protection Consultant Lindsay Shearer Emergency Response Personnel, Save the Children 2008 Intake Jackie Olanya IDP Camp Transitions Technical Advisor/Manager, World

Vision Haiti Earthquake Relief Response Fanny Asselineau Child Protection Consultant Mazen Haber Child Protection Consultant Sara Mabger Field Coordinator, MSF Morocco Janis Ridsdel Child Protection in Emergencies Specialist, Plan International Paul Murphy Deputy Head, Primary School UK Nicolas Martin-Achard Education and Child Protection Programme Manager, Save the

Children, DRC Hellen Nyangoya Area Manager, Kenya Programme Emergency Response, Save

the Children Hannah Newth Programme Manager, Save the Children, Ivory Coast 2007 Intake Anna Bryant Humanitarian Security Consultant, Control Risk Group Bridget Steffen Course Coordinator, Diploma in Humanitarian Assistance

(DHA), MSF, South Africa (part time), and Child Protection Consultant (currently in Kenya with Save the Children)

Carron Beaumont Senior Programme Officer, Mercy Corps, European HQ Esther Lieu Project and Grants Manager in Child Trafficking & Child

Labour, Advice on Individual Rights in Europe Centre, UK

31 This list is based on information contained in the February 2011 CPTS newsletter and was updated with information gained during interviews and complemented by recent email communication with former trainees. However, it still is a snapshot of mid 2011 rather than an accurate update of the current situation.

40

Lucy Bachelor Child Protection Regional Senior Specialist Middle East and Eurasia, Save the Children, USA

Mathilde Lemoine Programme Officer for West Africa (Education & Child Protection), Plan, UK

Myriam Marcuello-Lopez Child Protection Consultant Shona Bezanson Child Protection Advisor, Save the Children, Sierra Leone Sarah Hildrew Child Protection Technical Adviser, Save the Children Kenya Camilla Jones Currently studying for a Master’s in the UK (Sussex University) 2006 Fiona Bukirwa Child Protection Manager, Save the Children, Liberia Chiara Tamponi Child Protection Consultant Victoria Whitaker Emergency Child Protection Manager, Save the Children,

Tunisia Emmanuel Tuju Child Protection Officer, PACT Tanzania, Hannah Thompson Child Protection Consultant Tamirat Demessie Child Protection Advisor, Save the Children, Sweden Nancy Purdy Trafficking Project Worker, Barnardo’s, UK Claire Whiting Coordinator of Pakistan Humanitarian Forum Rivka van Deijk Integration Caseworker, Dutch Council for Refugees, The

Netherlands Jessica Chaix Humanitarian Programme Coordinator, Oxfam, Burma 2005 (pilot year) Hanna-Tina Fischer Rapid Response Team, Child Protection Working Group

(CPWG) Alyson Eynon Child Protection in Emergencies Advisor, Save the Children,

UK Jennifer Morgan FTR Coordinator, IRC, Haiti Rebecca Musgrove Sexual Health Adviser, Brooke Advisory, UK Asmara Figue International Protection Manager, Action Aid, Kenya,

Mozambique and Ghana Géraldine Liénart Currently looking for work in child protection

41

Annex 4 – questionnaire used for semi-structured interviews and focus group discussions

Brief introduction to the evaluation and consultant About the respondent Are you: a current/former trainee, current/former coach, country director who has hosted at

least one CPTS trainee, current/former SCUK line manager of the scheme, SCUK staff working at HQ on issues around child protection and/or capacity building, other working on child protection in emergencies, other indicate all that apply

Have you completed the online questionnaire? Overall success of the scheme Has the CPTS helped ensure that major humanitarian responses have an adequate child

protection component? If so, how? If not, why not? Please provide examples. Has the CPTS improved the speed of deployment of child protection in humanitarian

emergencies? If so, how? If not, why not? Please provide examples. Has the CPTS increased the capacity of your organisation (Save the Children and its partners)

to protect children in emergencies? If so, how? If not, why not? Please provide examples. Has the CPTS improved inter-agency collaboration in the area of child protection in

humanitarian emergencies? If so, how? If not, why not? Please provide examples. Output 1 Quality of the scheme Have you come across CPTS graduates (other than yourself, if you are one) in your work? If

so, how would you evaluate their knowledge, experience and skills in regard to child protection in emergencies?

What are the scheme’s key strengths? What are the scheme’s key weaknesses? Trainees What aspects of the traineeship did you find most useful? Why? What aspects of the traineeship did you find least useful? Why Do you have any other comments on the quality of your traineeship? Hosts and coaches How do you define a good trainee?

42

How would you evaluate the knowledge, experience and skills of CPTS trainees you hosted or coached in regard to child protection in emergencies?

Did you go for a second trainee after having hosted a first one? If so, why? If not, why not? Did the CPTS cause a positive spillover effect in your programme in terms of increased

interest in, and knowledge on, child protection in emergencies by staff other than trainees and coaches (but including yourself)?

Do you have any other comments on your experience with trainees? Recruitment and diversity

Please comment on gender and ethnic diversity of the CPTS

Trainees Line managers of the scheme How does the CPTS recruitment process, in particular the selection weekend, impact on

diversity of successful candidates? How could diversity in the scheme’s application and selection process be improved? Please comment on the overall value for money of running a selection weekend for the CPTS Do you have any other comments on the CPTS recruitment system? Blended Learning approach Trainees What aspects of the scheme provided the greatest opportunity for learning? Why? Please

mention key strengths and weaknesses of each element. (London-based training events; coaching/mentoring; first placement; second placement; competency-based learning; self-study – only mention if no recall)

Which of the three training events was the most useful? Why? Please mention key strengths

and weaknesses. (induction, including security training; annual learning event; end of cycle workshop - only mention if no recall)

Do you feel interagency participation in these events has made a difference? If so, how? If

not, why not? Do you have any other comments on the Blended Learning Approach? Matching process How do you define a ‘successful match’ between a trainee and host placement Trainees Please rate the overall matching experience with your two placements. What worked, what

didn’t?

43

(How) could the trainee/host placement matching process be improved? Do you have any other comments on the matching process? Hosts and coaches How would you rate the CPTS ‘matching’ process? Was the trainee well matched to your

programme? Why, or why not? How about management of the process? Coaching How do you define a good coach?

Trainees Hosts Line managers What was the quality of coaches in the scheme, both in terms of coaching and mentoring as

well as technical skills? How did good coaches differ from less effective ones? What, if any, was the difference between coaches who had been trained versus those that

had not been trained by the project? Do you have any other relevant comments on coaching? Coaches Did you ever perform the role of a coach before joining the CPTS? Did you receive training to be a CPTS coach? If so, what was the added value? If not, why

not? How many CPTS trainees did you coach? If more than one, what made you do this? Have you worked as a coach since? If so, what aspects of the training can you apply? Which

ones are less relevant? Is there anything else you would like to say about your coaching experience? Career paths Trainees Which agencies did you work for during your traineeship, and where? Do you plan to continue working in the area of child protection in emergencies? If so, why?

For how long? If not, why not? Former trainees/graduates only Have you worked in the field of child protection in emergencies since graduating from the

scheme? If so, which agencies have you worked with and where? If not, why not? Do you feel that your CPTS training has helped you significantly in finding employment?

44

What aspects of your CPTS training have you been able to apply? Where do you identify gaps in the CPTS training (i.e. what should the training have provided

to help you do your job better)? Do you have any other relevant comments on your career? Output 2 Coaching Toolkit Are you aware of the Coaching Toolkit? Have you ever used the Toolkit? How? If so, how? If not, why not? What are the toolkits strengths and weaknesses? Do you have any other comments on the Toolkit?

Competency Framework Has the CPTS facilitated the development of an interagency competency framework? If so,

how? If not, why not? How would you evaluate the usefulness of the competency framework? What is it used for?

What else could it be used for? What are the strengths and weaknesses of the framework? Do you have any other comments on the Framework? Output 3 Good Practice Network Are you aware of the Good Practice Network? What is its added value? (How) have you used it? Or, if not, why haven’t you? What other professional networks in the area of child protection and/or humanitarian

emergencies are you a member of? (How) has the informal network that emerged from the CPTS been of use? § (How) is the CPTS Good Practice Network linked up with other professional networks, such

as the Child Protection Working Group Do you have any other comments on the Good Practice Network? Overall Conclusion and outlook

45

Please comment on the necessity of running a scheme focusing on child protection in emergencies specifically (as opposed to a scheme focusing on child protection in both humanitarian and development contexts)

What other capacity building initiatives relating to child protection and/or humanitarian emergencies do you know?

Which ones have used (elements of) the CPTS as a model? If there was one thing you could change about the CPTS…

What can future child protection or humanitarian capacity building schemes learn from

the CPTS? Do you have any other comments on the future of child protection in humanitarian

emergencies?

Other

Do you have any other comments, suggestions or questions?

Many thanks for participation in the interview!

46

Annex 5 – CPTS logical framework CPTS Objective Objective Indicators Related Output

Increased numbers of skilled and experienced child protection staff are deployed within humanitarian emergencies Major humanitarian responses have an adequate child protection specific component Speed of deployment of child protection response improves in each humanitarian emergency

1. A cadre of at least 46 highly skilled child protection personnel is deployed within Save the Children’s programmes and throughout the humanitarian system, promoting the protection of children and generating programmes which enhance protection for children

Availability and use of coaching toolkit

Availability and use of new operational tools in child protection

2. A first class Coaching Toolkit for Child Protection in emergencies, developed in collaboration with key stakeholders, facilitates personnel development essential to growth of sector

Enhance the capacity of the international humanitarian system to protect children in emergencies

Increased capacity of Save the Children and its partners in terms of knowledge and skills on how to protect children in emergencies.

Increase influence and involvement in the Child Protection Working Group and other humanitarian reform processes

3. A good practice network of child protection practitioners is established, linking experts within a range of humanitarian organisations and contexts and facilitating learning and promoting quality for the sector as a whole.