2013 dispa meeting – dublin 23rd & 24th may 2013europa.eu/eas/dispa/docs/draft report dispa...

38
DISPA REPORT Developing a culture of quality in training Report of the Meeting of the Directors of Institutes and Schools of Public Administration of the European Union (DISPA) during the Luxembourgish Presidency of the Council of the EU

Upload: dangdien

Post on 31-Jan-2018

215 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

DISPA REPORT

Developing a culture of quality in training

Report of the Meeting of the Directorsof Institutes and Schools of Public Administration

of the European Union (DISPA) during the Luxembourgish Presidency of the Council of the EU

Luxembourg, 26-27 October 2015

Page 2: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

2

DISPA Meeting – Luxembourg 26-27 October 2015

Page 3: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

3

DISPA Meeting – Luxembourg 26-27 October 2015

THE MEETING OF DIRECTORS OF INSTITUTES AND SCHOOLS OF PUBLIC ADMINISTRATION

(DISPA) ORGANISED UNDER THE AUSPICES OF THE LUXEMBOURGISH PRESIDENCY OF THE

COUNCIL OF THE EUROPEAN UNION

Developing a culture of quality in training

The meeting was hosted by the Luxembourgish Institute of Public Administration (INAP). A "Troika" preparatory meeting took place in Luxembourg on 10 July 2015 attended by the Luxembourgish Institute of Public Administration (INAP), the Latvian School of Public Administration (LSPA) and the and the European School of Administration (EUSA).

The theme was selected by INAP and the agenda was drawn up in such a way as to contain a mix of presentations, discussions and workshops.

The programme can be found in annex I and the list of participants in annex II. Copies of the speakers' PowerPoint presentations have been made accessible at the DISPA wiki at http://europa.eu/eas/dispa/pages/home.html.

The meeting was chaired by Romain Kieffer, Director of the INAP.

Page 4: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

4

DISPA Meeting – Luxembourg 26-27 October 2015

Presentation of subject and meeting (David)

Romain Kieffer, Director INAP

8th European Quality Conference: highlights

Guy Wagener, Ministry of Civil Service and Administrative Reform

Mr Wagener as member of the EUPAN network reminded that the network exists since 1980 at the level of ministers, but meetings are rarely organized at that level. A meeting was held in Rome in December 2014, while the previous one was held 7 years ago. In this context it is worth looking at what the ministers identified as key challenges:

What are the key drivers and challenges in public sector modernization, based on countries’ experiences?

Are there new opportunities for European cooperation in the field of public administration?

In their final resolution, the ministers emphasized the strong relationship between the quality of public administration and the economic and social objectives with the following quote: “Quality of public administration strongly affects competitiveness, growth, social cohesion, and therefore the need to modernize administrative systems is a top priority.”

The second key message was on public sector modernization, to restore trust in public institutions. For such an objective to be achieved the main strategy is the one of innovation and development of new capabilities as well as effectiveness through evidence-based policy decisions.

Mr Wagener noticed a fundamental shift concerning the place of Public sector in the agenda setting of the meetings in 2007 and 2014. The Lisbon strategy did not even mention the public sector in its mid-term review in 2007, whereas in 2014 the public sector has a central place in the agenda, not only from the perspective of cost and budget but also as a central field of the political agenda.

The main priority identified by the HRWG for public administrations was to build trust and to use innovation to enhance HRM strategies, policies and practices and hence 4 main activities were identified:

Page 5: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

5

DISPA Meeting – Luxembourg 26-27 October 2015

study on the most promising trends, challenges and practices in OECD countries which support innovation

survey and analytical paper on enhancing innovation through an effective management of a more diverse workforce (OECD)

Comparative survey on innovative security and data protection measures

a quality conference as an opportunity to share best practices in these fields.

A task force was set up to look at ways and means of working better together. The recommendations will be presented in December and will be focused on 6 items, among which the continuity through the troïkas and the distinction between ‘DG level’ and ‘Working level’ meetings & activities.

One of the main challenges was to have all the concepts listed in the presentation in the same study. The overall conceptual model has at its heart the open government principles with a special attention to trust and transparency. It is based on two kinds of practices: HRM and PSD (Public services delivery) with a linking mechanism consisting in «innovation capacities».

The model enables to identify key innovation enhancing HRM practices and PSD practices and reads itself from left to right as the process of public value creation with 10 results (5 in each category) and their corresponding practices.

These results are in return generated by specific HR bundles and PSD bundles of practices that foster these results.

The recommendation was to have on the agenda of the ministers the following question: what can be done to enforce innovation culture?

Taking into account the highly increased importance of public administration in the political agenda a reflection was conducted to break down the agenda of the 8th QC («Strengthening the capacity of public administration in tackling current and future challenges”) in two more concrete reflections:

Breaking down overall administrative capacity in more concrete realities

What do high quality services require?

These questions led to 5 key activities listed in the power point presentation as trends and building blocks of the development of administrative capacity.

One of the highlights of the conference was the use of a new tool for design thinking in cooperation with the ‘Danish Design Center’ and ‘Luxembourg Institute of Science and Technology’

In terms of methodology the result is a new model of exchanges within EUPAN based on an interactive and collaborative approach.

In terms of content the outcome is a series of key messages:

Page 6: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

6

DISPA Meeting – Luxembourg 26-27 October 2015

the common problem for all administrations is to tackle complex challenges

the common added value of public administrations is to create solutions and concrete actions with impact

these solutions can be developed via the implementation of participatory design practices

the sustainability of the system relies on ensuring a continuous improvement of public administrations' practices

Sylvain Cottong, Ministry of Civil Service and Administrative Reform

M. Sylvain Cottong started his presentation by stating that the world has radically changed during the last 30 years, with a range of trends and events such as globalization, migration, environmental risks, financial pressure, digital transformation and ubiquitous connectivity having a strong impact on the political, economic and social context. He underlined among various consequences the increasing speed and complexity, the population heterogeneity, the heterogeneity of needs, the higher expectations for quality.

In this changing world "Crisis is the new normal" going from climate change to financial crisis, Euro crisis, debt crisis, Ukraine, Syria, IS, refugees, cyber threats. And above all it’s impossible to foresee what could come next.

The result is an increasing complexity which can be described through the concept of a VUCA world (Volatility, Uncertainty, Complexity, Ambiguity) where complex situations don’t have an immediate answer, but where systematic experimentation is needed to discover what works (CYNEFIN model, Davis Snowden). At the same time, and since the advent of internet the way we live, communicate, work and collaborate has fundamentally changed, people connect on the fly in different networks to collaborate, exchange and create value at almost no cost and without the need of an organization.

The problems can be sorted out in two types: those of public administration per say and the problems of the economic and social fabric. In this context the main role of PA is to manage the unexpected.

Page 7: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

7

DISPA Meeting – Luxembourg 26-27 October 2015

The consequences on the organisations are that more and more often, long established, mostly hierarchical and rigid management and organisational models (as well as business models) are disrupted and swept away (sometimes overnight…). In this respect monocultures appear to be weaker at absorbing external shocks. In fact, governance models at all levels are challenged and affected: politics, business, and social systems.

Mr Cottong focused then his presentation on the impact of this situation for the Public Administration training and its contribution to Public Service Quality which he referred to as "created by people, processes and relationships, forming together an organization’s intellectual capital and intangible assets». He mentioned that the contribution from DISPA to EUPAN is not on the agenda but the question of the coordination of the respective agenda was raised throughout a questionnaire.

The new success factors for organizations are dynamic capabilities, absorptive capacity, resilience, agility and the big shift is the move from institutions designed for scalable efficiency to institutions designed for scalable learning. Leaders must get comfortable with living in a state of continual becoming.

In a concrete manner, Mr Cottong identified the areas in which PA’s need to strengthen their capacities via organizational learning. They range from Strategic management including performance management, risk management, intellectual capital management and design thinking to experientialism which is according to Mr Cottong a valuable concept to take from philosophy to design the new approach.

Quality of Public Administration: A toolbox for practitioners (David)

Nick Thijs, European Commission

Page 8: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

8

DISPA Meeting – Luxembourg 26-27 October 2015

A training quality culture

Claude Curzietti, Professor at INAP

Training is a common word employed in a variety of expressions ranging from physical training to happiness training. However there is a need to distinguish between:

training: the process of bringing a person to an agreed standard of proficiency by practice and instruction

learning: the act of gaining knowledge.

Training has a clear focus on performance whereas learning has a more passive connotation.

In life-long « learning », the managers are in fact mostly interested in the return on investment. In this performance driven approach the central question becomes the one of quality with its objective dimension (is the course well documented, organized …) and its much more volatile subjective dimension (how do people feel about the training).

Looking at the world from the learner perspective leads us to move from the objective quality to the subjective one. The importance of this shift is reflected in the consequences it has for the ISPA’s in their approach to learning (putting in brackets the ROI) and their positioning on the market (mapping themselves out according to the recognised value and capability).

To develop a training culture 3 main dimensions are to be taken into account:

1) relevance : This dimension shows several possible stages.

-chaotic design (typically the one of a start-up company)

-reactive model (the Institute has a demand driven agenda, the limit being that the ISPA can have a limited view of competences required, and as a result there can be a lot of fire-fighting)

-pro-active (service driven model, the Institute would co-define the agenda)

Page 9: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

9

DISPA Meeting – Luxembourg 26-27 October 2015

-predictive (value driven model, the Institute is able to determine which competences are needed in the up-coming years)

2) competitiveness: This dimension aims at taking the learner’s perspective, not only to satisfy people but to delight them, hence creating reputation, brand appeal.

3) effectiveness: The ROI, is focused on the transfer in real life, however it's often an illusion (because of the number of influencing variables ranging from individual willingness to contextual influences) that many Institutes pursue as a holy grail.

As a result, even if training quality is still important to work on, the introduction of a learning culture needs to shift the emphasis on the institutional investment rather than the individual.

Workshop- Cases and conclusions

1 st case:

Dafodilia is a EU member state. The supervisory authority of Dafodilia’s Institute of Administration, Politics, Education and Research (DIAPER) has asked the school to re-focus their learning program. The reason is that the school’s current program is very focused on academic, administrative, traditional training. Lars Knäckebrot, the Head of DIAPER, said at a press conference: « Today’s civil servants must become more knowledgeable in current economic, social and political affairs, such as : refugees & immigration, the Greek and other debt crises, the war in Ukraine and its consequences, the rise of nationalisms, TTIP, cyber-security, modern transportation, etc. » As a think tank and task force, you have been invited to provide input for a new, relevant training content for DIAPER. How would you outline its main features?

Page 10: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

10

DISPA Meeting – Luxembourg 26-27 October 2015

2nd case:

iBAK (The international Board for Administrative Knowledge), a newly created European body, has

called upon the DISPA network to discuss and investigate the learning needs of European civil servants in 2020 and beyond. Miranda Futura, the Head of iBAK, is looking for the following input: 1. What are the major trends on other continents? 2. What new competencies will civil servants need in the future? 3. Are ISPAs equipped and positioned to deliver these new competencies? 4. How can ISPAs help shape tomorrow’s civil service? You have joined forces to provide some answers to Mrs Futura’s questions.

Page 11: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

11

DISPA Meeting – Luxembourg 26-27 October 2015

3rd case:

Propolis is a new European member state. Its Institute for Professional Administration (IPAd) can be described as, and compared with, a typical European school for administration. IPAd has a reasonable course offering, positioning, capability, quality, reputation. The school is neither the best nor the worst, it is « market standard ». Ivan Goodwill, Propolis’ new minister for public service, is facing criticism about the quality and value of IPAd overall. His opponents believe that other providers can do better, at a competitive price. He is planning to meet his opponents next week. You have been invited to help Ivan fight for the survival of IPAd. What he needs is a rock-solid competitive case.

4rth case:

The United Academy (TUnA) is a state and privately-owned administration academy, for civil servants of Xantia, a quiet EU member state. TUnA has had a good reputation for years: it is an old-fashioned but well-run institution, its courses are well designed and the faculty is recognised. The new Chairwoman, Annabella Cat, wishes to reposition the academy’s brand to keep competitors out. She briefed her advertiser using the following words: value, technology, degrees, qualification, research, attractive, desirable, blended learning, business school, value proposition, unique selling point. You have been invited to help Mrs Cat redesign the brand. What suggestions would you make, what concept would you propose?

Page 12: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

12

DISPA Meeting – Luxembourg 26-27 October 2015

5th case:

Bethadin is a new European member state. John Cosmic, the chairman of Bethadin’s School for Administration (SAd) wishes the school to broaden its network of alliances, so as to provide a moreeffective learning platform for all of civil servants. Mr. Cosmic wants to deflect the multiplication ofservice offerings, which could be detrimental or fatal to SAd. The idea is to federate learning offerings from the country’s universities, public and private institutes into an effective, powerful, transparent, learning platform for civil servants. Based on your experience, what key alliances would you recommend for Mr Cosmic? How would you make them work?

Page 13: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

13

DISPA Meeting – Luxembourg 26-27 October 2015

6th case:

Kindahoo, a new EU member state, conducted a survey on the perception of their Public Administration School (PAS). The results showed that course participants viewed PAS was an old-fashioned, disorganized institute.They complained about: slow registration, weak learning value, unprofessional service at the school, out-of-date teaching methods, lousy facilities and horrible coffee. Admittedly participants usually enrolled into courses because they had to, not because theywished to. Giorgina Mogul , the Head of PAS, has desperately asked DISPA members for help: sheneeds 10 great, high-impact ideas to quickly and visibly improve quality.

Page 14: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

14

DISPA Meeting – Luxembourg 26-27 October 2015

Page 15: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

15

DISPA Meeting – Luxembourg 26-27 October 2015

The reform of relevance initial training at ENA

Dauphinelle Clément, Ecole Nationale d'Administration

In ENA the assumption and conviction is that offering high education is key to have a high level public administration.

The ENA was created in October 1945 and hence is celebrating its 70th anniversary. Its core business is to broaden the access to the civil service, to train the leaders of tomorrow as well as today’s managers.

ENA’s main responsibilities are: to recruit and train the top civil service and to enable it to adapt to changing times. Its key values are: responsibility, neutrality, commitment to public service.

In changing times delivering high quality service is a constant challenge for ENA, which itself has to adapt constantly.

1) Some lessons of striving for excellence for 70 years:

ENA has developed an integrated quality management system whose components are strongly interrelated. All along the 24 months curriculum a variety of tools is used for evaluation purposes:

It starts with the annual report of the jury in charge of the selection; it’s a very valuable tool on the general profiles of the students.

On the students' profiles: evaluation to determine specific development and training needs to develop special skills.

The professors: ENA has only one permanent teacher (for sports). All others are constantly changing which is a very important source of renewal of the contents and input. The training methodology is largely practical and based on case studies. The content and teachers themselves are evaluated by the students.

The traineeships: Half of the training programmes consist of internships, which are constantly evaluated by the Director for internships.

Page 16: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

16

DISPA Meeting – Luxembourg 26-27 October 2015

Quality control procedure via ex post evaluation and impact assessment via the students associations: questionnaires are sent to those who left ENA in the last 14 months and to their administrations as well to get feed-back on the former students. The outcome is then used to improve the curricula.

The lesson driven from experience is that stakeholders' involvement is the key factor and in ENA’s case it means to include the end user: the employer.

2) ENA’s reform:

From the beginning ENA has been in a constant reform process, it has gone through 24 major reforms since 1945. However the current one is probably the most important one, it aims at changing the skills, the competences and the way to serve.

This deep reform will bring major changes in the competition and the curriculum:

a collective exam is introduced to measure the behavioral skills

a demanding test of public financing is introduced as it can no longer be considered as a specific area

the famous « grand oral » will be changed to make sure it really fits to a recruitment process, professional achievements for instance will be better taken into account. (The underlying rationale for this very special exam is to open public service to diversity by avoiding if possible the elite reproduction mechanisms).

the reform introduces at its heart the need to better adapt the students to the needs of Public Sector of tomorrow. The most challenging remains to teach public action.

The core of the reform is the central place of the management with a focus on public ethics and values as the public opinion in France is waiting for exemplarity and not only excellence.

3) Developing an integrated strategy for leadership and talent management:

A brand new program, mixing participants from a variety of departments was developed by ENA. It is an 8 day over 9 month program designed around individual and group development activities.

The aim is to allow participants to be better equipped for senior positions by sharing a managerial culture targeted to national interests.

Participants are also offered the space and tools to reflect at their future career development.

Question from the audience:

"You’ve mentioned diversity; can you elaborate a bit on this aspect?"

Page 17: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

17

DISPA Meeting – Luxembourg 26-27 October 2015

The Director is strongly committed to the issue of gender balance which is particularly challenging in high civil service. She even wrote a quite successful book on her own experience whose title is « Choose everything! ». In this field, one of the difficulties is that there is self-censorship even in the jury, so the jury is trained to fight against their own socially decided beliefs.

Performance management in Austria

Klaus Hartmann, Director, Federal Chancellery, Public Service and Administrative Reform

Performance management in Austria was presented by Mr Hartmann through a concrete example of a performance-based budgetary reform. The starting point was that due to an over fragmented structure, the budget turned to be inflexible, not transparent and missing the central question: what do we want to achieve with the taxpayers money?

The answer to these challenges was to introduce an "outcome orientation" in the Constitution as well as political aims and objectives in the budget for the first time in 2013.

Since the project aims at changing the administrative culture and not only to introduce words in the budget, it is a medium to long term process based on a performance structure (political objectives- input-activities-output-outcome). The connection of the budget structure with the performance structure is ensured by having a correspondence of the elements of each.

Due to its cycle dimension it is also a learning process for the organization in itself, following the principle of plan-do-check-act.

To illustrate the outcome orientation, which is the central objective of the reform, one example is the information cycle. The desired outcomes defined by the government are made transparent to the Parliament and interested public; the appropriateness and the level of ambitions of objectives can then be discussed and allow a better link between the citizens or stakeholders and their administration.

The added value of the project cannot be evaluated of course; however there are already feedback mechanisms and an assessment of consequences for the economic and social fabric as well as the environment.

Page 18: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

18

DISPA Meeting – Luxembourg 26-27 October 2015

Lastly, this year the technical foundations of the system have been greatly improved to support line ministries in their Performance Management capabilities and transparent information to the parliamentarians and the interested public.

Learning and development: acting as an agent of change within the public sector arena (David)

Leo Smits, Director, PBLQ

Impact, innovation and authenticity: a tool to guarantee our learning solution’s quality

Sandra Schillemanns & Julie Camerman, Training Institute of the Federal Administration

Mrs Schillemanns talked about the major change that her organization went through and a major tool which was used to review this change.

EIFA organises certified trainings since 2003. In all the trainings the participants go through exams in order to get a pay rise. In 2013 there was a political decision to stop the certified trainings from 01/01/2017. The certified trainings take more than half of EIFA's activity (resources and budget) and

Page 19: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

19

DISPA Meeting – Luxembourg 26-27 October 2015

that is why although the staff was happy with the decision they also felt uncomfortable. The decision of EIFA was to reposition themselves as a training institute and this is why a big change process was put in action.

Apart from the certified trainings, EIFA has also an open catalogue, on demand training, induction and management trainings. But certified trainings were EIFA's first priority and suddenly the School was confronted with a situation of change.

So how did they go about it? First they analysed their professional environment and entered into a discussion with clients to determine what they would do, for whom and how. They had to make choices on how to offer an added value for their clients. This exercise was very difficult for the staff because many of them were working for years for the certified trainings, which is a standardized process.

After the first exercise to review EIFA's portfolio, what became evident was that the offer was more or less the same with some small changes in the calalogue. So what was done was to go back to the customers to identify their needs.

4 categories of clients were identified: 1) individual civil servants, 2) organisations, 3) professional networks and 4) the political level.

and 3 criteria were identified for the project managers: 1) we want to learn with impact, 2) we want to be innovative, 3) authenticity (every civil servant who follows a training has to recognize that the training is really related and relevant to his professional environment)- shift to be closer to the workplace.

"it’s not about training it’s about learning"- that is the major shift in the mindset.

So all the project managers were asked to develop their new portfolio based on the 3 types of clients and 3 criteria mentioned above.

In total 30 criteria were identified.

A tool was also developed to help them meet these 3 criteria, which consists of 3 parts:

1. a memo sheet -to give an overview of all the criteria

2. a booklet- gives a small definition for all the sub criteria, a list of potential situations where the learning solution can fit, some clue of how to use the 2nd column- possible situations (check slide)

3. an excel file- to be able to use the tool and make notes about the learning solutions/ proposals

The tool is used to develop the new learning offer, to determine the criteria for specifications, to analyse the supplier's offer, to co-create the learning solutions with the supplier and to present the offer to the clients.

Page 20: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

20

DISPA Meeting – Luxembourg 26-27 October 2015

The criteria are being used in 2 ways:

1. to help the participant to transfer learning to the workplace- they have prepared some tips for how you can prepare before the training- during and after the training. One of the things done to increase the transfer is to include the participant's environment e.g. in management trainings the manager of the participant is involved, so that the participant has a supporting environment where he/she can develop his/her competencies.

2. to try to evaluate the transfer through survey (different kinds of surveys)

In relation to the management courses, there is a clear decision not to offer open courses but offer the programmes only in organisational level.

Don’t measure the impact of training, improve it (Yves)

Yves Caelen, European School of Administration

Being able to evaluate the impact of training is important for trainers and training designers who want to improve their products but also for managers who are always interested in giving evidence about the value they add to the organisation.

It is possible to find in the market several methods of impact assessment for training (a very well-known one is the "ROI approach" developed by Jack Phillips) but they are generally very expensive to implement. Even their promotors advise to use them only for the most important training programmes (in terms of strategy or budget allocated) only.

On the top of this, these methods are generally not well fitted to the specific context of a School of Administration. The most common situation in this kind of context is indeed not to train people that will all implement their learning in the same context and with the same objectives. The clients of our Schools generally go back in their administration where they use what they have learned for a variety of purposes and in a variety of contexts. It would therefore be illusory to believe that we can easily measure how they perform and transform it in a direct measure of impact.

Page 21: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

21

DISPA Meeting – Luxembourg 26-27 October 2015

In order to overcome these difficulties, the method developed in the European School of Administration actually consists in replacing these expensive, direct, and badly adapted methods of impact evaluation by another approach based on an indirect evaluation of the impact of a course.

"Indirect" meaning that if we accept that we cannot really measure the impact as such, we still however have a possibility to measure the factors that will create this impact.

If the participants confirm to us that all these factors are there, we can reasonably infer that the impact will occur.

According to the classical framework developed by Donald Kirkpatrick, these factors are the following:

1. A learning activity actually occurred (level 1)

2. Something has been learned (level 2)

3. What has been learnt is used on the job (level 3)

4. This learning/skill/ behaviour also appears to be important for the specific job of the participant. It makes a difference (level 4)

Based on this model we propose the following "formula for impact":

Indicator of impact = LEARNING ACTIVITY x ACTUAL LEARNING x FREQUENCY OF USE x IMPORTANCE FOR THE JOB

Considering that any training course should actually be based on a set of objectives that could actually refer to elements that people should be able to do (or to do better) on the job, we evaluate each of the elements of the formula according to the principles that we describe below:

For each of the main objectives that has been set for the course (e.g. "after the course the participants will be able to deal effectively with her/his emails") we consider that:

1. "LEARNING ACTIVITY" is normally 1 ("yes the learning activity happened") but we can also check in the questionnaire a verification of this ("do you remember that this was tackled during the course?") and bring back this result to 0 if the answer is no.

2. ACTUAL LEARNING is the difference between how the participant evaluated his/her level of competence before and after the course (on a scale from 1 to 10). For example if the participant considers that his/her level was 6 and that after the course it was raised to 8 the result for this factor will be 8 – 6 = 2.

3. FREQUENCY OF USE will be 0 if the participant declares that he almost never use this specific competence. It will be 1 if the competence is used sometimes and 2 if it is used often or very often.

4. IMPORTANCE FOR THE JOB will be rated 0, 1 or 2 according to a similar scale (0 if the participant says it is not very importance, 2 if the participant says that it is very important and 1 in the middle).

Page 22: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

22

DISPA Meeting – Luxembourg 26-27 October 2015

Based on the answers received from participants and the indicators that will have been calculated for each of them on each of the competencies that the course aimed to develop, a rating will be made.

The threshold currently used in the School for this purpose is currently the following:

Indicator of impact

Level Example

0 – 3,99 Some impact

Learning + 1 Frequency 2 Importance 1 => 2

4 – 7,99 High impact Learning + 2 Frequency 1 Importance 2 => 4

8 + Very high impact

Learning + 2 Frequency 2 Importance 2 => 8

Based on this threshold, it is possible to check the proportion (in %) for whom the level of impact is high or very high.

These indicators can then be used among other things for the following purposes:

- to compare the proportion of high or very high impact among the various objective of a course. Is there one objective that receives significantly higher or lower scores than others?

- to compare the global level of the courses between them. Do you have in your offer a course that consistently scores high for the indicators of impact? Or another one that constantly scores low.

- If you try to improve a course, check also the evolution of the impact indicator. Did you manage to improve the impact?

- When a course has a low impact indicator, go into the detail. Is this due to a low level of learning? to a low possibility to use the competence on the workplace? To a low importance of the subject for the participants?

- When analysing the results, do not forget to read the comments written by the participants. Can they explain some low or high scores.

Page 23: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

23

DISPA Meeting – Luxembourg 26-27 October 2015

INAP management training concept (David)

Marc Ant, Professor at INAP

marc

before- participants would choose randomnly between 12 courses of mgtnew way- strategic mgt process

starting from the basics-actual situation and vision+objectivesthey ask the participants to follow this scheme for their depts. And link this to a concept of mgt

6 days theory and 6 days practicegeneral mgtstrategic mgtproject mgtorganizational behaviorleadershipcommunication

org. strategydepartmental strategymanagerial theorypersonal strategy

try to reflect on what we do in order to improve our behavior

pedagogical approach- they use films- 12 angry men

tailor-made approach to administrations and for public servants

MBA level 12 days spread on 12 months with homework and time to reflectpresnetations on their own cases- interactivepersonal contributions

challenge: there was a change of head of administration at 3 of 4 cases where every time the strategy was changing – new strategy:

DISPA matters

Page 24: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

24

DISPA Meeting – Luxembourg 26-27 October 2015

David Walker, Director of the European School of Administration (EUSA)

Invitation to the next meeting

Romain Kieffer passed the floor to Mariette Baptist, the Director of PBLQ, who made a small presentation of the next DISPA meeting. She then warmly invited all DISPA members to the next meeting of the network which will take place in the Hague on 25 and 26 April 2016. The Troika will meet in January to discuss the agenda.

Concluding remarks

After a final message of greetings and thanks to all participants addressed by Romain Kieffer, the meeting was declared closed.

* * * *

Page 25: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

25

DISPA Meeting – Luxembourg 26-27 October 2015

Annex I

Programme

Sunday, 25th October 201519.30 – 22.00 Welcome dinner

Monday, 26th October 2015 Speaker

9.00 – 9.30 Registration

9.30 – 9.45 Presentation of subject and meeting Romain Kieffer(INAP)

9.45– 10.30 “8th European Quality Conference: highlights”

Guy WagenerSylvain CottongBob Greis(Ministry of Civil Service and Administrative Reform)

10.30 – 11.15 “Quality of Public Administration:A Toolbox for Practitioners”

Nick Thijs(European Commission)

11.15 – 11.45 Coffee break

11.45 – 12.30 Keynote – “A training quality culture” Claude CurziettiProfessor at INAP

12.30 – 12.45 Family photo12.45 – 14.15 Lunch14.15 – 16.00 Relevance: acting as an agent of change

“The Reform of Relevance Initial Training at ENA”

“Performance management in Austria”

WorkshopCoffee break

Dauphinelle Clément(France)Klaus Hartmann(Austria)

16.00 – 16.15 Workshop conclusions16.15 – 17.45 Competitiveness – Creating value through uniqueness and

customer experience

“Learning and development: acting as an agent of change within the public sector arena”

WorkshopWorkshop conclusions

Leo Smits(Netherlands)

19.00 Bus departure19.30 Dinner

Page 26: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

26

DISPA Meeting – Luxembourg 26-27 October 2015

Tuesday, 27th October 2015 Speaker08.45– 10.45

Effectiveness – delivering unbeatable ROI

“Impact, innovation and authenticity: a tool to guarantee our learning solution’s quality” “Don’t measure the impact of training, improve it”

WorkshopCoffee break

Sandra Schillemanns(Belgium)Yves Caelen(EUSA)

10.45 – 11.00 Workshop conclusions

11.00 – 12.15 “INAP management training concept” Marc AntProfessor at INAP

12.15 – 12.30 “DISPA Matters” David WalkerEUSA

12.30 – 12.45 Invitation to the next DISPA Meeting in the Netherlands Mariette Baptist-Fruin(NL)

12.45 – 13.00 Closing remarks Romain Kieffer(INAP)

13.00 – 14.15 Lunch14.15 Bus departure14.30 – 16.00 Cultural visit

Page 27: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

27

DISPA Meeting – Luxembourg 26-27 October 2015

Annex II

List of participants

Austria

Bundeskanzleramt (Federal Chancellery) Sektion III Öffentlicher Dienst und Verwaltungsreform (Unit III Public Service and Administrative Reform)

Mr Klaus HARTMANN

Head of delegation

[email protected]

Belgium

Institut de Formation de l'Administration Fédérale

Opleidingsinstituut van de Federale Overheid

(Training Institute of the Federal Administration)

Mrs Sandra SCHILLEMANS

Head of delegation

[email protected]

Mrs Julie CAMERMAN

[email protected]

EIPA

European Institute of Public Administration

Mrs Cristiana TURCHETTI

Head of delegation

[email protected]

Mr Nick THIJS

[email protected]

EUSA

European School of Administration

Mr David WALKER

Head of delegation

[email protected]

Mrs Karine AURIOL

[email protected]

Mrs Fay GIANNAROU

[email protected]

Mr Yves CAELEN

[email protected]

Page 28: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

28

DISPA Meeting – Luxembourg 26-27 October 2015

Finland

HAUS kehittämiskeskus Oy

(HAUS Finnish Institute of Public Administration)

Mrs Anneli TEMMES

Head of delegation

[email protected]

France

Ecole Nationale d'Administration

(National School of Administration)

Mrs Dauphinelle CLEMENT

Head of delegation

[email protected]

Mrs Aurélie ROYET-GOUNIN

[email protected]

Georgia

Zurab Zhvania School of Public Administration

Mrs Ketevan JAKELI

Head of delegation

[email protected]

Mrs Teona SHEKILADZE

[email protected]

Germany

Bundesakademie für öffentliche Verwaltung

(Federal Academy of Public Administration)

Mr Paul Erich HUNOLD

Head of delegation

[email protected]

Mr Ernst WILZEK

[email protected]

Greece

Εθνικό Κέντρο Δημόσιας Διοίκησης και Αυτοδιοίκησης (National Centre for Public Administration and Local Government)

Mrs Naja VRETTAKOU

Head of delegation

[email protected]

Mrs Kasimi CHRYSAFO

[email protected]

Hungary

Kormányzati Személyügyi Szolgáltató és Közigazgatási Képzési Központ

(Government Centre for Public Administration and Human Resources)

Mr Norbert KIS

Head of delegation

[email protected]

Ireland

Institute of public administration

Mrs Marian O'SULLIVAN

Head of delegation

[email protected]

Page 29: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

29

DISPA Meeting – Luxembourg 26-27 October 2015

Italy

Scuola Nazionale dell'Amministrazione

(School of Public Administration)

Mr Alberto PETRUCCI

Head of delegation

[email protected]

Latvia

Valsts administracijas skola

(Latvian School of Public Administration)

Mrs Edite KALNINA

Head of delegation

[email protected]

Lithuania

Lietuvos viešojo administravimo institutas

(Lithuanian Institute of Public Administration)

Mrs Renata LATVENIENE

Head of delegation

[email protected]

Lithuania

Lietuvos viešojo administravimo institutas

(Lithuanian Institute of Public Administration)

Mr Arturas ARBATAUSKAS

[email protected]

Mrs Skaidrė KATILIAVIENE

[email protected]

Luxembourg

Institut national d'administration publique

(National Institute of Public Administration)

Mr Romain KIEFFER

Head of delegation

[email protected]

Mr Philippe DIEDERICH

[email protected]

Mrs Nadia DJEBBAR

[email protected]

Mrs Laura JUNGERS

[email protected]

Mr Claude CURZIETTI

[email protected]

Mr Guy WAGENER

[email protected]

Mr Sylvain COTTONG

[email protected]

Mr Marc ANT

[email protected]

Malta Mrs Joanna GENOVESE

Page 30: 2013 DISPA Meeting – Dublin 23rd & 24th May 2013europa.eu/eas/dispa/docs/Draft report DISPA Lux.docx  · Web viewThe programme can be found in annex I and the list of participants

30

DISPA Meeting – Luxembourg 26-27 October 2015

Centre for Development, Research and Training - Office of the Prime Minister

Head of delegation

[email protected]

Netherlands

PBLQ-ROI

(Dutch Institute for Public Administration)

Mr Leo SMITS

Head of delegation

[email protected]

Mrs Mariette BAPTIST-FRUIN

[email protected]

Poland

Krajowa Szkola Administracji Publicznej

(National School of Public Administration)

Mr Jan PASTWA

Head of delegation

[email protected]

Mrs Klaudia WOJCIECHOWSKA

[email protected]

Slovenia

Ministrstvo za javno upravo

(Ministry of Public Administration)

Mrs Sandra SEKETIN LESTAN

Head of delegation

[email protected]

Spain

Instituto Nacional de Administración Pública

(Spanish School of Public Administration)

Mrs Carmen GONZALEZ SERRANO

Head of delegation

[email protected]

Sweden

Universitets och Högskolerådet - UHR

(Swedish Council for Higher Education)

Mrs Petra GÖRANSSON

Head of delegation

[email protected]

Switzerland

Institut de hautes études en administration publique (IDHEAP)

(Swiss Graduate School of Public Administration)

Mr Martial PASQUIER

Head of delegation

[email protected]

Ukraine

National Academy of Public Administration under the President of Ukraine (NAPA)

Mr Yurii KOVBASIUK

Head of delegation

[email protected]