report of the panel of the review of the finnish higher education evaluation council ·...

ISBN ---- (print edition)ISBN ---- (pdf)ISSN -

Finnish Higher Education Evaluation CouncilP.O. Box (Meritullinkatu ) HELSINKIFINLANDTel. + Fax +

www.finheec.fi

Jon Haakstad • Peter FindlayTia Loukkola • Maria Helena NazaréChristian Schneijderberg

Report of the panel of the reviewof the Finnish Higher EducationEvaluation Council

PUBLICATIONSOF THE FINNISHHIGHER EDUCATIONEVALUATION COUNCIL

:

This is the report of the review of the Finnish Higher Education EvaluationCouncil (FINHEEC) undertaken in May for the purpose of determiningwhether the agency meets the criteria for Full membership of the EuropeanAssociation for Quality Assurance in Higher Education (ENQA).

9 789522 061850

Report of the panel of the review of FIN

HEEC

ISBN 978-952-206-185-0 (print edition) ISBN 978-952-206-186-7 (pdf)ISSN 1457-3121

Publisher: Finnish Higher Education Evaluation Council

Cover: Juha IlonenLayout: Pikseri Julkaisupalvelut

Tammerprint OyTampere 2011

Report of the panel of the external review of the Finnish Higher Education Evaluation Council (FINHEEC)

This is the report of the review of the Finnish Higher Education Evaluation Council (FINHEEC) undertaken in May 2010 for the purpose of determining whether the agency meets the criteria for Full membership of the European Association for Quality Assurance in Higher Education (ENQA). The membership provisions are listed in Appendix 6.5 to the report.

Contents

Background and outline of the review processBackground and outline of the review processBackground and outline of the review processBackground and outline of the review processBackground and outline of the review process __________________________________________________________________________________________________________________________________

GlossaryGlossaryGlossaryGlossaryGlossary _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IntroductionIntroductionIntroductionIntroductionIntroduction ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ . Higher Education in Finland __________________________________________ . The quality assurance framework in the Finnish HE system and

the work of FINHEEC _______________________________________________ .. The national quality assurance framework ________________________ .. The constitution of FINHEEC ___________________________________ .. The work of FINHEEC _________________________________________

. ENQA membership __________________________________________________ . The review process __________________________________________________

FindingsFindingsFindingsFindingsFindings _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ . ENQA criterion ____________________________________________________

.. ENQA Criterion / ESG Part : External quality assurance processes __ .. ENQA Criterion / ESG ., .: Activities _________________________

. ENQA criterion / ESG .: Official status ______________________________ . ENQA criterion / ESG .: Resources _________________________________ . ENQA criterion / ESG .: Mission Statement _________________________ . ENQA criterion / ESG .: Independence ______________________________ . ENQA criterion / ESG .: External quality assurance criteria

and processes _______________________________________________________ . ENQA criterion / ESG .: Accountability procedures ___________________ . ENQA criterion / Miscellaneous _____________________________________

Conclusion and developmentConclusion and developmentConclusion and developmentConclusion and developmentConclusion and development _____________________________________________________________________________________________________________________________________________________________________________________________________________ . Recommendations __________________________________________________

AnnexesAnnexesAnnexesAnnexesAnnexes _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ . Schedule for the review ______________________________________________ . Site Visit Programme ________________________________________________ . Government Decree ____________________________________________ . Standing order of the Finnish Higher Education Evaluation Council _______ . ENQA Membership Provisions ________________________________________

7

1 Background and outline of the review process

ENQA’s regulations require all Full member agencies to undergo an external cyclical review, at least once every five years, in order to verify that they fulfil the membership provisions. In November 2004, the General Assembly of ENQA agreed that the third part of the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) should be incorporated into the membership provisions of its regulations. Substantial compliance with the ESG thus became the principal criterion for Full membership of ENQA. The ESG were subsequently adopted at the Bergen ministerial meeting of the Bologna Process in 2005.

The third part of the ESG covers the cyclical external review of quality assurance and accreditation agencies. In accordance with the principle of subsidiarity, external cyclical reviews for ENQA membership purposes are normally conducte d on a national level and initiated by national authorities in an EHEA State, but carried out independently from them. However, external reviews can also be coordinated by ENQA if they cannot be nationally organised. This may be the case, for instance, when no suitable or willing national body can be found to coordinate the review. In that event, ENQA plays an active role in the organisation of the review, being directly involved as coordinator, whereas, in the case of national reviews, it is only kept informed of progress throughout the whole process.

The external review of FINHEEC was conducted in line with the process described in the Guidelines for external reviews of quality assurance agencies in the European Higher Education Area and in accordance with the timeline set out in the Terms of Reference.

The review panel for the external review of FINHEEC was composed of the following members:

Jon Haakstad, Norway, (Chairman)Peter Findlay, UK, (Secretary)Tia Loukkola, BelgiumMaria Helena Nazaré, EUA nominationChristian Schneijderberg, ESU nomination

8

FINHEEC produced a self-evaluation report (March 2010) which helpfully provided a background to the work of the Agency, and discussed in detail its compliance with the ENQA criteria and the ESG. The self-evaluation report (SER) was developed by the Agency’s Secretariat and approved formally by its Council. The SER and accompanying documentation submitted by FINHEEC provided a substantial portion of the evidence that the panel used to form its conclusions. The panel conducted a site-visit to validate fully the self-evaluation and clarify any points at issue. Finally, the review panel produced the present final report on the basis of the self-evaluation report, site-visit and its findings. In doing so it provided an opportunity for FINHEEC to comment on the factual accuracy of the draft report. The review panel confirms that it was given access to all documents and people it wished to consult throughout the review.

9

2 Glossary

ECTS: European Credit Transfer and Accumulation System

EHEA: European Higher Education Area

ENQA: European Association for Quality Assurance in Higher Education

ESG: Standards and Guidelines for Quality Assurance in the European Higher Education Area, ENQA 2009

FINHEEC; the Council: the Finnish Higher Education Evaluation Council

HEI: Higher Education Institution

QA: Quality assurance

UAS: Finnish Universities of Applied Sciences

SER: The self-evaluation report submitted by FINHEEC to ENQA: External Review of Finnish Higher Education Council, Self-evaluation report, March 2010

10

3 Introduction

3.1 Higher Education in Finland

The Finnish HE system consists of two parallel sectors: universities and universities of applied sciences (UAS). Finland is strongly committed to participation in the EHEA and reformed the higher education awards framework and degree structures in 2005, introducing a strategy for the internationalisation of its higher education institutions for 2009–2015.

Universities confer Bachelor’s, Master’s and Doctoral degrees; UAS also confer degree awards. All awards carry ECTS credits. Research degrees are to a large extent conducted at doctoral schools. These are supported by the Academy of Finland, which also carries out peer review assessment of research performance.

Universities and UAS receive most of their funding from the Finnish government. Resources granted consist of core funding and performance-based funding. Research is largely funded through the Academy of Finland. In addition, there are also other public and private funding sources.

HEI activities are governed by performance agreements signed with the Ministry of Education and Culture, following assessment and performance negotiations. Performance of universities and UAS is monitored and steered through the maintenance of statistical databases maintained by the Ministry of Education, covering a wide range of performance indicators.

This is a period of rapid change in higher education in Finland. The most recent legislation on higher education is the Universities Act (588/2009), which sets out the functions, organisation, governance, and operational framework for universities. A number of institutions were merged. The Act extends the autonomy of universities by giving them independent legal status, greater financial autonomy and responsibility for human resources, changing the contractual position of university staff from civil servants to employees of the institution. The Act includes reference to the evaluation of university activities, at Section 47. This also refers to the continuing status and authority of FINHEEC; a subsequent associated governmental decree gives a detailed specification for the work of the Evaluation Council (see Appendix, 6.3).

11

3.2 The quality assurance framework in the Finnish HE system and the work of FINHEEC

3.2.1 The national quality assurance framework

The SER explained that the overall framework for quality assurance in Finnish HEIs is distributed between three main players. Institutions have the key responsibility for the quality of education, research and outreach work; it is their responsibility to establish effective quality assurance systems which are appropriate to the institutional context. The Ministry of Education deploys performance-based quantitative funding, informed by its databases and the quantitative data derived from them. The role of FINHEEC is to provide a qualitative external dimension which will complement and enrich the quantitative assessment carried out by the Ministry; it does this by assisting higher education institutions in the development of quality assurance systems and by identifying excellence and good practice across the HE sector.

3.2.2 The constitution of FINHEEC

FINHEEC was founded in 1996, succeeding the Finnish Higher Education Council (1966–1996). From the first, it was established with the intention to separate evaluation of higher education from the Ministry and economic decision-making. The central decision-making body is the Evaluation Council (the Council), of which the 12 members are selected for a four-year term of office. The appointment is made by the Ministry of Education, following nomination from universities, UAS and Students’ Unions; representatives of the world of work are also appointed. The Council also includes one Swedish-speaking member. The constitution of the Council provides for equal opportunity considerations to be taken into account.

The Council independently elects its Chair and Vice-Chair. The work and decisions of the Council are implemented on a day-to-day basis by an 11-member Secretariat which is led by the Secretary General. The constitution and terms of reference of the Council are laid down in Government Decree.

3.2.3 The work of FINHEEC

The initial work of FINHEEC was concerned with the development of quality assurance and evaluation within HEIs, providing consultancy and training for institutional evaluation. It has consistently maintained the aim of enhancement-led quality assurance. In the late 1990s, FINHEEC carried

12

out evaluations of selected areas of university activity at the request of the institution. It also made recommendations on the accreditation of the newly established universities of applied sciences. Since 2004 FINHEEC has worked in three main fields: audits of quality assurance systems; evaluations for Centres of Excellence and thematic evaluations. In addition, its role has been to promote and communicate good practice in quality assurance matters through benchmarking activity, through national and international networking, and through close co-operation with stakeholder groups such as the Rectors’ Conferences from the two sectors, and Students’ Unions. The main goal of the Council is to support the development of the quality of Finnish higher education and its international competitiveness. FINHEEC’s work follows clearly established aims and principles, which flow from its mission. The most important of these are: independence of action; an enhancement led approach to evaluation which engenders ongoing institutional development; international bridge-building; and proactive influencing within the European context.

FINHEEC conducts thematic evaluations (for instance in relation to doctoral-level study) or discipline-based evaluations in significant, rapidly developing or problematic subject areas. These may be commissioned by the Ministry of Education and result in sector-wide reports.

FINHEEC carries out evaluations of higher education ‘centres of excellence’ which are conducted separately across the university and UAS sectors. FINHEEC identifies excellent provision and communicates relevant information to the Ministry for each performance agreement period. This is then taken into account in funding decisions. These evaluations also contribute to benchmarking and sharing of good practice between institutions. The ENQA review panel was provided with the FINHEEC report Centres of Excellence in Finnish University Education 2010–2012 (March 2009); this provides a summative overview of the excellence in selected institutions for the period 2010–2012. It shows good evidence of a systematic and supportive method of enquiry, and records a range of good practice examples.

FINHEEC carries out periodic audits of the quality assurance systems of institutions. The audit methodology is described in the Audit Manual. In 2010, the first round of audits and re-audits was still in progress, and the second cycle of audits (for 2012 onwards) was in the planning stages. It is the audit activity that will be examined in detail in this report, as it is that work which contributes most directly to the fulfilment of the ESG (see 4.1.1 below). However, it should be borne in mind that FINHEEC’s full range of activities extends considerably further, and embraces a range of other evaluative, support and enhancement functions. These include, in addition to

13

those already outlined, the development of evaluation methodology, training provision, and publication and information services.

3.3 ENQA membership

FINHEEC is a full and active member of ENQA and the current review’s recommendations are intended to inform the renewal of membership. The mission of FINHEEC includes the specific clause that it should ‘participate in international evaluation activities’. The ESG can be seen implicitly to inform and underlie much of the Council’s work. FINHEEC builds and maintains links with evaluation organisations and networks in other countries, and serves consciously as a ‘bridge-builder’ between national HE institution and those within the EHEA (see also paragraphs 4.1.1 and 4.8 below).

3.4 The review process

FINHEEC produced a self-evaluation report (March 2010) (SER) which helpfully provided a background to the work of the Agency, and discussed in detail its compliance with the individual ENQA criteria and the expectation of the ESG Part 3. The evaluation was developed by the Agency’s Secretariat and approved formally by its Council.

The review panel conducted a site-visit on May 3–4, 2010 to validate fully the self-evaluation and clarify any points at issue (see Appendix 6.2 for the site visit programme). In the course of the visit, the panel met with representatives of the Ministry of Education and Culture, with members of the Council, members of the Secretariat, representatives of higher education institutions and Students’ Unions, with former evaluation group members who had taken part in FINHEEC evaluations, and with a number of other stakeholders. The panel much appreciated the readiness of the FINHEEC Secretariat to make the necessary arrangements for the interviews, and the ready response to requests for additional information and documentation.

Finally, the review panel produced the present final report on the basis of the self-evaluation report, the site-visit and its findings. In doing so it provided an opportunity for FINHEEC to comment on the factual accuracy of the draft report. The review panel confirms that it was given access to all documents and all those persons whom it wished to consult, throughout the review process.

14

4 Findings

4.1 ENQA criterion 1

4.1.1 ENQA Criterion 1/ ESG Part 2: External quality assurance processes

The criterion refers to Part 2 of the ESG, which itself includes, as its first standard, reference to Part 1. The panel understands this to mean that the external agency should concern itself in its work with the institutional implementation of the standards and guidelines contained in Part 1 of the ESG.

The expectations of the ESG under Part 2 are therefore now considered.

[ESG 2.1] External quality assurance

The review panel examined the specification of the audit in the Audit Manual for 2008–2011 (2009), and discussed implementation with FINHEEC officers, with institutions and with audit group members. The Audit Manual describes the processes and procedures for audit, and its Appendix gives a list of audit criteria which are intended to guide the audit group in its work. The panel also read a number of audit reports and considered the overview of audit outcomes produced by FINHEEC.

FINHEEC conducts external audits of the operation of quality assurance systems in HEIs. The audit methodology operated by FINHEEC is fully described in the Audit Manual, which develops and revises the model first introduced in 2005. The Audit Manual refers to the ESG as providing the basis for the FINHEEC audit model.

The main target of the FINHEEC audit is the QA system developed by each institution, starting from its own premises and objectives. FINHEEC states that the audit assesses the comprehensiveness, performance, transparency and effectiveness of the QA system, and the way in which its operation is monitored and developed by the institution. Thus both the audit group enquiry and the subsequent audit report focus on the individual institutional quality assurance system and the extent to which it is effective in operation. The audit approach is therefore tailored towards individual institutional systems, and may consequently vary from one institution to another with regard to the detailed focus of enquiry.

15

The scope of the audit has a broad focus on the quality system and quality management, giving a number of specified ‘targets’ which in addition to the assurance of academic provision (included under ‘degree education’) includes, for instance, strategic management, quality assurance of research, and impact on society and regional development.

The audit evaluation is supported by explicit criteria, which describe the characteristics of each main target field under the headings for four development stages: ‘absent’, ‘emerging’, ‘developing’, and ‘advanced’. Audits are conducted by an audit group, appointed by FINHEEC and composed of five members, three peers from higher education institutions, a student representative and a ‘working life’ representative.

Auditors are trained in a general one day session, and also fully briefed for each individual audit in preparatory meetings.

The audit process is governed by a registration and institutional agreement procedure, and follows the broadly conventional pattern of submission with self-evaluation, a preparatory meeting, the site visit, followed by draft and final versions of the report, publication and feedback. The institution is expected to submit documentation relating to the quality procedures in action, which will include the institutional quality manual, a description of the QA system, a SWOT analysis of the system, and samples to illustrate the operation of the system in practice.

The recommendation of the panel, arising from the audit, is either ‘pass’ or ‘re-audit’; a pass may be proposed if all audit targets meet the ‘emerging’ standard in the criteria and the QA system as a whole is judged by the panel to be at the ‘developing’ standard. Should a re-audit be proposed and confirmed by the Council, then the institution enters consultation with FINHEEC and the timetable and agreed essential development needs development plan for the re-audit are agreed.

The review panel considered the extent to which the audit criteria and the audit process were aligned with the ESG standards. It found that, in line with the character of the FINHEEC audit model, the criteria were at a general level, and focused mainly on broad quality management aspects and the operation of the institutional quality assurance system as a whole. The panel recognised the value of the flexible and enhancement-led approach, but found that it did not include explicit advice relating to some of the individual ESG standards. Thus for instance, while the internal quality assurance of support services and resources, and staffing and staff development, was clearly included in the criteria, there was no direct mention of the assurance of learning and teaching, programme approval procedures, monitoring and review of programmes, or of the management of assessment.

16

The review panel was partly reassured in this respect by its discussions with FINHEEC and with representatives of institutions, which suggested to the panel that there was an implicit understanding that all aspects identified in the ESG standards were essential to an effective quality assurance system, and would therefore naturally be expected to form part of an institutional audit. Nevertheless, reading of audit reports suggested to the panel that the extent to which the ESG Part 1 standards for institutional quality assurance had been addressed during the process was indeed variable. It was therefore difficult to find consistent evidence that all the relevant ESG standards had been integrated into the external quality assurance processes applied by FINHEEC.

The panel concluded from discussions with audit group members and institutional representatives that it was likely that institutions did in fact routinely address the expected quality assurance procedures laid down in ESG Part 1. The panel also accepted from its reading of reports that these procedures were included to an extent in the overview of institutional processes that was taken during a FINHEEC audit, at least at a superficial level. However, the panel also considered that neither of these implicit expectations was made sufficiently explicit in the Council’s documentation. In the view of the panel, the Audit Manual needed to express more transparently and clearly the expectation that the criteria laid down in the standards of the ESG part 1 would be included in institutional systems, and audited by FINHEEC.

The panel therefore recommends that as it carries out its review of the audit method in preparation for the forthcoming ‘second round’ of audits (from 2012), FINHEEC make explicit reference within its audit criteria to the ESG on the expected components of internal quality assurance, and their evaluation, as laid down Part 1 and Part 2 of the ESG.

The panel considered that the requirements of the standard ESG 2.1 were partially met.

[ESG 2.2] Development of processes

The panel was provided with evidence of careful and thorough development of the processes adopted by FINHEEC. This had included: policy decisions relating to the distribution of formal responsibilities between institutions, the Ministry, and the Agency; extensive consultation with stakeholders including a major seminar; piloting and revision of the Audit Manual; a self-evaluation reviewing the operation of audits and audit outcomes, and ongoing review discussions in the FINHEEC Council.

17

The panel was satisfied that the requirements of the standard were fully met.

[ESG 2.3] Criteria for decisions

All decisions are based on clear criteria published in the Audit Manual. These provide guidance on the assignment of grades for different aspects of the audit and on the overall weighting of grades leading to a decision. Recommendations from the audit group are evidence-based and checked by officers of the FINHEEC Secretariat. The final decision lies with the FINHEEC Council. The Council receives the written report of the audit group, with an oral report from the chair of the group, and an opportunity is provided at the Council meeting for members to ask questions and clarify particular issues arising from the report. The Council provides for consistency of judgement. Consistency is also supported by audit training for all audit groups.


[2.4] Processes fit for purpose

The panel reviewed the processes for managing and implementing the audits carried out by FINHEEC. It was able to confirm that:– Audit groups are appointed formally by the FINHEEC Council– Experts who are members of the audit group must meet relevant stated

criteria, which include appropriate knowledge, experience and skills– The audit group includes a student members and a member

representing employer stakeholders– Training of the audit group is arranged– The model for the audit (as described above under ESG 2.1) follows

broadly the pattern described in the ESG i.e. presentation of evidence / site visit / draft report / published report / follow upThe panel noted that, rather than a full self-evaluation, institutions

were asked to present for the purposes of the audit authentic material from the institutional system, providing examples and evidence together with a brief SWOT commentary, so that the audit group can gain a picture of the organisation and the operation of its quality assurance system. The panel considered that this was an acceptable equivalent to a formal self-evaluation document.

The panel observed that there was a level of ambivalence within FINHEEC’s procedures regarding the use of international experts. These

18

procedures provide that HEIs may have a choice of a national or international audit group. Normally only an international group would include international experts. The majority of audits conducted to date have been through a nationally-based audit group (17 of the 19 conducted up to 2008) and therefore some of these may not strictly have met the relevant ESG guideline relating to international membership. In some audits it had been possible to use Swedish audit group members. In discussion it became clear that the alternative would be to conduct the audit in English, but because the audit materials reviewed by the panel would normally be in the Finnish language, the translation demands would be considerable to provide materials for an international group. In general, institutions also naturally preferred to conduct the audit in Finnish, particularly during this first cycle of audits. The panel acknowledged that it would be challenging to recruit Finnish speaking international experts. The panel was told that FINHEEC expected to have more international audit groups in the next round of audits, when there was greater familiarity with the process.

Notwithstanding these points, the panel considered that it would be possible and highly desirable to strengthen the international dimension within the FINHEEC processes. International members could be included within the Council. The relevant Government decree (see Appendix 6.3, below) provides for the establishment of an international advisory body or consultative committee, but this had not yet been established. A small number of Finnish-speaking international experts might be identified who could form a pool to draw upon for membership of audit groups. Furthermore, it should be noted that audit groups could consist of a mixture of national and international experts and thus the national experts could check the material that is not available in English. During the visit the panel formed the view that conducting the site visits in English should not normally pose a problem, considering the high level of knowledge of English. In taking these views, the review panel was mindful of the value of making national quality assurance systems accessible and open to discussion within an international forum, and the possible disadvantages that could arise from too great a level of national isolation.

The panel therefore recommends that FINHEEC give continuing attention to the question of international expert participation in its processes, including consideration of international membership of the Council and the establishment of the proposed international advisory committee.

Overall, the review panel found that FINHEEC substantially met the requirements of the standard.

19

[2.5] Reporting

The panel read a number of FINHEEC audit reports on institutions, both in English and in Finnish.

It was confirmed that the reports follow a clear structure, in accordance with the ESG guidance. The audit reports include a full description of the audit process, an account of the institutional quality assurance system, audit findings listed under aims, and conclusions with a recommendation relating to the judgement of the audit. FINHEEC publishes all its reports in full on its website. The majority of reports are printed and distributed to relevant stakeholders. A summary abstract is also published in Finnish, Swedish and English for each report.


[2.6] Follow-up procedures

As described above, FINHEEC audits result in a judgement that the quality assurance system should pass, with the consequent award of an audit certificate, or that it should be re-audited. A re-audit takes place two years after the audit, and focuses on identified recommendations for improvement. For institutions which pass, a collective follow-up seminar with open participation is held.

The panel reviewed reports on re-audit, together with accounts of the follow-up seminars.

For institutions that pass the audit, there are clear follow-up procedures. Three years after the audit, the institution is asked to prepare a short self-evaluation follow-up report, outlining actions taken and developments since the audit. This report is then submitted for discussion at a collective seminar, which all institutions audited at that point in the cycle attend and exchange reports, identifying matters of common interest in the development of their quality assurance systems, and to share good practice. These follow-up seminars, organised by FINHEEC, take place each year. The panel agreed that the follow-up seminars were an excellent procedure, reflecting the level of openness and trust within the Finnish system.


20

[2.7] Periodic reviews

The FINHEEC procedures require an audit of the institutional quality assurance system to take place every six years. The development of the second round of audits is currently underway.


[2.8] System-wide analysis

FINHEEC regularly publishes summary reports relating to its various activities. In 2004 the Council published an extensive analysis of the impact of programme-related evaluations carried out between 1997–2003. From 2010 FINHEEC will publish an annual report on its range of activities, which will include a summary of key findings from evaluations. The panel read a recent publication Analysis of audit outcomes 2005–2008. This was based on 19 audit reports from the first cycle and it provided a useful descriptive overview of audit findings, with detailed content analysis providing lists of good practice in the various audit aspects. While strongly welcoming this initiative and recognising the quality of the research contributing to the publication, the panel felt that it could have been further strengthened by an assessment of the effectiveness of the audit procedures themselves, and recommendations for enhancement activity based on the analysis. However, the panel noted that FINHEEC has also financed two additional external research projects which aimed to assess the impact of audits in HEIs, and that these would provide additional evidence on which to base an overall analysis when they were completed in December 2010.


Summary

The panel reviewed the overall picture regarding the satisfaction by FINHEEC of the expectations of ENQA Criterion 1. The panel found that, in the majority of aspects, the expectations of ESG Parts 1 and 2 were addressed fully by the FINHEEC activities and audit procedures. The only significant reservation here relates to the absence in the procedures of any explicit reference to the ESG Part 1 guidance for internal quality assurance in institutions (see comments under ESG 2.1 above). Finally, while recognising the difficulties occasioned by language issues, the panel recommends that FINHEEC finds ways of strengthening the level of international participation in its processes.

21

Overall, the panel considered that FINHEEC is substantially compliant with the criterion, and thus with the relevant standards under ESG Parts 1 and 2.

4.1.2 ENQA Criterion 1 / ESG 3.1, 3.3: Activities

The panel worked from the full description of the FINHEEC activities provided in the self-evaluation and the Quality Manual; it discussed the range of activities with Secretariat officers, Council members, and institutional representatives, both staff and students.

As already described, FINHEEC carries out institutional audits of all Finnish HEIs on a six-year cycle. It also conducts Centres of Excellence evaluations on a regular basis. In addition, FINHEEC undertakes thematic evaluations and evaluations of specific study fields. It also carries out system-wide studies of specific quality-related aspects, for instance the evaluation of doctoral level postgraduate study (2006) and the ongoing evaluation of the national implementation of the Bologna process.

FINHEEC organises seminars and workshops for supporting the quality assurance activities in institutions by benchmarking and providing for the exchange of good practice. It sees its work as being an international ‘bridge builder’, scoping the latest developments in its field and importing information on recent trends to its various stakeholders. While the work of FINHEEC is deliberately and explicitly devoted to enhancement, development and support of institutions, it clearly also carries out an external quality assurance role and supports the development of internal quality assurance systems in Finnish HEIs.

The panel was confident that FINHEEC’s range of activities is relevant and appropriate to the work of a national quality assurance agency, combining effectively assurance and enhancement activities.

The panel therefore found that FINHEEC is fully compliant with the standard.

4.2 ENQA criterion 2 / ESG 3.2: Official status

The panel read formal government decrees and Acts relating to the work of FINHEEC and discussed its status with Secretariat officers and with representatives of the Ministry of Education.

FINHEEC is formally recognised as the primary organisation responsible for the evaluation of higher education in Finland. Its position and range of functions are stipulated in governmental decrees (1320/1995, 465/1998,

22

548/2005, 965/2007 and 794/2009). The last of these has nine sections describing in detail the mission, composition, organisation and activities of the Council. Included here is the obligation to submit an annual report on its evaluations, and to ‘participate in international evaluation of its own activities on a regular basis’.

The role of FINHEEC with regard to the accountability for internal quality assurance of universities and universities of applied sciences is further laid down in the recent Universities Act 2009 and the Polytechnics Act (amended 2009). These Acts have identical articles stating that ‘universities [and Universities of Applied Sciences] shall ...take part in external evaluation of their activities and quality assurance systems on a regular basis’ and referring to FINHEEC as the responsible ‘independent expert body’ which will carry out evaluations.

The panel was therefore able to confirm that FINHEEC is fully compliant with the expectations of the standard.

4.3 ENQA criterion 3 / ESG 3.4: Resources

The panel reviewed the range of work undertaken by the Council and discussed the adequacy of resources and level of activity with a range of stakeholder representatives.

The activities of FINHEEC are funded entirely by the Ministry of Education out of the state budget allocation. The budget provides for premises, staffing, and the funding of all audits and evaluations (see also 4.5 below). The majority of direct expenditure is on labour costs. The Secretariat of the Council comprises the secretary general, three chief planning officers, six senior advisers and an administrative assistant.

The panel met with these staff, and undertook a tour of the Council’s accommodation and facilities. The FINHEEC premises occupy one floor of the Ministry of Education; they provide modern, well-appointed office accommodation, and include separate kitchen facilities and a common resources area.

The panel took note of the range of experience represented in the Council and in members of the Secretariat, which extended over thirteen years of higher education evaluation and had included systematic in-house training and participation in international networks.

The panel concluded that FINHEEC is fully compliant with the expectations of the standard.

23

4.4 ENQA criterion 4 / ESG 3.5: Mission Statement

The panel referred to the Self-Evaluation Report, to the Universities Act 2009, and to the FINHEEC Quality Manual. It discussed the mission and its realisation with Secretariat officers and Council members.

The mission statement of FINHEEC is laid down in the Government Decree 2009. This states:

‘The mission of the Finnish Higher Education Evaluation Council shall be to1 assist higher education institutions and the Ministry of Education in matters

pertaining to evaluation2 conduct evaluations relating to the activities and quality assurance systems of

higher education institutions3 support quality assurance and enhancement in higher education institutions4 participate in international evaluation activities and cooperation concerning

evaluation’In its self-evaluation statement, FINHEEC made reference to its

published Quality Manual, which outlines the key operational goals and activities which are derived from the mission statement. These are summarised under three main headings: 1) effective evaluation processes and relevant reports, consolidating institutions’ culture of continuous development; 2) stakeholder cooperation and collaboration through networks, seminars and workshops, giving a lead in the provision of information and development to the HE sector relating to quality assurance and evaluation work; and 3) well-established internal principles and values in the work of the Council. The Quality Manual provides a clear and helpful outline of the structure, organisation and main activities of the Council; its values, goals and processes; and its structure of accountability, including resolution and appeals procedures.

For each period of office of the Council, a detailed Action Plan is drawn up, which defines the focus areas and policies relating to activities during that period; this is then revised annually and supplemented by an annual work plan drawn up by the Secretariat. The schedule of evaluations is included within the action plan. The action plan is published in the FINHEEC publication series, and has an associated annual budget drawn up in consultation with the Ministry of Education.

The panel therefore found that there was a clear and official statement of mission, which provided for external evaluation to be a major activity of the agency. The Quality Manual provides detail of the ways in which the general aims of the mission statement are to be put into effect in terms of

24

operation and management, and effective and detailed management planning is in place.

The panel considered that FINHEEC is fully compliant with the standard.

4.5 ENQA criterion 5 / ESG 3.6: Independence

The panel discussed the question of the level of independence with Council members and with the Ministry of Education. It referred to the SER, the Universities Act, the Audit Manual and Quality Manual.

In its self-evaluation, FINHEEC described its position as ‘an independent expert body, operating in conjunction with the Ministry of Education’. The Council membership appointed by the Ministry, after consultation with institutions. FINHEEC is located within the offices of the Ministry, and its staff are formally employees of the Ministry; it uses the support infrastructure of the Ministry, such as IT services, human resource management, payroll and financial management services. It is clear, then, that there is a very close association, in organisational support terms, between the Council and the Ministry. The panel therefore devoted careful attention to the question of the independence of the Council’s work.

With regard to the question of official independent status, the panel’s attention was drawn to a statement in the preamble to the Universities Act as follows:

‘...Independence means that the evaluation findings, conclusions and recommendations are independent from the influence of third parties, such as the higher education institutions, ministries, or other parties concerned. In operating the within the Evaluation Council, the members of the Finnish Higher Education Evaluation Council are independent experts...’ The panel was therefore able to confirm that independence of operation was enshrined in legislation.

The panel found evidence in the documentation provided, and through discussion, that this independence in the Council’s operational activities was then reflected in fully independent decision-making with regard to the design and implementation of evaluations, methods used, appointment of evaluation and audit groups, scheduling, the contents of reports and the judgements and recommendations of the Council. The reports of FINHEEC are public documents, but not directed to or accountable to the Ministry. The Council may initiate evaluations at the suggestion of institutions or at the request of the Ministry, but the conduct of evaluations is managed quite separately and independently. The panel found no evidence of any direct involvement of the

25

Ministry in the day-to-day operations, decision- making or judgements of the Council.

The panel was able to discuss the question of independence with representatives of the Ministry of Education. The understanding of the need for separation of powers was very clearly expressed. The Ministry understood its role as providing material support, but otherwise remaining at a distance from the work of FINHEEC as a matter of principle. Consultation and collaboration between the two organisations was deliberately limited, covering a reporting meeting each year, one of which was concerned with discussion of the annual report.

On the basis of this evidence, the panel formed the view that the independence of FINHEEC was guaranteed both in principle through the legal framework, and in practice in its operational arrangements. The relationship with the Ministry, while close and providing the necessary material and organisational support, had no influence on FINHEEC’s principal activities.


4.6 ENQA criterion 6 / ESG 3.7: External quality assurance criteria and processes

The processes and procedures developed by FINHEEC in its external quality assurance and enhancement work, in particular its audits, have already been described in detail above, (paragraph 3.2.3).

With regard to this standard, on the basis of the evidence in the publications reviewed (Audit Manual, Quality Manual, Audit reports, Follow-up report) and discussions with institutions and with audit group members, the panel found as follows:– institutions present for scrutiny a representative selection of materials

exemplifying the internal quality assurance system in operation, together with a commentary; this can be seen as equivalent to a self-evaluation report

– in the audits conducted by FINHEEC, an external assessment is carried out by a group of expert peers appointed by the Council; the group includes a student representative member and students are trained so as to be able to contribute effectively to the process

– final decision on the outcome of an audit is made by the Council following recommendation by the audit group

26

– the full report of the audit, including all recommendations, is published on the FINHEEC website, and the majority of reports are also published in hard copy

– FINHEEC has a follow-up procedure in place: institutions produce a brief follow-up report after three years; a seminar takes place, organised by FINHEEC, at which institutions share and discuss action plans and their own follow-up report, exchanging views on the process and sharing good practiceThe panel found that the position regarding appeals against Council

decisions on institutional audit outcomes was more complex. In its audits FINHEEC is essentially making enhancement-related decisions and judgements, which have no direct formal political or economic implications for institutions. In its self-evaluation, FINHEEC explained that a formal direct appeal against a decision of the Council (which would be for ‘pass’ or ‘re-audit’ with regard to the effectiveness of the institution’s quality assurance system) was not possible. This is because the Council’s decisions were not, in legal terms, ‘administrative decisions’, and therefore the appeals procedures provided for under Finnish legislation do not apply. The only possible route for formal objection would be under individual citizen rights to complain to the Parliamentary Ombudsman. FINHEEC took the view that its use of feedback on processes from audited institutions provided a level of opportunity for commentary and criticism. The panel recognised the legal constraints, but it considered that it would nevertheless be possible, and beneficial, to introduce an opportunity for institutions to make a representation to the Council in any cases where they considered that the experience and outcomes of the audit had not adhered to the agreed procedures, specifications and schedule. The report could then be referred back to the team. However, the panel agreed that the status of decisions made by the Council had no formal consequences, and that the issue of appeals did not therefore affect compliance with the standard.


4.7 ENQA criterion 7 / ESG 3.8: Accountability procedures

The panel was guided by the account of internal quality assurance provided in the SER and the Quality Manual. The review panel then read reports, papers of the Evaluation Council, saw feedback questionnaires from institutions and analysis of the feedback, and discussed the internal quality assurance processes

27

with officers of the Secretariat, with members of the Council, and with representatives of institutions.

The self-evaluation described a range of processes and activities which contributed to the internal assurance and accountability of FINHEEC. The most important of these are outlined in the Council’s Quality Manual. This document is available on the FINHEEC website, in the Finnish version, and an English translation in hard copy was made available to the panel.

The Manual explains that accountability is understood by the Evaluation Council in large part to involve transparency of goals and methods, together with internal quality assurance procedures which will secure the effective operation of the Council and the achievement of the stated goals. The Evaluation Council itself has a quality assurance role, as it is responsible for the overall standards and consistency of audit procedures and reports. Members of the Council receive draft copies of reports in advance of meetings. On the basis of reading the audit report, the Council may, in the interest of consistency or impartiality, differ in its decision from the recommendations of the audit group.

The quality assurance framework implemented by FINHEEC is based on the Deming ‘quality circle’ of Plan–Do–Check–Act. This is a dynamic planning and development process, which includes a range of quality management activities. For the purposes of the ESG standard, the following are significant:– The Quality Manual explicitly links mission and goals to processes,

analysis of performance and results.– The Council has in place procedures, as laid down by the Finnish

Administrative Procedure Act, which ensure that there is a no-conflict-of-interest mechanism. In particular this provides that no-one involved at any stage in an audit should have prior involvement with the institution being audited.

– Internal accountability and feedback takes the form of a periodic evaluation. The Secretariat uses the Common Assessment Framework model for internal reflection and produces a self-evaluation. This then contributes to a general self-evaluation by the retiring Council, which is drawn up in a report setting out an evaluation of the success of the evaluation method, good practice and development targets.

– The internal evaluation is in part informed by an external feedback mechanism which takes place for each audit. The Secretariat sends feedback questionnaires to the HEI participating in an audit, and also to the audit group which carried out the audit. Feedback data are collected relating to the experience of the evaluation method, and to the impact and usefulness of the audit process and report.

28

– Each year the Secretariat holds a development seminar for internal reflection and review, part of which is discussion of institutional and audit group feedback. The seminar produces recommendations for development and improvement. These recommendations are then reported to the Council in its respective development seminar, which contributes directly to development work and is normally held after that of the Secretariat.

– A summary of development activities agreed by the Council is then produced and this is circulated to institutions and to auditors.

– FINHEEC is committed by virtue of its founding legislation to undergo a mandatory cyclical external review at least once every five years in order to confirm its continuing membership of ENQA.In addition to these major components of its systematic quality assurance

framework for internal evaluation and improvement, the Quality Manual lists a range of contributory procedures which serve to support its quality of operation. These include: annual appraisal review of Secretariat staff; weekly and monthly meetings of the Secretariat for planning and communication; induction and orientation of Council members; training and preparation of audit groups

The panel was therefore able to confirm that all those aspects of accountability listed under the standard were addressed in the work of FINHEEC. The panel concluded that FINHEEC has in place thorough and effective arrangements for the quality assurance of its work, for its onward development, and for external accountability.

Accordingly, the panel considered that FINHEEC is fully compliant with the standard.

4.8 ENQA criterion 8 / Miscellaneous

Professionalism and consistency in judgements

The quality of the FINHEEC documents, seen in institution reports, Manuals and guidance documentation, and internal working documentation, attests to the professionalism of its work. The Secretariat consists of professional administrators with experience of higher education, who provide for a high standard of planning and organisation.

Institutional representatives and other stakeholders met by the review panel had a high regard for the professionalism of FINHEEC in its evaluation and support activities.

29

Decisions and judgements are made in the Evaluation Council on recommendation from audit and evaluation groups. The Council provides the experience, expertise and continuity that is necessary to safeguard consistency in decision-making. The panel learnt of two example cases where the recommended decision had been overturned by the Council in order to ensure proper procedure and consistency of judgement; in one case a senior representative of the institution met by the panel had in retrospect welcomed this decision by the Council as appropriate and constructive.

Appeals

In its audits, FINHEEC is essentially making enhancement-related decisions and judgements, which have no direct formal political or economic implications for institutions. The legal framework within which FINHEEC operates does in any event make formal appeal difficult (see also 4.6 above). The panel considered that it would nevertheless be possible, and beneficial, to introduce an opportunity for institutions to make a representation to the Council in any cases where they considered that the experience and outcomes of the audit had not adhered to the agreed procedures, specifications and schedule.

The agency is willing to contribute actively to the aims of ENQA

FINHEEC is committed by virtue of its formal mission statement to ‘participate in international evaluation activities and cooperation concerning evaluation’. Its stated principles and values are strongly aligned with those of the Bologna process and thus with those of ENQA. There was clear evidence available that FINHEEC is active within European networks, and itself offers conferences and workshops at the European level. For example, FINHEEC has recently successfully hosted an ENQA ‘Audit Group’ development and networking event, and will host the ENQA General Assembly in September 2010. Its location in Helsinki provides for direct contact and liaison with ENQA officers.

The panel was confident that FINHEEC meets the expectations of this ENQA criterion.

30

5 Conclusion and development

Overall findings

In the light of the documentary and oral evidence considered by it, the Review Panel is satisfied that, in the performance of its functions, the Finnish Higher Education Council (FINHEEC) is in compliance with the ENQA Membership Provisions. The Panel therefore recommends to the Board of ENQA that FINHEEC should have its Full Membership of ENQA confirmed for a further period of five years.

The review panel found many commendable features in the work of FINHEEC. These included: the professional and committed work of the Secretariat; the high quality of audit reports; the open dialogue in feedback to and from institutions; and the readiness of the Evaluation Council to carry out self-critical analysis in the course of its work. The panel was clear that FINHEEC is contributing to the development of Finnish HE and has the full support of the sector, based on an excellent working relationship with its stakeholders).

5.1 Recommendations

The review panel considered that, as FINHEEC continues to develop its audit procedures, its work would benefit from consideration of the following aspects, discussed in this report:

5.1.1 to make explicit reference to the standards and guidelines of ESG Part 1 within the FINHEEC Audit Manual, the audit process and the audit reports (paragraph 4.1.1 [ESG 2.1])

5.1.2 to give continuing attention to the question of international expert participation in its processes, including consideration of international membership of the Council and the establishment of the proposed international advisory committee. (paragraph 4.1.1 [ESG 2.4])

5.1.3 to allow for a form of representation to the Council, subsequent to the audit report, with reference back to the team, in cases where an institution disputed the Council’s decision on procedural grounds (paragraph 4.6)

31

6 Annexes

6.1 Schedule for the review

Appointment of review team members by ENQA November 2009

FINHEEC submission of self-evaluation document and accompanying documentation February 2010

Briefing of the review panel by ENQA (teleconference) 8 March 2010

Preparation of site visit schedule, planning of meetings and agreement on indicative timetable for site visit to FINHEEC March–April 2010

Expert panel site visit to FINHEEC Helsinki May 2–4 2010

Panel correspondence and agreement on the draft report July 2010

Commentary on the draft report received from FINHEEC August 2010

Submission of the final report to ENQA August 2010

Consideration of the final report by ENQA September 2010

6.2 Site Visit Programme

External review of Finnish Higher Education Evaluation Council site-visit programme

Venue: Meeting room Aapo & Tuomas, (2nd floor) Ministry of Education, Meritullinkatu 1, Helsinki

MONDAY 3rd May 2010

9:00–9.15 Introductions

Chair Riitta Pyykkö, FINHEEC, University of Turku

Secretary-General Helka Kekäläinen, FINHEEC

Senior advisor Matti Kajaste, FINHEEC

9:15–10:15 Interview: FINHEEC

Chair Riitta Pyykkö, FINHEEC, University of Turku



10:15–10:30 Break

32

10:30–11:30 Interview FINHEEC Secretariat

Chief Planning Officer Karl Holm, FINHEEC

Chief Planning Officer Marja-Liisa Saarilammi, FINHEEC

Chief Planning Officer Sirpa Moitus, FINHEEC

Senior advisor Hannele Seppälä, FINHEEC

Senior advisor Kirsi Hiltunen, FINHEEC

Senior advisor Kirsi Mustonen, FINHEEC

11:30–12:30 Lunch break

12:30–13:30 Interview: FINHEEC Council members

Vice-chair, rector Pentti Rauhala, Laurea University of Applied Sciences

Managing director Mikko Luoma, JTO School of Management

Vice-rector Riitta Rissanen, Savonia University of Applied Sciences

Principal lecturer Jan-Erik Krusberg, Arcada University of Applied Sciences

Educational Officer Juhana Harju, The National Union of University Students

13:30–14:30 Break – Private meeting of the panel

14:30–15:30 Interview: representatives of HEIs

Secretary-General Liisa Savunen, Universities Finland

Vice-Chair, Rector Henrik Wolff, Arene, Arcada University of Applied Sciences

Quality manager Aimo Virtanen, University of Helsinki

Director of Development Marjo-Riitta Järvinen, Lahti University of Applied

Sciences

Rector Marja-Liisa Tenhunen, Central Ostrobothnia University of

Applied Sciences

Rector Eeva-Liisa Antikainen, Humak University of Applied Sciences

Vice-President Martti Raevaara, Aalto University

Rector Gustav Djupsjöbacka, Sibelius Academy

15:30–15:45 Break

15:45–16:45 Interview: Former evaluation group members

Research Director Timo Aarrevaara, University of Helsinki

Researcher Anu Yanar, Aalto University

Director Katariina Raij, Welfare and Pedagogical Competence, Laurea UAS

Professor Ilkka Virtanen, University of Vaasa

Professor Turo Virtanen, University of Helsinki

Student Milja Seppälä, University of Oulu

Juuso Leivonen, The Finnish Association of Business School Graduates

16.45–17.00 Break

17.00–18.00 Interview: Ministry of Education

Director General Sakari Karjalainen, Ministry of Education

Director Anita Lehikoinen, Ministry of Education

Counsellor of Education Birgitta Vuorinen, Ministry of Education

Counsellor of Education Maija Innola, Ministry of Education

33

TUESDAY 4th May 2010

9:00–10:30 Private meeting of the Panel

10:30–11:30 Interview: Students’ unions

Chair Simo Takanen Union of Students in Finnish Universities of Applied

Sciences (SAMOK)

Secretary General Jani Hyppänen, SAMOK

Secretary General Tuomas Viskari, National Union of University Students in

Finland (SYL)

Executive Board Member Jukka Vornanen, (SYL) Adviser Mikko Heinikoski,

SAMOK

Student Päivi Keränen, Metropolia University of Applied Sciences

Student Johanna Ahola, University of Helsinki

11:30–12:45 Lunch break

12:45–13:45 Interview: Other stakeholders

Senior advisor Marita Aho, Confederation of Finnish Industries (EK)

Chief Planning Officer Anu Räisänen, Education Evaluation Council

Head of Unit Anne Siltala, Centre for International Mobility (CIMO) Project

Planning Officer Arvo Alajoki, Academy of Finland

Professor Jussi Välimaa, Finnish Institute for Educational Research, University of

Jyväskylä

Research Director Jussi Kivistö, Higher Education Group, University of Tampere

13:45–15:00 Private meeting of the panel

15:00–15:15 Break

15:15–16:15 Final feedback session

Vice-chair, rector Pentti Rauhala, Laurea University of Applied Sciences


CPO Karl Holm, FINHEEC

CPO Marja-Liisa Saarilammi, FINHEEC

CPO Sirpa Moitus, FINHEEC


Senior advisor Hannele Seppälä, FINHEEC

Senior advisor Kirsi Hiltunen, FINHEEC

Senior advisor Kirsi Mustonen, FINHEEC

Senior advisor Johanna Mattila, FINHEEC

Administrative assistant Arja Bilund, FINHEEC

34

6.3 Government Decree 2009

Unofficial translation

Government Decree on the Finnish Higher Education Evaluation Council

Section 1. Mission of the Finnish Higher Education Evaluation Council

1. The mission of the Finnish Higher Education Evaluation Council shall be to assist the higher education institutions and the Ministry of Education in matters pertaining to evaluation;

conduct evaluations relating to the activities and quality assurance systems of higher education institutions; support quality assurance and enhancement in higher education institutions; and participate in international evaluation activities and cooperation concerning evaluation.

2. Further, the Higher Education Evaluation Council shall perform duties assigned to it in the Presidential Decree concerning university of applied sciences degrees awarded in the Province of Åland (548/2005).

3. The Higher Education Evaluation Council may accept other commissions relating to evaluation from Finnish and foreign operators.

4. The Higher Education Evaluation Council must participate in international evaluation of its own activities on a regular basis. The Higher Education Evaluation Council must annually submit an account of its own activities and the salient findings of the evaluations.

Section 2. Composition of the Higher Education Evaluation Council

1. The Higher Education Evaluation Council shall have a maximum of 12 members, who must be versed in higher education evaluation. They must represent expertise in the activities and operational environment of higher education institutions and the world of work and the majority of them must be affiliated with higher education institutions. The Ministry of Education shall appoint the members for a maximum of four years at a time after hearing the higher education institutions and stakeholders.

2. The Higher Education Evaluation Council shall elect a chairperson and a vice-chairperson from amongst its members for its term of office.

3. Where a member of the Higher Education Evaluation Council becomes unable to perform his or her duties in the middle of the term of office,

35

the Ministry of Education shall appoint a new member in his or her place for the remainder of the term.

Section 3. Subcommittees

1. The Higher Education Evaluation Council may set up subcommittees to prepare matters which come before the Council. The chairperson, vice- chairperson and members of the subcommittee shall be appointed by the Higher Education Evaluation Council.

2. A member assigned to a subcommittee may also be external to the Higher Education Evaluation Council.

Section 4. International advisory body

1. The Higher Education Evaluation Council may appoint a consultative committee composed of international and national experts and chaired by the chairperson of the Evaluation Council to assist it in its operation and development and to enhance international cooperation.

Section 5. Meetings

1. The Higher Education Evaluation Council shall convene at the invitation of the chairperson or, when he or she is prevented, by the vice-chairperson.

2. The Higher Education Evaluation Council shall form a quorum when at least half of the members, including the chairperson, are present. Matters shall be decided by a simple majority. If the votes are equal, the chairperson shall have the casting vote.

Section 6. Secretariat

1. For the preparation of matters which come before the Evaluation Council, there shall be a secretariat whose officials are appointed by the Ministry of Education after hearing the Higher Education Evaluation Council.

2. The work of the secretariat shall be led by a secretary-general, the qualification requirement for whom shall be a doctoral degree, familiarity with the field concerned and proven leadership skills.

3. The qualification requirement for the officials in the expert posts of the secretariat shall be a Master’s degree and familiarity with the field concerned.

36

Section 7. Fees and remunerations

1. The grounds for paying fees and remunerations to the chairperson, members and experts of the Evaluation Council and its subcommittees and the advisory body shall be determined by the Ministry of Education.

Section 8. Standing orders of the Finnish Higher Education Evaluation Council

1. The tasks and competence of the Higher Education Evaluation Council and its subcommittees, advisory body and secretariat and the processing of matters shall be laid down in the standing orders issued by the Evaluation Council.

Section 9. Coming into force

1. This Decree shall come into force on the first of January 2010.

2. Measures needed to implement the Decree may be taken before the coming into force of the Decree.

6.4 Standing order of the Finnish Higher Education Evaluation Council

Adopted on 17.12.2009In force: 1.1.2010 – until further notice

1. General

The operation of the Finnish Higher Education Evaluation Council is governed by Section 87.2 of the Universities Act (558/2009), Section 9.2. of the Polytechnics Act (351/2003; Amendment 564/2009) and the Government Decree on the Finnish Higher Education Evaluation Council (794/2009).

Under the Universities Act and the Polytechnics Act, ”Attached to the Ministry of Education is an independent expert body called the Higher Education Evaluation Council, further provisions on which shall be enacted by Government Decree.”

In the preamble to the Universities Act, the independence of FINHEEC is described in the following way:

According to the provision, the Finnish Higher Education Evaluation Council is an independent expert body, which means independent responsibility for its operation. Further, ’independence’ means that the evaluation findings, conclusions and recommendations are independent from the influence of third parties, such as the higher education institutions, ministries or other parties concerned. In operating in the Evaluation Council, the members of the Finnish

37

Higher Education Evaluation Council are independent experts. They do not represent their own background organisation but seek to promote quality enhancement in all the higher education institutions under review and the development of the Finnish higher education system as a whole.

The Higher Education Evaluation Council (subsequently FINHEEC) is attached to the Ministry of Education and uses the Ministry’s infrastructure and follows the Ministry’s regulations in regard of the financial and information management, facilities, official journeys, annual and other leave, and occupational health care.

Further, the operation of FINHEEC is governed by the national evaluation plan and the FINHEEC Quality Manual.

2. FINHEEC organisation and the tasks and competence of its parts

In this standing order, the term ’Higher Education Evaluation Council’ denotes the entity composed of the Evaluation Council and the Secretariat. The FINHEEC organisation further includes possible subcommittees and an international advisory board. For the planning and implementation, FINHEEC appoints planning and evaluation groups for a fixed term.

Duties of the Evaluation Council

Under the Government Decree, the mission of FINHEEC is to:1) assist higher education institutions and the Ministry of Education in matters

relating to evaluation;2) organise evaluations relating to the operation and quality assurance systems of

higher education institutions;3) support quality assurance and enhancement in higher education institutions;

and4) participate in international evaluation activities and cooperation.

In addition, FINHEEC performs the duties assigned to it in the Presidential Decree (548/2005) on polytechnic degrees awarded in the Province of Ahvenanmaa/Åland.

FINHEEC may accept other assignments relating to evaluation from Finnish or foreign operators.

FINHEEC shall regularly participate in an international evaluation of its own operation. FINHEEC shall submit an annual report on its own activities and the salient findings of the evaluations.

Organisation

The Ministry of Education appoints the members of the Evaluation Council for four years after consulting the higher education institutions and different interest groups. At its first meeting after appointment, the Evaluation Council shall elect a chairperson and a vice-chairperson from amongst its members for the four-year term. Before the election of the chairperson, the Evaluation Council shall be chaired by its longest serving member or, should there be several long-serving members, by the oldest in age.

38

To begin with, the Evaluation Council shall elect a chairperson. The members may propose chairpersons. An election shall be held if there are more than one seconded candidate. If none of the candidates receives a simple majority in the first round, another round shall be held between the two chairperson candidates with the most votes. If the votes are equal, the election of the chairperson shall be decided by lot.

The elected chairperson shall chair the meeting. The vice-chairperson shall be elected. The members may propose vice-chairpersons. If there are more than one seconded candidate, an election shall be held. If none of the candidates receives a simple majority in the first round, another round shall be held between the two vice-chairperson candidates with the most votes. If the votes are equal, the chairman shall have the casting vote.

Meetings

The Evaluation Council shall convene at the invitation of the chairperson or, if he/she is prevented, at the invitation of the vice-chairperson. The Evaluation Council shall have a quorum when six members, including the chairperson, are present. Matters shall be decided by simple majority. When the votes are equal, the chairman shall have the casting vote.

Minutes shall be written of the Evaluation Council meeting. The minutes shall be signed by the chairperson of the meeting and confirmed by the secretary. The minutes shall be checked at the following meeting. Documents drawn up by a decision of the Evaluation Council shall be signed by the chairperson or vice-chairperson and the Secretary General. Project-specific documents shall be signed by the chairperson and the Secretary General or one of these and the Chief Planning Officer, a Senior Adviser or the Administrative Assistant.

Operation

The Evaluation Council shall decide on■ the FINHEEC standing order and quality manual■ the action plan for the term of office and the annual work plan and budget and

adopt the annual report■ its subcommittees and international advisory board■ the evaluations to be undertaken■ the project plans for the evaluations■ the chairpersons and members of the planning groups■ the chairpersons and members of the evaluation groups■ the result and consequences of an audit based on the reports■ centres of excellence in education based on an evaluation, on Ministry of

Education assignment■ evaluation and education subsidies granted to higher education institutions■ proposals to the Ministry of Education concerning the filling of vacancies in the

Secretariat■ opinions given in the name of the Evaluation Council that have primary

consequence in terms of higher education policy or the operation of FINHEEC.FINHEEC shall keep in regular contact with Finnish and foreign stakeholders

named in a separate stakeholder document.

39

Subcommittees

Working committee

The Evaluation Council shall elect from amongst its members a working committee composed of a chairperson, a vice-chairperson and representatives of the polytechnic and university sectors and students.

The working committee shall prepare matters for the Council meetings and handle matters delegated to it by the Council, when needed.

Other subcommittees

The Evaluation Council may appoint subcommittees to prepare matters that come before the Council. The Evaluation Council shall appoint the chairperson, vice-chairperson and members of the subcommittee. Where needed, separate guidelines shall be issued concerning the duties of the subcommittee. The subcommittees shall report on their activities to the Evaluation Council in conjunction with the annual report.

Persons external to FINHEEC may also be appointed to the subcommittees.

International advisory board

In order to support and develop its operation and to boost international cooperation, the Evaluation Council may appoint an advisory board composed of international and national experts and chaired by the chairperson of the Evaluation Council. The international advisory board shall convene at the invitation of the chairperson. The advisory board shall support and provide consultancy to FINHEEC on its operation.

Secretariat

For the preparation and implementation of matters deliberated by it, the Evaluation Council has a Secretariat. The Secretariat shall be directed by a Secretary General. The Secretariat shall comprise chief planning officers and senior advisors responsible for evaluation projects. In addition, the Secretariat shall have an administrative assistant who acts as secretary at the meetings, secretary to the Secretary General and manages financial and administrative matters. The members of the Secretariat shall have job descriptions, which are reviewed at the annual performance and development review meeting. The division of work among the Secretariat shall be recorded in a work assignment document. Each project is assigned a senior advisor responsible for it and a deputy. The chief planning officers, together with the Secretary General, shall be responsible for annual reporting on the activities.

The Secretary General shall direct and develop the activities of the Secretariat, be responsible for the administration and finances of the unit, the planning of the activities, the preparation of matters and overseeing their implementation, act as editor of the FINHEEC publication series and make proposals concerning fixed-term employees for the Secretariat.

The Secretary General shall conduct annual performance and development reviews with the personnel and make performance assessments, and in this connection also agree on personal training schedules.

40

Each new employee in the Secretariat shall be assigned a personal induction mentor. The Secretariat shall maintain induction materials for new employees.

Planning and evaluation groups

The Evaluation Council shall appoint planning and evaluation groups for evaluation projects and chairpersons to the groups. Deputies may also be appointed for the group members. The chairperson of a planning group is generally elected from the members of the Evaluation Council, which promotes the flow of information between the planning group and the Council. Student organisations usually propose the student members to the planning and evaluation groups.

In the composition of the groups, the aim shall be as diverse and comprehensive expertise as possible in regard of the field, theme, higher education institution or higher education sector to be evaluated. The members shall be representatives of higher education institutions, stakeholders, the labour market and student organisations. Depending on the evaluation, the evaluation group may also have foreign experts as members. Written contracts shall be concluded with the group members on the assignment. In regard of disqualification, the appointment of members to the groups shall be governed by the provisions of the Administrative Procedure Act (434/2003, Chapter 5, Sections 27–29).

The planning group shall chart the area to be evaluated and make a proposals as to its delimitation. In addition the planning group shall define the aims, content and method of the evaluation project, in other words, draw up a project plan submitted to the Evaluation Council for approval, propose the composition of the evaluation group and arrange a discussion and orientation event for the evaluation group, where the project plan is talked through.

In its conclusions, the evaluation group shall be independent and autonomous. Depending on the evaluation, the evaluation group shall■ acquaint themselves with the assignment, specify the evaluation assignment in

more detail and define possible evaluation criteria■ acquaint themselves with the target of evaluation with the help of the project

plan adopted by the Evaluation Council and background material provided by the Secretariat

■ participate in the orientation arranged by FINHEEC■ scrutinise the self evaluation reports or other materials submitted by the higher

education institutions■ based on the self evaluation reports, specify the matters to be inspected during

the site visit■ determine possible need for additional information■ organise the site visit and possible other events relating to the evaluation

process, such as thematic discussions■ make the site visits■ based on the self-evaluation reports, other materials and site visits, formulate a

view of the quality of the phenomenon under review both in the national scale and in the higher education institutions participating in the evaluation

■ put forward recommendations for quality enhancement relating to the object of review

■ prepare a final report or evaluation and development feedback.

41

The Evaluation Council may also appoint steering groups for its projects. The purpose of the steering group is to follow the project as it proceeds and direct it with its expertise, where needed.

3. Processing of matters

The Evaluation Council shall make decisions on referral. The matters are presented by the Secretary General and/or an official appointed by him/her for each matter. Matters concerning the Secretary General shall be presented by the chairperson of the Evaluation Council. Otherwise the presenting of matters shall be governed by established administrative procedures.

A matter to be submitted to the Evaluation Council shall be delivered to the decision-makers sufficiently in advance. The meeting materials shall be posted one week before the date of the meeting.

4. Finances and fees

The Higher Education Evaluation Council shall receive its annual appropriations as part of the state budget in an item for educational evaluation. In addition, it may accept other assignments relating to evaluation from Finnish and foreign operators.

The fees and compensation for experts used in evaluations shall be decided annually as part of the FINHEEC budget.

The grounds for determining the fees and compensation of the chairperson and members of the Evaluation Council and its subcommittees shall be determined by the Ministry of Education.

The Secretary General shall approve the invoices arising from the operation of FINHEEC and issue travel assignments. Travel assignments to the Secretary General shall be issued by the Director of the Division for Higher Education and Science of the Department for Education and Science Policy of the Ministry of Education.

5. Appeals

The decisions issued by FINHEEC concerning the results and consequences of evaluations are expert opinions by nature. They are not administrative decisions and cannot be appealed against. They do not contain an address for appeals. In audits of quality assurance systems, the institutions under review and FINHEEC conclude a written agreement, which sets out possible consequences to which the parties commit themselves.

The FINHEEC Secretariat members are civil servants and the Secretariat shall follow good administrative practice.

6. Review of the FINHEEC standing order

The FINHEEC standing order shall be revised by a decision of the Evaluation Council, when needed.

42

6.5 ENQA Membership Provisions

MEMBERSHIP PROVISIONS

CHAPTER I. CRITERIA FOR FULL MEMBERSHIP

Full Membership of ENQA is open to quality assurance agencies in the field of higher education from EHEA member states that have been operating and conducting actual evaluation activities for at least two years.

Before being accepted as a Full Member, an applicant agency must satisfy the Board that it meets the seven criteria, listed below. The applicant agency will thereby also meet the European Standards and Guidelines for Quality Assurance in the European Higher Education Area as adopted by the European Ministers in charge of higher education in Bergen in 2005. The Board may modify the details of the procedures at its discretion.

Each criterion is followed by guidelines (in italics) which provide additional information about good practice and in some cases explain in more detail the meaning and importance of the criteria. Although the guidelines are not part of the criteria themselves, the criteria should be considered in conjunction with them.

Criterion 1 – Activities

A Full Member will undertake external quality assurance activities (at institutional or programme level) on a regular basis. In undertaking its activities, the member should take into account the presence and effectiveness of the external quality assurance processes described in the European Standards and Guidelines for Higher Education in the European Higher Education Area1.

The external quality assurance activities may involve evaluation, review, audit, assessment, accreditation or other similar activities and should be part of the core functions of the member.

Criterion 2 – Official status

A Full Member should be formally recognised by competent public authorities in the European Higher Education Area as an agency with responsibility for external quality assurance and should have an established legal basis. It should comply with any requirements of the legislative jurisdiction(s) within which it operates.

Criterion 3 – Resources

A Full Member should have adequate and proportional resources, both human and financial, to enable it to organise and run its external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of its processes, procedures and staff.

1 Standards and Guidelines for Quality Assurance in the European Higher Education Area, ISBN 952-5539-04-0, Helsinki: ENQA, 2005.

43

Criterion 4 – Mission statement

A Full Member should have clear and explicit goals and objectives for its work, contained in a publicly available statement.

This statement should describe the goals and objectives of the member’s quality assurance processes, the division of labour with relevant stakeholders in higher education, especially the higher education institutions, and the cultural and historical context of its work. The statement should make clear that the external quality assurance process is a major activity of the member and that there exists a systematic approach to achieving its goals and objectives. There should also be documentation to demonstrate how the statement is translated into a clear policy and management plan.

Criterion 5 – Independence

i. A Full Member should be independent to the extent both that it has autonomous responsibility for its operations and that the conclusions and recommendations made in its reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders.

The member will need to demonstrate its independence through measures, such as:■ its operational independence from higher education institutions and governments

is guaranteed in official documentation (e.g. instruments of governance or legislative acts);

■ the definition and operation of its procedures and methods, the nomination and appointment of external experts and the determination of the outcomes of its quality assurance processes are undertaken autonomously and independently from governments, higher education institutions, and organs of political influence;

■ while relevant stakeholders in higher education, particularly students/learners, are consulted in the course of quality assurance processes, the final outcomes of the quality assurance processes remain the responsibility of the member.

Criterion 6 – External quality assurance criteria and processes used by the members

i. The processes, criteria and procedures used by the member should be pre-defined and publicly available.

ii. These processes will normally be expected to include:■ a self-assessment or equivalent procedure by the subject of the quality assurance

process;■ an external assessment by a group of experts, including, as appropriate, (a)

student member(s), and site visits as decided by the member;■ publication of a report, including any decisions, recommendations or other

formal outcomes;■ a follow-up procedure to review actions taken by the subject of the quality

assurance process in the light of any recommendations contained in the report.

44

The member may develop and use other processes and procedures for particular purposes. The member should pay careful attention to its declared principles at all times and should ensure both that its requirements and processes are managed professionally and that its conclusions and decisions are reached in a consistent manner, even though the decisions are formed by groups of different people. A member that makes formal quality assurance decisions, or conclusions which have formal consequences, should have an appeals procedure. The nature and form of the appeals procedure should be determined in the light of the constitution of each member.

Criterion 7 – Accountability procedures

The member should have in place procedures for its own accountability.

These procedures are expected to include the following:

i. a published policy for the assurance of its own quality, made available on its website;

ii. documentation which demonstrates that:■ the member’s processes and results reflect its mission and goals of quality

assurance;■ the member has in place, and enforces, a no-conflict-of-interest mechanism in

the work of its external experts, Committee/Council/Board and staff members;■ the member has reliable mechanisms that ensure the quality of any activities and

material produced by subcontractors, if some or all of the elements in its quality assurance procedure are subcontracted to other parties;

■ the member has in place internal quality assurance procedures which include an internal feedback mechanism (i.e. a means to collect feedback from its own staff and council/board); an internal reflection mechanism (i.e. means to react to internal and external recommendations for improvement); and an external feedback mechanism (i.e. means to collect feedback from experts and reviewed institutions for future development) in order to inform and underpin its own development and improvement.

iii. a mandatory cyclical external review of its activities at least once every five years which includes a report on its conformity with the membership criteria of ENQA.

CHAPTER II. APPLICATION FOR MEMBERSHIP

Application form and documentation

Applications for membership of ENQA shall be in the form specified by the Board (details obtainable from the Secretary General). Applications for membership are considered and decided upon by the Board on the basis either of submitted documentation alone, or of submitted documentation and a visit to the applicant body. Applications for Full Membership will only be considered where an independent external review report on the agency’s conformity with the membership criteria, carried out in a manner and to a standard acceptable to the Board, is received (see 7iii above).

45

Candidate Membership procedure

If the Board decides, in the light of the application, that the applicant agency does not meet the above mentioned criteria for Full Membership, but is likely to be able to meet the criteria within two years of the Candidate Membership being granted, it may grant, at its discretion, Candidate Membership for a maximum of two years. At the end of that period (or sooner, if the Candidate Member so requests), the Board will require the submission of an external review report which demonstrates that the applicant meets the criteria. If, in the opinion of the Board, and following the submission of the evidence, the criteria are still not met, the application will lapse and the applicant will, by the decision of the Board, not be allowed to reapply for membership until a further period of two years has elapsed. During this period the agency will remain on the ENQA mailing list to ensure information dissemination on the activities of ENQA. If, following the request for further evidence, the Board grants Full Membership, the agency will be required to undergo an external review within five years of the date on which Candidate Membership was granted. If, however, the applying organisation does not have the intention or capacity to fulfil the Full Membership criteria, it can apply to become an Associate or Affiliate of ENQA (see Title III of the ENQA Regulations).

External reviews

As indicated in criterion 7 above, it is a condition of membership that all Full Members of ENQA undergo an external review at least once every five years. If a member does not undergo an external review within five years of Full Membership being granted or reconfirmed, it will, by decision of the General Assembly, cease to be a member of ENQA. If, as a result of an external review, a member is judged not to meet the membership criteria by the Board, it will be given two years to conform with the criteria, during which time the agency will be designated as a Candidate Member of ENQA. A further review will be carried out by the Board, or its nominated reviewers, at the end of the two-year period (or sooner, if the member agency so requests). An agency that, in the opinion of the Board, and following the further review, remains in breach of ENQA’s membership criteria will, by confirmation of the General Assembly, be debarred from ENQA. A debarred agency will be permitted to reapply for membership after a further period of two years.

Notification and Appeal

Applicants that are not accepted for membership or which are offered Candidate Membership, shall be notified of the reasons by the President of ENQA and shall be informed of the areas where the Board considers that further development or changes are required or advised. A body whose application for membership is not accepted by the Board, or which is granted Candidate, rather than Full Membership, or which is redesignated from Full Membership to Candidate Membership against its wishes, may appeal in writing to the Board, indicating why it believes the Board’s decision to be wrong. Appeals should be addressed to the Secretary General. The deadline for appeals is two calendar months from the date of the notification of the Board’s decision. The Board shall ask the Appeals and Complaints Committee (see article 8 of the ENQA Regulations) to review

46

the decision, and the Board’s decision on the appeal shall take into account the Committee’s report. The Board’s decision on appeals is final.

CHAPTER III. TRANSITIONAL MEMBERSHIP ARRANGEMENTS

The Regulations describe the objectives, membership, structure and funding arrangements of the European Association for Quality Assurance in Higher Education. ENQA was established on 4 November 2004 in Frankfurt, Germany when ENQA succeeded its predecessor body, the European Network for Quality Assurance in Higher Education, which existed from 29 March 2000 until 4 November 2004, and which itself was founded in fulfilment of Council Recommendation 98/561/EC of 24 September 1998 on European co-operation in quality assurance in higher education.

At the first General Assembly of the European Association for Quality Assurance in Higher Education held in Frankfurt, Germany, on 4 November 2004, it was agreed that the organisations that were Full Members of the Network at the point of dissolution, should be designated as Full Members of ENQA, subject to their agreeing to undergo a review, to the satisfaction of the Board, as described in the section on external reviews (under Chapter II) of the present document, within the first five years of ENQA’s existence (i.e. by 19 September 2010). It was further agreed that organisations that were Associate members of the Network and that wished to continue in membership of ENQA, should be invited to make an application for Candidate Membership and be subject to the provisions of Title III, sections I and III of ENQA’s Regulations and of the present document. Existing Candidate Members would continue in that category and be subject to the provisions for achieving Full Membership shown in the present document.

KORKEAKOULUJEN ARVIOINTINEUVOSTON JULKAISUJAPL 133, 00171 HELSINKI • Puh. 09-1607 6913 • Fax 09-1607 6911 • www.kka.fi

1:2000 Lehtinen, E., Kess, P., Ståhle, P. & Urponen, K.: Tampereen yliopiston opetuksen arviointi2:2000 Cohen, B., Jung, K. & Valjakka, T.: From Academy of Fine Arts to University. Same name, wider ambitions3:2000 Goddard, J., Moses, I., Teichler, U., Virtanen, I. & West, P.: External Engagement and Institutional Adjustment:

An Evaluation of the University of Turku4:2000 Almefelt, P., Kekäle, T., Malm, K., Miikkulainen, L. & Pehu-Voima, S.: Audit of Quality Work. Swedish

Polytechnic, Finland5:2000 Harlio, R., Harvey, L., Mansikkamäki. J., Miikkulainen, L. & Pehu-Voima, S.: Audit of Quality Work. Central

Ostrobothnia Polytechnic6:2000 Moitus, S. (toim.): Yliopistokoulutuksen laatuyksiköt 2001–20037:2000 Liuhanen, A.-M. (toim.): Neljä aikuiskoulutuksen laatuyliopistoa 2001–20038:2000 Hara, V. , Hyvönen, R. , Myers, D. & Kangasniemi, J. (Eds.): Evaluation of Education for the Information

Industry9:2000 Jussila, J. & Saari, S. (Eds.): Teacher Education as a Future-moulding Factor. International Evaluation of

Teacher Education in Finnish Universities10:2000 Lämsä, A. & Saari, S. (toim.): Portfoliosta koulutuksen kehittämiseen. Ammatillisen opettajankoulutuksen

arviointi11:2000 Korkeakoulujen arviointineuvoston toimintasuunnitelma 2000–200312:2000 Finnish Higher Education Evaluation Council Action Plan for 2000–200313:2000 Huttula, T. (toim.): Ammattikorkeakoulujen koulutuksen laatuyksiköt 200014:2000 Gordon, C., Knodt, G., Lundin, R., Oger, O. & Shenton, G.: Hanken in European Comparison. EQUIS

Evaluation Report15:2000 Almefelt, P., Kekäle, T., Malm, K., Miikkulainen, L. & Kangasniemi, J.: Audit of Quality Work. Satakunta

Polytechnic16:2000 Kells, H.R., Lindqvist, O. V. & Premfors, R.: Follow-up Evaluation of the University of Vaasa. Challenges of a

small regional university17:2000 Mansikkamäki, J., Kekäle, T., Miikkulainen, L. , Stone, J., Tolppi, V.-M. & Kangasniemi, J.: Audit of Quality Work.

Tampere Polytechnic18:2000 Baran, H., Gladrow, W. , Klaudy, K. , Locher, J. P. , Toivakka, P. & Moitus, S.: Evaluation of Education and

Research in Slavonic and Baltic Studies19:2000 Harlio, R. , Kekäle, T. , Miikkulainen, L. & Kangasniemi, J.: Laatutyön auditointi. Kymenlaakson ammatti-

korkeakoulu20:2000 Mansikkamäki, J., Kekäle, T., Kähkönen, J., Miikkulainen, L., Mäki, M. & Kangasniemi, J.: Laatutyön auditointi.

Pohjois-Savon ammattikorkeakoulu21:2000 Almefelt, P., Kantola, J., Kekäle, T., Papp, I., Manninen, J. & Karppanen, T.: Audit of Quality Work. South Carelia

Polytechnic

1:2001 Valtonen, H.: Oppimisen arviointi Sibelius-Akatemiassa2:2001 Laine, 1., Kilpinen, A., Lajunen, L., Pennanen, J., Stenius, M., Uronen, P. & Kekäle, T.: Maanpuolustuskorkea-

koulun arviointi3:2001 Vähäpassi, A. (toim.): Erikoistumisopintojen akkreditointi4:2001 Baran, H., Gladrow, W. , Klaudy, K. , Locher, J. P. , Toivakka, P. & Moitus, S.: |kspertiza obrazowaniq i

nau^no-issledowatelxskoj raboty w oblasti slawistiki i baltistiki (Ekspertizaobrazovanija i naucno-issledovatelskoj raboty v oblasti slavistiki i baltistiki)

5:2001 Kinnunen, J.: Korkeakoulujen alueellisen vaikuttavuuden arviointi. Kriteerejä vuorovaikutteisuudenarvottamiselle

6:2001 Löfström, E.: Benchmarking korkeakoulujen kieltenopetuksen kehittämisessä7:2001 Kaartinen-Koutaniemi, M.: Korkeakouluopiskelijoiden harjoittelun kehittäminen. Helsingin yliopiston,

Diakonia-ammattikorkeakoulun ja Lahden ammattikorkeakoulun benchmarking-projekti8:2001 Huttula, T. (toim.): Ammattikorkeakoulujen aluekehitysvaikutuksen huippuyksiköt 20019:2001 Welander, C. (red.): Den synliga yrkeshögskolan. Ålands yrkeshögskola.

10:2001 Valtonen, H.: Learning Assessment at the Sibelius Academy11:2001 Ponkala, O. (toim.): Terveysalan korkeakoulutuksen arvioinnin seuranta12:2001 Miettinen, A. & Pajarre, E.: Tuotantotalouden koulutuksen arvioinnin seuranta

^

PUBLICATIONS OF THE FINNISH HIGHER EDUCATION EVALUATION COUNCILP. B. 133, 00171 HELSINKI, Finland • Tel. +358-9-1607 6913 • Fax +358-9-1607 6911 • www.finheec.fi

1:2000 Lehtinen, E., Kess, P., Ståhle, P. & Urponen, K.: Tampereen yliopiston opetuksen arviointi2:2000 Cohen, B., Jung, K. & Valjakka, T.: From Academy of Fine Arts to University. Same name, wider ambitions3:2000 Goddard, J., Moses, I., Teichler, U., Virtanen, I. & West, P.: External Engagement and Institutional Adjustment:

An Evaluation of the University of Turku4:2000 Almefelt, P., Kekäle, T., Malm, K., Miikkulainen, L. & Pehu-Voima, S.: Audit of Quality Work. Swedish

Polytechnic, Finland5:2000 Harlio, R., Harvey, L., Mansikkamäki. J., Miikkulainen, L. & Pehu-Voima, S.: Audit of Quality Work. Central

Ostrobothnia Polytechnic6:2000 Moitus, S. (toim.): Yliopistokoulutuksen laatuyksiköt 2001–20037:2000 Liuhanen, A.-M. (toim.): Neljä aikuiskoulutuksen laatuyliopistoa 2001–20038:2000 Hara, V. , Hyvönen, R. , Myers, D. & Kangasniemi, J. (Eds.): Evaluation of Education for the Information

Industry9:2000 Jussila, J. & Saari, S. (Eds.): Teacher Education as a Future-moulding Factor. International Evaluation of

Teacher Education in Finnish Universities10:2000 Lämsä, A. & Saari, S. (toim.): Portfoliosta koulutuksen kehittämiseen. Ammatillisen opettajankoulutuksen

arviointi11:2000 Korkeakoulujen arviointineuvoston toimintasuunnitelma 2000–200312:2000 Finnish Higher Education Evaluation Council Action Plan for 2000–200313:2000 Huttula, T. (toim.): Ammattikorkeakoulujen koulutuksen laatuyksiköt 200014:2000 Gordon, C., Knodt, G., Lundin, R., Oger, O. & Shenton, G.: Hanken in European Comparison. EQUIS

Evaluation Report15:2000 Almefelt, P., Kekäle, T., Malm, K., Miikkulainen, L. & Kangasniemi, J.: Audit of Quality Work. Satakunta

Polytechnic16:2000 Kells, H.R., Lindqvist, O. V. & Premfors, R.: Follow-up Evaluation of the University of Vaasa. Challenges of a

small regional university17:2000 Mansikkamäki, J., Kekäle, T., Miikkulainen, L. , Stone, J., Tolppi, V.-M. & Kangasniemi, J.: Audit of Quality Work.

Tampere Polytechnic18:2000 Baran, H., Gladrow, W. , Klaudy, K. , Locher, J. P. , Toivakka, P. & Moitus, S.: Evaluation of Education and

Research in Slavonic and Baltic Studies19:2000 Harlio, R. , Kekäle, T. , Miikkulainen, L. & Kangasniemi, J.: Laatutyön auditointi. Kymenlaakson ammatti-

korkeakoulu20:2000 Mansikkamäki, J., Kekäle, T., Kähkönen, J., Miikkulainen, L., Mäki, M. & Kangasniemi, J.: Laatutyön auditointi.

Pohjois-Savon ammattikorkeakoulu21:2000 Almefelt, P., Kantola, J., Kekäle, T., Papp, I., Manninen, J. & Karppanen, T.: Audit of Quality Work. South Carelia

Polytechnic

1:2001 Valtonen, H.: Oppimisen arviointi Sibelius-Akatemiassa2:2001 Laine, 1., Kilpinen, A., Lajunen, L., Pennanen, J., Stenius, M., Uronen, P. & Kekäle, T.: Maanpuolustuskorkea-

koulun arviointi3:2001 Vähäpassi, A. (toim.): Erikoistumisopintojen akkreditointi4:2001 Baran, H., Gladrow, W. , Klaudy, K. , Locher, J. P. , Toivakka, P. & Moitus, S.: |kspertiza obrazowaniq i

nau^no-issledowatelxskoj raboty w oblasti slawistiki i baltistiki (Ekspertizaobrazovanija i nauc’´no-issledovatelskoj raboty v oblasti slavistiki i baltistiki)

5:2001 Kinnunen, J.: Korkeakoulujen alueellisen vaikuttavuuden arviointi. Kriteerejä vuorovaikutteisuudenarvottamiselle

6:2001 Löfström, E.: Benchmarking korkeakoulujen kieltenopetuksen kehittämisessä7:2001 Kaartinen-Koutaniemi, M.: Korkeakouluopiskelijoiden harjoittelun kehittäminen. Helsingin yliopiston,

Diakonia-ammattikorkeakoulun ja Lahden ammattikorkeakoulun benchmarking-projekti8:2001 Huttula, T. (toim.): Ammattikorkeakoulujen aluekehitysvaikutuksen huippuyksiköt 20019:2001 Welander, C. (red.): Den synliga yrkeshögskolan. Ålands yrkeshögskola.

10:2001 Valtonen, H.: Learning Assessment at the Sibelius Academy11:2001 Ponkala, O. (toim.): Terveysalan korkeakoulutuksen arvioinnin seuranta12:2001 Miettinen, A. & Pajarre, E.: Tuotantotalouden koulutuksen arvioinnin seuranta

13:2001 Moitus, S., Huttu, K., Isohanni, I., Lerkkanen, J., Mielityinen, I., Talvi, U., Uusi-Rauva, E. & Vuorinen, R.:Opintojen ohjauksen arviointi korkeakouluissa

14:2001 Fonselius, J., Hakala, M.K. & Holm, K. : Evaluation of Mechanical Engineering Education atUniversities and Polytechnics

15:2001 Kekäle, T. (ed.): A Human Vision with Higher Education Perspective.Institutional Evaluation of theHumanistic Polytechnic

1:2002 Kantola, I. (toim.): Ammattikorkeakoulun jatkotutkinnon kokeilulupahakemusten arviointi2:2002 Kallio, E.: Yksilöllisiä heijastuksia. Toimiiko yliopisto-opetuksen paikallinen itsearviointi?3:2002 Raivola, R., Himberg, T., Lappalainen, A., Mustonen, K. & Varmola, T.: Monta tietä maisteriksi. Yliopisto-

jen maisteriohjelmien arviointi4:2002 Nurmela-Antikainen, M., Ropo, E., Sava, I. & Skinnari, S.: Kokonaisvaltainen opettajuus. Steiner-

pedagogisen opettajankoulutuksen arviointi5:2002 Toikka, M. & Hakkarainen, S.: Opintojen ohjauksen benchmarking tekniikan alan koulutus-

ohjelmissa. Kymenlaakson, Mikkelin ja Pohjois-Savon ammattikorkeakoulut6:2002 Kess, P., Hulkko, K., Jussila, M., Kallio, U., Larsen, S. , Pohjolainen,T. & Seppälä, K.: Suomen avoin

yliopisto. Avoimen yliopisto-opetuksen arviointiraportti7:2002 Rantanen, T., Ellä, H., Engblom, L.-Å., Heinonen, J., Laaksovirta, T., Pohjanpalo, L., Rajamäki, T.&

Woodman, J.: Evaluation of Media and Communication Studies in Higher Education in Finland8:2002 Katajamäki, H., Artima, E., Hannelin, M., Kinnunen, J., Lyytinen, H. K., Oikari, A. & Tenhunen, M.-L.:

Mahdollinen korkeakouluyhteisö. Lahden korkeakouluyksiköiden alueellisen vaikuttavuuden arviointi9:2002 Kekäle, T. & Scheele, J.P: With care. Institutional Evaluation of the Diaconia Polytechnic

10:2002 Härkönen, A., Juntunen, K. & Pyykkönen, E.-L. : Kajaanin ammattikorkeakoulun yrityspalveluidenbenchmarking

11:2002 Katajamäki, H. (toim.): Ammattikorkeakoulut alueidensa kehittäjinä.Näkökulmia ammatti-korkeakoulujen aluekehitystehtävän toteutukseen

12:2002 Huttula, T. (toim.): Ammattikorkeakoulujen koulutuksen laatuyksiköt 2002–200313:2002 Hämäläinen, K. & Kaartinen-Koutaniemi, M. (toim.): Benchmarking korkeakoulujen

kehittämisvälineenä14:2002 Ylipulli-Kairala, K. & Lohiniva, V. (eds.): Development of Supervised Practice in Nurse Education. Oulu

and Rovaniemi Polytechnics15:2002 Löfström, E., Kantelinen, R., Johnson, E., Huhta, M., Luoma, M., Nikko, T., Korhonen, A., Penttilä, J.,

Jakobsson, M. & Miikkulainen, L.: Ammattikorkeakoulun kieltenopetus tienhaarassa. Kielten-opetuksen arviointi Helsingin ja Keski-Pohjanmaan ammattikorkeakouluissa

16:2002 Davies, L., Hietala, H., Kolehmainen, S., Parjanen, M. & Welander, C.: Audit of Quality Work. VaasaPolytechnic

17:2002 Sajavaara, K., Hakkarainen, K. , Henttonen, A., Niinistö, K., Pakkanen, T. , Piilonen, A.-R. & Moitus, S.:Yliopistojen opiskelijavalintojen arviointi

18:2002 Tuomi, O. & Pakkanen, P.: Towards Excellence in Teaching. Evaluation of the Quality of Education andthe Degree Programmes in the University of Helsinki

1:2003 Sarja, A., Atkin, B. & Holm, K.: Evaluation of Civil Engineering Education at Universities andPolytechnics

2:2003 Ursin, J. (toim.): Viisi aikuiskoulutuksen laatuyliopistoa 2004–20063:2003 Hietala, H., Hintsanen, V., Kekäle, T., Lehto, E., Manninen, H. & Meklin, P.: Arktiset haasteet ja

mahdollisuudet. Rovaniemen ammattikorkeakoulun kokonaisarviointi4:2003 Varis, T. & Saari, S. (Eds.): Knowledge Society in Progress – Evaluation of the Finnish Electronic

Library – FinELib5:2003 Parpala, A. & Seppälä, H. (toim.): Yliopistokoulutuksen laatuyksiköt 2004–20066:2003 Kettunen, P., Carlsson, C., Hukka, M., Hyppänen, T., Lyytinen, K., Mehtälä, M., Rissanen, R., Suviranta, L.

& Mustonen, K.: Suomalaista kilpailukykyä liiketoimintaosaamisella. Kauppatieteiden ja liiketaloudenkorkeakoulutuksen arviointi

7:2003 Kauppi, A. & Huttula, T. (toim.): Laatua ammattikorkeakouluihin8:2003 Parjanen, M. : Amerikkalaisen opiskelija-arvioinnin soveltaminen suomalaiseen yliopistoon

9:2003 Sarala, U. & Seppälä, H.: (toim.): Hämeen ammattikorkeakoulun kokonaisarviointi10:2003 Kelly‚ J., Bazsa, G. & Kladis, D.: Follow-up review of the Helsinki University of Technology11:2003 Goddard, J., Asheim, B., Cronberg, T. & Virtanen, I.: Learning Regional Engagement. A Re-evaluation of

the Third Role of Eastern Finland universities12:2003 Impiö, 1., Laiho, U.-M., Mäki, M., Salminen, H., Ruoho, K.,Toikka, M. & Vartiainen, P.: Ammatti-

korkeakoulut aluekehittäjinä. Ammattikorkeakoulujen aluekehitysvaikutuksen huippuyksiköt 2003–2004

13:2003 Cavallé, C., de Leersnyder, J.-M., Verhaegen, P. & Nataf, J.-G. : Follow-up review of the Helsinki Schoolof Economics. An EQUIS re-accreditation

14:2003 Kantola, I. (toim.): Harjoittelun ja työelämäprojektien benchmarking15:2003 Ala-Vähälä, T.: Hollannin peili. Ammattikorkeakoulujen master-tutkinnot ja laadunvarmistus16:2003 Goddard, J., Teichler, U., Virtanen, I., West, P. & Puukka, J.: Progressing external engagement. A re-

evaluation of the third role of the University of Turku17:2003 Baran, H., Toivakka, P. & Järvinen, J.: Slavistiikan ja baltologian koulutuksen ja tutkimuksen arvioinnin

seuranta

1:2004 Kekäle, T., Heikkilä, J., Jaatinen, P., Myllys, H., Piilonen, A.-R., Savola, J., Tynjälä, P. & Holm, K.:Ammattikorkeakoulujen jatkotutkintokokeilu. Käynnistysvaiheen arviointi

2:2004 Ekholm, L., Stenius, M., Huldin, H., Julkunen, I., Parkkonen, J., Löfström, E., Metsä, K.: NOVA ARCADA –Sammanhållning, decentralisering, gränsöverskridande. Helhetsutvärdering av Arcada – Nylandssvenska yrkeshögskola 2003

3:2004 Hautala, J.: Tietoteollisuusalan koulutuksen arvioinnin seuranta4:2004 Rauhala, P., Karjalainen, A., Lämsä, A.-M., Valkonen, A., Vänskä, A. & Seppälä, H.: Strategiasta

koulutuksen laatuun. Turun ammattikorkeakoulun kokonaisarviointi5:2004 Murto, L., Rautniemi, L., Fredriksson, K., Ikonen, S., Mäntysaari, M., Niemi, L., Paldanius, K., Parkkinen,

T., Tulva, T., Ylönen, F. & Saari, S.: Eettisyyttä, elastisuutta ja elämää. Yliopistojen sosiaalityön jaammattikorkeakoulujen sosiaalialan arviointi yhteistyössä työelämän kanssa

6:2004 Ståhle, P., Hämäläinen, K., Laiho, K., Lietoila, A., Roiha, J., Weijo, U. & Seppälä, H.:Tehokas järjestelmä –elävä dialogi. Helian laatutyön auditointi

7:2004 Korkeakoulujen arviointineuvoston toimintakertomus 2000–20038:2004 Luopajärvi, T., Hauta-aho, H., Karttunen, P., Markkula, M., Mutka, U. & Seppälä, H.: Perämerenkaaren

ammattikorkeakoulu? Kemi-Tornion ammattikorkeakoulun kokonaisarviointi9:2004 Moitus, S. & Seppälä, H.: Mitä hyötyä arvioinneista? Selvitys Korkeakoulujen arviointineuvoston

1997–2003 toteuttamien koulutusala-arviointien käytöstä10:2004 Moitus, S. & Saari, S.: Menetelmistä kehittämiseen. Korkeakoulujen arviointineuvoston arviointi-

menetelmät vuosina 1996–200311:2004 Pratt, J., Kekäle, T., Maassen, P., Papp, I., Perellon, J. & Uitti, M.: Equal, but Different – An Evaluation of

the Postgraduate Studies and Degrees in Polytechnics – Final Report

1:2005 Niinikoski, S. (toim.): Benchmarking tutkintorakennetyön työkaluna2:2005 Ala-Vähälä, T.: Korkeakoulutuksen ulkoisen laadunvarmistuksen järjestelmät Ranskassa3:2005 Salminen, H. & Kajaste, M. (toim.): Laatua, innovatiivisuutta ja proaktiivisuutta. Ammatti-

korkeakoulujen koulutuksen laatuyksiköt 2005–20064:2005 Korkeakoulujen laadunvarmistusjärjestelmien auditointi. Auditointikäsikirja vuosille 2005–20075:2005 Auditering av högskolornas kvalitetssäkringssystem. Auditeringshandbok för åren 2005–2007

1:2006 Dill, D.D., Mitra, S. K., Siggaard Jensen, H., Lehtinen, E., Mäkelä, T., Parpala, A., Pohjola, H., Ritter, M. A.& Saari, S.: PhD Training and the Knowledge-Based Society. An Evaluation of Doctoral Education inFinland

2:2006 Antikainen, E.-L., Honkonen, R., Matikka, O., Nieminen, P., Yanar, A. & Moitus, S.: Mikkelin ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

3:2006 Kekäle, T., Ilolakso, A., Katajavuori, N., Toikka, M. & Isoaho, K.: Kuopion yliopiston laadunvarmistus-järjestelmän auditointi

4:2006 Audits of Quality Assurance Systems of Finnish Higher Education Institutions. Audit Manual for2005–2007

5:2006 Rauhala, P., Kotila, H., Linko, L., Mulari, O., Rautonen, M. & Moitus, S.; Keski-Pohjanmaan ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

6:2006 Hämäläinen, K., Kantola, I., Marttinen, R., Meriläinen, M., Mäki, M. & Isoaho, K.: Jyväskylän ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

7:2006 Kekäläinen, H.: (toim.)Neljä aikuiskoulutuksen laatuyliopistoa 2007–20098:2006 Yliopistokoulutuksen laatuyksiköt 2007–20099:2006 Ojala, I. & Vartiainen, P.: Kolmen yliopiston opetuksen kehittämistoiminnan vaikuttavuus. Lapin

yliopiston, Lappeenrannan teknillisen yliopiston ja Vaasan yliopiston opetuksen kehittämistoiminnanvaikuttavuuden benchmarking-arviointi

10:2006 Lappalainen, M. & Luoto, L.: Opetussuunnitelmaprosessit yliopistoissa11:2006 Levänen, K., Tervonen, S., Suhonen, M. & Stigell, L.: Verkko-opintojen mitoituksen arviointi12:2006 Vuorela, P., Kallio, U., Pohjolainen, T., Sylvander, T. & Kajaste, M.; Avoimen yliopiston arvioinnin seuranta-

raportti13:2006 Käyhkö, R., Hakamäki, S., Kananen, M., Kavonius, V., Pirhonen, J., Puusaari, P., Kajaste, M. & Holm, K.:

Uudenlaista sankaruutta. Ammattikorkeakoulujen aluekehitysvaikutuksen huippuyksiköt 2006–200714:2006 Malm, K., Lavonius, H., Nystén, P., Santavirta, N. & Cornér, S.: Auditering av Svenska yrkeshögskolans

kvalitetssäkringssystem15:2006 Papp, I., Carolan, D., Handal, G., Lindesjöö, E., Marttinen, R., Mustonen, V. & Isoaho, K.: Audit of the

quality assurance system of Seinäjoki Polytechnic16:2006 Alaniska, H. (toim.): Opiskelija opetuksen laadunarvioinnissa.17:2006 Pyykkö, R., Keränen, P., Lahti, M., Mikkola, A., Paasonen, S. & Holm, K.: Media- ja viestintäalan

seuranta

1:2007 Karppanen, E., Tornikoski, E., Töytäri, R., Urponen, H., Uusitalo, T., Holm, K.: Lahden ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

2:2007 Liljander, J.-P., Heikkilä, J., Lappalainen, M., Nystén, P., Sulameri, T. & Kajaste, M.: Savonia-ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

3:2007 Wahlbin, C., Heikkilä, J., Hellberg, M., Lindroos, P., Nybom, J. & Cornér, S.: Auditering av Svenskahandelshögskolans kvalitetssäkringssystem

4:2007 Jokinen, T., Malinen, H., Mäki, M., Nokela, J., Pakkanen, P. & Kekäläinen, H.: Tampereen teknillisenyliopiston laadunvarmistusjärjestelmän auditointi

5:2007 Saari, S. (toim.): Korkeakouluopiskelija yhteiskunnallisena toimijana. Kansallinen benchmarking-arviointi

6:2007 Korkeakoulujen laadunvarmistusjärjestelmien auditointi. Uusinta-auditoinnin käsikirja 2007–2009 –Auditering av högskolornas kvalitetssäkringssystem. Handbok för förnyad auditering 2007–2009 –Audits of the quality assurance systems of higher education institutions. Manual for Re-Audits2007–2009

7:2007 Korkeakoulujen laadunvarmistusjärjestelmien auditointi. Auditointikäsikirja vuosille 2008–20118:2007 Seppälä, K., Rinne, R. & Trapp, H. (eds.): Connecting Research and Client. Finnish Experience of

Quality Enhancement in University Lifelong Learning9:2007 Auditering av högskolornas kvalitetssäkringssystem. Auditeringshandbok för åren 2008–2011


11:2007 Toikka, M., Aarrevaara, T., Isotalo, J., Peltokangas, N., Raij, K., Hiltunen, K. & Holm, K.: Kajaaninammattikorkeakoulun laadunvarmistusjärjestelmän auditointi

1:2008 Ståhle, P., Karppanen, E., Kiiskinen, N., Okkonen, T., Saxén, H., Uusi-Rauva, E., Holm, K.& Seppälä, H.:Teknillisen korkeakoulun laadunvarmistusjärjestelmän auditointi

2:2008 Vuorio, E., Huttula, T., Kukkonen, J., Kurtakko, K., Malm, K., Mikkola, A., Mäki, M., Rekilä, E., Yanar, A.,Kekäläinen, H., Moitus, S. & Mustonen, K.: Helsingin yliopiston laadunvarmistusjärjestelmän auditointi

3:2008 Aaltonen, E., Anoschkin, E., Jäppinen, M., Kotiranta, T., Wrede, G. H. & Hiltunen, K.: Sosiaalityön jasosiaalialan koulutuksen nykytila ja kehittämishaasteet – Yliopistojen sosiaalityön ja ammatti-korkeakoulujen sosiaalialan koulutuksen seuranta-arviointi

5:2006 Rauhala, P., Kotila, H., Linko, L., Mulari, O., Rautonen, M. & Moitus, S.; Keski-Pohjanmaan ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

6:2006 Hämäläinen, K., Kantola, I., Marttinen, R., Meriläinen, M., Mäki, M. & Isoaho, K.: Jyväskylän ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

7:2006 Kekäläinen, H.: (toim.)Neljä aikuiskoulutuksen laatuyliopistoa 2007–20098:2006 Yliopistokoulutuksen laatuyksiköt 2007–20099:2006 Ojala, I. & Vartiainen, P.: Kolmen yliopiston opetuksen kehittämistoiminnan vaikuttavuus. Lapin

yliopiston, Lappeenrannan teknillisen yliopiston ja Vaasan yliopiston opetuksen kehittämistoiminnanvaikuttavuuden benchmarking-arviointi

10:2006 Lappalainen, M. & Luoto, L.: Opetussuunnitelmaprosessit yliopistoissa11:2006 Levänen, K., Tervonen, S., Suhonen, M. & Stigell, L.: Verkko-opintojen mitoituksen arviointi12:2006 Vuorela, P., Kallio, U., Pohjolainen, T., Sylvander, T. & Kajaste, M.; Avoimen yliopiston arvioinnin seuranta-

raportti13:2006 Käyhkö, R., Hakamäki, S., Kananen, M., Kavonius, V., Pirhonen, J., Puusaari, P., Kajaste, M. & Holm, K.:

Uudenlaista sankaruutta. Ammattikorkeakoulujen aluekehitysvaikutuksen huippuyksiköt 2006–200714:2006 Malm, K., Lavonius, H., Nystén, P., Santavirta, N. & Cornér, S.: Auditering av Svenska yrkeshögskolans

kvalitetssäkringssystem15:2006 Papp, I., Carolan, D., Handal, G., Lindesjöö, E., Marttinen, R., Mustonen, V. & Isoaho, K.: Audit of the

quality assurance system of Seinäjoki Polytechnic16:2006 Alaniska, H. (toim.): Opiskelija opetuksen laadunarvioinnissa.17:2006 Pyykkö, R., Keränen, P., Lahti, M., Mikkola, A., Paasonen, S. & Holm, K.: Media- ja viestintäalan

seuranta

1:2007 Karppanen, E., Tornikoski, E., Töytäri, R., Urponen, H., Uusitalo, T., Holm, K.: Lahden ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

2:2007 Liljander, J.-P., Heikkilä, J., Lappalainen, M., Nystén, P., Sulameri, T. & Kajaste, M.: Savonia-ammatti-korkeakoulun laadunvarmistusjärjestelmän auditointi

3:2007 Wahlbin, C., Heikkilä, J., Hellberg, M., Lindroos, P., Nybom, J. & Cornér, S.: Auditering av Svenskahandelshögskolans kvalitetssäkringssystem

4:2007 Jokinen, T., Malinen, H., Mäki, M., Nokela, J., Pakkanen, P. & Kekäläinen, H.: Tampereen teknillisenyliopiston laadunvarmistusjärjestelmän auditointi

5:2007 Saari, S. (toim.): Korkeakouluopiskelija yhteiskunnallisena toimijana. Kansallinen benchmarking-arviointi

6:2007 Korkeakoulujen laadunvarmistusjärjestelmien auditointi. Uusinta-auditoinnin käsikirja 2007–2009 –Auditering av högskolornas kvalitetssäkringssystem. Handbok för förnyad auditering 2007–2009 –Audits of the quality assurance systems of higher education institutions. Manual for Re-Audits2007–2009

7:2007 Korkeakoulujen laadunvarmistusjärjestelmien auditointi. Auditointikäsikirja vuosille 2008–20118:2007 Seppälä, K., Rinne, R. & Trapp, H. (eds.): Connecting Research and Client. Finnish Experience of

Quality Enhancement in University Lifelong Learning9:2007 Auditering av högskolornas kvalitetssäkringssystem. Auditeringshandbok för åren 2008–2011


11:2007 Toikka, M., Aarrevaara, T., Isotalo, J., Peltokangas, N., Raij, K., Hiltunen, K. & Holm, K.: Kajaaninammattikorkeakoulun laadunvarmistusjärjestelmän auditointi

1:2008 Ståhle, P., Karppanen, E., Kiiskinen, N., Okkonen, T., Saxén, H., Uusi-Rauva, E., Holm, K.& Seppälä, H.:Teknillisen korkeakoulun laadunvarmistusjärjestelmän auditointi

2:2008 Vuorio, E., Huttula, T., Kukkonen, J., Kurtakko, K., Malm, K., Mikkola, A., Mäki, M., Rekilä, E., Yanar, A.,Kekäläinen, H., Moitus, S. & Mustonen, K.: Helsingin yliopiston laadunvarmistusjärjestelmän auditointi

3:2008 Aaltonen, E., Anoschkin, E., Jäppinen, M., Kotiranta, T., Wrede, G. H. & Hiltunen, K.: Sosiaalityön jasosiaalialan koulutuksen nykytila ja kehittämishaasteet – Yliopistojen sosiaalityön ja ammatti-korkeakoulujen sosiaalialan koulutuksen seuranta-arviointi

4:2008 Leppisaari, I., Ihanainen, P., Nevgi, A., Taskila, V.-M., Tuominen, T. & Saari, S.: Hyvässä kasvussa –Yhdessä kehittäen kohti ammattikorkeakoulujen laadukasta verkko-opetusta

5:2008 Hiltunen, K. & Kekäläinen, H.: Benchmarking korkeakoulujen laadunvarmistusjärjestelmien kehittämi-sessä – Laadunvarmistusjärjestelmien benchmarking-hankkeen loppuraportti

6:2008 Rauhala, P., Liljander, J.-P., Mulari, O. & Moitus, S.: Keski-Pohjanmaan ammattikorkeakoulun laadun-varmistusjärjestelmän uusinta-auditointi

7:2008 Korkeakoulujen arviointineuvoston toimintasuunnitelma 2008–20098:2008 Hintsanen, V., Höynälänmaa, M., Järvinen, M.-R., Karjalainen, A., Peltokangas, N. & Hiltunen, K.: Vaasan

ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi9:2008 Rekilä, E., Heikkilä, J., Kääpä, P., Seppälä, M., Virtanen, T., Öberg, J., Moitus, S. & Mustonen, K.: Tampe-

reen yliopiston laadunvarmistusjärjestelmän auditointi10:2008 Luoma, M., Daniel, H.D., Kristensen, B., Pirttilä, A., Vaisto, L., Wahlén, S., Mustonen, K. & Seppälä, H.:

Audit of the quality assurance system of Helsinki School of Economics11:2008 Stenius, M. Ansala, L., Heino, J., Käyhkö, R., Lempa, H., Niemelä, J., Holm, K. & Seppälä, H.: Turun

yliopiston laadunvarmistusjärjestelmän auditointi

1:2009 Helander, E., Ahola, J., Huttunen, J., Lahtinen, M., Okko, P., Suomalainen, H., Virtanen, I., Holm, K. &Mustonen, K.: Lisää yhteistyötä alueiden parhaaksi. Yliopistokeskusten arviointi

2:2009 Saarela, M., Jaatinen, P., Juntunen, K., Kauppi, A., Otala, L., Taskila, V.-M., Holm, K. & Kajaste, M.:Ammattikorkeakoulujen koulutuksen laatuyksiköt 2008–2009

3:2009 Hiltunen, K. (ed.): Centres of Excellence in Finnish University Education 2010–20124:2009 Harmaakorpi, V., Furu, P., Takala, M., Tenhunen, M.-L., Westersund, C. & Holm, K.: Turun kauppakorkea-

koulun laadunvarmistusjärjestelmän auditointi5:2009 Pirttilä, A., Keränen, P., Pirnes, H., Tiilikka, A.-M., Virtanen, A. & Seppälä, H.: Tampereen ammattikorkea-

koulun laadunvarmistusjärjestelmän auditointi6:2009 Malinen, H., Hallikainen, J., Karttunen, P., Majander, M., Pudas, M. & Mustonen, K.: Satakunnan

ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi7:2009 Suntioinen, S., Myller, E., Nieminen, P., Pohjolainen, S., Wahlgrén, A., Kajaste, M. & Moitus, S.: Lappeen-

rannan teknillisen yliopiston laadunvarmistusjärjestelmän auditointi8:2009 Urponen, H., Kinnunen, J., Levä, K., Nieminen, R., Raij, K., Seppälä, M. & Hiltunen, K.: Jyväskylän

yliopiston laadunvarmistusjärjestelmän auditointi9:2009 Papp, I., Lindesjöö, E., Töytäri, R. & Seppälä, H.: Re-audit of the Quality Assurance System of the

Seinäjoki University of Applied Sciences10:2009 Kantola, I., Keto, U. & Nykänen, M: Avaimia arvioinnin tehokkaampaan hyödyntämiseen – Turun ja

Mikkelin ammattikorkeakoulujen benchmarking11:2009 Heikkilä, J., Lappalainen, M., Mulari, O. & Kajaste, M: Savonia-ammattikorkeakoulun laadunvarmistus-

järjestelmän uusinta-auditointi12:2009 Hulkko, P., Virtanen, A., Lampelo, S., Teckenberg, T., Vieltojärvi, M., Saarilammi, M.-L. & Mustonen, K.:

Diakonia-ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi13:2009 Hiltunen, A.-M, Uusitalo, E., Hietanen, O., Hyyryläinen, T., Kettunen, S. & Söderlund, S.: Dynaaminen

laatunäkemys – kolmen yliopistoverkoston kehittävä vertaisarviointi14:2009 Moitus, S.: Analyysi korkeakoulujen laadunvarmistusjärjestelmien auditointien tuloksista vuosilta

2005–200815:2009 Järvinen, M.-R., Granö, P., Huhtamo, E., Kettunen, A., Laaksonen, E., Holm, K. & Holopainen, H.:

Taideteollisen korkeakoulun laadunvarmistusjärjestelmän auditointi16:2009 Andersson, Ö., Cornér, S., Heikkilä, J., Huldin, H.,Lejonqvist, G.-B. & Lundin, K.: Auditering av kvalitets-

säkringssystemet vid Högskolan på Åland17:2009 Antikainen, E.-L., Eskelinen, H., Mäki, M., Nykänen, M., Taskila, V.-M. & Mustonen, K.: Rovaniemen

ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi18:2009 Aarrevaara, T., Toikka, M., Apajalahti, H., Huttula, T., Mäkilä, M., Kajaste, M. & Saarilammi, M.-L: Lapin

yliopiston laadunvarmistusjärjestelmän auditointi

1:2010 Auvinen, P., Kauppi, A., Kotila, H., Loikkanen, A., Markus, A., Holm, K. & Kajaste, M.: Ammatti-korkeakoulujen koulutuksen laatuyksiköt 2010–2012

2:2010 Hintsanen, V., Luukka, M.-R.,Lounasmeri, T., Majander, M., Renvall, J., Holopainen, H. & Hiltunen, K.:Turun ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi

3:2010 External Review of Finnish Higher Education Evaluation Council. Self-evaluation report4:2010 Lundqvist, R., Löfström, E., Hokkanen, A., Lindesjöö, E., Westermarck, C.-M., Raaheim, A. & Lundin, K.:

Auditering av kvalitetssäkringssystemet vid Åbo Akademi5:2010 Korkeakoulujen arviointineuvoston toimintakertomus toimikaudelta 2008–20096:2010 Okko, P., Pirttilä, A., Ansala, L., Immonen, H., Uusitalo, T. & Saarilammi, M.-L.: Oulun yliopiston laadun-

varmistusjärjestelmän auditointi7:2010 Virtanen, T., Ahonen, H., Ahonen, H., Koski, P., Lähteenmäki, J. & Mustonen, K.: Teatterikorkeakoulun

laadunvarmistusjärjestelmän auditointi8:2010 Korkeakoulujen arviointineuvoston toimintasuunnitelma 2010–20139:2010 Rådet för utvärdering av högskolorna: Verksamhetsplan 2010–2013

10:2010 Finnish Higher Education Evaluation Council: Plan of action 2010–201311:2010 Karppanen, E., Kiiskinen, N., Urponen, H., Uusi-Rauva, E., Holm, K. & Mattila, J.: Teknillisen korkea-

koulun laadunvarmistusjärjestelmän uusinta-auditointi12:2010 Varmola, T., Granö, P., Hyvönen, U., Klemettinen, T., Lippus, U., Salo, T., Mattila, J., Seppälä, H.: Sibelius-

Akatemian laadunvarmistusjärjestelmän auditointi13:2010 Virtanen, A., Aaltonen, M., Markus, A., Oresto, J., Rytkönen, P. & Saarilammi, M.-L.: HAAGA-HELIA

ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi14:2010 Lähdeniemi, M., Hulkko, P., Lappalainen, A., Mäkitalo, J., Suviranta, L. & Mustonen, K.: Kemi-Tornion

ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi15:2010 Moitus, S.: Analysis on FINHEEC Audit Outcomes 2005–200816:2010 Korkeakoulujen laatujärjestelmien auditointikäsikirja vuosiksi 2011–201717:2010 Niemelä, J., Ahola, S., Blomqvist, C., Juusola, H., Karjalainen, M., Liljander, J.-P., Mielityinen, I.,

Oikarinen, K., Moitus, S. & Mattila, J.: Tutkinnonuudistuksen arviointi 201018:2010 Lampelo, S., Kainulainen, S., Turunen, J., Viljanen, J., Yanar, A., Mattila, J. & Saarilammi, M.-L.:

Laurea-ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi

1:2011 Tornikoski, E., Korhonen, K., Okkonen, E., Rantakangas, T.-M., Tarkkanen, J., Holm, K. & Mattila, J.:Saimaan ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi

2:2011 Okko, P., Immonen, H., Kolehmainen, S., Levä, K., Seppälä, M., Kajaste, M. & Mustonen, K.:Maanpuolustuskorkeakoulun laadunvarmistusjärjestelmän auditointi

3:2011 Audit manual for the quality systems of Finnish higher education institutions 2011–20174:2011 Auditeringshandbok för högskolornas kvalitetssystem för åren 2011–20175:2011 Aarrevaara, T., Aaltonen, M., Ansala, L., Huttunen, J., Ryynänen-Karjalainen, L., Saarilammi, M.-L. &

Talvinen, K.: Itä-Suomen yliopiston laadunvarmistusjärjestelmän auditointi6:2011 Malinen, H., Puolanne, E., Sorvisto, M., Suomalainen, M., Takala, M., Mustonen, K. & Östman, K.:

Hämeen ammattikorkeakoulun laadunvarmistusjärjestelmän auditointi7:2011 Haapakorpi, A.: Auditointiprosessi ja sen vaikutukset yliopistossa8:2011 Ala-Vähälä, T.: Mitä auditointi tekee? Tutkimus korkeakoulujen laadunvarmistusjärjestelmien auditoin-

tien vaikutuksista