2 dius_self-assessment methodology
TRANSCRIPT
-
7/29/2019 2 DIUS_self-Assessment Methodology
1/18
DIUS: self-assessmentmethodology
Susanna Karakhanyan, PhD
DIUS National Coordinator
ANQAINQAAHE Board of Directors
World Bank Expert
DIUS FINAL CONFERENCESeptember, 2012
-
7/29/2019 2 DIUS_self-Assessment Methodology
2/18
OUTLINE
How it all started: getting to commonunderstanding
Description of processes run by the universities
Mapping a process: identification of gaps
Organization of self-assessment: major steps
Data collection: major sources of data
Production of a self-assessment report.
3/30/2013 DIUS FINAL CONFERENCE2
-
7/29/2019 2 DIUS_self-Assessment Methodology
3/18
How it all started: getting tocommon understanding
Need to establish a common understanding on suchconcepts as quality, quality assurance, self-evaluation and the like;
Quality assurance as fitness for purpose; Interpretation of ESG standards;
Needs analysis per ESG standard;
Allocation of roles among the partners.
Two ESGs per institution; ANQA - external quality assurance;
EU partnersinternational peers in an expert panel;
MoESobservers and evaluators.
3/30/2013 DIUS FINAL CONFERENCE3
-
7/29/2019 2 DIUS_self-Assessment Methodology
4/18
Description of processes run by university
3/30/2013 4
QUALITY QUALITY
ASSURANCE
PERFORMANCE
-
7/29/2019 2 DIUS_self-Assessment Methodology
5/18
Quality management
Planning
Implementing
Evaluationand
feedbackmechanisms
Improving
3/30/2013 5Public Accreditation Agency for Higher Education
-
7/29/2019 2 DIUS_self-Assessment Methodology
6/18
Mapping a process
3/30/2013 Public Accreditation Agency for Higher Education
-
7/29/2019 2 DIUS_self-Assessment Methodology
7/18
Example: ESG 1.4
1.4 Quality assurance of teaching staff:
Institutions should have ways of satisfyingthemselves that staff involved with the teaching
of students are qualified and competent to
do so. They should be available to those
undertaking external reviews, andcommented upon in reports.
3/30/2013 DIUS FINAL CONFERENCE7
-
7/29/2019 2 DIUS_self-Assessment Methodology
8/18
Example: ESG 1.4
3/30/2013 DIUS FINAL CONFERENCE8
-
7/29/2019 2 DIUS_self-Assessment Methodology
9/18
Example: mapping teacher
assessment
3/30/2013 Public Accreditation Agency for Higher Education
-
7/29/2019 2 DIUS_self-Assessment Methodology
10/18
Organization of self-assessment: major steps
Purpose identificationWhy do we need to engage in self-assessment?
Outcomes identificationWhat results do we need to achieve
with the self-assessment?
Measurements and evaluation designWhat methodology,tools and mechanisms for measurement should we use?
Data collectionWhat type of data do we need to obtain?
Analysis and evaluation - What are the methods we are goingto apply to analyse the data?
Decision-making regarding actions to be takenHow are wegoing to use the results of the analysis in the decision-makingregarding new strategies and developments?
3/30/2013 Public Accreditation Agency for Higher Education
-
7/29/2019 2 DIUS_self-Assessment Methodology
11/18
How to put together a valid self-
assessment ?
The questions to be answered are as follows:
What indicators do we have already?
Which indicators do I need to control my process?
Which indicators are out of my influence?
Which indicators do I need for doing the work?
Which indicators do we need for improvement andenhancement?
What is the cost-benefit ratio of collecting indicator data?
3/30/2013 Public Accreditation Agency for Higher Education
-
7/29/2019 2 DIUS_self-Assessment Methodology
12/18
Data collection: major sources
of data
Data collected from the feedback mechanisms, evaluationgroups, which provide for qualitative information;
e.g. questionnaires, which are a powerful element if devised the rightway;
discussion-oriented team workquality circles, peer-reviewers(external and internal);
focus groups, individual interviews and the like;
Statistical data available at the university units and/or statisticaloffice, that is key performance indicators (KPIs) (data like studentsuccess rate, drop-out rate, in-coming out-going students,number of teaching staff, budget allocations and the like);
Performance outcomes (samples) and other evidences (thesispapers, exam questions and respective answers, samples ofpublications, observations of classes and the like).
3/30/2013 Public Accreditation Agency for Higher Education
-
7/29/2019 2 DIUS_self-Assessment Methodology
13/18
Organising a self-assessment
3/30/2013 DIUS FINAL CONFERENCE13
DATE ACTIVITY
8 months before a
planned
assessment
Appoint the leader of the assessment process Compose the
assessment team, including students dividing up the subjects to
be dealt with
The following 6
months
Each person responsible for collecting information and data
collects that informationWriting drafts of the subjects
4 months after the
start
Discussion on the drafts in the group
Second draft
About 5 months
after the start
Discussion of the 2nd draft with all faculty staff and students
during an open hearing
6 months after the
start
Edit the comments of the hearing for the final draft
8 months after the
start
Carry out an (inter-collegial) assessment
-
7/29/2019 2 DIUS_self-Assessment Methodology
14/18
Format: introduction
One page history and current profile of theinstitution;
An overview of the management andorganisational structure, including an organisationchart (or equivalent);
A brief overview of the institutions qualityassurance arrangements and systems,
The scope of the audit, The fitness of QA arrangements for the context of the
institution's charter, profile/objectives and specialcharacteristics.
3/30/2013 14
-
7/29/2019 2 DIUS_self-Assessment Methodology
15/18
Format: the maindocument
The text should be developed per criterion/standard and include
In-depth analysis of quantitative and qualitative data and factualinformation not available in other materials (such as annualreports), cross- referenced to, and within, the main document.
entry qualifications, non-completion rates, degree results,graduate employment data,stakeholder satisfaction data,performance indicators and measures employed that relate tothe scope of the audit.
Effectiveness of a particular approach;
How this information is used in quality control and assuranceprocesses within the institution, giving examples where possible.
Examples of the effectiveness in the operation of qualityassurance systems and instances of particularly good practice
SWOT analysis.
3/30/2013 15
-
7/29/2019 2 DIUS_self-Assessment Methodology
16/18
Appendices
Appendices indicate
how this information is used in quality control
and assurance processes within the institution,giving examples where possible;
examples of the effectiveness in the operationof quality assurance systems and instances ofparticularly good practice;
The referenced documents (e.g. strategic plan,yearly and other plans);
Any other attachments required by the agency.
3/30/2013 16
-
7/29/2019 2 DIUS_self-Assessment Methodology
17/18
Challenges ahead
Re- development of policies and procedures that reallywork;
Re-development of the tools: questionnaires, focus
group discussions and the like;
Wider involvement of stakeholders in the process;
Analytical approach to self-assessment rather thandescriptive;
Using the major findings in the institutional planning andenhancement;
Establishment of a quality culture at institutions.
3/30/2013 DIUS FINAL CONFERENCE17
-
7/29/2019 2 DIUS_self-Assessment Methodology
18/18
THANK YOU and QUESTIONS
3/30/2013 18
mailto:[email protected]:[email protected]