semester 2 2014/2015 quality audit – audit kualiti

SEMESTER 2 SEMESTER 2 2014/20152014/2015

QUALITY AUDIT – AUDIT KUALITIQUALITY AUDIT – AUDIT KUALITI

Audit kualiti dalam pendidikan tinggi(Quality Audit in Higher Education)

APRIL 2013

Intended Learning Outcomes – student be able to:Explain the meaning of quality auditDescribe the components / approaches of quality audit Propose suitable approaches to conduct quality audit in HEI

Quality AuditQuality Audit

Audit kualiti dalam pendidikan tinggi(Quality Audit in Higher Education)

A Quality Audit has been defined as:A systematic and independent determination of whether: the planned arrangements are suitable to achieve the objectives; the actual quality activities conform to the planned arrangements; and the arrangements are being implemented effectively.(Australian Universities Quality Agency (AUQA), 2009)

Relevant web site: http://www.auqa.edu.au


British Standards British Standards

Quality audit is a systematic and Quality audit is a systematic and independent examination to determine independent examination to determine whether quality activities and related whether quality activities and related results comply with planned arrangements results comply with planned arrangements and whether these arrangements are and whether these arrangements are implemented effectively and are suitable to implemented effectively and are suitable to achieve objectives. achieve objectives.

Quality Audit

The procedural controls that ensure all quality providers are adequately following

the required procedures as prescribed.


Dimensions of evaluation

AUQA methods of Evaluation: Evaluates institution quality assurance arrangements:ApproachDeployment ResultsImprovement

Does not impose an externally prescribed set of standards upon auditees, but uses it as primary starting point for audit of organization’s own objectives.


AUQA AUQA Objectives

AUQA has several objectives:Arrange and manage a system of periodic audits of the quality of academic activities, the quality assurance arrangements, institutional compliance to criteria set out in the National Protocols for Higher Education Approval Processes. Monitor, review, analyze and provide public reports on the quality of outcomes in Australian universities and higher education institutions.Arrange and manage a system of periodic audits of the quality assurance processes, procedures, and outcomes.Publicly report periodically on matters relating to quality assurance.


Approach

The trail from an organization’s mission, vision and values (the overall objectives). Audit questions include:What is this organization about?What outcome is it trying to achieve?What, if any, reference points are used in establishing the organization’s objectives? How does the organization plan to achieve its objectives?Does it understand its context and capabilities?Are the organization’s objectives set against appropriate benchmark?What risk management processes does it have in place?Is the approach aligned and communicated throughout the organization and more widely?


Deployment

The deployment dimension considers whether and how effectively, the approach is being put into effect. Audit questions include:Is the approach being deployed in the best possible manner? According to whom?What standards and benchmarks is the organization using to assess this?If the approach is not being deployed, why not, and how is this managed?Are staff appropriately trained and resources appropriately deployed, to fulfill the approach?


Results

The result dimension looks at results as a means of determining how ell the deployment is achieving the planned approach. Audit questions include:Is the organization achieving its intended objectives and outcomes?Does the organization understand why and how it achieved those particular results?How are the results reported and used within the organization?


Improvement

The improvement dimension focuses on whether the organization is actively and continuously engaged with understanding its performance in each of the A-D-R dimensions, and is using this understanding to bring about improvements . Audit questions include:Does the organization know how it can improve?How does it know this (e.g. through the use of external benchmarks)?How is it acting upon this knowledge?Does the organization have a sustained history of improvement?


Operating Quality Audit

Pre-defined standards are determined Pre-defined standards are determined Quality requirements for specific work and Quality requirements for specific work and implementation will be identified in advance implementation will be identified in advance Specific procedures have to be followed at all Specific procedures have to be followed at all stages of implementationstages of implementationQuality Methods must be defined and followed Quality Methods must be defined and followed Completed work and process implementation Completed work and process implementation must be reviewed for compliance. must be reviewed for compliance.


The Practice in MalaysiaThe Practice in MalaysiaMQA – Malaysian Qualifications AgencyMQA – Malaysian Qualifications Agency

MQA established under the Malaysian Qualifications MQA established under the Malaysian Qualifications Agency Act 2007. It was the merging of LAN (Lembaga Agency Act 2007. It was the merging of LAN (Lembaga Akreditasi Negara) and QAD (Quality Assurance Division Akreditasi Negara) and QAD (Quality Assurance Division of the Ministry of Higher Education).of the Ministry of Higher Education).

MQF (Malaysian Qualifications Framework) was MQF (Malaysian Qualifications Framework) was developed to unify and harmonize all Malaysian developed to unify and harmonize all Malaysian qualifications.qualifications.

Code of Practice for Program Accreditation (COPPA) Code of Practice for Program Accreditation (COPPA) and Code of Practice for Institutional Audit (COPIA) are and Code of Practice for Institutional Audit (COPIA) are guidelines, standards and code of practice to ensure guidelines, standards and code of practice to ensure quality in the academic performance and institutional quality in the academic performance and institutional effectiveness.effectiveness.

The codes contain an overview of the Malaysian quality The codes contain an overview of the Malaysian quality assurance system for higher education. assurance system for higher education.


MQA – Malaysian Qualifications AgencyMQA – Malaysian Qualifications Agency definitionsdefinitions

Institutional Audit – an evaluation of an institution to – an evaluation of an institution to determine whether it is achieving its mission and goals, to determine whether it is achieving its mission and goals, to identify strengths and areas of concern, and to enhance identify strengths and areas of concern, and to enhance quality.quality.

Internal Quality Audit – an evaluation conducted – an evaluation conducted internally by a higher education provider (e.g. the internally by a higher education provider (e.g. the university) to determine whether it is achieving its mission university) to determine whether it is achieving its mission and goals; to identify strengths and areas of concern, and and goals; to identify strengths and areas of concern, and to enhance quality. It generates a Self-Review Portfolio.to enhance quality. It generates a Self-Review Portfolio.

Program Accreditation – an exercise to accredit the – an exercise to accredit the teaching, learning and all other related activities to a teaching, learning and all other related activities to a program provided by a higher education provider. program provided by a higher education provider.


MQA – Malaysian Qualifications Agency definitions

Quality Assurance – comprises all those planned and Quality Assurance – comprises all those planned and systematic actions (policies, strategies, attitudes, systematic actions (policies, strategies, attitudes, procedures and activities) necessary to provide procedures and activities) necessary to provide adequate demonstration that quality is being maintained adequate demonstration that quality is being maintained and enhanced, and the products and services meet the and enhanced, and the products and services meet the specified quality standards. In higher education, quality specified quality standards. In higher education, quality assurance is the totality of systems, resources and assurance is the totality of systems, resources and information devoted to maintaining and improving the information devoted to maintaining and improving the quality and standards of teaching, scholarship and quality and standards of teaching, scholarship and research as well as students’ learning experience.research as well as students’ learning experience.


MQA – 7 responsibilities

To implement the MQF as a reference point for the To implement the MQF as a reference point for the Malaysian qualifications.Malaysian qualifications.

To develop standards and criteria and all other relevant To develop standards and criteria and all other relevant instruments as a national references for the conferment of instruments as a national references for the conferment of awards with the co-operations of stakeholders.awards with the co-operations of stakeholders.

To quality assure higher education providers and To quality assure higher education providers and programs.programs.

To accredit programs that fulfill a set of criteria and To accredit programs that fulfill a set of criteria and standards.standards.

To facilitate the recognition and articulation of qualificationsTo facilitate the recognition and articulation of qualifications To establish and maintain the Malaysian Qualifications To establish and maintain the Malaysian Qualifications

Register (MQR).Register (MQR). To advise the Minister on matters relating to quality To advise the Minister on matters relating to quality

assurance in higher education. assurance in higher education.


MQA Council

MQA is headed by a Council comprising a Chairman and 16 MQA is headed by a Council comprising a Chairman and 16 members. The functions of the Council are to:members. The functions of the Council are to:

Approve plans and policies for the management of the Approve plans and policies for the management of the Agency.Agency.

Approve any amendment or update of the MQF.Approve any amendment or update of the MQF. Approve policies and guidelines relating to audit processes Approve policies and guidelines relating to audit processes

and accreditation of programs, qualifications and higher and accreditation of programs, qualifications and higher education providers.education providers.

Receive and monitor reports and other information relating Receive and monitor reports and other information relating to accreditation, institutional audit and evaluation.to accreditation, institutional audit and evaluation.

Continuously guide the Agency in its function as a quality Continuously guide the Agency in its function as a quality assurance body.assurance body.


Malaysian Qualifications Framework - MQF

The MQF is an instrument that classifies qualifications The MQF is an instrument that classifies qualifications based on a set of criteria that are approved nationally and based on a set of criteria that are approved nationally and benchmarked against international best practices. benchmarked against international best practices.

The framework clarifies the academic levels, learning The framework clarifies the academic levels, learning outcomes and credit systems based on student academic outcomes and credit systems based on student academic load.load.

The MQF integrates with and links all national The MQF integrates with and links all national qualifications.qualifications.

3 types of quality assurance: 3 types of quality assurance: Provisional Accreditation; Full Accreditation; and Institutional Audit.


COPIA, COPPA, HEP, MQA, MQF

COPIACOPIA – Code of Practice for Institutional – Code of Practice for Institutional AuditAudit

COPPACOPPA – Code of Practice for Programme – Code of Practice for Programme AccreditationAccreditation

HEP HEP – Higher Education Provider / Pemberi – Higher Education Provider / Pemberi Pendidikan TinggiPendidikan Tinggi

MQAMQA – Malaysian Qualification Agency / – Malaysian Qualification Agency / Agensi Kelayakan MalaysiaAgensi Kelayakan Malaysia

MQFMQF – Malaysian Qualifications Framework / – Malaysian Qualifications Framework / Kerangka Kelayakan Malaysia (KKM) Kerangka Kelayakan Malaysia (KKM)


Code of Practice for Institutional Audit (COPIA)

Contains guidelines on criteria & Standards for Higher Education Provider (HEP).

Define requirements for higher education in broad terms. Divided into 9 areas of evaluation. Standards are categorized into 2 attainment levels: Benchmark – expressed as a “must”. Enhanced – expressed as a “should” Purpose: multipurpose audit code , comprehensive. Guided information template. Explains preparations for audits, roles of audit panels, MQA Explains processes, responsibilities, audit criteria, report

writing formats.


Pendidikan tinggi merupakan pemangkin utama penjanaan sumber tenaga manusia.

Pelan Strategik Pendidikan Tinggi Negara menggariskan 7 tuntutan cabaran pendidikan tinggi:Peranan universiti dan ahli akademikPerkembangan kurikulum mengikut keperluan pasaranPenyelidikan, pembangunan dan pengkomersialan dalam sistem inovasi kebangsaanKaedah pengajaran dan pembelajaranGlobalisasi dan piawaian melalui pemeringkatan dan penarafanPeluang guna tenaga


Senario pengajian tinggi negara

Findings and Judgments of COPIA

For Academic Performance Audit (APA): highlight affirmation, commendation and areas of concern. Indicate performance of institution on 9 areas of evaluation. No specific decision of pass or fail. For continuation or cessation of Program Accreditation: Maintain accreditation Cease accreditation For Self-accreditation Status: Grant Self Accreditation Status Not granted Self Accreditation status MQA Institutional Audit Committee will make decisions base

on proposal by the panel of assessors.


Malaysian Qualifications Framework - MQF

Provisional AccreditationProvisional Accreditation – seeking approval from the – seeking approval from the Ministry of Higher Education (MOHE) to conduct new Ministry of Higher Education (MOHE) to conduct new programs.programs.

Full AccreditationFull Accreditation – a conferment to denote that a program – a conferment to denote that a program has met all the criteria and standards set for that purpose.has met all the criteria and standards set for that purpose.

Institutional AuditInstitutional Audit – periodic academic performance audit – periodic academic performance audit on all universities or to establish the continuation or on all universities or to establish the continuation or maintenance of program accreditation status. It could take maintenance of program accreditation status. It could take the form of an exercise for purposes of verifying data, the form of an exercise for purposes of verifying data, public policy input or for rating and ranking of institutions public policy input or for rating and ranking of institutions and programs. The highest form of IA is the self-and programs. The highest form of IA is the self-accreditation audit which can lead to a conferment of a accreditation audit which can lead to a conferment of a self-accreditation status for the institution whereby the self-accreditation status for the institution whereby the institution can accredit its own programs.institution can accredit its own programs.


9 Areas of evaluation COPIA

Vision, mission, educational goals and learning outcomesVision, mission, educational goals and learning outcomes Curriculum design and deliveryCurriculum design and delivery Assessment of studentsAssessment of students Student selection and support servicesStudent selection and support services Academic staffAcademic staff Educational resourcesEducational resources Program monitoring and reviewProgram monitoring and review Leadership, governance and administrationLeadership, governance and administration Total continual quality improvement Total continual quality improvement


2 levels of standards attainment 2 levels of standards attainment

Benchmark standards – are standards that must be met – are standards that must be met and compliance demonstrated during institutional audit. and compliance demonstrated during institutional audit. Benchmark standards are expressed as a “MUST”.Benchmark standards are expressed as a “MUST”.

Enhanced standardsEnhanced standards – are standards that should be met – are standards that should be met as the institution strives to continuously improve itself. as the institution strives to continuously improve itself. Enhanced standards reflect international and national Enhanced standards reflect international and national consensus on good practices in higher education. consensus on good practices in higher education. Enhanced standards are expressed by a “SHOULD”. Enhanced standards are expressed by a “SHOULD”.

The standards define the expected level of attainment for The standards define the expected level of attainment for each criterion and serve as a performance indicator.each criterion and serve as a performance indicator.


9 Areas of evaluation – Benchmark standards

Statements of Vision, Mission, and Educational goals: Set the direction, guide academic planning and Set the direction, guide academic planning and

implementation, and bring members towards achievement implementation, and bring members towards achievement of world class status and tradition of excellence.of world class status and tradition of excellence.

HEP HEP must state vision, mission and educational goals and state vision, mission and educational goals and make known to internal and external stakeholders.make known to internal and external stakeholders.

Mission statements and goals Mission statements and goals must reflect crucial reflect crucial elements of the processes and outcomes of higher elements of the processes and outcomes of higher education.education.

Must formulate goals consistent with vision and mission. formulate goals consistent with vision and mission. Publications Publications must accurately cite the current vision and accurately cite the current vision and

mission.mission.


9 Areas of evaluation – Enhanced standards

Statements of Vision, Mission, and Educational goals: Mission and goals Mission and goals should encompass leadership qualities encompass leadership qualities

in the areas of social responsibility, research attainment, in the areas of social responsibility, research attainment, community involvement, ethical values, profesionalism and community involvement, ethical values, profesionalism and knowledge creation.knowledge creation.

Should demonstrate that planning and evaluation demonstrate that planning and evaluation processes, educational programs, educational support processes, educational programs, educational support services, financial and physical resources, administrative services, financial and physical resources, administrative processes are adequate and appropriate to fulfill stated processes are adequate and appropriate to fulfill stated purpose of mission and goals.purpose of mission and goals.



Curriculum Design and Delivery: Must have a clearly defined process by which the

curriculum is established, reviewed and evaluated. All Educational programs must be considered only after a

needs assessment have been identified. All Educational programs must be considered only after

resources to support the programs have been identified. All aspects of educational programs must be related to

HEP’s vision and mission. Must show that educational content and approach,

teaching learning methods are appropriate, consistent with and support the attainment of learning outcomes.

Teaching learning methods must ensure that students take responsibility for their own learning and prepare for life long learning.



Curriculum design and delivery: The curriculum The curriculum should encourage multi-disciplinary encourage multi-disciplinary

approaches to enhance personal development through approaches to enhance personal development through electives, study pathways and other means which should electives, study pathways and other means which should be monitored and appraisedbe monitored and appraised

There There should be co-curricular activities that will enrich be co-curricular activities that will enrich students’ experiences, foster personal development and students’ experiences, foster personal development and responsibility.responsibility.



Assessment of students: Assessment and Learning Assessment principles, processes, methods and practices

must be aligned with learning outcomes and program content.

The process and methods of assessment must reflect the change to the program outcomes from any review exercise.

The assessment must be consistent with the levels defined in the MQF and the MQF eight domains of learning outcomes.



Assessment of students: Assessment and Learning Assessment methodology Assessment methodology should be reviewed periodically be reviewed periodically

to ensure currency with development in best practices. to ensure currency with development in best practices.



Assessment of students: Assessment methods Frequency and methods of student assessment including

grading criteria and award must be documented and communicated to students.

The assessment methods must comprise of summative and formative purposes compatible with learning outcomes.

A variety of methods and tools must be used appropriately for assessing the given learning outcomes and competencies (communication, problem solving, teamwork and self-directed learning).

There must be mechanisms to ensure the validity, reliability and fairness of the assessment system.



Assessment of students: Assessment methods Methods of assessing Methods of assessing should be comparable to be comparable to

international best practices.international best practices. The assessment systems The assessment systems should beshould be reviewed periodically. reviewed periodically.



Student selection and support services: admission and selection – 13 standards (here are five examples)

The HEP must have an admission policy with clear statements on the criteria and processes of student selection including transferring students.

The criteria and processes of selection must be published and disseminated to students.

If a selection interview is used, the HEP must demonstrate that it is objectively and fairly structured.

Student selection must be free from discrimination and bias. The HEP must provide evidence that it selects students

whose capabilities are consistent with the admission policies.



Student selection and support services: admission and selection

The admission policy should be reviewed periodically. The outcome of selection and student performance should

be monitored to improve the selection process. The size of student intake should be reviewed in

consultation with stakeholders. Students intake should include a consideration of gender

and ethnic balance and social responsibilities such as policies for disadvantaged students.


The Practice: Academic Performance Audit – Audit Prestasi Akademik

Effectiveness of the internal management systems for quality by which institutions manage their core functions with resources and manpower available.

Strength & areas of concern in the academic performance of the universities.

Improvements needed in the areas of concern. Action plans by institutions. Institutions demonstrate that they have systems and

resources in place for institutional effectiveness.


The Practice: Academic Performance Audit – Audit Prestasi Akademik

The audit is in the form of an external quality assurance in which institutions perform self evaluation against standards and criteria.

The quality assurance standards and criteria are intended to help develop, enhance, and strengthen the institutional effectiveness.

The results do not “pass” or “fail”. Reports are not intended to penalize or reward institutions. Information will give trends, differences between institutions,

indication of impact of policy implementation, constraints and expectations.


Some Comparisons between COPIA & COPPA

Area 1: Vision, Mission, Educational Goals & Learning Outcomes

1.1 Statements of vision, mission and educational goals: COPIA – sets the direction of the HEP and guides

educational planning & implementation. COPPA – at the program level, it is more directed to see

how a specific program supports the larger institutional vision.




1.3 Academic Autonomy: COPIA – specifies the autonomy of HEP over academic

matters. COPPA – autonomy in designing the curriculum of a specific

program.




1.4 Learning Outcomes (LO): COPIA – relates LO and competencies to vision and

mission of HEP. COPPA – emphasis on learning outcomes of a particular

program.



Area 2: Curriculum Design and Delivery 2.1 Curriculum Design and Teaching Learning Methods: COPIA – perspective of curriculum design is largely about

institutional policies, structures, processes and practices related to curriculum development across the institution.

COPPA – refers to specific description, content and delivery of a particular program.


Higher education as catalyst for human capital development

National Higher Education Strategic Plan:Academics and University RolesCurriculum development in line with current needs Research, development and commercialization Teaching and learning methodologyGlobalization and standardizationOpportunities for human resources utilization


Higher education scenario

Universiti awam dikategorikan kepada tiga kumpulan iaitu universiti penyelidikan, universiti berfokus (teknikal, pendidikan, pengurusan dan pertahanan), universiti komprehensif.

Universiti penyelidikan:Ciri-ciri – bidang pengajian fokus kepada penyelidikan, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 50:50.UMUSMUPMUKMUTM



Public universities into several categories: APEX university, Research universities, Comprehensive universities, Focus universitiesResearch universities:Focus on research, intake competitive, quality academics, ratio undergrad and postgrad 50:50.UMUSMUPMUKMUTM



Universiti komprehensif:Ciri-ciri – pelbagai bidang pengajian, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 70:30.UiTMUIAMUMSUnimas



Comprehensive universities:Diversified of program of studies, quality academics, intake competitive, ratio undergrad postgrad 70:30.UiTMUIAMUMSUnimas



Universiti berfokus: Ciri-ciri – bidang pengajian berfokus, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 70:30.UUMUPSIUTHMUTeMUMPUMTUDMUMKUPNM



Focus universities

Focus program of studies, quality academics, intake competitive, ratio undergrad postgrad 70:30.UUMUPSIUTHMUTeMUMPUMTUniszaUMKUPNM



Public universitiesPublic universities Private universitiesPrivate universities Branch campus universities – nottinghamBranch campus universities – nottingham Universities wakaf / Fund By Foundation– Universities wakaf / Fund By Foundation–

Albukhari University Albukhari University kolej universitykolej university


Area 2: Curriculum Design and Delivery 2.2 Curriculum Content and Structure: COPIA – relates to HEP’s processes in developing all

program content in general. COPPA – focuses on the processes of developing one

particular program.



Area 3: Assessment of Students 3.3 Management of Student Assessment: COPIA – looks at the HEP’s students assessment system. COPPA – how is the students assessment system

implemented for a specific program.



Area 4: Student Selection and Support Services 4.1 Admission and Selection: COPIA – how the institution comply to the MOHE admission

and selection policies. COPPA – how are the policies cascaded down (diturunkan)

and implemented at the departmental level for a particular program.



Area 4: Student Selection and Support Services

4.5 Alumni:COPIA – incorporating alumni into curriculum

development of HEP.COPPA – involvement of alumni in

developing a specific program



Area 5: Academic StaffCOPIA – looks at HEP policies on academic

staff recruitment, appointment and promotion, and academic staff development.

COPPA – looks at policies and requirements for academic staff involved in teaching a specific program.


Conclusion

Quality Audit In Quality Audit it is expected that all

procedural controls are adequately followed according to the prescribed procedures.

The impact of Quality Audit will be across all parts of the quality cycles.


semester 2 2014/2015 quality audit – audit kualiti

Documents

quality audit approach

quality of outcomes

quality providers

audit of organizations

audit questions

quality assurance processes

quality of academic

actual quality activities