Total quality managementbased on self-assessment

Andre Vyt

PARTNERS IN QUALITY MANAGEMENT

The experience with in Flanders

PROZA exists in different versions

• Higher education• Secondary education• Vocational training for handicapped

persons

• Adult education & training centers• Health Care & Social care centers

The goal of is

• to diagnose strengths and weaknesses of all relevant aspects in the organization

• inspired by an all-encompassing and recognized framework (EFQM Excellence model)

• by the process-owners (lecturers, researchers, administrators, technicians ...) in quality teams

• together with all parties involved: supervisors, colleagues, and clients (students, qualified professionals…)

• to set up well-supported specific actions for improvement

Inspired by EFQM Excellence

Basic conditions

Results

Core processes

Core processes

Leadership

Human resources

policy

Appreciation by society

Performance of the

organization

Resources and co-operation

Student satisfaction

Policy and strategy

Personnel satisfaction

The 9 areas of attention

The content of PROZA

• It contains more than 60 focal points over the 9 areas of attention

• In a focal point 25 specific items are to be checked/answered

• The 25 items are grouped into 5 phases, according to the level of quality management

Areas and points of attention

Focal points in core processes

• Admissions coaching• Communication with students• Curriculum design• Design of course units• Implementation of the curriculum• Teaching and tutoring• Student coaching

Focal points in core processes

• Practice-based learning• Internationalisation• Testing and assessment• Graduation and alumni-oriented

activities• Postgraduate education• Public service provision• Research• Development and practice of arts

The 5 phases of development

• Quality is variableQuality is linked to the individual

• Initiation of process-based thinking/actingStarting to work systematically

• (High level of) ProfessionalismQuality is guaranteed

• Systematic innovationContinuous improvement

• Aiming for excellenceRecognition of expertise and high level of quality

The PDCA-cycle of QM (Deming)

quality

plan

do

act

check

time

improvementassurance

evaluate - control

The higher the level attained,

the better the

• Formulation and management of processesMeasurements and information gathering

• Systematisation and reliability Openness and transparency

• Attunement and integration Innovation and improvement

• Involvement of different persons Client and external orientation

1: Quality linked to the individual

• Quality is the result of individual effort

• Short-term thinking• Problems are resolved when they

arise• Limited exchange of information• Lack of follow-up commitments• A minimum of quality is present

2: Initiating process-based thinking/acting

• Formulation of needs and priorities• Overall commitments enforceable to a

limited extent• Increasing attention to information

management• Real causes of most problems are

known• Performance is evaluated regularly• Improvement is clearly noticeable

3: (High level of) Professionalism

• Planned approach• Availability of reliable data• Binding by written commitments• Continuity of service is guaranteed• Level of quality achieved is

supported by assessment data• Information flow is effective

4: Systematic innovation

• Systematic analysis and assessment of operations

• Use of quality indicators as a policy instrument

• Structural problem detection & handling

• Innovative thinking• Emergence of synergy effects

5: (Aiming for) excellence

• Frames of reference, indicators and working methods internationally recognized as best practice

• Broad base of support for moving beyond current levels of quality

• All quality indicators are moving in a positive direction

• Large number of personnel are internationally recognized experts

E.g., the focal point “course units” has items

such as:

1. Do lecturers know the function of their course unit in the curriculum?

2. Do colleagues work together in a team to design course components?

3. Is the design of nearly all course units geared to the frame of reference of the department?

4. Are goals of improvement defined on a yearly basis for all course units?

5. Is the design of specialized course units entrusted to experts?

Testing and assessment

1. Do all students receive a copy of the exam regulations in time?

2. Is there an attempt to safeguard objective assessment by involving more assessors?

3. Are assessment methods for each course component clearly geared to the objectives?

4. Are assessment methods systematically adjusted following evaluation?

5. Is the quality of the assessment methods regularly tested by external experts?

Student admission coaching

Answering the questions

• Answering categories: yes / no / not applicable / impossible to answer

• Lack of measurement data provides relevant information on the need to professionalize the management of information

• Concrete questions provide immediate goals of improvement

The method used in PROZA

• The checklists are filled out individually by the team members

• They gather to agree in consensus on the scores of the items and deduce the level of quality management

• but also to identify immediately specific goals of improvement

• that are selected according priority and written down in a self-evaluation report

Score (in consensus)

• Actual level: phase within which 3 items are positive

• Potential or growth level: phase within which 1 item is positive

• … each time on condition that in previous phases at least 3 items are positive

Chart of scores within an area

Chart of scores over the 9 areas

The PROZAself-assessment process

• Preparation• Diagnosis• Improvement

Preparation

• Policy decision to use PROZA and formation of a steering group

• Getting people on board by engagements and good communication

• Initial screening with quickscan to determine which areas (and foci) to address

• Formation of self-assessment groups• Agreement on working method and prior

clarification of terminology and gradation words

Screening with the quickscan

• 200 statements that probe the overall level of quality (20 per area, 40 for core processes)

• To be answered with– Adequate (A)– Must be improved / not adequate (NA)– Important (I)– Not so important (NSI)

• Preferably by the steering group or key persons

The Quickscan

The Quickscan

Carrying out the diagnosis

• Relevant information on the foci can be made/kept available

• Gaining a view on hot issues by an inventory of individual scores before the meeting

• Value of individual comments on the questions

• Short-circuiting endless discussions:– Right/wrong is less essential than identifying

improvement– Eventual agreement on skipping items on which

only one member answered negatively

From diagnosis to improvement

• Diagnosis and proposals for improvement are formulated on a report form

• Assignment of a priority value• Definitive selection of ultimate priorities

by steering group or management • For each selected goal, an action plan

must be drawn up on a predefined form by an improvement team

Work sheets in PROZA

Action plan

• Concrete goal• Responsible person• Actions that have to be taken• Persons involved• Time plan• Expected results / deliveries• Budget

Client-orientation of PROZA

• Evaluation criteria and items were tested on content and applicability in different HE institutions.

• Assessment questions have grown out of practical experience and are based on a consensus between TQ managers

• The checklists are accompanied by a users guide and a list of concepts

• The checklists are delivered electronically, so that institutes can adjust words to match the structure of the organisation

Some pro’s of PROZA

• Exhaustive and flexible: separate areas and focal points can be analysed for different units in the organisation

• It is easy to deduce questionnaires• More than 1000 items result in very

specific goals of quality improvement• Based on individual analysis and on

consultation with others in teams

Links between self-evaluations and external

quality assurance

Quality management policyHuman resourcesMeans and financesLearning goalsProgramme content and organizationOutside school practice trainingInternational policy and activitiesTraining methods and toolsAssessment and examinationCounseling and guidanceStudy load and effectivenessResearch, arts and societal servicesGraduates

Themes for auditsSelf-evaluation

Implementation requirements

• Improvement as the final goal: diagnosis and scoring is only the starting point.

• Explicit policy and engagement on all levels of the management.

• Agreement on aspects of confidentiality and openness of reports

• Explicit rules about implementation of selected goals, follow-up and feedback

Avoiding pitfalls

• If not properly introduced and coached, it may be perceived and labeled as a mass of paperwork which demands more energy than that it should help to improve quality.

• If not complemented by a leading and steering management, self-evaluation may bog down in the diagnostic process.

• If the instrument is put forward as to be used primarily on teaching processes , it may result in adverse effects of “shooting the piano-player” instead of improving the piano.

Guaranteeing the achievement

of goals

• Diagnoses by self-evaluation teams can exist alongside the management instead of in the heart of the management, leading to quality management on paper instead of effective improvement in reality

• Avoid misuse by management to “solve” management problems, or to evaluate employees.

• Diagnoses must not only result in clear objectives of improvement but must also be made concrete in realistic project planning according to SMART rules.

Criteria for project goals

•Relevant

•Understandable

•Measurable

•Behaviorable

•Attainable

•Specific

•Measurable

•Agreed/acceptable

•Relevant/realistic

•Time-related

For further information…

www.prose.be

www.prose.be

Info@prose.be