AEC PUBLICATIONs 2007

handbook

internal quality assurance in

higher music education

evert bisschop boele

The Polifonia project has been funded with support from the European Commission. This publication reflects the views of its authors and the Commission cannot be held responsible for any use which might be made of the information contained therein.

A free electronic version of this handbook is available through www.polifonia-tn.org.

handbook

internal quality assurance in

higher music education

evert bisschop boele

Contents

Foreword1. introduction 2. some preliminary remarks on internal quality assurance 2.1 Quality 2.2 Assurance 2.3 Internal3. aiming for quality 3.1 Introduction:amodel 3.2 Theproduct 3.3 Theprocess 3.4 Theorganisation 3.5 Thequalityassurancesystem 3.6 Summaryandconclusion4. measuring quality 4.1 Hardfactsandsatisfactionstatements 4.2 Hardfacts 4.3 Satisfactionstatements 4.4 Summaryandconclusions5. setting up an internal quality assurance system in your conservatoire 5.1 Introduction 5.2 Step1 Stateyourqualitygoals 5.3 Step2 Pickyourperformanceindicators 5.4 Step3 Addoutsidedemands 5.5 Step4 Picktheinstruments 5.6 Step5 Formulatetargetresults 5.7 Step6 Makeanimplementationplan 5.8 Step7 Describe 5.9 Step8 Carryout 5.10 Step9 Reviewthesystemregularly 5.11 Summaryandconclusionsappendicesappendix a InternalQualityAssuranceinConservatoires–SomeExamples A.1 Introduction A.2 ThequalityassurancesystemoftheRoyalCollegeofMusic,London(UK) A.3 ThequalityassurancesystemofthePrinceClausConservatoire,Hanze UniversityofAppliedSciences,Groningen(NL)appendix b TheENQA2005StandardsandGuidelinesforQualityAssuranceintheEuropean HigherEducationArea(ESG)–Summaryversionappendix c ListofAbbreviationsUsed

5788911131314161719192121222326282828303131323233333334

37

373742

45

48

5

5

Foreword

TheERASMUSThematicNetwork forMusic “Polifonia”1, the largestEuropeanprojectonprofes-sionalmusictrainingtodate,involved67organisationsinprofessionalmusictrainingandthemusicprofessionfrom32Europeancountriesand30expertsin5connectedworkinggroupsinanintensive3-yearworkprogrammefromSeptember2004–October2007.Theproject,whichwascoordinatedjointlybytheMalmöAcademyofMusic–LundUniversityandtheAssociationEuropéennedesCon-servatoires,AcadémiesdeMusiqueetMusikhochschulen(AEC),receivedsupportfromtheEuropeanUnionwithintheframeworkoftheERASMUSProgramme.Theaimsoftheprojectwere:1. TostudyissuesconnectedtotheBolognaDeclarationProcess,suchasthedevelopmentoflearn-

ingoutcomesfor1st(Bachelor),2nd(Master)and3rdcyclestudiesthroughthe‘Tuning’method-ology2,theuseofcreditpointsystems,curriculumdevelopment,mobilityofstudentsandteach-ers,andqualityassuranceinthefieldofmusicinhighereducation.

2. Tocollect informationon levels inmusiceducationother than the 1st (Bachelor)and the2nd(Master)studycycles,inparticularpre-collegetrainingand3rdcycle(Doctorate/PhD)studiesinthefieldofmusic.

3. Toexploreinternationaltrendsandchangesinthemusicprofessionandtheirimplicationsforprofessionalmusictraining.

WiththeaimtoparticipateinthediscussionstakingplaceinthehighermusiceducationsectorandintheframeworkoftheBolognaprocess,theAECformedwithin“Polifonia”agroupwiththefollow-ingexperts:

• JeremyCox(Chair-RoyalCollegeofMusic,London)• HannuApajalahti(SibeliusAcademy,Helsinki)• EvertBisschopBoele(HanzehogeschoolGroningen)• CristinaBritodaCruz(EscolaSuperiordeMúsicadeLisboa)• BrunoCarioti(ConservatorioStatalediMusica“AlfredoCasella”,L’Aquila)• GrzegorzKurzynski(K.LipinskiAcademyofMusic,Wroclaw)• JörgLinowitzki(MusikhochschuleLübeck)• JacquesMoreau(CNSMDdeLyon)

1 Seeformoreinformationabout“Polifonia”www.polifonia-tn.org.

2 Formoreinformationaboutthe“Tuning”methodologypleaseseehttp://www.tuning.unideusto.org/tuningeu/index.php?option

=content&task=view&id=172&Itemid=205.

6 7

InordertoassisthighermusiceducationinstitutionswiththerequirementsproposedbytheBolognaprocess,the“Polifonia”projectissuedaseriesofpublicationsthatcanbeusedbytheinstitutionsinthedevelopmentoftheirstudyprogrammes:

• Severalpracticalhandbookson:- CurriculumDesignandDevelopmentinHigherMusicEducation- ImplementationandUseofCreditPointsinHigherMusicEducation- InternalQualityAssuranceinHigherMusicEducation• Adocumententitled“SummaryofTuningFindings–HigherMusicEducation”,whichcontainsthe

AECLearningOutcomesforthe1st,2ndand3rdcycles,aswellasthe“Polifonia/DublinDescrip-tors”

• Atrilingualwebsitecalled“BolognaandMusic”(www.bologna-and-music.org),whereallrele-vantdocumentationinrelationtotheBolognaprocessseenfromtheperspectiveofhighermusiceducationcanbefound.

Inaddition,theAECproject“AccreditationinEuropeanProfessionalMusicTraining”3producedsev-eralimportantdocumentsaddressingexternalqualityassuranceandaccreditationinmusic.

3 Moreinformationaboutthisprojectcanbefoundatwww.bologna-and-music.org/accreditation.

6 7

1 . IntroduCtIon

1.1 Thishandbookismeanttobeashortguideforconservatoireswhowanttodevelopasystemforin-ternalqualityassurance.

1.2 Itdoesnotaimatacompleteintroductionintotheworldofqualityassurance,withitsmanydifferentsortsofsystemsanditselaborateandsometimesconfusingjargon.Thishandbookexplainsinthefirstfourchaptersinasimplewaythemainelementsofapossibleinternalqualityassurancesystem.ChapterfiveandappendixAgiveyousomepracticalassistancebypresentingasimpleprocedurewhichyoumightuseindevelopingafirstsystemforinternalqualityassurance,andbypresentingthesystemsforinternalqualityassuranceoftwoEuropeanconservatoires.

1.3 Thispublication iswrittenforpeopleworkingwithinaconservatoire,withagoodoverviewof theinstitute,maybewithmanagementresponsibilitiessuchasaheadofdepartmentormaybeevenaprinciple,orinothercasesasastaffmemberwhois,onbehalfofthemanagement,askedtodevelopaninternalqualityassurancesystem.

1.4 Workon thishandbookhasstartedwithgatheringsome insight in thecurrentstateofaffairsofinternalqualityassuranceinconservatoiresbymeansofaquestionnaire.69conservatoiresfilledoutthequestionnaire.Roughly,twothirdofthemstatedexplicitlytoworkwithorworkonasystemofinternalqualityassurance4.Furtherexaminationoftheanswersgiveninthequestionnaires,andsubsequentdiscussionswithpeoplefromwithinvariousconservatoires,suggestedhoweverthatac-tuallymuchmoreconservatoiresareworkingoninternalqualityassurance,butmanydonotlabeltheiractivitiesonthefosteringofqualityintermsofinternalqualityassurance.IthankthoseoftenunknowncolleaguesalloverEuropewhoansweredthequestionnaireandthosewhoengagedwithmeindiscussionsonqualityassurance.

1.5 IalsothankthemembersoftheTuningworkinggroupofthePolifoniaproject,andmycolleaguesfromHanzeUniversityofAppliedScience inGroningen, theNetherlands, for their insightfuldis-cussionsand theircarefulcommentsonvariousdraftsof thishandbook. I finally thankmydearcolleaguesfromtheAECOfficeinUtrechtfortheirpatienceandlovingtenacity.Withoutthemthishandbookwouldprobablyneverhavebeenfinished.

4 Theresultsofthisquestionnaireoninternalqualityassurancecanbefoundat:http://www.bologna-and-music.org/internalqa.

9

2. some PrelImInary remarks on Internal QualIty assuranCe

2.1 quality

2.1.1 “Quality”,andspecificallyartisticormusicalquality,hasbeenaboutthemostimportantconceptofconservatoireeducationsinceitsverybeginning.Musicalqualitytendstotaketheshapeofanidealwhichcanneverbefullyreached,butmustbeaimedatcontinuously.Ittakesdifferentformsandshapes,andisoftenimmediatelyrecognizedbuthardtoputinwords.

2.1.2 Assuringthatqualityisofferedandbeingaccountableforthequalityyoudeliverhasalwaysbeencentraltoconservatoireeducation.Inarecentdocument“Quality,Assurance,Accountability:ABrief-ingPaper”5itisputasfollows:

Music study is permeated with accountability. (…) In practice sessions, rehearsal, and even in perform-

ance, constant evaluation and adjustment are the norm. The success of professional music study is evalu-

ated in the light of the high standards and high expectations of the larger musical world. Tours, recordings,

and international competition continue to define professional expectations by exchange of work at the

highest levels. In music, we have standards because we have art, not art because we have standards.

2.1.3 Thenormalwayofassuringthatthehighestmusicalqualityisdeliveredistohavethequalitytestedandverifiedbymusicianswhoareacknowledgedasoutstanding,bytheirpeersandbysociety ingeneral.Becausemusicalqualityissuchafuzzyconceptandcantakedifferentformsandshapes,inmanycasestheassessmentofmusicalqualityistrustedtonotone,butseveraloutstandingmusi-ciansatthesametime.Acommitteeorjurydeliberatesinallfreedomandtriestoreachaconclu-sionthatcanbesharedbyall.Musicalqualityisthusformulatedin,orthrough,an“inter-subjectivedebate”.

2.1.4 Qualityassurance,includinginternalqualityassurance,isatpresentakindofbuzzwordamongsthighereducationpolicymakers. Ithassince thebeginningbeena topic in theBolognaprocess6,hasgrowninimportanceovertheyearsandisspecificallyreferredtointheBergenCommuniqué(2005)7andfurtherdevelopedinthedocument“StandardsandGuidelinesforQualityAssuranceintheEuropeanHigherEducationArea”8bytheEuropeanAssociationforQualityAssuranceinHigher

5 Seewww.arts-accredit.org/site/docs/pdf/10-MSMAP-Quality-Assurance-Accountability-BriefingPaper.pdf.Thepaperwaspub-

lishedwithinthe“MusicStudy,MobilityandAccountabilityProject”,ajointprojectoftheEuropeanAssociationofConservatoires

(AEC)andtheNationalAssociationofSchoolsofMusic(NASM;UnitedStates).

6 FormoreinformationontheBologna-processandhighermusiceducation,see:www.bologna-and-music.org.

7 Seewww.bologna-bergen2005.no/Docs/00-Main_doc/050520_Bergen_Communique.pdf.

8 Seewww.enqa.eu/files/BergenReport210205.pdf.

8

9

EducationENQA,adoptedinBergenaswell.Manykindsoforganisationsareinonewayoranotherdealingwithit,andithasitsownspecialistsanditsownjargon.

2.1.5 Thispublicationwillfocusoninternalqualityassuranceandwilltrytorelatetheinternationaldis-cussionon(internal)qualityassuranceinhighereducationtohighermusiceducationastaughtinconservatoiresalloverEurope.Itisinevitablethatsomeofthequalityassurancejargonwilltrickleintothispublication,andthatatsometimesyou,thereader,mayfindhismindabittroubled–“Arewestilltalkingaboutqualityhere?”,isaquestionthatmightoccur.

2.1.6 Itisthereforegoodtoremindourselvesatthebeginningofthishandbookthatwearenottalkingaboutatotallynewconceptcalled“quality”thatneedstobeintroducedtoconservatoiresbecausetheyhaveneverthoughtaboutitthemselves.Conservatoireshavebeentalkingaboutqualitycon-tinuously,andasmusicianswequiteoftenrecognizeiteasilyandfromalongdistance.Eventuallyeverythingdescribedinthispublicationshouldservethegoaloffosteringthisprimaryqualitywithinoureducation.

2.2 assurance

2.2.1 Inessence,theassuranceofqualityisnothingmorethansecuringthatoneisofferingthequalityonepromisestooffer.This,again,isnothingnewforconservatoires.Foralongtime,assuringthegen-eralpublicthatoutstandingqualitywasofferedbyconservatoireswasnotproblematic.Conserva-toireswereexpectedtooffermusicaleducationofthehighestpossiblequality,andthegeneralpublicmostlyassumedtheydid,unlessthecontrarywasproven.Theyassumedthisprobablybecauseinconservatoiresoutstandingmusicianswereteaching,musicalqualitywasmeasuredbyoutstandingpeers,andtheworld’sbestmusiciansmostlyhadreceivedtheirtrainingataconservatoire.Ifthemusiciansthemselveswouldnotbeabletoguaranteemusicalquality,whowould?

2.2.2 Thingshavechanged.Unconditionalbeliefinprofessionalauthorityhasgivenwaytoamodern,moreeconomicaldrivenviewonanyformofhighereducation,andthereforealsoonconservatoireeduca-tion.Conservatoireeducationisviewednotonlyashigh-qualityeducationperse,butalsoasapubliccommodity,aserviceofferedonthemarketplacetocustomers.Ifitispaidforbythegovernment(read:thetaxpayers),thegovernmenthastoshowthatthemoneyinvestedinconservatoireeduca-tioniswell-spent.Ifitisofferedbyaprivateinstitution,quiteoftenits“customers”aregiventherighttobeshownthatthemoneypaidisworthitsvalue.Doconservatoiresdelivertheresultstheyhavepromisedandforwhichtheyarepaidforbytaxpayersand/orcustomers?Cantheyensuretheywillcontinuetodosointhefuture?

2.2.3 Infact,thatiswhatthe“assurance”bitinqualityassuranceisabout.Itisaboutlookingwhetherornotcertainpromisedresultshavebeenobtained,andwhetheronereasonablymayexpectthattheseresultswillcontinuetobeobtainedinthefuture.Forsomeofustheideathatconservatoiresare

11

heldresponsiblefortheirpromisesonqualityisslightlyworryingbecauseitmaybeperceivedasintrusionfromoutsidersintotheautonomyoftheconservatoire.However,wemustrememberthatthesocietyweliveinrightfullymayaskusquestionsaboutwhoweareandwhatwedo,asoneofourgoalsistoservethissocietywiththeeducationofexcellentmusicians.Wemustalsorememberthatinmostnationalsystemstheautonomyoftheindividualinstitutiontomaintainitsownsystemofinternalqualityassurance,andthereforeitsowndefinitionsofquality,isleftuntouchedandactuallyhighlyregarded.Externalqualityassurancesystemswillbeincreasinglyfocusingupontheinternalqualityassurancesystemsoftheinstitution,asisalsostressedinENQA’s“StandardsandGuidelinesforQualtyAssuranceintheEuropeanHigherEucationArea”9.

2.2.4Thebasicprincipleofqualityassurance:lookingwhetherornotcertainpromisedresultshavebeenobtained,andwhetheronereasonablymayexpectthattheseresultswillcontinuetobeobtainedinthefuture,cantakemanyshapes.Theresultstobeobtainedmaybedefinedbythegovernment,byanationalinstitution,byconservatoirescollectively,orbyasingleconservatoire.Theymaybedefinedintermsofminimumresults,orofaverageresults,oroftargetresults.Theresultsmaybemeasuredbytheinstituteitself,byacollectiveofinstitutes,byanindependentbody,ordirectlybythegovern-ment.Theymaybeveryconcreteorveryvague;theymayevenbedefinedonameta-level,sothatnottheactualresultsbutthewaysinwhichinstitutionssecurethattheseresultswillbeobtainedaremeasured.

2.2.5 Inthishandbookwedefinethreekeyelementsinanyformofinternalqualityassurance:• definingyourgoals,• measuringyourresults,• relatinggoalsandresultsinwhatisoftenconceptualizedas“thePDCA-circle”.

2.2.6Thechaptersthreeandfourwilldiscussgoalsandresults.ThePDCA-circleisabasicconceptbehindthetwowhichwillbeexplainedhere.Itliesattheveryheartofanyqualityassurancesystem.Simplyput:• Planwhatyouaregoingtodo;• thenDoit;• afterwardsCheckifyouhavedonewhatyoumeanttodoandifitdeliveredgoodresults;• reflectontheresultsofyourcheckingandprepareAdaptationsforfutureactions;• thenstartagain:Plan–Do–Check–Adapt10.

2.2.7 Aconcreteexampleofdevelopinganewmoduleinacurriculummayexplainthis:• developthemoduleandplanhowyouaregoingtodeliverittostudents(Plan);

9 Seewww.enqa.eu/files/BergenReport210205.pdf,p.7,point2.1.SeealsoAppendixBonpage45.

10 The“A”inPDCAmayalsobereferredtois“Act”.Weprefer“Adapt”becausethedifferencebetweenDoandActmayseem

confusinglysmall.

10

11

• deliverthemodule(Do);• check,forexamplebystudentquestionnairesandinterviewswiththeteachers,ifthemodulehas

workedoutthewayyoumeantit(Check);• onthebasisoftheremarksofstudentsandteachers,decidewhichchangesshouldbemadein

themoduleandadaptthemodule(Adapt);• then,startthecircleagainbyplanningtheteachingofthemoduleforthesecondtime(Plan);• etcetera.

2.2.8Ofcoursetheideabehindthisisnotsimplythatyouexecutethiscircleendlessly.Theideaisthat,becauseofstructuralcheckingandadapting,yourresultswillbecomebetterandbetter,andyouwillfindyourselfnotinthetreadmillofthecirclebutonastaircaseintheformofaspiral,aspiringtoeverbetterquality.Or,inthetheoreticalcasethatyourqualitygoalshavebeenreached,maintainingyourqualityonthesamehighlevel.

2.2.9Sobeforegoingintotheintricaciesofinternalqualityassurance,itisgoodtoremindourselvesthatinternalqualityassuranceisnothingmorethanelaboratingawayofmakingsurethatwhatyoude-liverlivesuptothesetstandards.Furtheroninthisbrochurewewillseeintosuchquestionsaswhatwemeanby“whatyoudeliver”,whatwemeanby“standards”andonwho“sets”thosestandards.Untilthen,thingsareassimplyastheyarestatedhere.

2.3 Internal

2.3.1 Generally,twoformsofqualityassurancearediscerned:internalandexternal.Externalqualityas-surancereferstoqualityassurancesystemsoperatedbyoutsiders(suchasgovernmentsoraccredi-tationagencies).Internalqualityassurancereferstoqualityassurancesystemssetupandoperatedbytheinstitutionitself.

2.3.2Thishandbookwillconcentrateoninternalqualityassuranceandwillpayminimalattentiontoex-ternalqualityassurance.Inexternalqualityassurance,manynational,Europeanandglobaldevel-opmentstakeplace.Theywillnotbedescribedinthishandbook.Foranyoneinterestedinexternalqualityassuranceanditsconsequencesforconservatoires,wereferyoutotheresultsoftheAEC-project“AccreditationinEuropeanProfessionalMusicTraining”11.

2.3.3Ashasbeenstatedearlier, internalqualityassuranceliesat thecoreofconservatoireeducation.Fromitsverybeginning,everyconservatoirehasbeenveryconsciousofthemusicalqualityitoffered.Internalqualityassurancesystemsinconservatoiresnowadayscanbefoundinverymanydifferentforms,fromanearlyimplicitorintuitivewayofsecuringthemusicalqualitydelivereduptoelaborate(andsometimesratherbureaucratic)systemstakingintoaccountqualityonvariouslevelsinarigor-ous,continuouslyrepeatedanddenselydocumentedprocess.

11 Seewww.bologna-and-music.org/accreditation.

12 13

2.3.4Internalqualityassuranceisatpresentoftenrelatedtoexternalqualityassurance.Werepeatherewhatwealreadystatedinparagraph2.2.3:itisincreasinglythecasethatexternalqualityassuranceisexpectedtotakeintoaccounttheinternalqualityassurancesystemsoftheindividual institute,andthatthereforeinstitutionalautonomyandinternalqualityassuranceshouldformthebasisofthinkinginqualityassuranceterms.NotonlyENQA’s“StandardsandGuidelines”,butalsothe2005BergenCommuniquémentions“thesystematicintroductionofinternalmechanismsandtheirdirectcorrelationtoexternalqualityassurance”.

2.3.5 Inmanycountriesconservatoireshavetoshowtooutsiders–beitdirecttotheMinistryortoasepa-rateofficialbody-onaregularbasisthattheyofferthequalitytheypromisetooffer.Quiteoftenonefindsthatinternalqualityassurancesystemsareinsuchcasesnotonly(andmaybenoteveninthe

firstplace)maintainedbecauseitisfeltasanintrinsicneed,butthattheyservealso,andsometimesmainly,asameansofpreparingoneselfforexternalqualityassuranceprocedures.

2.3.6Thatisfine:earlierwealreadystatedthatsocietyisentitledtoquestionusonquality.Theremayhowever lurkadanger there.Ashasbeensaidbefore, theconceptofquality liesat theheartofconservatoireeducation,andconservatoireshavealways,albeitsometimes intuitiveand implicit,hadtheirownformsofinternalqualityassurance.Thisisanimportantfact,anditscoreshouldbemaintained.Adequateinternalqualityassuranceissomethingonedoesnotbecausesomeonetellsyouithastobedoneoronlybecauseitisapreparationforexternalqualityassurance.Internalqualityassuranceissomethingthatcomesasanaturalconcerntoanyonewhotakespartinconservatoireeducation,beitmanagement,teachers,studentsorexternalrelations.Thisisso,simplybecausetheconceptofqualityliesalwaysatthecoreofanymusician’swork.

2.3.7 Building an internal quality assurance system in any conservatoire therefore never starts fromscratch.Therealwayswillbeasoundcoreofideasconcerningqualityfromwhichaconservatoirecanstartandwhichgraduallycanbebuiltoutabitifnecessaryinordertofitthemoreexpandedmeaningoftheword“quality”nowadaysorinordertoadjust itmoretothedemandsofexternalqualityassuranceprocedures.Inessence,however,internalqualityassuranceisanaturalfeatureofconservatoireeducation.

12 13

3. aImIng For QualIty

3.1 Introduction: a model

3.1.1 Thekeywordinqualityassuranceis“quality”-asitisinconservatoireeducation–butqualityisinitselfanabstractconcept.Notone,butmanydefinitionsexist;arecentpublicationoftheEuropeanUniversityAssociationEUAmentionsamongstothers“fitnessforpurpose”,“customersatisfaction”and“excellence”12.Inthispublication,wewillnotchooseforonesingledefinition,butwillbaseour-selvesonanoverarchingideaofquality.

3.1.2 Oneelementisstandardinanydefinitionofquality:tomakeitconcrete, itmustbeattachedtoa“something”. If we want to express our quality goals, we are always talking about the quality ofsomething.Inthispublication,weproposeamodelinwhichqualitygoalscanbeformulatedonfourdifferentlevels:wemaybetalkingaboutthequalityoftheproduct,theprocess,theorganisationorthequalityassurancesystem.

3.1.3 Thefourlevelsmaybeexpressedinamodelofconcentriccircles.Centralinthemodelistheproduct.Thefinalqualityofwhataconservatoireofferstotheworldliesinitsproduct:themusician,com-poser,musicteacheretcetera.Tomakesuretheproductiseventuallythere,learningandteachingprocessesarecarriedout.Thisisthesecondlevel.Thelearningandteachingprocessesarecarriedoutwithinanorganisation, thethird levelof themodel.Ona fourth (meta-)levelyouwill findthesystemsdevisedforassuringthequalityoftheproducts,theprocessesandtheorganisation.ThemodelthusencompassesthetwomainapproachesoflookingatqualityasdefinedintheEUA-reportmentionedearlier:anapproachthatfocusesonqualityofoutputsandanapproachthatfocusesonqualityofprocesses–theoutputoftheproductbeingrightinthecentre,andthestressshiftingfromoutputtoprocessasonemovesawayfromthecentre.

12 Seehttp://www.eua.be/eua/jsp/en/upload/Quality_Culture_2002_2003.1150459570109.pdf.

Product(music)

Process

(learningandteaching)

Organisation

(TheConservatoire)

QualityAssurance

14 15

3.1.4 Themodelisinawayahierarchicalmodel:thelevelsbecomemoreandmoreabstractaswemovefromtheinsidetotheoutsideofthemodel,andeachcircleisinawayapreconditionforthecirclethatlieswithinit.Theproductisattheheartbecauseeventuallytheproductistheultimategoal.Learningandteachingprocessesmaybeanimportantissueontheirownbutthefinalaimofthemistheproductwedeliver.Inthesamewaytheorganisationisneveragoalinitselfbutismeanttofosterthepropercarryingoutofteachingandlearningprocesseswhichinitsturnareonlymeanttoleadtothehighestqualityoffinalproducts.Andqualityassurancesystemsareimportant,butonlytoassurethequalityoforganisation,processesand,finallyandmostimportant,products.

3.1.5 Aremarkonthesimplicityofthemodelisnecessary,becauseitactuallyoversimplifiesreality.Forexample,thekeyconceptineachofthelevels-“product”,“process”etcetera-seemsratherbigandmonolithicterms,butinrealitycanonlybehandledifthetermsarefurtherspecified,oftenbybreak-ingthemdowninseparatebutrelatedelements.Talkingaboutthequalityof“theproduct”or“theprocess”canbequitemeaninglessifwedon’tspecifywhichelementsoftheproductortheprocessweexactlymean.Also,theterm“quality”quiteoftenhastobespecifiedabitmore,astheremaybemanydifferentkindsofquality,ashasbeenmentionedbefore.Intheparagraphsbelowwewillmakeashortinvestigationofeachofthefourlevelsandwillcommentonthepossiblefurtherspecificationofthekeytermsaswellasonthevariouskindsofqualityonemaydiscern.

3.1.6 Finally,awordofcautionmustbemadehereandwillberepeatedagainandagaininthispublica-tion.It isverytemptinginqualityassurancetocoverthecompletefieldandworkin“totalqualitymanagement”terms:tryingtomonitorallfourlevels,withinalllevelseverysortofthinkablesubdi-visionsinvariouselements,andfromallelementallkindsofquality.Especiallywhensettingupaninternalqualityassurancesystem,itmaybewisetoworkfromadifferentperspective.Donotaimatcompleteness(asitwillneverbereachedinthefirstplace,andonemaydrownintheprocessasaside-effect),butpickselectivelywithwhichelementsyouwillmonitorwhichkindsofquality.Makeyourchoiceonthebasisofwhatyouthinkarethecoreconceptsofyourconservatoireeducation.

3.2 the product

3.2.1 Ashasbeensaid,informertimesconservatoiresdefinedqualityasmusicalqualityandwerehappyiftheycouldshow,bymeansofintersubjectiveprocedures,thatmusicalqualitywasmaintainedfromyeartoyear.Whatactuallywas“measured”wasthemusicalqualityofa“musicalproduct”:mostoftentheperformanceofoneormorepiecesofmusicduringarecital.Thedefinitionofwhatthe“musicalproduct”was,wasquitestrictlyconfinedtotheaudiblemusic.Althougharecitalhasalsotheconnotationofaspecificmusicianplayingforaspecificaudienceataspecificplaceandaspecifictime,actuallyinstandardconservatoirepracticewhatwasmeasuredwaspurely“themusic”.Theaudienceandthepersonofthemusicianwereleftoutofconsideration:weareremindedofthisat-titudebythepowerfulimageoftheorchestralauditionbehindacurtain.

14 15

3.2.2Thisverynarrowdefinitionofthemusicalproductisnowadaysinawaystillcentralatthequalityconceptinhighermusiceducation,buthasbecomebroaderinthreedirections.Tobeginwith,notonlythetypicalrecitalofclassicalmusiccanbethefinalproductofconservatoireteaching.Otherstyleshaveofcoursecomein:jazz,pop,variousgenresofworldmusic.Acompositionorportfolioofcompositionshas,fromtheearlystartoftheconservatoire,beenwidelyacceptedasanothertypeofmusicalproduct,ashasbeentheconductingofaconcert.Nowadays,alsoteachingaclassmaybeinsomecasesbeconsideredamusicalproduct,asmaybethefinalproductofaneducationalproject,astudiorecording,amusictherapysessionorinsomecasessimplysomekindofthesis.

3.2.3Thesecondsenseinwhichthedefinitionofthemusicalproducthasbroadened,isthatevenifwetaketheclassicalmusicperformanceasanexample,theviewontheperformancehasoftenbecomebroader.Theactual,aestheticqualityoftheperformanceofthemusicstillistheheartofthematter.Butinaddition,otherkindsofqualitymaybetakenintoconsiderationandformalisedinassessmentcriteria.Forexample,thewaythemusicianinteractswithhisaudiencebefore,whileandafterplay-ingmaybeconsidered(“interactionalquality”mightbeatermforthis),asmayinsomecasestherecitalasanorganisedevent(“organisationalquality”)–uptothepointwhereeventhelay-outoftheprogrammeleafletmaybeapointofdiscussionintheexaminationcommittee.Also“entrepreneurialquality”maycomein,inwhichwelookattheperformanceinmarketing-terms:wouldthisbeaprod-uctthatattractspublic,that“sells”?

3.2.4Finally,thedefinitionofthemusicalproducthasshiftedinathirdsense,relatedtotheprevioustwo.Moreandmore,itisacknowledgedthattheactualgoalofconservatoiretrainingisnottheperform-ance(orcomposition,or…)perse,buttheperformanceasanexpressionoftheabilities,orcompe-tencies,oftheeducatedmusician.Inaway,themeaningof“product”hasshiftedfromthemusictothemusician,fromtheperformancetotheperformer,fromthecompositiontothecomposer.

3.2.5Summingup,thecentralproductofconservatoireeducationisacomplexnotion.Adefinitionmightbe“thecompetenciesofanykindofmusicianinanygenreandstyleofmusic”.Thosecompetenciesusuallywillbeexpressedandassessedinaso-called“criticalsituation”–asituationwhichisspecifi-callycreatedinordertoassessthecompetenciesoneseesasthegoalofconservatoireeducation.Thereisawiderangeofpossiblecriticalsituationsinconservatoireeducation,andnotoneexhaus-tivemodelwhichdescribesthemanypossibleproductsandkindsofquality.Ifonechoosestotakeproductqualityintoaccount(anobviouschoiceinconservatoireeducation),itishoweverwisetobeawareofthedifferentsortsofproductsanddifferentkindsofqualityonemaydiscern.Finally,onehastobeselective:itisnousetomonitorallkindsofqualityintheproducts,itwouldbewiseifonewouldfocusonthosekindsofqualitywhichonethinksarethemostimportant.

16 17

3.3 the process

3.3.1 Quiteoftenwhenpeopletalkaboutqualityinconservatoireeducationtheconceptisnotonlyappliedtothequalityoftheproduct.Asinanyorganisation,thefinalproductistheresultofaprocess.Intheconservatoire,thefinalmusicalproduct–themusician-istheresultoflearningandteachingprocesses:thestudentlearnsinmanydifferentways,stimulatedbytheteachingoftheteachers,byco-operationandcompetitionwithotherstudents,bymusicalexperiencesoutsidetheirformallearningenvironment,etcetera.Itisnowadaysnotuncommontonotonlyfocusonthequalityofthefinalproduct,butalsoonthequalityofthelearningandteachingprocessesleadingtothisproduct.

3.3.2Justas“theproduct”,“thelearningandteachingprocess”isaratherbigconceptaswell,anditsqualityishardtoassessinonestroke.Whatthereforeoftenhappensisthattheteachingandlearningprocessisdividedintosub-processes.Thereisnotoneuniformmodelforthissubdivision.Differ-entviewsonlearningandteachingmayleadtodifferentsubdivisionsofthelearningandteachingprocess.Forexample,inastrictlyteacher-ledlearningprocessotherelementswillbediscernedtomeasurequalitybythaninstudent-centred,competence-basedlearning.

3.3.3Twomodelswillbepresentedinordertogiveafeelingforthis.Thefirstisaratherstraightsubdivi-siondevelopedinthemiddleofthelastcenturyforuseinprimaryandsecondaryeducationbuteasilytransferabletohighereducation13.Ittakesasitsbasicunitofdescription“thelesson”andstatesthatinplanningtheteachingprocessonehastotakeintoaccountthefollowingelements:• startingsituation,• goals,• learningprocesses(withinthestudent),• didacticalprocesses(teaching),• useofmedia,• learningcontent,• formsofstudentgrouping,• evaluation.

3.3.4Anothermodel,currentlyusedinaqualityassurancemodelinhighereducation14,ismoregearedtowardscompetence-basedteachingmodelsonthelevelofacompletecurriculumanddiscernsthefollowingelements:• researchonthestakeholders’wishesanddemands,• compilationofthespecifications,• compilationoftheintra-andextra-muralcurriculum,• designofstudycomponents,• designofassessment,• designofthelearningenvironment,

13 ModeldevelopedbyVanGelder;sourceE.DeCortea.o.,Beknopte Didaxologie,Groningen:Wolters-Noordhoff,1981.

14 ExpertgroupHBO,Method for improving the quality of higher education based on the EFQM model,Groningen/Eindhoven:

HanzehogeschoolGroningen/FontysHogescholen,2006.

16 17

• studentandteacheractivities,• monitoring.

3.3.5Andagain,differentkindsofqualitymaybediscernedinthevariouselements.Onemightlookforexampleonthelevelofthedesignofstudycomponentssuchasseparatemodulesatthe(musical)qualityofthecontent,butalsoatthequalityofthedesignprocess,atthequalityofthedocumentationofthemodulesetcetera.

3.3.6Ifonewantsitsinternalqualityassurancesystemnotonlytomonitorthequalityofthemusicalprod-uct(asdescribedinparagraph3.2)butalsothequalityofthelearningandteachingprocessitisagainimportanttorealisethatonehastochoosewhichelementsaremonitored.Intheorymanydifferentaspectscanbediscernedinlearningandteaching,andallofthemcanbemonitored.Unlessonehasendlesshourstospendonbuildingupandmaintaininganall-encompassingsystem,itisveryimportanttofocusonthoseelementsoneconsiderscrucialformonitoringthequalityofthelearningandteachingprocess.

3.3.7Summarizing,thelearningandteachingprocesseswithinaconservatoireareimportantprecondi-tionsforthequalityoftheproducteventuallydelivered.Learningandteachingprocessesmaybebrokendownintoseparatebutrelatedelementsinvariousways,dependingononesviewoneduca-tion,andvarioussortsofprocessqualitycanbedistinguished.Ifonechoosestotakeprocessqualityintoaccount,itisimportantagaintobeselective:focusonlyonthoseelementsoftheprocessoneconsiderscrucialfortheoverallqualityoftheinstitute.

3.4 the organisation

3.4.1 Inmanyqualityassurancesystems,notonlythequalityoftheproductandtheprocessesleadingtotheproductaremonitored.Onealsokeepstrackofthequalityoftheorganisationwhichorganisesprocessesinordertodelivertheproduct.Inconservatoireterms:onelooksintothequalityoftheconservatoireasanorganisationthatorganiseslearningandteachingprocessesinordertoletstu-dentsdelivermusicofhighqualityattheendoftheprogramme.

3.4.2Asintheprecedingparagraphontheprocess,“theorganisation”istoocomplexaconcepttomakesimplestatementsonitsquality.Inordertosaysomethingworthwhileaboutitspossiblequality,ithastobebrokendowninorganisationalelements.Awidelyusedwayoflookingatanorganisationisthefollowing15:- centralintheorganisationaretheprimaryprocesses.Inthecaseofconservatoirestheprimary

processisbasically“learningandteaching”,insomecasesalsoresearch.Wehaveelaboratedontheprocessalreadyaboveinparagraph3.3,andtheprimaryprocessesleadeventuallytotheproductsofparagraph3.2;

- supportiveelementsfortheprimaryprocessarepolicy/strategy,personnel/staff,andresources(amongstwhichfinances);

- thewholeisrunbyleadership.

15 Seee.g.www.efqm.org/uploads/introducing_english.pdf.

18 19

Inascheme:

Everyelementoftheschememaythenagainbebrokendownintosmallerelements,leadingtopro-

nouncementsonwhatqualityinanorganisationis.

3.4.3Togiveanexampleofanotherpossiblewayoflookingattheorganisationallevel:McKinsey’s“7s-model”16discernssevenfactors,allbeginningwithan“s”,whicharecrucialforthequalityanyor-ganisationdelivers. Thesevenfactorsareinterdependentandsomeare“harder”(e.g.structure,systems)thanothers(style,sharedvalues).Inascheme:

3.4.4Againthoseschemesandtheirbreakingupinelementsarejusttwoexamplesofhowonemaylookatorganisations–theremaybeothersjustasworthwhile.Andagain,itispossibletodiscernvariouskindsofquality:thequalityofanorganisationmayforsomebelyinginitsthoroughdocumentationofprocesses,forothersinitsdemocraticpossibilitiesforcreativeexchangeofinnovativeideas.

Lead

ersh

ip

Proc

esse

s

lead

ing

toa

pro

duct

People

(personell/staff)

Policy&Strategy

Partnerships&Resources

structure

strategy systems

skills style

staff

shared values

16 Seee.g.www.12manage.com/methods_7S.html.

18 19

Finally,againwemuststressthatonehastobeselective.Itisnousetomonitorthetotalqualityofallelementsofanorganisation,onepicksthoseelementsthatstronglyconnecttothequalityidealsoftheconservatoire.

3.4.5Summarizing,thequalityoftheconservatoireasanorganisationisanimportantpreconditionforthequalityoftheprocessescarriedoutwithinitandtheproducteventuallydelivered.Itispossibletolookattheseparatebutrelatedelementsoftheorganisationinvariousways,andvarioussortsoforganisationalqualitycanbedistinguished.Ifonechoosestomakeorganisationalqualitypartoftheinternalqualityassurancesystem,onehastobeselectiveandfocusonlyontheelementsdirectlyconnectedtothequalityidealsoftheconservatoire.

3.5 the quality assurance system

3.5.1 Finally,a“meta-level”issometimesintroduced,inwhichnotthequalityoftheproduct,theprocessortheorganisationismonitored,butonlythequalityofthequalityassurancesystem(s)itself.Thismeta-levelwillonlybementionedhereshortlyasitisactuallyhardlyofanyuseforstrictlyinternalqualityassurancesystems–itpresupposestheexistenceofaninternalqualityassurancesystemandthereforecanneverexcuseyoufromtakingintoaccounttheproduct,processand/ororganisa-tionalquality.

3.5.2Externalparties,though,maybeinterestedinthisratherproceduralformofinternalqualityassur-ance.Insomecountriesgovernmentwantstocontrolthequalitydeliveredbyinstitutionsbutdoesnotwanttointerferedirectlywitheducationoronlywantstohavecontrolonthemainfeatures.Insuchcasesitmaybecontentedbyscreeningtheinternalqualityassuranceproceduresofaninstitutewithoutgoingintotheactualqualitativedetailsoforganisation,processorproduct.Thecurrentsitu-ationintheUnitedKingdom,whereinstitutionsundergoanInstitutionalAuditfortheQualityAssur-anceAgencyandmaythenreceivedegree-awardingpower,mayserveasanexample.

3.5.3 Ifaninstitutionisbigenoughandaconservatoireonlyasmallpartofit,itmightbethatthecentralapparatuswillnottrytotakeactualcontrolonthedeliveranceoftheconservatoirebutonlymonitoritsqualityassurancesystem.Inthatcasethismightbeconsideredaformofinternalqualityassur-anceontheleveloftheinstitution–ontheleveloftheconservatoirethatformsapartoftheinstitu-tionitwillnodoubtbefeltasexternal.Somemoreremarksontherelationbetweeninternalandexternalqualityassurancewillbefoundinchapter5.

3.6 summary and conclusion

3.6.1 Inthischapterwehavebeentalkingaboutinternalqualityassurancefromtheperspectiveofthe“what”:whenwearetalkingaboutquality,qualityofwhat?Whatisitwearemeasuringthequalityof?

20 21

3.6.2Weformulatedanswersonfourpossiblelevels:- thequalityoftheproduct,whichmaybeastudentperformingarecital,presentingacomposition

orteachingalesson,andmaynotberestrictedtothe“strictlymusical”qualitybutmaytakeintoaccountotherkindsofqualityoftheproduct(communicativequality,forexample);

- thequalityoftheprocessesleadingtotheproduct,inaconservatoiretheprocessesoflearningandteaching,whichoftenwillbedividedintosub-processesinawaythatreflectsonesphiloso-phyofteachingandlearning;

- thequalityoftheorganisationinwhichtheprocessesleadingtoaproducttakesplace,mostlybrokendownintothevariouselementsonecandiscerninanorganisation;

- thequalityoftheinternalqualityassurancesystemthatmonitorsorganisation,processesandproduct.Thislastlevelactuallyliesmostlyoutsidethescopeofthispublicationasitstopsbeingaformofinternalqualityassurance–mostoftenitis,orcomesverynearto,anexternalqualityassurancesystem.

3.6.3Eachlevelmaybeanalysedorbrokendowninseparatebutrelatedelementsinvariouswaysandvariouskindsofqualitymaybediscerned. It is importanttorememberthatonemaystrive in itsexternalqualityassurancesystemnotforcompleteness,butselectscarefullythoseitemsthataretightlyrelatedtotheoverallideasofqualitytheconservatoirefosters.

20 21

4. measurIng QualIty17

4.1 Hard facts and satisfaction statements

4.1.1 Intheprecedingchapterwehaveproposedafour-level-modeltoformulateourqualitygoalsin,inordertoanswerthequestion:aboutthequalityofwhatarewetalking?Anotherquestionisaboutjustasimportant:whatwillbetheevidenceofquality?Howdowemeasurewhetherornotwereachourqualitativegoals?Irrespectiveofthequestionwhetheryourqualitygoalsareformulatedonthelevelofaproduct,aprocess,anorganisationoraqualityassurancesystem,youaremeasuringitbylookingatinformationthatwillgiveyouaclueaboutthequality.Basically,therearetwotypesofinformationonecandrawconclusionsfrom:hardfactsandsatisfactionstatements.

4.1.2 Hardfacts(inbusiness,youwouldcallthemyourbusinessresults)canbeeitherfinancialoropera-tional.Financialresultsinaconservatoiremaybeforexamplesolvency,liquidity,totalannualbudget,costsperstudent,costsperteacherfulltimeequivalent,totalcostsperrealizedstudycreditetce-tera.Operationalresultscouldbeforexamplethenumberofenrolledstudents,thenumberofnewlyadmittedstudents,thepercentageofstudentsthatdropsoutduringthecourse,thepercentageofstudentsthatdropsoutinthefirstyear,theaveragestudydurationofdrop-outs,etcetera.

4.1.3 Satisfactionstatements,ontheotherhand,arelinkedto(groupsof)individualswhoareexpressingtheirlevelofsatisfactionwiththequalityofferedbytheconservatoire.Typicalexampleswouldbe:peerswhostatetheyaresatisfiedwiththequalityofthefinalexaminations(product);studentswhostatetheyaresatisfiedwithexaminationpracticesthroughthecurriculum(process);teacherswhostatetheyarenotsatisfiedwiththeavailableclassroomsandinstruments(organization);oremploy-ersfromtheprofessionwhostatethatyourcurriculumdoesnotpreparestudentsfortheirfutureprofessionenough(product).

4.1.4 Tobecomeabitmore“technical”:hardfactsandsatisfactionstatementscanserveinrelationtoyourqualitygoalsas“performanceindicators”,theyindicatehowyouareperforming.Youmaypickoneorseveralperformanceindicatorsforeveryqualitygoalyouhaveformulated.Foreveryperformanceindicatoryouthendefineameasuringinstrumentwhichyouwilluseformeasuring,andameasur-ingunit.Youthenstateatargetresult,youmeasureyouractualresult,andaftercomparingthetwoyoudrawconclusionsandactonthem.InthePDCA-circle(seepar.2.2),workingwithperformanceindicators,targetresultsandactualresultsfitspecificallyintheCheck-phaseandmakesurethatonecanAdaptonthebasisofreliablefacts,therewith“closing”thePDCA-circle.

17 Muchofthecontentandmanyoftheexamplesinthischaptercomefrom:ExpertgroupHBO,Method for improving the quality of

higher education based on the EFQM model,Groningen/Eindhoven:HanzehogeschoolGroningen/FontysHogescholen,2006.

22 23

4.1.5 Asimpleexamplewouldbe:

- qualitygoal:yourfinalassessmentmethodsarestateoftheart;- performanceindicator:satisfactionoftheprofessionalfieldconcerningyourfinalassessment

method;- measuringinstrument:threespecificquestionsinaquestionnairefortheprofessionalfield;- measuringunit:percentageofanswersindicatingsatisfaction;- targetresult:70%satisfaction;- actualresult:80%;- compare:youscorehigherthenyourtarget;- conclude:noadaptationsneeded.

4.2 Hard facts

4.2.1 Consideringthehardfacts,thebusinessresults,onecanofcoursecompileanendlesslistofcatego-riesofdata.Forexample,onemightmakethefollowinglistofcategoriesofdata:enrolledstudents,inflow,throughput,outflow(allsplitintotargetgroupswhereappropriate),businessoperation,in-novation,staff,andexternalassessment.Eachcouldbesubdividedagain;businessoperation forexampleinstudycoursedemandfactor,efficiencyratiomeasurement,costsoffailure,students/staffratio, investments,accommodation.Accommodation in turncouldbesubdivided into theaveragenumberoffunctionalsquaremetresinuse,accommodationcostspersquaremetre,andfunctionalsquaremetreperstudent.Andsoonandsoon.

4.2.2Evenwithununlimitedamountoftime,peopleandmoneyyouwouldneverbeabletobecomplete.Andtobesure,completenessisnotthegoalyoushouldbelookingfor.Anendlesssetofdata ismeaningless.Databecomemeaningfulonlywhentheytellyousomethingaboutyourqualitygoals(i.e.whentheyareusedasperformanceindicators),andwhenyoucananalyzetheminthelightoftargetresults.Ifyouwanttoanalysedata,youneedcriteriaonthebasisofwhichyouanalyseyourdata.

4.2.3Thebasicthingtodoistotaketheeightstepsalreadydescribedintheexampleintheparagraphabove:1. decidewhichgoalsyoucouldformulateregardingthequalityyouwanttooffer;2. decidewhichdataarethemosttellingaboutthisquality;inotherwords:pickyourperformance

indicators;3. foreveryperformanceindicator,defineameasuringinstrument;4. defineyourmeasuringunit;5. stateatargetresult;6. measureyourcurrentresult;7. comparetargetandactualresult,analyse,drawconclusions;8. changethingsifyouthinkyouhaveto.

Aftersometime,thecyclemayberepeated:measureagainifyourresultshaveprogressed.

22 23

4.2.4Anexample:1. yourgoalistoofferacurriculumthatisfeasibleforstudents;2. you think themost tellingdatawillbe theamountofstudentsgraduatedwithin thestandard

amountofyears,saythreeyears;3. yourmeasuringinstrumentwillbeacountbyyouradministration;4. themeasuringunitwillbethepercentagegraduatedafterthreeyearsfromthestudentsenrolled

inaparticularyear;5. yourtargetresultis80%;6. yourcurrentresultturnsouttobe60%;7. onfurtheranalysisyoufindthatmostoftheremaining40%dropoutbeforetheydoafinalexami-

nation.Fromthisyouconcludethateitheryourentranceexaminationisnotselectiveenoughorthereissomethingwrongwithyourcurriculum.Youdecidetodoaquicksurveyamongasmanystudentsaspossiblewhodroppedoutinthelasttwoyears.Theyinformyouthemainreasonfordroppingoutearlyisthattheycouldnotmanagethecurriculumbecauseitwasoverloaded,frag-mentedandincoherent;

8. youstartworkingontheimprovementofyourcurriculumandeveryyearmonitorthegraduationpercentageafterthreeyears,hopingthatyoureffortswillleadtoagradualincreaseoftheper-centage.

4.2.5 Themainthingthisexampleshowsishowyouworkwith“hardfacts”ininternalqualityassurance:theyaretelling,butonlyifyouareselectiveandconnectthemwithyourmostimportantgoals,ifyoustatetargetsandcompareyourownresultswiththem,ifyoutakeactionsafterwardandcontroliftheyhaveresults.Ifoneofthoseelementsismissing,eventuallyyourdatawillneverbemorethandata:hardfactswithoutmeaning.

4.2.6Onemoreremarkcanbemadehere.Theexampleabovesuggestsanannualrepetitionofthegath-eringofhardfacts.Thefrequency,however,isnotfixedtoonceayear.Sometimes,itmaybewisetogatheryourdatamoreoften.Forexample,ifyouofferacompletelynewcurriculum,youmightinthestartingphasedecidetomonitorsomeelementsthreeorfourtimesayear.If,however,youarequitesatisfiedwithyourresults,youmightdecidetogathercertaindataonlyonceeverytwooreventhreeyears.

4.3 satisfaction statements

4.3.1 Satisfactionstatementsarestatementsof(groupsof)individualswhoareabletotellsomethingaboutyourcurriculum–oftentheyarecalled“stakeholders”.Aswithhardfacts,therearemanydifferent(groupsof)individualswhocouldmakesuchstatements.

4.3.2 Thefirstoptionis,ofcourse,yourself.Inthenarrowestsensemeaning:“you-theprincipaloftheconservatoire”or“you–theheadofstudies”or“you–theprincipalteacher”.Inthatcasegathering

24 25

satisfactionstatementsinaninternalqualityassurancesystemwouldbequitesimple:forceyourselfforexampleonceayeartotakethequalityofwhatyou’redeliveringunderclosescrutiny,drawcon-clusions,actuponthemandyou’reready.

4.3.3 Actually,thisiswhatweareconstantlydoingbymeansofourassessmentprocedures.Assessmentiscrucialtothefosteringofqualitywithinconservatoires.Forinternalqualityassurance,however,thiscannotbeenough.Qualityassurance,asexplainedinchapter1,isoftennotjustmeanttocon-stantlyimproveone’sownquality,butalsotojustifyonesqualitytowardsexternalparties.“You-theprincipalteacher”or“you-theheadofstudies”willundoubtedlybeanexpertonquality,buttheremaybesomebiasinsomeofyourconclusionsandyoumayhavedevelopedsomeblindanglesovertheyears.Yourinternalqualityassurancesystemwillbenefit ifotherswillalsomakestatementsaboutthequalityyoudeliver,sothatyoucancomparethemwithyourownthoughtsaboutitandletyourconsequentactionsbeinformedbythem.

4.3.4 Thatiswhyqualityassuranceproceduresoftenskipthelevelofconcreteassessmentsandlookforoutsiders’satisfactionstatements.Thefirst,wellknownandwidelyusedother is the“peer”: theoutsideexpertfromwithinyourfield,some-onechosenforhisexpertknowledge.Infact,thebring-inginofpeerstojudgequalitymakesyourownassessmentofthequalityyoudeliverlesssubjective.Qualityremarksarenever,andcertainlynot inmusic,objective,butwithpeerstheymaybecome“inter-subjective”.

4.3.5 Youcanthenbroadenthesetofpeopleyouwanttotakepartinyourqualityassessmentbybringinginvariousstakeholders.Inpractice,thefollowingaremostusedininternalqualityassurance:- students;- (recent)alumni;- theprofessionalfield(especiallypeersandemployers);- staff;- thepartyultimatelyresponsiblefor(andpayingfor)theconservatoire(mostoftenthisisgovern-

ment);- thegeneralpublic.

4.3.6 Eachofthesecategoriesmaybesubdividedagain:togatherspecificdatayoumightwanttoaskaspecificcategoryofstudents(firstyear’sstudents,forexample,orstringstudents),aspecificcat-egoryofalumni(lastyear’salumni,forexample,orjazzalumni),etcetera.

4.3.7 Aswiththehardfacts,satisfactionstatementsareveryusefulaslongasyouareselective.Itisnousetryingtogatherasmuchsatisfactionstatementsaspossiblefromasmanyindividualsaspossibleonasmanyelementsaspossible–again,evenifonehadanunlimitedamountoftime,peopleandmoneyavailableonewouldendupwithanenormouspileofdataoutofwhichitwouldbestillveryhardtoextracttheinformationyouwouldwant.Andaswiththehardfacts,youmightconsiderthe

24 25

frequencywithwhichyougatherdata:maybeyouwanttorepeatafirstyear’sintervieweveryyear,butyoumightconsideraskingalumnionceeverytwoyearsandsendaquestionnairetoemployersintheprofessionalfieldonlyonceeverythreeyear.

4.3.8 Sohereagainweshouldworkintheeightstepsdescribedbefore:1. decidewhichgoalsyoucouldformulateregardingthequalityyouwanttooffer;2.decidewhichsatisfactionstatementsarethemosttellingaboutthisquality;inotherwords:pick

yourperformanceindicators;3.foreveryperformanceindicator,defineameasuringinstrument;4.defineyourmeasuringunit;5.stateatargetresult;6.measureyourcurrentresult;7.comparethem,analyse,drawconclusions;8.changethingsifyouthinkyouhaveto. Aftersometimemeasureagainwhetheryourresultshaveprogressed.

4.3.9 Anexamplewouldbethefollowing:1. yourgoalistoofferajazzprogrammethatisgearedtowardsprofessionalpractice;2. youdecidethatthebestwaytotelliswhenjazzstudentsaresatisfiedwiththecurriculumwhen

theylookbackonitaftergraduationoncetheyareintheprofessionalpractice;3. youdecidetomeasurethiseveryyearwithalumniwhograduatedtwoyearsbefore.Youdothis

bymeansofashortinterviewbytelephone;4. Intheinterviewyouaskthemtogivehecurriculumagradeforpreparationonprofessionalprac-

ticeonascalefrom1(verypoor)to10(excellent);5. yourtargetresultisanaveragesatisfactiongradeof7;6. youfindoutthataveragesatisfactionis6.8;7. youdecidetocarryouta limitedamountof longer interviewsbytelephoneandonthatbasis

decidethatespeciallyaspectsofbusinessandfinancialsaremissingfromthecurriculum;8. afteryouhavebuildacourseonthattopicintothelastyearofthecurriculumyoufindoutsatis-

factionratesareindeedrisingtowellabovea7onaverage.

4.3.10 Thisexampleshowshowtoworkwithsatisfactionstatementsininternalqualityassurance.Aswithhardfacts,theyaretelling,butagainonlywhenyouareselective,relatethemtoyourowngoals,statecleartargets,makeagoodcomparisonwithyouractualresults,takeactionsafterwardandcontroliftheyhaveresults.Beware:satisfactionstatementsareeventuallyindividualandthereforerathersubjective.Oneofthewaysaroundthissubjectivityistoaskenoughindividualstomakeastate-ment.Ifyoursampleisbigenough,youcanmoreorlessabstractfromtheindividual’ssubjectivity.

4.3.11 Herewetouchupononeofthepeculiaritiesofconservatoireeducation:one-to-oneteachingwithitscloseteacher/student-relationship.Togatherreliablestudentsatisfactionstatementsconcern-

26 27

ingteachingonthemaininstrumentisoneofthemostimportantandsensitiveissuesininternalqualityassuranceinconservatoires,andmanyinstitutionshavefoundtheirownwaysinhandlingthismatter.Someuseaformalandanonymoussystem,whichenablesstudentstogivefeedbackontheirteacher;thisisthendiscussedbetweenteacherandheadofdepartment,forexampleduringannualstaffappraisalinterviews.Otherconservatoiresarelookingmoreintoredefiningandprofes-sionalisingtheirsystemsofmentoringandtutoring18inordertotakeawaythepossibleproblemstowhichone-to-oneteachingsometimescanlead.Thepointismaybenotsomuchwhichsystemaconservatoiredevelopsinthisrespect,butthatasystemisdevelopedthatworksinthespecificcontextoftheinstitutionconcerned.

4.3.12 However,anotheraspectthatcomesiniswhatsometimesiscalled“managementofexpectations”:theindividualsyouaskactuallymayexpectmorefromyouthenreasonablycanbeexpected,whichmakesthemunhappyaboutcertainresults,whereasthoseresultsasexpressedin“hardfacts”arenotsobadatall.Afamousexampleisstudents’satisfactionintermsofthemarkingandreturnofwork.Youmayaskstudentsiftheyaresatisfiedwiththeamountoftimebetweenawrittenexamina-tionandthemarkingbyateacher.Teacherscanbesobusywithwrittenexaminationsthattheycanonlybeexpectedtohavegradedwrittenworkwithintwoweeks’time.Ifstudentsdon’trealisethis,theymaybedissatisfiedifmarkingtakeslongerthenaweek,becausetheyarekeentoknowtheirresults.Ifstudentsthereforegivenegativesatisfactionstatementsbuthardfactsshowthatmarkingisdonewithinthetimeyougiveyourteachers,youmayaskyourselfwhetheryouwanttopressyourteacherstoworkevenfaster,orwhetheryouwilltrytomanagetheexpectationsofyourstudentsbetterbytellingthemthegoalyouhavesetyourselfistohavemarkswithinatwoweeks’termbe-causeyoucan’tdemandmorefromyourteachers.

4.3.13 Comparableexamplesmaybegivenonmanymoresatisfactionstatements,reasontohandlethemwithacertaincare.

4.4 summary and conclusions

4.4.1 Inthischapterwehavebeendiscussingthequestionwhatexactlytellsyouaboutquality.Wemadeadivisionintwoseparatekindsofperformanceindicators:ontheonehand“hardfacts”,ontheotherhand“satisfactionstatements”.

4.4.2Hardfactscanbeeitherfinancialresultsoroperationalresults.Examplesoffinancialresultscouldbethetotalannualbudgetoftheconservatoireorthecostsperfte.Examplesofoperationalresultscouldbethenumberofenrolledstudentsorthepercentageofstudentsthatdropsoutduringthecourse.

18 SeePeterRenshaw,“thePlaceofMentoring”,http://www.lifelonglearninginmusic.org/BeheerContentPagina.asp?id=170.

26 27

4.4.3Satisfactionstatementsarestatementsmadeby(groupsof)individualsonthequalityofyourcon-servatoire.Apart fromyourself,manyotherpeoplecanmakesuchstatements.Wedistinguished students,alumni,theprofessionalfield(especiallypeersandemployers),staff,thepartyultimatelyresponsiblefortheconservatoire(mostlygovernment),andthegeneralpublic.

4.4.4Forbothhardfactsandsatisfactionstatementsitisimportanttorememberthattheamountofpossi-bledatatogatherisendless.Onehastochoosecarefulwhichperformanceindicatoronethinkstellsmostaboutthequalitygoalsonewantstoreach,defineameasuringinstrumentandameasuringunit,formulateatargetresult,gatherthedatatofindoutwhatyouractualresultis,comparetargetandactualresult,ifnecessarytakeactionandthenmeasureagaintofindoutwhetherornottheac-tionshaveledtomorepositiveresults.

4.4.5Finally,althoughhardfactsaswellassatisfactionstatementsaremoreorless“objective”datafromoutsidetheconservatoire,onehastorememberthattheyonlygetmeaningoncetheyhavebeenrelatedtoyourgoals,youranalysisandyourdecisions.Thatisthebasisofallinternalqualityassur-ance:itremainsdrivenfromtheinsideofheconservatoire.Thereisanuancetobemadeonthis,butthatwillbedoneinthenextchapter.

2928

5. settIng uP an Internal QualIty assuranCe system In your ConservatoIre

5.1 introduction

5.1.1 Ashasbeensaidbefore,almosteveryconservatoirehasatleastsomethingofanimplicitinternalqualityassurancesystemupandrunning.Itmaynotbeanintegralsystem,itmaynotbeastrictlydocumentedsystem,itmaynotevenbeaverysystematicsystem,butqualityis,bythenatureoftheconservatoire,anaturalconcerntoallworkinginit.

5.1.2 Inthischapter,however,wewillstartfromscratchandpretendweareworkinginanewconserva-toirethathastosetupanewsystemofinternalqualityassurance.Basically,wewouldinsuchacasetakeninesteps,asfollows:1. Stateyourqualitygoals:pickcarefullytheelementsonwhichyouwantinformationconsidering

thequalityyouareoffering;2. Pickcarefullyyourperformanceindicators:which“hardfacts”and“satisfactionstatements”you

thinkarenecessaryinordertogetinformationonthequality;3. Addexternallyformulatedqualitygoalsandperformanceindicators:whatisnecessarytodobe-

causeofpossibledemandsfromoutsidetheconservatoireconcerningyoursystemofinternalqualityassurance19;

4. Picktheinstrumentswithwhichyouwillgatherinformationanddefineyourmeasuringunits;5. Formulatetargetresults;6. Makeanimplementationplanforyoursystem;7. Describecarefullyforeveryactionthatwillbeimplementedwhoisresponsible,whatyoumeas-

ureetcetera;8. Startyourmeasurements,analysetheresultsandsuggestadaptationsifnecessary,implement

theadaptations,measureagainetcetera(makeyourPDCA-circleafullcircle!);9. Reviewyourinternalqualityassurancesystemregularly.

5.2 step 1: state your quality goals

5.2.1 Forconservatoireeducation,thereisno“logicallevel”perseforinternalqualityassurance.OnemaylookinvariousconservatoiresalloverEuropeandfindthatonestrictlystickstoproductmeasuringonly,anotheronestressesprocessesandmaintainsthatonceyouhaveguaranteedthequalityofthelearningandteachingprocesstheproductwillinevitablybeaqualitativegoodproduct,andathirdonewillfocusheavilyonorganisationalquality.

19 Theitemstwoandthreemaybereversedinordersothatonefirstinventarisestheexternaldemandstotheexternalqualityas-

surancesystemandthenaddswhatoneconsiderstobemissing.Wedo,however,prefertheoriginalorder,asitstartswiththe

intrinsicpreferencesoftheinstituteitselfandonlylateraddsthedemandsoftheoutsideworld.

29

5.2.2Whatonealsowill find isthat inmostof theconservatoireswithsomehistory informal internalqualityassurance,abalancedpositionistaken.Therewillbegeneralagreementonthefactthatinconservatoireeducation,strictmonitoringoftheproductqualitybypeersfromtheprofessionisthebasisoftheartisticlevelofthemusicprofession,buttherewillalsobegeneralagreementonthefactthatinordertomaintainthehighestqualityoftheproductoneneedstokeeptrackofthequalityofthemainprocessesinlearningandteaching,andthatforasoundqualityofprocessesandproductsatleastthemostbasicfeaturesoftheorganisationaldevelopmentmustbemonitoredaswell.Itisinfindingthebalancebetweenthesethreelevels,andinchoosingwherethefocuswillliewithineachlevel,thateachinstitutionmakesitsownchoicesandthusgivesitsinternalqualityassurancesystemapersonal“face”.

5.2.3Thefirststepinanewconservatoirewouldbetocarefullypickoutthoseelementsonwhichyouwantinformationconsideringthequalityyouoffer.Asdescribedinchapter3,youmaystarttoconsiderwhetheryouwanttomonitoryourqualityontheleveloftheproduct,theprocessortheorganisa-tion(monitoringthequalityofthequalityassurancesystemwouldbeanotheroptionbutasstatedinchapter3wewillleavethisoutofconsiderationintherestofthishandbook).

5.2.4Thechoicesherearenotexclusive,anditmightbewisetogoforamixofoptions.Agoodwaytodeterminethismixmightbetotakeagoodlookatthetotalityoftheproposedmodel,sketchinga“map”ofthepossibleelementsfor internalqualityassuranceandindicatingwhichelementsyouhavepickedout.Suchasketchcouldforexamplelooklikethis:

5.2.5Thesketchtellsyouthatyoutakeintoaccountelementsfromproduct,processandorganisation.Italsomakesyouawarethatyouhavedefinedintheproductwhatyouthinkisimportant:themusicalquality,theentrepreneurialqualityandthepresentationalquality.Intheprocessandtheorganisa-tion,itatoncemakesclearthatyouhaveonlyselectedafewitemstolookat:forexample,intheprocessyouhavedecidedthatteacheractivities,media/environmentandevaluationareespeciallyimportant,andthatyouleaveoutsuchaspectsasstartingsituationorlearningcontent.

30 31

5.2.6Elaboratingabituponthis,youmayendupwiththefollowingelements: Product:

- musicalqualityoffinalexamination;- entrepreneurialqualityoffinalexamination;- presentationalqualityoffinalexamination.

Process:- qualityofevaluations/assessments,specificallyinthemainsubjectareas;- qualityofteachingandcoaching;- qualityoflearningenvironment,specificallyclassrooms,rehearsalrooms,instruments,audio-

visualsetcetera.

Organisation:- qualityofartisticleadership;- qualityofteachingstaff.

5.3 step 2: Pick your performance indicators

5.3.1 Onceyouhavedeterminedyourqualitygoals,youdeterminewhichperformanceindicatorsyouneedinordertodeterminethequality.

5.3.2Youroverviewcouldthenlooklikethis:

Qualitygoals: Performanceindicators:

Product: Product:

-musicalqualityoffinalexamination -satisfactionofpeers

-entrepreneurialqualityoffinalexamination -satisfactionofpeers

-presentationalqualityoffinalexamination -satisfactionofaudiencepresent

Process: Process:

-qualityofevaluations/assessments -satisfactionofprofessionalfield

-qualityofteachingandcoaching -satisfactionofstudents

-qualityoflearningenvironment -satisfactionofstudents

Organisation: Organisation:

-qualityofartisticleadership -satisfactionofgeneralpublic

-qualityofteachingstaff -percentage of teaching staff with own professional

practice

30 31

5.4 step 3: add outside demands

5.4.1 Upuntilnowyouhavebeenconcernedwithyourowncrucialqualitygoalsandperformanceindica-torstomonitor.Ofcourse,internalqualityassuranceinmostinstitutesisnotentirelyselfdriven.Quiteoften“outsiders”willaskyoutomonitoryourownqualityaswell.Suchanoutsidermightbethegovernmentwho,whenaccreditingyou,willlookintoyourinternalqualityassurancesystemandmayspecificallyaskforcertainelements.Forconservatoireswithinbiggeruniversities,thegeneraluniversitylevelmayplayacomparablerole.

5.4.2Anexamplemightbethe“StandardsandGuidelinesforQualityAssuranceintheEuropeanHigherEducationArea”20,adoptedbytheEuropeanMinstersofEducationin2005.AlthoughthisdocumenthasonaEuropeanlevelthestatusnotofalawbutofarecommendation,onemayexpectthatincer-taincountriesindividualinstitutionswillbeexpectedtocomplywiththisdocument.Inthatcasetheymayamongstothersbeobligedtocoverthefollowingitems:- studentprogressionandsuccessrates;- employabilityofgraduates;- students’satisfactionwiththeirprogrammes;- effectivenessofteachers;- profileofthestudentpopulation;- learningresourcesavailableandtheircosts.

5.4.3Ittheniswisetoincorporatethisintheinternalqualityassurancesystem.Ifthatisnotdone,onemayeveryroundofaccreditationagainhavetoproduce,withmucheffort,thesamesortofinformation.

5.5 step 4: pick the instruments

Nowyouknowwhichhardfactsandsatisfactionstatementsyouneed,youhavetodeterminehowyouwillgathertheseandwhatyourmeasuringunitwillbe.Forexample:

Qualitygoals: Instruments: Measuringunits:

Product:

-musicalqualityoffinalexamination -countbyadministrativeunit -gradesgivenbypeersatexamination(scale1-10)

-entrepreneurialqualityoffinal

examination

-countbyadministrativeunit -gradesgivenbypeersatexamination(scale1-10)

-presentationalqualityoffinal

examination

-audiencequestionnaire -levelofsatisfactionaudience(scale1-10)

Process:

-qualityofevaluations/assessments -questionnaireprofessional

field

-percentageofsatisfactionprofessionalfield

-qualityofteachingandcoaching -questionnairestudents -percentageofsatisfactionstudents

20 FormoreinformationabouttheEuropeanStandardsandGuidelines,pleaseseeAppendixB.

32 33

-qualityoflearningenvironment -questionnairestudents -percentageofsatisfactionstudents

Organisation:

-qualityofartisticleadership -countbymarketingunit -totalnumberofpositiveitemsinregional/national

newspapers

-qualityofteachingstaff -countbystaffunit -percentageofteachingstaffwithownprofessional

practice

5.6 step 5: formulate target results

5.6.1 Finallyyouhavetoformulatetargetresultsforyourperformanceindicators.Thismaybeahardthingtodo.Sometimesyouwillquitepreciselyormoreonaveragenowwhatkindsofresultsyouwouldliketoobtain.Sometimesitismoreanintuitivefeeling,maybesecondedbyresultofcomparableinstitutesyouknowormeasuredagainstabenchmarkinstituteyouhighlyrespect.Inourexample,youmightcomeupwiththefollowing:

Qualitygoals: Measuringunits Targetresult

Product:

-musicalqualityoffinalexamination -gradesgivenbypeersatexamination(scale1-10) -average7

-entrepreneurialqualityoffinal

examination

-gradesgivenbypeersatexamination(scale1-10) -average7

-presentationalqualityoffinalexamination -levelofsatisfactionaudience(scale1-10) -average7

Process:

-qualityofevaluations/assessments -percentageofsatisfactionprofessionalfield -70%

-qualityofteachingandcoaching -percentageofsatisfactionstudents -80%

-qualityoflearningenvironment -percentageofsatisfactionstudents -80%

Organisation:

-qualityofartisticleadership -totalnumberofpositiveitemsinregional/

nationalnewspapersperyear

-regional:10;national:2

-qualityofteachingstaff -percentageofteachingstaffwithown

professionalpractice

-90%

5.7 step 6: make an implementation plan

Onceyouhavemadesurewhatyouwantinyourinternalqualityassurancesystemandwhatotherswantintoit,makeanimplementationplan.Donottrytoimplementacompletesystemfromscratchinoneyear. Ifyouhavemanythingstostart, takeafewyears ifpossible.Quiteoften it turnsoutthatonceyou’vebuildupsomeexperiencewithquestionnaires,interviewsorthegatheringof“hardfacts”,introductionofnewelementstakeslesstime;whereasacompleteintroductionatoncemaycauseyoutodrawninaswampofdata.

32 33

5.8 step 7: describe

5.8.1 Onceyouhavecomethisfar,itiswisetotakeastepyoumighttendtoforgetbutwhichcanbeofenormousvaluefurtheronintheprocess.Make,foreveryelementofyourinternalqualityassur-ancesystem,amethodicaldescription:whatareyourgoals,whatareyourperformanceindicators,whichinstrumentdoyouuse,whatisthetargetresult,howoftenwillyoumeasure,whoisrespon-sibleformeasuring,whatistheprocedureyoufollowinordertomakethePDCA-circleafullcircle.

5.8.2Adescriptionofthiskindmightlooklikethis:

goal musicalqualityoffinalexamination

performanceindicator(s) satisfactionofpeers

instrument/measuringunitused gradingofexternalmembersofexaminationcommittee,scale1-10

targetresult average7

howoftenmeasurement twiceayear,aftertheexaminationroundsinJanuaryandJune

responsibleformeasuring administrativeunit,onthebasisoffilledoutexaminationprotocols

PDCAfullcircle aftermeasurement:feedbacktoprincipalandheadofstudies;resultsandmeasurestobe

takenonthebasisoftheresultstobeannouncedintheTeachersNewsletter

5.9 step 8: carry out

5.9.1 Yoursystemisnowfinishedanddocumented.Carryoutwhatyouplanned.Gatheractualresultsbymeasuring,comparethemtoyourtargetresults,analysethediscrepanciesandsearchforpos-siblereasons,makeadaptationsinyourcurriculumoryourorganisation.Thengathernewresults,analysetheminordertofindifyouhavemadesatisfactoryprogress.Ifnot,analyseagaincarefully;ifyouhavemadeprogress,monitorwhetheryouhavekeptyourstandardsup.

5.10 step 9: review the system regularly

5.10.1 Finally,asystemisnotmeantforeternity.Whatyouwilloftenfind,isthatasystemstartsverybasic,butwillgrowquitefastbecausequiteoftentheanalysisofyourresultsmakesyoufeelyouneedotherkindsofresultsaswellinordertomaketherightdecisions.Youmayendupwithanenormoussystemofdata-gatheringwhichintheendmaymakeithard,orevenvirtuallyimpossible,tokeepyouranalysisofgoodqualityandthereforetotakeanysounddecisionsatall.

5.10.2 Thatmaybethepointtomakesuretogetridofsomeofthemechanismswhichstillatfirstsightmayseemtobeusefulbutstrictlyspeakinghavenocrucialstrategicmeaninganymore.Youmaydecideeithertodividesuchanenormousinternalqualityassurancesystemintwoparts:thebasisisagreatsetofmeasurementsyoudo,buttherealanalysistakesintoaccountonlythoseelementsyouconsiderofstrategicimportance.Theotherpossibilityistostopmeasuringatallallthoseele-mentsyoudonotconsiderascrucialanymore.

34 35

5.11 summary and conclusions

5.11.1 Inthischapterwehave,onthebasisoftheprecedingchapters,proposedawayofdevisinganinter-nalqualityassurancesystem.Wesketchedanine-stepprocedurewhichenablesyoutoformulatequalitygoalsandperformanceindicators,choosemethodsandunitsofmeasuringandformulatetargetresults,plantheimplementationanddescribethesystem,carryoutthesystemandeventu-allyrevisethesystemifnecessary.

5.11.2 Tworemarksaretobemade.Oneisthatthisnine-steppathtoaninternalqualityassurancesystemmaydifferfrommuchoftheliteratureyouwillencounter.QuiteoftenliteraturewillstressthevirtuesofwhatiscalledTotalQualityManagement–systemswhichpretendto(andmaybeindeeddo)covercompleteorganisations.Wehavenothingagainstsuchelaboratesystemsandifyouconsiderim-plementingtheminyourconservatoire,donothesitate(butbeawareoftheamountofbureaucracythatmaybeattachedtoit).Wehave,however,chosenthestandpointthatformanyitwillbeeasiertostartbottomup,withfewbutcrucialquestionsoneasksoneself.

5.11.3 Theotherremarkisthatwehavedeliberatelynotpresentedtwoorthree“complete”systemsofinternalqualityassuranceinconservatoireshere.Itisourfirmbeliefthatagoodinternalqualityassurancesystemhasthelocalasbasis,andthesectoral/national,Europeanandglobalascontext.Becausesuchasystemissodependentoncontext,itishardlyeverpossibletointegrallytransportitfromoneinstitutetotheother,evenwithincountries.Wedohavehowever,inAppendixA,compiledasetofinstrumentsusedininternalqualityassuranceinordertogiveyouanideaofthepossibili-ties.

34 35

aPPendICes

3736

37

aPPendIx a. Internal Quality assurance in Conservatoires – some examples

a.1 Introduction

A.1.1 Inthisappendixwepresentelementsofthesystemsofinternalqualityassuranceoftwoconserva-toires:theRoyalCollegeofMusicinLondon,UK,andthePrinceClausConservatoire,partoftheHanzeUniversityofAppliedSciences inGroningen,theNetherlands.TheexampleswereathandastheauthorisconnectedtothePrinceClausConservatoireandthechairofthePolifoniaTuningworkinggroupistotheRoyalCollegeofMusic.Intermsofqualityassuranceproceduresbothareinterestingandbotharedifferentfromeachother.

A.1.2 TheRoyalCollegeofMusicisanindependentcollege(slightlyover600students)withitsownde-gree-awardingpower,offeringonlymusicprogrammes.Ithascarefullydeviseditsqualityassurancesystemsforthepurposeofrigorousqualitycontrolcompletelygearedtowardsthespiritofmusiceducation.

A.1.3 ThePrinceClausConservatoireisoneoftheeighteenschoolsofabigmultisectoraluniversityofap-pliessciences(about22.000students)withaseparateandstrongcentralqualityassurancedepart-mentworkingforalltheacademies.TheinternalqualityassurancesystemisexplicitlydevisedintermsofaformofTotalQualityManagement,withavarietyofgeneralinstrumentstobeusedwhichcanpartlybegearedtowardstheneedsofthevariousacademies.

a.2 the Quality assurance system of the royal College of music, london (uk)

A.2.1 TheRoyalCollegeofMusichas itssystemofqualityassurance laiddown inaQualityAssuranceHandbookofapproximately25pages1.Thehandbooknotonlygivesanoverviewofthequalityassur-anceproceduresoftheRCMbutstatestheprinciplesbehindthemaswell.Whatfollowsbelowisatextconsistingofquotesfromthehandbook.ThetextdoesnotgivethecompletepictureofthequalityassurancesystemoftheRoyalCollegeofMusicbuthighlightsthegeneralprinciplesaswellassomeverymusic-specificelements.

general features

A.2.2“Theobjectivesofqualityassuranceat theRoyalCollegeofMusicare: toassuretheprovisionofthehighestqualitymusicconservatoireprogrammespossible,withinavailableresources,ofanin-ternationalstandardrelevanttothemusicprofessionandtoenhancethequalityoflearningandteach-ingbyprovidinganenvironmentwhichsupportstheirdevelopment.

1 Seewww.rcm.ac.uk/cache/fl0000907.pdf.

38

A.2.3The quality assurance processes encompass: programmes, including teaching and performanceactivitiesandadmissionsprocesses;learningandteachingsupportservicesandresources;assess-mentandstandardsofoutcomes,includingperformancestandards.(…)

A.2.4IndeliveringtheseobjectivestheRCM’squalityassurancesystemshavefivearms:- programmedevelopment,approval,reviewandmonitoring;- externalexaminers,externalspecialists,internalexaminers(includingauditionsforentry);- studentengagement;- reviewofperformancestandards;- professionaldevelopment,appraisalandpeerobservation.(…)

A.2.5PeriodicallytheCollegeevaluatestheeffectivenessofitsapproachtoqualityassurance.Incommonwithotherhighereducationinstitutions,theCollegeissubjecttoinstitutionalauditbythenationalqualityassurancebody(currentlytheQualityAssuranceAgencyforHigherEducation).Inpractice,therefore,forreasonsofefficiency,itconductsreviewsofitsapproachtoqualityassurancewiththecycleofQAAaudit,seeingtheQAAaudititselfastheculminationofthereviewprocess.TheCollegewelcomesthisexternalprocessasanimportantpartofitsownreviewprocess,providingavaluableexternalperspective.”(p.2-3)

programme approval and programme review

A.2.6“Initialprogrammeapprovalandsubsequentprogrammereviewbothinvolveadevelopmentprocessleadingtoanextended,usuallyfullday,meetingofapanelofexternalacademicsandmusicprofes-sionalsandinternalpeerswhichrecommendseithertheapprovalorotherwiseoftheprogrammeandanyrecommendationsorconditions,followingdiscussionswithprofessorialstaff,administra-tors,representativesofthelearningresources,andstudents.AreportisproducedbythepanelandsubmittedtotheBoardofProfessorsforapproval.Therelevantprogrammescommitteeconsidersandrespondstoanyconditionsand/orrecommendations.Programmereviewisdesignedtobeakeyqualityenhancementprocess.”(p.3)

A.2.7“Programmereviewisaprocessofreflectivereviewconductedeveryfive-sixyears.Itdrawsonan-nual monitoring and provides an opportunity for making significant structural changes to a pro-gramme.”(p.5)

criteria for programme approval and review

A.2.8“Theissuesbelowshouldbeaddressedbyprogrammedeveloperswhenpreparingforinitialapprov-alandreview.Initialapprovalandreviewpanelswillwishtosatisfythemselvesthattheyhavebeenadequatelyaddressedeitherbythedocumentationoraspartoftheirdiscussionswithprogrammedevelopmentteams.Ineffect,thesearethecriteriaforinitialorcontinuingprogrammeapproval.

Admission

- Dotheadmissionrequirementsrelatetotheaimsandobjectivesandcontentoftheprogramme?- DotheadmissionrequirementsandcriteriarelatetotheCollege’sequalopportunitiespolicyand

admissionspolicy?- Whatisthescopeforadmissionwithacademiccreditfromanotherinstitution?

Learning outcomes

- Whataretheintendedlearningoutcomesfortheprogramme?- Howdotheyrelatetoexternalreferencepointsincludingthemusicsubjectbenchmarkingstate-

ment(inthecaseoftheBachelorofMusic)andthenationalqualificationsframework?- Howdotheyrelatetotheoverallaimsoftheprogramme?- Aretheyappropriatetotheaims?- Howdoesthedesignandorganisationofthecurriculumpromotestudentlearningandachieve-

mentoftheintendedlearningoutcomes?- Howaretheintendedoutcomesoftheprogrammecommunicatedtostaff,studentsandexternal

examiners?- Dothestudentsknowwhatisexpectedofthem?

Achievement of the intended learning outcomes

- Howdoestheprogrammeencourageachievementoftheintendedlearningoutcomesintermsofknowledgeandunderstanding,specificpracticalskills,transferableskills,progressiontoca-reersasperformersorcomposersand/ortofurtherstudy,andpersonaldevelopment?

- Istheprogrammecontentanddesigninformedbyrecentdevelopmentsintechniquesofteachingandlearning,bycurrentresearchandscholarship,andbychangesinthemusicprofession?

Assessment process and standards

- Doestheassessmentprocessenablestudentstodemonstrateachievementoftheintendedout-comes?

- Aretheresatisfactorycriteriatoenableinternalandexternalexaminerstodistinguishbetweendifferentcategoriesofachievement?

- Cantherebefullconfidenceinthesecurityandintegrityofassessmentprocedures?- Doestheassessmentstrategyhaveanadequateformativefunctionindevelopingstudentabili-

ties?- Whatevidenceistherethatthestandardsachievedbystudentsmeettheminimumexpectations

fortheaward,asmeasuredagainstrelevantsubjectbenchmarksandthequalificationsframe-work?

- Howdoestheprogrammedevelopmentteamreviewandseektoenhancestandards?

Learning opportunities and effective teaching

- Howeffectiveisteachinginrelationtocurriculumcontentandprogrammeaims?- How effectively do professors draw upon their performance practice, composition, research,

scholarshiporotherprofessionalactivitytoinformtheirteaching?

39

40 41

- Isthereeffectiveengagementwithandparticipationbystudents?- Isthequalityofteachingmaintainedandenhancedthrougheffectiveprofessionaldevelopment,

peerappraisalofteaching,effectiveinductionofnewstaff?- Arestudentworkloadsmanageable?

Student progression, academic support and equal opportunities

- Howeffectivelyislearningfacilitatedbyacademicguidance,feedbackandsupervisoryarrange-ments?

- Arethearrangementsforacademictutorialsupportclearandgenerallyunderstoodbystaffandstudents?

- Whatarrangementsareinplacetosupportstudentswithdisabilities?- Howdoestheprogrammetakeaccountofthedutytopromoteequalopportunities/diversity,in-

cludingracialequality?

Learning resources

- Isthecollectiveexpertiseoftheprofessorialstaffsuitableandavailableforeffectivedeliveryoftheprogramme,fortheoverallteaching,learningandassessmentstrategyandfortheachieve-mentoftheintendedlearningoutcomes?

- Areappropriateprofessionaldevelopmentopportunitiesavailable?- Isappropriateadministrativesupportavailable?- HoweffectivelyislearningpromotedbytheCollege’slearningresources?- Issuitableteachingandlearningaccommodationavailable?- Arethelibrarystocksandservicesappropriateandaccessible?- AresuitablespecialistequipmentandappropriateITfacilitiesavailabletostudents?- Isthecareerguidanceserviceadequateandappropriatefortheprogramme?

Programme management

- Whatarethearrangementsforprogrammemanagement?- Dotheyensuretheeffectivedivisionofresponsibilitiesfordifferentaspectsoftheprogramme’s

operation?- Wheretheproposalinvolvesliaisonwithotherinstitutions,whatarethemechanismsformain-

tainingandmonitoringtheserelationships?

Resources

- Are theresources identified in theresourcestatementadequateandappropriate for thepro-gramme?”(p.9-11)

programme monitoring

A.2.9 “TheCollege’sannualprogrammemonitoringsystemisaprocessofqualitycontrol,whichfeedsintoperiodicprogrammereviews.Itisintendedtobeaconcisemechanismwhichreflectsonthepreviousyear’sstandardsofachievement,addressespointsinexternalexaminerreports,andsets

40 41

anactionlistforthecomingacademicyear.Likeprogrammereview,annualmonitoringisdesignedtobeakeyqualityenhancementprocess.Therelevantheadofprogrammescompilesthereport,whichissubmittedtotherelevantprogrammescommittee.”(p.3)

student participation

A.2.10 “Avarietyofstudentengagementandconsultationmethodsareused:- questionnaire-basedfeedback- consultationsessionswithstudents(yearorpathwaygroupsorrepresentatives,forexampleby

externalexaminers)- studentrepresentationonacademiccommittees- sessionswithstudentsaspartofinitialapprovalandreviewevents- regularliaisonwiththeStudents’Association,viatheStudentServicesManager- Director’sweeklyopensurgeriesforstaffandstudents.”(p.16)

external examiners

A.2.11 “The College has procedures for the appointment of external examiners and a published policystatementabout theirdutiesandresponsibilities.Externalexaminernominationsoriginate fromtheprogrammescommittee,andaresubjecttotheapprovaloftheBoardofProfessors.ExternalexaminersarerequiredtosubmitannualreportstotheDirectorofAcademic&AdministrativeAf-fairs,whocirculatesthemtotheDirector,Dean,therelevantheadofprogrammes,andtherelevantregistryadministrator.Therelevantheadofprogrammesisresponsibleforrespondingtoreports,usuallyinthecontextoftheannualprogrammemonitoringreport,acopyofwhichisprovidedtotheexternalexamineronceithasbeenfinalisedbytherelevantprogrammescommittee.

A.2.12 Externalspecialistsareincludedonallpanelsforgraduationorfinalrecitalassessments.Theyarenominatedbyheadsoffaculty.Internalexaminersforallpracticalexaminationsandforentranceauditionsarenominatedbyheadsoffaculty.DetailednotesfortheguidanceofexaminersandforthoseconductingauditionsarereviewedeachyearbytheAAMG[ArtisticandAcademicManage-mentGroup]andprovidedforallexaminers.Copiesofthecriteriatobeappliedaremadeavailableforstudentstoensurethattheyarefullyawareoftheparametersagainstwhichtheywillbeas-sessed.”(p.4)

review of performance standards

A.2.13 “Highperformancestandardsareacornerstoneofworldclassconservatoireeducation.Forthisreason,evidenceofthelevelofstandardsachievediscarefullymonitoredandreviewedonperiodicbasis.Theevidencebaseusedcomprisesreportsrequestedfromvisitingconductors,thosegivingmasterclassesorworkshops,prizecompetitionadjudicators,andexternalspecialistexaminers.

42 43

Annualoverviewreports,which includeananalysisof thequantitativeandqualitativedata fromreports,togetherwithexternalpressreportsofRCMperformancesandarecordofkeystudentsuc-cessesinexternalcompetitionsandotherperformance-relatedactivities,arecompiledbytheDeanandaresubmittedtotheBoardofProfessorsforapproval.”(p.4)

staff

A.2.14“Theknowledge,experienceandskillofmembersofstaff isoneof theCollege’smostvaluableassets.Tofulfilitsresponsibilitiestostudents,theCollegemustensurethatthequalityofthisre-sourceiscontinuallyenhanced.TheCollegeprofessionaldevelopmentpolicyincludesprovisionforthesupportofindividualandgroupproject-basedstaffprofessionaldevelopmentactivities.Profes-sionaldevelopmentisacorepartofstaffappraisalsystems.Monitoringandevaluationreportsonprofessional development are considered annually by the Board of Professors. The College alsosupportspeerobservationthroughitsLearningandTeachingStrategy.”(p.4)

a.3 the quality assurance system of the Prince Claus Conservatoire, Hanze university of applied sciences, groningen (nl)

A.3.1 ThePrinceClausConservatoirehasitssystemofinternalqualityassurancelaiddowninanannualQualityAssurancePlan(approx.35pages)2.TheQualityAssurancePlaniswrittenconformingtoauniversity-wideformat,whichlinkstheplantoanarrayofotherdocuments,proceduresandinstru-ments,bothuniversity-wideasacademy-specific.

2 HanzeUniversityofAppliedScience,PrinceClausConservatoire:“Kwaliteitszorgplanenprocedurehandboek2007-2008”,

version0.2.

42 43

A.3.2TheHanzeUniversityofAppliedSciencehasdefinedqualityassuranceasthepermanent,system-aticandcyclicalattentionformeasuring,monitoringandfosteringquality.IthasbaseditssystemofqualityassuranceonthePDCA-circle,representedvisuallyasfollows:

3 ManagementDashboard:aninstrumentshowinginrealtimetherelationbetweenactualresultsandtargetresultsforavariety

ofperformanceindicators

UniversityStrategicPlan

AcademyStrategicPlan

AcademyYearPlan

Managementcontract

Academyspecific

measurements

Academyreports Managementdashboard3

Prestation-Evaluation

Satisfaction

statements

Results

(hardfacts)

adapt

check

Prestation measurements

do

plan

Goals Performanceindicators

Goals Performanceindicators

Goals Performanceindicators

Activities

Goals Performanceindicators

44 45

A.3.3 Crucial inthisschemeistheconnectionbetweenthephaseofplanningandthephasesofcheckandadapt.Intheplanningphase,fortheuniversityasawholeaswellasforitseighteenacademiesgoalsandperformanceindicatorsareformulatedinvariousdocuments,rangingfromtheuniversitystrategytotheyearlymanagementcontractsbetweentheuniversityboardandthedeanofeachoftheacademies.

A.3.4 Inthecheckphase,satisfactionstatementsaswellasresults(“hardfacts”)aregatheredrelatedtothevariousgoalsandperformanceindicators.Thisispartlydonecentrallyforthewholeuniversitywithgenericinstruments,andpartlyhastobedonebytheacademiesthemselves.Satisfactionstate-mentsandresultsarepresentedinthegeneralmanagementdashboardandinthevariousreportsonacademylevel.Thephaseofadaptconsistsofstudyanddiscussionofthesedocumentsinordertodecideonfutureactionswhicharethanfedbackintotheplanningphase.

A.3.5 Thiswholecycleisforeveryacademy,soalsoforthePrinceClausConservatoire,laiddownintheQualityAssurancePlan.TheQualityAssurancePlancontains:- ageneralintroductiontothesystemofinternalqualityassurance;- thequalitygoalsandperformanceindicatorsfortheyearconcerned- anoverviewofthepersonsplayingaroleinqualityassurance,ofthedecisionmakingprocesses,

oftherelevantdocumentsandofthecommunicationplanconnectedtoqualityassuranceactivi-ties

- atimepathforthequalityassuranceactivities- schemesforeveryindividualqualityassuranceactivity:

Instrument

Responsibleperson

Goaloftheactivity

Normvalue

Reportby

Reportedto

Reportedbymeansof

Frequencyanddeadline(s)

Shortdescriptionofactivity(plan/do) Date Activity Executedby

Procedureleadingtoqualityimprove-

ment(check/adapt)

Remarks

44 45

aPPendIx B the enQa 2005 standards and guidelines for Quality assurance in the european Higher education area (esg) – summary version

ThissummarylistofEuropeanstandardsforqualityassuranceinhighereducationisdrawnfromChapter2of the full reportand isplacedhere foreaseofreference. Itomits theaccompanyingguidelines.Thestandardsareinthreepartscoveringinternalqualityassuranceofhighereducationinstitutions,externalqualityassuranceofhighereducation,andqualityassuranceofexternalqual-ityassuranceagencies.

Part 1: european standards and guidelines for internal quality assurance within higher education institutions

B.1.1 Policyandproceduresforqualityassurance:Institutionsshouldhaveapolicyandassociatedproce-duresfortheassuranceofthequalityandstandardsoftheirprogrammesandawards.Theyshouldalsocommitthemselvesexplicitlytothedevelopmentofaculturewhichrecognisestheimportanceofquality,andqualityassurance,intheirwork.Toachievethis,institutionsshoulddevelopandim-plementastrategyforthecontinuousenhancementofquality.Thestrategy,policyandproceduresshouldhaveaformalstatusandbepubliclyavailable.Theyshouldalsoincludearoleforstudentsandotherstakeholders.

B.1.2 Approval, monitoring and periodic review of programmes and awards: Institutions should haveformal mechanisms for the approval, periodic review and monitoring of their programmes andawards.

B.1.3 Assessment of students: Students should be assessed using published criteria, regulations andprocedureswhichareappliedconsistently.

B.1.4 Qualityassuranceofteachingstaff:Institutionsshouldhavewaysofsatisfyingthemselvesthatstaffinvolvedwiththeteachingofstudentsisqualifiedandcompetenttodoso.Theyshouldbeavailabletothoseundertakingexternalreviews,andcommenteduponinreports.

B.1.5 Learningresourcesandstudentsupport:Institutionsshouldensurethattheresourcesavailableforthesupportofstudentlearningareadequateandappropriateforeachprogrammeoffered.

B.1.6 Informationsystems:Institutionsshouldensurethattheycollect,analyseanduserelevantinforma-tionfortheeffectivemanagementoftheirprogrammesofstudyandotheractivities.

B.1.7 Publicinformation:Institutionsshouldregularlypublishuptodate,impartialandobjectiveinforma-tion,bothquantitativeandqualitative,abouttheprogrammesandawardstheyareoffering.

46 47

Part 2: european standards for the external quality assurance of higher edu- cation

B.2.1 Useofinternalqualityassuranceprocedures:ExternalqualityassuranceproceduresshouldtakeintoaccounttheeffectivenessoftheinternalqualityassuranceprocessesdescribedinPart1oftheEuropeanStandardsandGuidelines.

B.2.2Developmentofexternalqualityassuranceprocesses:Theaimsandobjectivesofqualityassuranceprocessesshouldbedeterminedbeforetheprocessesthemselvesaredeveloped,byallthosere-sponsible(includinghighereducationinstitutions)andshouldbepublishedwithadescriptionoftheprocedurestobeused.

B.2.3Criteriafordecisions:Anyformaldecisionsmadeasaresultofanexternalqualityassuranceactivityshouldbebasedonexplicitpublishedcriteriathatareappliedconsistently.

B.2.4Processesfitforpurpose:Allexternalqualityassuranceprocessesshouldbedesignedspecificallytoensuretheirfitnesstoachievetheaimsandobjectivessetforthem.

B.2.5Reporting:Reportsshouldbepublishedandshouldbewritteninastyle,whichisclearandread-ilyaccessibletoitsintendedreadership.Anydecisions,commendationsorrecommendationscon-tainedinreportsshouldbeeasyforareadertofind.

B.2.6Follow-upprocedures:Qualityassuranceprocesseswhichcontainrecommendationsforactionorwhichrequireasubsequentactionplan,shouldhaveapredeterminedfollow-upprocedurewhichisimplementedconsistently.

B.2.7Periodicreviews:Externalqualityassuranceofinstitutionsand/orprogrammesshouldbeunder-takenonacyclicalbasis.Thelengthofthecycleandthereviewprocedurestobeusedshouldbeclearlydefinedandpublishedinadvance.

B.2.8System-wideanalyses:Qualityassuranceagenciesshouldproducefromtimetotimesummaryre-portsdescribingandanalysingthegeneralfindingsoftheirreviews,evaluations,assessmentsetc.

Part 3: european standards for external quality assurance agencies

B.3.1 Useofexternalqualityassuranceproceduresforhighereducation:Theexternalqualityassuranceofagenciesshouldtakeintoaccountthepresenceandeffectivenessoftheexternalqualityassur-anceprocessesdescribedinPart2oftheEuropeanStandardsandGuidelines.

46 47

B.3.2Officialstatus:AgenciesshouldbeformallyrecognisedbycompetentpublicauthoritiesintheEu-ropeanHigherEducationAreaasagencieswithresponsibilitiesforexternalqualityassuranceandshouldhaveanestablishedlegalbasis.Theyshouldcomplywithanyrequirementsofthelegislativejurisdictionswithinwhichtheyoperate.

B.3.3Activities:Agenciesshouldundertakeexternalqualityassuranceactivities(atinstitutionalorpro-grammelevel)onaregularbasis.

B.3.4Resources: Agencies should have adequate and proportional resources, both human and finan-cial,toenablethemtoorganiseandruntheirexternalqualityassuranceprocess(es)inaneffectiveandefficientmanner,withappropriateprovisionforthedevelopmentoftheirprocessesandproce-dures.

B.3.5Missionstatement:Agenciesshouldhaveclearandexplicitgoalsandobjectivesfortheirwork,con-tainedinapubliclyavailablestatement.

B.3.6Independence:Agenciesshouldbeindependenttotheextentboththattheyhaveautonomousre-sponsibilityfortheiroperationsandthattheconclusionsandrecommendationsmadeintheirre-portscannotbeinfluencedbythirdpartiessuchashighereducationinstitutions,ministriesorotherstakeholders.

B.3.7Externalqualityassurancecriteriaandprocessesusedbytheagencies:Theprocesses,criteriaandproceduresusedbyagenciesshouldbepre-definedandpubliclyavailable.Theseprocesseswillnormallybeexpectedtoinclude:- Aself-assessmentorequivalentprocedurebythesubjectofthequalityassuranceprocess;- Anexternalassessmentbyagroupofexperts,including,asappropriate,(a)studentmember(s),

andsitevisitsasdecidedbytheagency;- Publicationofareport,includinganydecisions,recommendationsorotherformaloutcomes;- Afollow-upproceduretoreviewactionstakenbythesubjectofthequalityassuranceprocessin

thelightofanyrecommendationscontainedinthereport.

B.3.8Accountabilityprocedures:Agenciesshouldhaveinplaceproceduresfortheirownaccountability.

48

aPPendIx C. list of abbreviations used

AEC AssociationEuropéennedesConservatoires,AcadémiesdeMusiqueetMusikhochschulen

EFQM EuropeanFoundationforQualityManagement

ENQAEuropeanAssociationforQualityAssuranceinHigherEducation

EUA EuropeanUniversityAssociation

PDCA Plan-Do-Check-Adapt

QAA QualityAssuranceAgency

RCM RoyalCollegeofMusic

Association Européenne des Conservatoires,Académies de Musique et Musikhochschulen (AEC)PO Box 805 NL-3500AV Utrecht The Netherlands

Tel +31.302361242 Fax +31.302361290Email aecinfo@aecinfo.org Website www.aecinfo.org