aec handbook - internal quality assurance in higher music education
Post on 08-Apr-2016
224 Views
Preview:
DESCRIPTION
TRANSCRIPT
AEC PUBLICATIONs 2007
handbook
internal quality assurance in
higher music education
evert bisschop boele
The Polifonia project has been funded with support from the European Commission. This publication reflects the views of its authors and the Commission cannot be held responsible for any use which might be made of the information contained therein.
A free electronic version of this handbook is available through www.polifonia-tn.org.
handbook
internal quality assurance in
higher music education
evert bisschop boele
Contents
Foreword1. introduction 2. some preliminary remarks on internal quality assurance 2.1 Quality 2.2 Assurance 2.3 Internal3. aiming for quality 3.1 Introduction:amodel 3.2 Theproduct 3.3 Theprocess 3.4 Theorganisation 3.5 Thequalityassurancesystem 3.6 Summaryandconclusion4. measuring quality 4.1 Hardfactsandsatisfactionstatements 4.2 Hardfacts 4.3 Satisfactionstatements 4.4 Summaryandconclusions5. setting up an internal quality assurance system in your conservatoire 5.1 Introduction 5.2 Step1 Stateyourqualitygoals 5.3 Step2 Pickyourperformanceindicators 5.4 Step3 Addoutsidedemands 5.5 Step4 Picktheinstruments 5.6 Step5 Formulatetargetresults 5.7 Step6 Makeanimplementationplan 5.8 Step7 Describe 5.9 Step8 Carryout 5.10 Step9 Reviewthesystemregularly 5.11 Summaryandconclusionsappendicesappendix a InternalQualityAssuranceinConservatoires–SomeExamples A.1 Introduction A.2 ThequalityassurancesystemoftheRoyalCollegeofMusic,London(UK) A.3 ThequalityassurancesystemofthePrinceClausConservatoire,Hanze UniversityofAppliedSciences,Groningen(NL)appendix b TheENQA2005StandardsandGuidelinesforQualityAssuranceintheEuropean HigherEducationArea(ESG)–Summaryversionappendix c ListofAbbreviationsUsed
5788911131314161719192121222326282828303131323233333334
37
373742
45
48
5
5
Foreword
TheERASMUSThematicNetwork forMusic “Polifonia”1, the largestEuropeanprojectonprofes-sionalmusictrainingtodate,involved67organisationsinprofessionalmusictrainingandthemusicprofessionfrom32Europeancountriesand30expertsin5connectedworkinggroupsinanintensive3-yearworkprogrammefromSeptember2004–October2007.Theproject,whichwascoordinatedjointlybytheMalmöAcademyofMusic–LundUniversityandtheAssociationEuropéennedesCon-servatoires,AcadémiesdeMusiqueetMusikhochschulen(AEC),receivedsupportfromtheEuropeanUnionwithintheframeworkoftheERASMUSProgramme.Theaimsoftheprojectwere:1. TostudyissuesconnectedtotheBolognaDeclarationProcess,suchasthedevelopmentoflearn-
ingoutcomesfor1st(Bachelor),2nd(Master)and3rdcyclestudiesthroughthe‘Tuning’method-ology2,theuseofcreditpointsystems,curriculumdevelopment,mobilityofstudentsandteach-ers,andqualityassuranceinthefieldofmusicinhighereducation.
2. Tocollect informationon levels inmusiceducationother than the 1st (Bachelor)and the2nd(Master)studycycles,inparticularpre-collegetrainingand3rdcycle(Doctorate/PhD)studiesinthefieldofmusic.
3. Toexploreinternationaltrendsandchangesinthemusicprofessionandtheirimplicationsforprofessionalmusictraining.
WiththeaimtoparticipateinthediscussionstakingplaceinthehighermusiceducationsectorandintheframeworkoftheBolognaprocess,theAECformedwithin“Polifonia”agroupwiththefollow-ingexperts:
• JeremyCox(Chair-RoyalCollegeofMusic,London)• HannuApajalahti(SibeliusAcademy,Helsinki)• EvertBisschopBoele(HanzehogeschoolGroningen)• CristinaBritodaCruz(EscolaSuperiordeMúsicadeLisboa)• BrunoCarioti(ConservatorioStatalediMusica“AlfredoCasella”,L’Aquila)• GrzegorzKurzynski(K.LipinskiAcademyofMusic,Wroclaw)• JörgLinowitzki(MusikhochschuleLübeck)• JacquesMoreau(CNSMDdeLyon)
1 Seeformoreinformationabout“Polifonia”www.polifonia-tn.org.
2 Formoreinformationaboutthe“Tuning”methodologypleaseseehttp://www.tuning.unideusto.org/tuningeu/index.php?option
=content&task=view&id=172&Itemid=205.
6 7
InordertoassisthighermusiceducationinstitutionswiththerequirementsproposedbytheBolognaprocess,the“Polifonia”projectissuedaseriesofpublicationsthatcanbeusedbytheinstitutionsinthedevelopmentoftheirstudyprogrammes:
• Severalpracticalhandbookson:- CurriculumDesignandDevelopmentinHigherMusicEducation- ImplementationandUseofCreditPointsinHigherMusicEducation- InternalQualityAssuranceinHigherMusicEducation• Adocumententitled“SummaryofTuningFindings–HigherMusicEducation”,whichcontainsthe
AECLearningOutcomesforthe1st,2ndand3rdcycles,aswellasthe“Polifonia/DublinDescrip-tors”
• Atrilingualwebsitecalled“BolognaandMusic”(www.bologna-and-music.org),whereallrele-vantdocumentationinrelationtotheBolognaprocessseenfromtheperspectiveofhighermusiceducationcanbefound.
Inaddition,theAECproject“AccreditationinEuropeanProfessionalMusicTraining”3producedsev-eralimportantdocumentsaddressingexternalqualityassuranceandaccreditationinmusic.
3 Moreinformationaboutthisprojectcanbefoundatwww.bologna-and-music.org/accreditation.
6 7
1 . IntroduCtIon
1.1 Thishandbookismeanttobeashortguideforconservatoireswhowanttodevelopasystemforin-ternalqualityassurance.
1.2 Itdoesnotaimatacompleteintroductionintotheworldofqualityassurance,withitsmanydifferentsortsofsystemsanditselaborateandsometimesconfusingjargon.Thishandbookexplainsinthefirstfourchaptersinasimplewaythemainelementsofapossibleinternalqualityassurancesystem.ChapterfiveandappendixAgiveyousomepracticalassistancebypresentingasimpleprocedurewhichyoumightuseindevelopingafirstsystemforinternalqualityassurance,andbypresentingthesystemsforinternalqualityassuranceoftwoEuropeanconservatoires.
1.3 Thispublication iswrittenforpeopleworkingwithinaconservatoire,withagoodoverviewof theinstitute,maybewithmanagementresponsibilitiessuchasaheadofdepartmentormaybeevenaprinciple,orinothercasesasastaffmemberwhois,onbehalfofthemanagement,askedtodevelopaninternalqualityassurancesystem.
1.4 Workon thishandbookhasstartedwithgatheringsome insight in thecurrentstateofaffairsofinternalqualityassuranceinconservatoiresbymeansofaquestionnaire.69conservatoiresfilledoutthequestionnaire.Roughly,twothirdofthemstatedexplicitlytoworkwithorworkonasystemofinternalqualityassurance4.Furtherexaminationoftheanswersgiveninthequestionnaires,andsubsequentdiscussionswithpeoplefromwithinvariousconservatoires,suggestedhoweverthatac-tuallymuchmoreconservatoiresareworkingoninternalqualityassurance,butmanydonotlabeltheiractivitiesonthefosteringofqualityintermsofinternalqualityassurance.IthankthoseoftenunknowncolleaguesalloverEuropewhoansweredthequestionnaireandthosewhoengagedwithmeindiscussionsonqualityassurance.
1.5 IalsothankthemembersoftheTuningworkinggroupofthePolifoniaproject,andmycolleaguesfromHanzeUniversityofAppliedScience inGroningen, theNetherlands, for their insightfuldis-cussionsand theircarefulcommentsonvariousdraftsof thishandbook. I finally thankmydearcolleaguesfromtheAECOfficeinUtrechtfortheirpatienceandlovingtenacity.Withoutthemthishandbookwouldprobablyneverhavebeenfinished.
4 Theresultsofthisquestionnaireoninternalqualityassurancecanbefoundat:http://www.bologna-and-music.org/internalqa.
9
2. some PrelImInary remarks on Internal QualIty assuranCe
2.1 quality
2.1.1 “Quality”,andspecificallyartisticormusicalquality,hasbeenaboutthemostimportantconceptofconservatoireeducationsinceitsverybeginning.Musicalqualitytendstotaketheshapeofanidealwhichcanneverbefullyreached,butmustbeaimedatcontinuously.Ittakesdifferentformsandshapes,andisoftenimmediatelyrecognizedbuthardtoputinwords.
2.1.2 Assuringthatqualityisofferedandbeingaccountableforthequalityyoudeliverhasalwaysbeencentraltoconservatoireeducation.Inarecentdocument“Quality,Assurance,Accountability:ABrief-ingPaper”5itisputasfollows:
Music study is permeated with accountability. (…) In practice sessions, rehearsal, and even in perform-
ance, constant evaluation and adjustment are the norm. The success of professional music study is evalu-
ated in the light of the high standards and high expectations of the larger musical world. Tours, recordings,
and international competition continue to define professional expectations by exchange of work at the
highest levels. In music, we have standards because we have art, not art because we have standards.
2.1.3 Thenormalwayofassuringthatthehighestmusicalqualityisdeliveredistohavethequalitytestedandverifiedbymusicianswhoareacknowledgedasoutstanding,bytheirpeersandbysociety ingeneral.Becausemusicalqualityissuchafuzzyconceptandcantakedifferentformsandshapes,inmanycasestheassessmentofmusicalqualityistrustedtonotone,butseveraloutstandingmusi-ciansatthesametime.Acommitteeorjurydeliberatesinallfreedomandtriestoreachaconclu-sionthatcanbesharedbyall.Musicalqualityisthusformulatedin,orthrough,an“inter-subjectivedebate”.
2.1.4 Qualityassurance,includinginternalqualityassurance,isatpresentakindofbuzzwordamongsthighereducationpolicymakers. Ithassince thebeginningbeena topic in theBolognaprocess6,hasgrowninimportanceovertheyearsandisspecificallyreferredtointheBergenCommuniqué(2005)7andfurtherdevelopedinthedocument“StandardsandGuidelinesforQualityAssuranceintheEuropeanHigherEducationArea”8bytheEuropeanAssociationforQualityAssuranceinHigher
5 Seewww.arts-accredit.org/site/docs/pdf/10-MSMAP-Quality-Assurance-Accountability-BriefingPaper.pdf.Thepaperwaspub-
lishedwithinthe“MusicStudy,MobilityandAccountabilityProject”,ajointprojectoftheEuropeanAssociationofConservatoires
(AEC)andtheNationalAssociationofSchoolsofMusic(NASM;UnitedStates).
6 FormoreinformationontheBologna-processandhighermusiceducation,see:www.bologna-and-music.org.
7 Seewww.bologna-bergen2005.no/Docs/00-Main_doc/050520_Bergen_Communique.pdf.
8 Seewww.enqa.eu/files/BergenReport210205.pdf.
8
9
EducationENQA,adoptedinBergenaswell.Manykindsoforganisationsareinonewayoranotherdealingwithit,andithasitsownspecialistsanditsownjargon.
2.1.5 Thispublicationwillfocusoninternalqualityassuranceandwilltrytorelatetheinternationaldis-cussionon(internal)qualityassuranceinhighereducationtohighermusiceducationastaughtinconservatoiresalloverEurope.Itisinevitablethatsomeofthequalityassurancejargonwilltrickleintothispublication,andthatatsometimesyou,thereader,mayfindhismindabittroubled–“Arewestilltalkingaboutqualityhere?”,isaquestionthatmightoccur.
2.1.6 Itisthereforegoodtoremindourselvesatthebeginningofthishandbookthatwearenottalkingaboutatotallynewconceptcalled“quality”thatneedstobeintroducedtoconservatoiresbecausetheyhaveneverthoughtaboutitthemselves.Conservatoireshavebeentalkingaboutqualitycon-tinuously,andasmusicianswequiteoftenrecognizeiteasilyandfromalongdistance.Eventuallyeverythingdescribedinthispublicationshouldservethegoaloffosteringthisprimaryqualitywithinoureducation.
2.2 assurance
2.2.1 Inessence,theassuranceofqualityisnothingmorethansecuringthatoneisofferingthequalityonepromisestooffer.This,again,isnothingnewforconservatoires.Foralongtime,assuringthegen-eralpublicthatoutstandingqualitywasofferedbyconservatoireswasnotproblematic.Conserva-toireswereexpectedtooffermusicaleducationofthehighestpossiblequality,andthegeneralpublicmostlyassumedtheydid,unlessthecontrarywasproven.Theyassumedthisprobablybecauseinconservatoiresoutstandingmusicianswereteaching,musicalqualitywasmeasuredbyoutstandingpeers,andtheworld’sbestmusiciansmostlyhadreceivedtheirtrainingataconservatoire.Ifthemusiciansthemselveswouldnotbeabletoguaranteemusicalquality,whowould?
2.2.2 Thingshavechanged.Unconditionalbeliefinprofessionalauthorityhasgivenwaytoamodern,moreeconomicaldrivenviewonanyformofhighereducation,andthereforealsoonconservatoireeduca-tion.Conservatoireeducationisviewednotonlyashigh-qualityeducationperse,butalsoasapubliccommodity,aserviceofferedonthemarketplacetocustomers.Ifitispaidforbythegovernment(read:thetaxpayers),thegovernmenthastoshowthatthemoneyinvestedinconservatoireeduca-tioniswell-spent.Ifitisofferedbyaprivateinstitution,quiteoftenits“customers”aregiventherighttobeshownthatthemoneypaidisworthitsvalue.Doconservatoiresdelivertheresultstheyhavepromisedandforwhichtheyarepaidforbytaxpayersand/orcustomers?Cantheyensuretheywillcontinuetodosointhefuture?
2.2.3 Infact,thatiswhatthe“assurance”bitinqualityassuranceisabout.Itisaboutlookingwhetherornotcertainpromisedresultshavebeenobtained,andwhetheronereasonablymayexpectthattheseresultswillcontinuetobeobtainedinthefuture.Forsomeofustheideathatconservatoiresare
11
heldresponsiblefortheirpromisesonqualityisslightlyworryingbecauseitmaybeperceivedasintrusionfromoutsidersintotheautonomyoftheconservatoire.However,wemustrememberthatthesocietyweliveinrightfullymayaskusquestionsaboutwhoweareandwhatwedo,asoneofourgoalsistoservethissocietywiththeeducationofexcellentmusicians.Wemustalsorememberthatinmostnationalsystemstheautonomyoftheindividualinstitutiontomaintainitsownsystemofinternalqualityassurance,andthereforeitsowndefinitionsofquality,isleftuntouchedandactuallyhighlyregarded.Externalqualityassurancesystemswillbeincreasinglyfocusingupontheinternalqualityassurancesystemsoftheinstitution,asisalsostressedinENQA’s“StandardsandGuidelinesforQualtyAssuranceintheEuropeanHigherEucationArea”9.
2.2.4Thebasicprincipleofqualityassurance:lookingwhetherornotcertainpromisedresultshavebeenobtained,andwhetheronereasonablymayexpectthattheseresultswillcontinuetobeobtainedinthefuture,cantakemanyshapes.Theresultstobeobtainedmaybedefinedbythegovernment,byanationalinstitution,byconservatoirescollectively,orbyasingleconservatoire.Theymaybedefinedintermsofminimumresults,orofaverageresults,oroftargetresults.Theresultsmaybemeasuredbytheinstituteitself,byacollectiveofinstitutes,byanindependentbody,ordirectlybythegovern-ment.Theymaybeveryconcreteorveryvague;theymayevenbedefinedonameta-level,sothatnottheactualresultsbutthewaysinwhichinstitutionssecurethattheseresultswillbeobtainedaremeasured.
2.2.5 Inthishandbookwedefinethreekeyelementsinanyformofinternalqualityassurance:• definingyourgoals,• measuringyourresults,• relatinggoalsandresultsinwhatisoftenconceptualizedas“thePDCA-circle”.
2.2.6Thechaptersthreeandfourwilldiscussgoalsandresults.ThePDCA-circleisabasicconceptbehindthetwowhichwillbeexplainedhere.Itliesattheveryheartofanyqualityassurancesystem.Simplyput:• Planwhatyouaregoingtodo;• thenDoit;• afterwardsCheckifyouhavedonewhatyoumeanttodoandifitdeliveredgoodresults;• reflectontheresultsofyourcheckingandprepareAdaptationsforfutureactions;• thenstartagain:Plan–Do–Check–Adapt10.
2.2.7 Aconcreteexampleofdevelopinganewmoduleinacurriculummayexplainthis:• developthemoduleandplanhowyouaregoingtodeliverittostudents(Plan);
9 Seewww.enqa.eu/files/BergenReport210205.pdf,p.7,point2.1.SeealsoAppendixBonpage45.
10 The“A”inPDCAmayalsobereferredtois“Act”.Weprefer“Adapt”becausethedifferencebetweenDoandActmayseem
confusinglysmall.
10
11
• deliverthemodule(Do);• check,forexamplebystudentquestionnairesandinterviewswiththeteachers,ifthemodulehas
workedoutthewayyoumeantit(Check);• onthebasisoftheremarksofstudentsandteachers,decidewhichchangesshouldbemadein
themoduleandadaptthemodule(Adapt);• then,startthecircleagainbyplanningtheteachingofthemoduleforthesecondtime(Plan);• etcetera.
2.2.8Ofcoursetheideabehindthisisnotsimplythatyouexecutethiscircleendlessly.Theideaisthat,becauseofstructuralcheckingandadapting,yourresultswillbecomebetterandbetter,andyouwillfindyourselfnotinthetreadmillofthecirclebutonastaircaseintheformofaspiral,aspiringtoeverbetterquality.Or,inthetheoreticalcasethatyourqualitygoalshavebeenreached,maintainingyourqualityonthesamehighlevel.
2.2.9Sobeforegoingintotheintricaciesofinternalqualityassurance,itisgoodtoremindourselvesthatinternalqualityassuranceisnothingmorethanelaboratingawayofmakingsurethatwhatyoude-liverlivesuptothesetstandards.Furtheroninthisbrochurewewillseeintosuchquestionsaswhatwemeanby“whatyoudeliver”,whatwemeanby“standards”andonwho“sets”thosestandards.Untilthen,thingsareassimplyastheyarestatedhere.
2.3 Internal
2.3.1 Generally,twoformsofqualityassurancearediscerned:internalandexternal.Externalqualityas-surancereferstoqualityassurancesystemsoperatedbyoutsiders(suchasgovernmentsoraccredi-tationagencies).Internalqualityassurancereferstoqualityassurancesystemssetupandoperatedbytheinstitutionitself.
2.3.2Thishandbookwillconcentrateoninternalqualityassuranceandwillpayminimalattentiontoex-ternalqualityassurance.Inexternalqualityassurance,manynational,Europeanandglobaldevel-opmentstakeplace.Theywillnotbedescribedinthishandbook.Foranyoneinterestedinexternalqualityassuranceanditsconsequencesforconservatoires,wereferyoutotheresultsoftheAEC-project“AccreditationinEuropeanProfessionalMusicTraining”11.
2.3.3Ashasbeenstatedearlier, internalqualityassuranceliesat thecoreofconservatoireeducation.Fromitsverybeginning,everyconservatoirehasbeenveryconsciousofthemusicalqualityitoffered.Internalqualityassurancesystemsinconservatoiresnowadayscanbefoundinverymanydifferentforms,fromanearlyimplicitorintuitivewayofsecuringthemusicalqualitydelivereduptoelaborate(andsometimesratherbureaucratic)systemstakingintoaccountqualityonvariouslevelsinarigor-ous,continuouslyrepeatedanddenselydocumentedprocess.
11 Seewww.bologna-and-music.org/accreditation.
12 13
2.3.4Internalqualityassuranceisatpresentoftenrelatedtoexternalqualityassurance.Werepeatherewhatwealreadystatedinparagraph2.2.3:itisincreasinglythecasethatexternalqualityassuranceisexpectedtotakeintoaccounttheinternalqualityassurancesystemsoftheindividual institute,andthatthereforeinstitutionalautonomyandinternalqualityassuranceshouldformthebasisofthinkinginqualityassuranceterms.NotonlyENQA’s“StandardsandGuidelines”,butalsothe2005BergenCommuniquémentions“thesystematicintroductionofinternalmechanismsandtheirdirectcorrelationtoexternalqualityassurance”.
2.3.5 Inmanycountriesconservatoireshavetoshowtooutsiders–beitdirecttotheMinistryortoasepa-rateofficialbody-onaregularbasisthattheyofferthequalitytheypromisetooffer.Quiteoftenonefindsthatinternalqualityassurancesystemsareinsuchcasesnotonly(andmaybenoteveninthe
firstplace)maintainedbecauseitisfeltasanintrinsicneed,butthattheyservealso,andsometimesmainly,asameansofpreparingoneselfforexternalqualityassuranceprocedures.
2.3.6Thatisfine:earlierwealreadystatedthatsocietyisentitledtoquestionusonquality.Theremayhowever lurkadanger there.Ashasbeensaidbefore, theconceptofquality liesat theheartofconservatoireeducation,andconservatoireshavealways,albeitsometimes intuitiveand implicit,hadtheirownformsofinternalqualityassurance.Thisisanimportantfact,anditscoreshouldbemaintained.Adequateinternalqualityassuranceissomethingonedoesnotbecausesomeonetellsyouithastobedoneoronlybecauseitisapreparationforexternalqualityassurance.Internalqualityassuranceissomethingthatcomesasanaturalconcerntoanyonewhotakespartinconservatoireeducation,beitmanagement,teachers,studentsorexternalrelations.Thisisso,simplybecausetheconceptofqualityliesalwaysatthecoreofanymusician’swork.
2.3.7 Building an internal quality assurance system in any conservatoire therefore never starts fromscratch.Therealwayswillbeasoundcoreofideasconcerningqualityfromwhichaconservatoirecanstartandwhichgraduallycanbebuiltoutabitifnecessaryinordertofitthemoreexpandedmeaningoftheword“quality”nowadaysorinordertoadjust itmoretothedemandsofexternalqualityassuranceprocedures.Inessence,however,internalqualityassuranceisanaturalfeatureofconservatoireeducation.
12 13
3. aImIng For QualIty
3.1 Introduction: a model
3.1.1 Thekeywordinqualityassuranceis“quality”-asitisinconservatoireeducation–butqualityisinitselfanabstractconcept.Notone,butmanydefinitionsexist;arecentpublicationoftheEuropeanUniversityAssociationEUAmentionsamongstothers“fitnessforpurpose”,“customersatisfaction”and“excellence”12.Inthispublication,wewillnotchooseforonesingledefinition,butwillbaseour-selvesonanoverarchingideaofquality.
3.1.2 Oneelementisstandardinanydefinitionofquality:tomakeitconcrete, itmustbeattachedtoa“something”. If we want to express our quality goals, we are always talking about the quality ofsomething.Inthispublication,weproposeamodelinwhichqualitygoalscanbeformulatedonfourdifferentlevels:wemaybetalkingaboutthequalityoftheproduct,theprocess,theorganisationorthequalityassurancesystem.
3.1.3 Thefourlevelsmaybeexpressedinamodelofconcentriccircles.Centralinthemodelistheproduct.Thefinalqualityofwhataconservatoireofferstotheworldliesinitsproduct:themusician,com-poser,musicteacheretcetera.Tomakesuretheproductiseventuallythere,learningandteachingprocessesarecarriedout.Thisisthesecondlevel.Thelearningandteachingprocessesarecarriedoutwithinanorganisation, thethird levelof themodel.Ona fourth (meta-)levelyouwill findthesystemsdevisedforassuringthequalityoftheproducts,theprocessesandtheorganisation.ThemodelthusencompassesthetwomainapproachesoflookingatqualityasdefinedintheEUA-reportmentionedearlier:anapproachthatfocusesonqualityofoutputsandanapproachthatfocusesonqualityofprocesses–theoutputoftheproductbeingrightinthecentre,andthestressshiftingfromoutputtoprocessasonemovesawayfromthecentre.
12 Seehttp://www.eua.be/eua/jsp/en/upload/Quality_Culture_2002_2003.1150459570109.pdf.
Product(music)
Process
(learningandteaching)
Organisation
(TheConservatoire)
QualityAssurance
14 15
3.1.4 Themodelisinawayahierarchicalmodel:thelevelsbecomemoreandmoreabstractaswemovefromtheinsidetotheoutsideofthemodel,andeachcircleisinawayapreconditionforthecirclethatlieswithinit.Theproductisattheheartbecauseeventuallytheproductistheultimategoal.Learningandteachingprocessesmaybeanimportantissueontheirownbutthefinalaimofthemistheproductwedeliver.Inthesamewaytheorganisationisneveragoalinitselfbutismeanttofosterthepropercarryingoutofteachingandlearningprocesseswhichinitsturnareonlymeanttoleadtothehighestqualityoffinalproducts.Andqualityassurancesystemsareimportant,butonlytoassurethequalityoforganisation,processesand,finallyandmostimportant,products.
3.1.5 Aremarkonthesimplicityofthemodelisnecessary,becauseitactuallyoversimplifiesreality.Forexample,thekeyconceptineachofthelevels-“product”,“process”etcetera-seemsratherbigandmonolithicterms,butinrealitycanonlybehandledifthetermsarefurtherspecified,oftenbybreak-ingthemdowninseparatebutrelatedelements.Talkingaboutthequalityof“theproduct”or“theprocess”canbequitemeaninglessifwedon’tspecifywhichelementsoftheproductortheprocessweexactlymean.Also,theterm“quality”quiteoftenhastobespecifiedabitmore,astheremaybemanydifferentkindsofquality,ashasbeenmentionedbefore.Intheparagraphsbelowwewillmakeashortinvestigationofeachofthefourlevelsandwillcommentonthepossiblefurtherspecificationofthekeytermsaswellasonthevariouskindsofqualityonemaydiscern.
3.1.6 Finally,awordofcautionmustbemadehereandwillberepeatedagainandagaininthispublica-tion.It isverytemptinginqualityassurancetocoverthecompletefieldandworkin“totalqualitymanagement”terms:tryingtomonitorallfourlevels,withinalllevelseverysortofthinkablesubdi-visionsinvariouselements,andfromallelementallkindsofquality.Especiallywhensettingupaninternalqualityassurancesystem,itmaybewisetoworkfromadifferentperspective.Donotaimatcompleteness(asitwillneverbereachedinthefirstplace,andonemaydrownintheprocessasaside-effect),butpickselectivelywithwhichelementsyouwillmonitorwhichkindsofquality.Makeyourchoiceonthebasisofwhatyouthinkarethecoreconceptsofyourconservatoireeducation.
3.2 the product
3.2.1 Ashasbeensaid,informertimesconservatoiresdefinedqualityasmusicalqualityandwerehappyiftheycouldshow,bymeansofintersubjectiveprocedures,thatmusicalqualitywasmaintainedfromyeartoyear.Whatactuallywas“measured”wasthemusicalqualityofa“musicalproduct”:mostoftentheperformanceofoneormorepiecesofmusicduringarecital.Thedefinitionofwhatthe“musicalproduct”was,wasquitestrictlyconfinedtotheaudiblemusic.Althougharecitalhasalsotheconnotationofaspecificmusicianplayingforaspecificaudienceataspecificplaceandaspecifictime,actuallyinstandardconservatoirepracticewhatwasmeasuredwaspurely“themusic”.Theaudienceandthepersonofthemusicianwereleftoutofconsideration:weareremindedofthisat-titudebythepowerfulimageoftheorchestralauditionbehindacurtain.
14 15
3.2.2Thisverynarrowdefinitionofthemusicalproductisnowadaysinawaystillcentralatthequalityconceptinhighermusiceducation,buthasbecomebroaderinthreedirections.Tobeginwith,notonlythetypicalrecitalofclassicalmusiccanbethefinalproductofconservatoireteaching.Otherstyleshaveofcoursecomein:jazz,pop,variousgenresofworldmusic.Acompositionorportfolioofcompositionshas,fromtheearlystartoftheconservatoire,beenwidelyacceptedasanothertypeofmusicalproduct,ashasbeentheconductingofaconcert.Nowadays,alsoteachingaclassmaybeinsomecasesbeconsideredamusicalproduct,asmaybethefinalproductofaneducationalproject,astudiorecording,amusictherapysessionorinsomecasessimplysomekindofthesis.
3.2.3Thesecondsenseinwhichthedefinitionofthemusicalproducthasbroadened,isthatevenifwetaketheclassicalmusicperformanceasanexample,theviewontheperformancehasoftenbecomebroader.Theactual,aestheticqualityoftheperformanceofthemusicstillistheheartofthematter.Butinaddition,otherkindsofqualitymaybetakenintoconsiderationandformalisedinassessmentcriteria.Forexample,thewaythemusicianinteractswithhisaudiencebefore,whileandafterplay-ingmaybeconsidered(“interactionalquality”mightbeatermforthis),asmayinsomecasestherecitalasanorganisedevent(“organisationalquality”)–uptothepointwhereeventhelay-outoftheprogrammeleafletmaybeapointofdiscussionintheexaminationcommittee.Also“entrepreneurialquality”maycomein,inwhichwelookattheperformanceinmarketing-terms:wouldthisbeaprod-uctthatattractspublic,that“sells”?
3.2.4Finally,thedefinitionofthemusicalproducthasshiftedinathirdsense,relatedtotheprevioustwo.Moreandmore,itisacknowledgedthattheactualgoalofconservatoiretrainingisnottheperform-ance(orcomposition,or…)perse,buttheperformanceasanexpressionoftheabilities,orcompe-tencies,oftheeducatedmusician.Inaway,themeaningof“product”hasshiftedfromthemusictothemusician,fromtheperformancetotheperformer,fromthecompositiontothecomposer.
3.2.5Summingup,thecentralproductofconservatoireeducationisacomplexnotion.Adefinitionmightbe“thecompetenciesofanykindofmusicianinanygenreandstyleofmusic”.Thosecompetenciesusuallywillbeexpressedandassessedinaso-called“criticalsituation”–asituationwhichisspecifi-callycreatedinordertoassessthecompetenciesoneseesasthegoalofconservatoireeducation.Thereisawiderangeofpossiblecriticalsituationsinconservatoireeducation,andnotoneexhaus-tivemodelwhichdescribesthemanypossibleproductsandkindsofquality.Ifonechoosestotakeproductqualityintoaccount(anobviouschoiceinconservatoireeducation),itishoweverwisetobeawareofthedifferentsortsofproductsanddifferentkindsofqualityonemaydiscern.Finally,onehastobeselective:itisnousetomonitorallkindsofqualityintheproducts,itwouldbewiseifonewouldfocusonthosekindsofqualitywhichonethinksarethemostimportant.
16 17
3.3 the process
3.3.1 Quiteoftenwhenpeopletalkaboutqualityinconservatoireeducationtheconceptisnotonlyappliedtothequalityoftheproduct.Asinanyorganisation,thefinalproductistheresultofaprocess.Intheconservatoire,thefinalmusicalproduct–themusician-istheresultoflearningandteachingprocesses:thestudentlearnsinmanydifferentways,stimulatedbytheteachingoftheteachers,byco-operationandcompetitionwithotherstudents,bymusicalexperiencesoutsidetheirformallearningenvironment,etcetera.Itisnowadaysnotuncommontonotonlyfocusonthequalityofthefinalproduct,butalsoonthequalityofthelearningandteachingprocessesleadingtothisproduct.
3.3.2Justas“theproduct”,“thelearningandteachingprocess”isaratherbigconceptaswell,anditsqualityishardtoassessinonestroke.Whatthereforeoftenhappensisthattheteachingandlearningprocessisdividedintosub-processes.Thereisnotoneuniformmodelforthissubdivision.Differ-entviewsonlearningandteachingmayleadtodifferentsubdivisionsofthelearningandteachingprocess.Forexample,inastrictlyteacher-ledlearningprocessotherelementswillbediscernedtomeasurequalitybythaninstudent-centred,competence-basedlearning.
3.3.3Twomodelswillbepresentedinordertogiveafeelingforthis.Thefirstisaratherstraightsubdivi-siondevelopedinthemiddleofthelastcenturyforuseinprimaryandsecondaryeducationbuteasilytransferabletohighereducation13.Ittakesasitsbasicunitofdescription“thelesson”andstatesthatinplanningtheteachingprocessonehastotakeintoaccountthefollowingelements:• startingsituation,• goals,• learningprocesses(withinthestudent),• didacticalprocesses(teaching),• useofmedia,• learningcontent,• formsofstudentgrouping,• evaluation.
3.3.4Anothermodel,currentlyusedinaqualityassurancemodelinhighereducation14,ismoregearedtowardscompetence-basedteachingmodelsonthelevelofacompletecurriculumanddiscernsthefollowingelements:• researchonthestakeholders’wishesanddemands,• compilationofthespecifications,• compilationoftheintra-andextra-muralcurriculum,• designofstudycomponents,• designofassessment,• designofthelearningenvironment,
13 ModeldevelopedbyVanGelder;sourceE.DeCortea.o.,Beknopte Didaxologie,Groningen:Wolters-Noordhoff,1981.
14 ExpertgroupHBO,Method for improving the quality of higher education based on the EFQM model,Groningen/Eindhoven:
HanzehogeschoolGroningen/FontysHogescholen,2006.
16 17
• studentandteacheractivities,• monitoring.
3.3.5Andagain,differentkindsofqualitymaybediscernedinthevariouselements.Onemightlookforexampleonthelevelofthedesignofstudycomponentssuchasseparatemodulesatthe(musical)qualityofthecontent,butalsoatthequalityofthedesignprocess,atthequalityofthedocumentationofthemodulesetcetera.
3.3.6Ifonewantsitsinternalqualityassurancesystemnotonlytomonitorthequalityofthemusicalprod-uct(asdescribedinparagraph3.2)butalsothequalityofthelearningandteachingprocessitisagainimportanttorealisethatonehastochoosewhichelementsaremonitored.Intheorymanydifferentaspectscanbediscernedinlearningandteaching,andallofthemcanbemonitored.Unlessonehasendlesshourstospendonbuildingupandmaintaininganall-encompassingsystem,itisveryimportanttofocusonthoseelementsoneconsiderscrucialformonitoringthequalityofthelearningandteachingprocess.
3.3.7Summarizing,thelearningandteachingprocesseswithinaconservatoireareimportantprecondi-tionsforthequalityoftheproducteventuallydelivered.Learningandteachingprocessesmaybebrokendownintoseparatebutrelatedelementsinvariousways,dependingononesviewoneduca-tion,andvarioussortsofprocessqualitycanbedistinguished.Ifonechoosestotakeprocessqualityintoaccount,itisimportantagaintobeselective:focusonlyonthoseelementsoftheprocessoneconsiderscrucialfortheoverallqualityoftheinstitute.
3.4 the organisation
3.4.1 Inmanyqualityassurancesystems,notonlythequalityoftheproductandtheprocessesleadingtotheproductaremonitored.Onealsokeepstrackofthequalityoftheorganisationwhichorganisesprocessesinordertodelivertheproduct.Inconservatoireterms:onelooksintothequalityoftheconservatoireasanorganisationthatorganiseslearningandteachingprocessesinordertoletstu-dentsdelivermusicofhighqualityattheendoftheprogramme.
3.4.2Asintheprecedingparagraphontheprocess,“theorganisation”istoocomplexaconcepttomakesimplestatementsonitsquality.Inordertosaysomethingworthwhileaboutitspossiblequality,ithastobebrokendowninorganisationalelements.Awidelyusedwayoflookingatanorganisationisthefollowing15:- centralintheorganisationaretheprimaryprocesses.Inthecaseofconservatoirestheprimary
processisbasically“learningandteaching”,insomecasesalsoresearch.Wehaveelaboratedontheprocessalreadyaboveinparagraph3.3,andtheprimaryprocessesleadeventuallytotheproductsofparagraph3.2;
- supportiveelementsfortheprimaryprocessarepolicy/strategy,personnel/staff,andresources(amongstwhichfinances);
- thewholeisrunbyleadership.
15 Seee.g.www.efqm.org/uploads/introducing_english.pdf.
18 19
Inascheme:
Everyelementoftheschememaythenagainbebrokendownintosmallerelements,leadingtopro-
nouncementsonwhatqualityinanorganisationis.
3.4.3Togiveanexampleofanotherpossiblewayoflookingattheorganisationallevel:McKinsey’s“7s-model”16discernssevenfactors,allbeginningwithan“s”,whicharecrucialforthequalityanyor-ganisationdelivers. Thesevenfactorsareinterdependentandsomeare“harder”(e.g.structure,systems)thanothers(style,sharedvalues).Inascheme:
3.4.4Againthoseschemesandtheirbreakingupinelementsarejusttwoexamplesofhowonemaylookatorganisations–theremaybeothersjustasworthwhile.Andagain,itispossibletodiscernvariouskindsofquality:thequalityofanorganisationmayforsomebelyinginitsthoroughdocumentationofprocesses,forothersinitsdemocraticpossibilitiesforcreativeexchangeofinnovativeideas.
Lead
ersh
ip
Proc
esse
s
lead
ing
toa
pro
duct
People
(personell/staff)
Policy&Strategy
Partnerships&Resources
structure
strategy systems
skills style
staff
shared values
16 Seee.g.www.12manage.com/methods_7S.html.
18 19
Finally,againwemuststressthatonehastobeselective.Itisnousetomonitorthetotalqualityofallelementsofanorganisation,onepicksthoseelementsthatstronglyconnecttothequalityidealsoftheconservatoire.
3.4.5Summarizing,thequalityoftheconservatoireasanorganisationisanimportantpreconditionforthequalityoftheprocessescarriedoutwithinitandtheproducteventuallydelivered.Itispossibletolookattheseparatebutrelatedelementsoftheorganisationinvariousways,andvarioussortsoforganisationalqualitycanbedistinguished.Ifonechoosestomakeorganisationalqualitypartoftheinternalqualityassurancesystem,onehastobeselectiveandfocusonlyontheelementsdirectlyconnectedtothequalityidealsoftheconservatoire.
3.5 the quality assurance system
3.5.1 Finally,a“meta-level”issometimesintroduced,inwhichnotthequalityoftheproduct,theprocessortheorganisationismonitored,butonlythequalityofthequalityassurancesystem(s)itself.Thismeta-levelwillonlybementionedhereshortlyasitisactuallyhardlyofanyuseforstrictlyinternalqualityassurancesystems–itpresupposestheexistenceofaninternalqualityassurancesystemandthereforecanneverexcuseyoufromtakingintoaccounttheproduct,processand/ororganisa-tionalquality.
3.5.2Externalparties,though,maybeinterestedinthisratherproceduralformofinternalqualityassur-ance.Insomecountriesgovernmentwantstocontrolthequalitydeliveredbyinstitutionsbutdoesnotwanttointerferedirectlywitheducationoronlywantstohavecontrolonthemainfeatures.Insuchcasesitmaybecontentedbyscreeningtheinternalqualityassuranceproceduresofaninstitutewithoutgoingintotheactualqualitativedetailsoforganisation,processorproduct.Thecurrentsitu-ationintheUnitedKingdom,whereinstitutionsundergoanInstitutionalAuditfortheQualityAssur-anceAgencyandmaythenreceivedegree-awardingpower,mayserveasanexample.
3.5.3 Ifaninstitutionisbigenoughandaconservatoireonlyasmallpartofit,itmightbethatthecentralapparatuswillnottrytotakeactualcontrolonthedeliveranceoftheconservatoirebutonlymonitoritsqualityassurancesystem.Inthatcasethismightbeconsideredaformofinternalqualityassur-anceontheleveloftheinstitution–ontheleveloftheconservatoirethatformsapartoftheinstitu-tionitwillnodoubtbefeltasexternal.Somemoreremarksontherelationbetweeninternalandexternalqualityassurancewillbefoundinchapter5.
3.6 summary and conclusion
3.6.1 Inthischapterwehavebeentalkingaboutinternalqualityassurancefromtheperspectiveofthe“what”:whenwearetalkingaboutquality,qualityofwhat?Whatisitwearemeasuringthequalityof?
20 21
3.6.2Weformulatedanswersonfourpossiblelevels:- thequalityoftheproduct,whichmaybeastudentperformingarecital,presentingacomposition
orteachingalesson,andmaynotberestrictedtothe“strictlymusical”qualitybutmaytakeintoaccountotherkindsofqualityoftheproduct(communicativequality,forexample);
- thequalityoftheprocessesleadingtotheproduct,inaconservatoiretheprocessesoflearningandteaching,whichoftenwillbedividedintosub-processesinawaythatreflectsonesphiloso-phyofteachingandlearning;
- thequalityoftheorganisationinwhichtheprocessesleadingtoaproducttakesplace,mostlybrokendownintothevariouselementsonecandiscerninanorganisation;
- thequalityoftheinternalqualityassurancesystemthatmonitorsorganisation,processesandproduct.Thislastlevelactuallyliesmostlyoutsidethescopeofthispublicationasitstopsbeingaformofinternalqualityassurance–mostoftenitis,orcomesverynearto,anexternalqualityassurancesystem.
3.6.3Eachlevelmaybeanalysedorbrokendowninseparatebutrelatedelementsinvariouswaysandvariouskindsofqualitymaybediscerned. It is importanttorememberthatonemaystrive in itsexternalqualityassurancesystemnotforcompleteness,butselectscarefullythoseitemsthataretightlyrelatedtotheoverallideasofqualitytheconservatoirefosters.
20 21
4. measurIng QualIty17
4.1 Hard facts and satisfaction statements
4.1.1 Intheprecedingchapterwehaveproposedafour-level-modeltoformulateourqualitygoalsin,inordertoanswerthequestion:aboutthequalityofwhatarewetalking?Anotherquestionisaboutjustasimportant:whatwillbetheevidenceofquality?Howdowemeasurewhetherornotwereachourqualitativegoals?Irrespectiveofthequestionwhetheryourqualitygoalsareformulatedonthelevelofaproduct,aprocess,anorganisationoraqualityassurancesystem,youaremeasuringitbylookingatinformationthatwillgiveyouaclueaboutthequality.Basically,therearetwotypesofinformationonecandrawconclusionsfrom:hardfactsandsatisfactionstatements.
4.1.2 Hardfacts(inbusiness,youwouldcallthemyourbusinessresults)canbeeitherfinancialoropera-tional.Financialresultsinaconservatoiremaybeforexamplesolvency,liquidity,totalannualbudget,costsperstudent,costsperteacherfulltimeequivalent,totalcostsperrealizedstudycreditetce-tera.Operationalresultscouldbeforexamplethenumberofenrolledstudents,thenumberofnewlyadmittedstudents,thepercentageofstudentsthatdropsoutduringthecourse,thepercentageofstudentsthatdropsoutinthefirstyear,theaveragestudydurationofdrop-outs,etcetera.
4.1.3 Satisfactionstatements,ontheotherhand,arelinkedto(groupsof)individualswhoareexpressingtheirlevelofsatisfactionwiththequalityofferedbytheconservatoire.Typicalexampleswouldbe:peerswhostatetheyaresatisfiedwiththequalityofthefinalexaminations(product);studentswhostatetheyaresatisfiedwithexaminationpracticesthroughthecurriculum(process);teacherswhostatetheyarenotsatisfiedwiththeavailableclassroomsandinstruments(organization);oremploy-ersfromtheprofessionwhostatethatyourcurriculumdoesnotpreparestudentsfortheirfutureprofessionenough(product).
4.1.4 Tobecomeabitmore“technical”:hardfactsandsatisfactionstatementscanserveinrelationtoyourqualitygoalsas“performanceindicators”,theyindicatehowyouareperforming.Youmaypickoneorseveralperformanceindicatorsforeveryqualitygoalyouhaveformulated.Foreveryperformanceindicatoryouthendefineameasuringinstrumentwhichyouwilluseformeasuring,andameasur-ingunit.Youthenstateatargetresult,youmeasureyouractualresult,andaftercomparingthetwoyoudrawconclusionsandactonthem.InthePDCA-circle(seepar.2.2),workingwithperformanceindicators,targetresultsandactualresultsfitspecificallyintheCheck-phaseandmakesurethatonecanAdaptonthebasisofreliablefacts,therewith“closing”thePDCA-circle.
17 Muchofthecontentandmanyoftheexamplesinthischaptercomefrom:ExpertgroupHBO,Method for improving the quality of
higher education based on the EFQM model,Groningen/Eindhoven:HanzehogeschoolGroningen/FontysHogescholen,2006.
22 23
4.1.5 Asimpleexamplewouldbe:
- qualitygoal:yourfinalassessmentmethodsarestateoftheart;- performanceindicator:satisfactionoftheprofessionalfieldconcerningyourfinalassessment
method;- measuringinstrument:threespecificquestionsinaquestionnairefortheprofessionalfield;- measuringunit:percentageofanswersindicatingsatisfaction;- targetresult:70%satisfaction;- actualresult:80%;- compare:youscorehigherthenyourtarget;- conclude:noadaptationsneeded.
4.2 Hard facts
4.2.1 Consideringthehardfacts,thebusinessresults,onecanofcoursecompileanendlesslistofcatego-riesofdata.Forexample,onemightmakethefollowinglistofcategoriesofdata:enrolledstudents,inflow,throughput,outflow(allsplitintotargetgroupswhereappropriate),businessoperation,in-novation,staff,andexternalassessment.Eachcouldbesubdividedagain;businessoperation forexampleinstudycoursedemandfactor,efficiencyratiomeasurement,costsoffailure,students/staffratio, investments,accommodation.Accommodation in turncouldbesubdivided into theaveragenumberoffunctionalsquaremetresinuse,accommodationcostspersquaremetre,andfunctionalsquaremetreperstudent.Andsoonandsoon.
4.2.2Evenwithununlimitedamountoftime,peopleandmoneyyouwouldneverbeabletobecomplete.Andtobesure,completenessisnotthegoalyoushouldbelookingfor.Anendlesssetofdata ismeaningless.Databecomemeaningfulonlywhentheytellyousomethingaboutyourqualitygoals(i.e.whentheyareusedasperformanceindicators),andwhenyoucananalyzetheminthelightoftargetresults.Ifyouwanttoanalysedata,youneedcriteriaonthebasisofwhichyouanalyseyourdata.
4.2.3Thebasicthingtodoistotaketheeightstepsalreadydescribedintheexampleintheparagraphabove:1. decidewhichgoalsyoucouldformulateregardingthequalityyouwanttooffer;2. decidewhichdataarethemosttellingaboutthisquality;inotherwords:pickyourperformance
indicators;3. foreveryperformanceindicator,defineameasuringinstrument;4. defineyourmeasuringunit;5. stateatargetresult;6. measureyourcurrentresult;7. comparetargetandactualresult,analyse,drawconclusions;8. changethingsifyouthinkyouhaveto.
Aftersometime,thecyclemayberepeated:measureagainifyourresultshaveprogressed.
22 23
4.2.4Anexample:1. yourgoalistoofferacurriculumthatisfeasibleforstudents;2. you think themost tellingdatawillbe theamountofstudentsgraduatedwithin thestandard
amountofyears,saythreeyears;3. yourmeasuringinstrumentwillbeacountbyyouradministration;4. themeasuringunitwillbethepercentagegraduatedafterthreeyearsfromthestudentsenrolled
inaparticularyear;5. yourtargetresultis80%;6. yourcurrentresultturnsouttobe60%;7. onfurtheranalysisyoufindthatmostoftheremaining40%dropoutbeforetheydoafinalexami-
nation.Fromthisyouconcludethateitheryourentranceexaminationisnotselectiveenoughorthereissomethingwrongwithyourcurriculum.Youdecidetodoaquicksurveyamongasmanystudentsaspossiblewhodroppedoutinthelasttwoyears.Theyinformyouthemainreasonfordroppingoutearlyisthattheycouldnotmanagethecurriculumbecauseitwasoverloaded,frag-mentedandincoherent;
8. youstartworkingontheimprovementofyourcurriculumandeveryyearmonitorthegraduationpercentageafterthreeyears,hopingthatyoureffortswillleadtoagradualincreaseoftheper-centage.
4.2.5 Themainthingthisexampleshowsishowyouworkwith“hardfacts”ininternalqualityassurance:theyaretelling,butonlyifyouareselectiveandconnectthemwithyourmostimportantgoals,ifyoustatetargetsandcompareyourownresultswiththem,ifyoutakeactionsafterwardandcontroliftheyhaveresults.Ifoneofthoseelementsismissing,eventuallyyourdatawillneverbemorethandata:hardfactswithoutmeaning.
4.2.6Onemoreremarkcanbemadehere.Theexampleabovesuggestsanannualrepetitionofthegath-eringofhardfacts.Thefrequency,however,isnotfixedtoonceayear.Sometimes,itmaybewisetogatheryourdatamoreoften.Forexample,ifyouofferacompletelynewcurriculum,youmightinthestartingphasedecidetomonitorsomeelementsthreeorfourtimesayear.If,however,youarequitesatisfiedwithyourresults,youmightdecidetogathercertaindataonlyonceeverytwooreventhreeyears.
4.3 satisfaction statements
4.3.1 Satisfactionstatementsarestatementsof(groupsof)individualswhoareabletotellsomethingaboutyourcurriculum–oftentheyarecalled“stakeholders”.Aswithhardfacts,therearemanydifferent(groupsof)individualswhocouldmakesuchstatements.
4.3.2 Thefirstoptionis,ofcourse,yourself.Inthenarrowestsensemeaning:“you-theprincipaloftheconservatoire”or“you–theheadofstudies”or“you–theprincipalteacher”.Inthatcasegathering
24 25
satisfactionstatementsinaninternalqualityassurancesystemwouldbequitesimple:forceyourselfforexampleonceayeartotakethequalityofwhatyou’redeliveringunderclosescrutiny,drawcon-clusions,actuponthemandyou’reready.
4.3.3 Actually,thisiswhatweareconstantlydoingbymeansofourassessmentprocedures.Assessmentiscrucialtothefosteringofqualitywithinconservatoires.Forinternalqualityassurance,however,thiscannotbeenough.Qualityassurance,asexplainedinchapter1,isoftennotjustmeanttocon-stantlyimproveone’sownquality,butalsotojustifyonesqualitytowardsexternalparties.“You-theprincipalteacher”or“you-theheadofstudies”willundoubtedlybeanexpertonquality,buttheremaybesomebiasinsomeofyourconclusionsandyoumayhavedevelopedsomeblindanglesovertheyears.Yourinternalqualityassurancesystemwillbenefit ifotherswillalsomakestatementsaboutthequalityyoudeliver,sothatyoucancomparethemwithyourownthoughtsaboutitandletyourconsequentactionsbeinformedbythem.
4.3.4 Thatiswhyqualityassuranceproceduresoftenskipthelevelofconcreteassessmentsandlookforoutsiders’satisfactionstatements.Thefirst,wellknownandwidelyusedother is the“peer”: theoutsideexpertfromwithinyourfield,some-onechosenforhisexpertknowledge.Infact,thebring-inginofpeerstojudgequalitymakesyourownassessmentofthequalityyoudeliverlesssubjective.Qualityremarksarenever,andcertainlynot inmusic,objective,butwithpeerstheymaybecome“inter-subjective”.
4.3.5 Youcanthenbroadenthesetofpeopleyouwanttotakepartinyourqualityassessmentbybringinginvariousstakeholders.Inpractice,thefollowingaremostusedininternalqualityassurance:- students;- (recent)alumni;- theprofessionalfield(especiallypeersandemployers);- staff;- thepartyultimatelyresponsiblefor(andpayingfor)theconservatoire(mostoftenthisisgovern-
ment);- thegeneralpublic.
4.3.6 Eachofthesecategoriesmaybesubdividedagain:togatherspecificdatayoumightwanttoaskaspecificcategoryofstudents(firstyear’sstudents,forexample,orstringstudents),aspecificcat-egoryofalumni(lastyear’salumni,forexample,orjazzalumni),etcetera.
4.3.7 Aswiththehardfacts,satisfactionstatementsareveryusefulaslongasyouareselective.Itisnousetryingtogatherasmuchsatisfactionstatementsaspossiblefromasmanyindividualsaspossibleonasmanyelementsaspossible–again,evenifonehadanunlimitedamountoftime,peopleandmoneyavailableonewouldendupwithanenormouspileofdataoutofwhichitwouldbestillveryhardtoextracttheinformationyouwouldwant.Andaswiththehardfacts,youmightconsiderthe
24 25
frequencywithwhichyougatherdata:maybeyouwanttorepeatafirstyear’sintervieweveryyear,butyoumightconsideraskingalumnionceeverytwoyearsandsendaquestionnairetoemployersintheprofessionalfieldonlyonceeverythreeyear.
4.3.8 Sohereagainweshouldworkintheeightstepsdescribedbefore:1. decidewhichgoalsyoucouldformulateregardingthequalityyouwanttooffer;2.decidewhichsatisfactionstatementsarethemosttellingaboutthisquality;inotherwords:pick
yourperformanceindicators;3.foreveryperformanceindicator,defineameasuringinstrument;4.defineyourmeasuringunit;5.stateatargetresult;6.measureyourcurrentresult;7.comparethem,analyse,drawconclusions;8.changethingsifyouthinkyouhaveto. Aftersometimemeasureagainwhetheryourresultshaveprogressed.
4.3.9 Anexamplewouldbethefollowing:1. yourgoalistoofferajazzprogrammethatisgearedtowardsprofessionalpractice;2. youdecidethatthebestwaytotelliswhenjazzstudentsaresatisfiedwiththecurriculumwhen
theylookbackonitaftergraduationoncetheyareintheprofessionalpractice;3. youdecidetomeasurethiseveryyearwithalumniwhograduatedtwoyearsbefore.Youdothis
bymeansofashortinterviewbytelephone;4. Intheinterviewyouaskthemtogivehecurriculumagradeforpreparationonprofessionalprac-
ticeonascalefrom1(verypoor)to10(excellent);5. yourtargetresultisanaveragesatisfactiongradeof7;6. youfindoutthataveragesatisfactionis6.8;7. youdecidetocarryouta limitedamountof longer interviewsbytelephoneandonthatbasis
decidethatespeciallyaspectsofbusinessandfinancialsaremissingfromthecurriculum;8. afteryouhavebuildacourseonthattopicintothelastyearofthecurriculumyoufindoutsatis-
factionratesareindeedrisingtowellabovea7onaverage.
4.3.10 Thisexampleshowshowtoworkwithsatisfactionstatementsininternalqualityassurance.Aswithhardfacts,theyaretelling,butagainonlywhenyouareselective,relatethemtoyourowngoals,statecleartargets,makeagoodcomparisonwithyouractualresults,takeactionsafterwardandcontroliftheyhaveresults.Beware:satisfactionstatementsareeventuallyindividualandthereforerathersubjective.Oneofthewaysaroundthissubjectivityistoaskenoughindividualstomakeastate-ment.Ifyoursampleisbigenough,youcanmoreorlessabstractfromtheindividual’ssubjectivity.
4.3.11 Herewetouchupononeofthepeculiaritiesofconservatoireeducation:one-to-oneteachingwithitscloseteacher/student-relationship.Togatherreliablestudentsatisfactionstatementsconcern-
26 27
ingteachingonthemaininstrumentisoneofthemostimportantandsensitiveissuesininternalqualityassuranceinconservatoires,andmanyinstitutionshavefoundtheirownwaysinhandlingthismatter.Someuseaformalandanonymoussystem,whichenablesstudentstogivefeedbackontheirteacher;thisisthendiscussedbetweenteacherandheadofdepartment,forexampleduringannualstaffappraisalinterviews.Otherconservatoiresarelookingmoreintoredefiningandprofes-sionalisingtheirsystemsofmentoringandtutoring18inordertotakeawaythepossibleproblemstowhichone-to-oneteachingsometimescanlead.Thepointismaybenotsomuchwhichsystemaconservatoiredevelopsinthisrespect,butthatasystemisdevelopedthatworksinthespecificcontextoftheinstitutionconcerned.
4.3.12 However,anotheraspectthatcomesiniswhatsometimesiscalled“managementofexpectations”:theindividualsyouaskactuallymayexpectmorefromyouthenreasonablycanbeexpected,whichmakesthemunhappyaboutcertainresults,whereasthoseresultsasexpressedin“hardfacts”arenotsobadatall.Afamousexampleisstudents’satisfactionintermsofthemarkingandreturnofwork.Youmayaskstudentsiftheyaresatisfiedwiththeamountoftimebetweenawrittenexamina-tionandthemarkingbyateacher.Teacherscanbesobusywithwrittenexaminationsthattheycanonlybeexpectedtohavegradedwrittenworkwithintwoweeks’time.Ifstudentsdon’trealisethis,theymaybedissatisfiedifmarkingtakeslongerthenaweek,becausetheyarekeentoknowtheirresults.Ifstudentsthereforegivenegativesatisfactionstatementsbuthardfactsshowthatmarkingisdonewithinthetimeyougiveyourteachers,youmayaskyourselfwhetheryouwanttopressyourteacherstoworkevenfaster,orwhetheryouwilltrytomanagetheexpectationsofyourstudentsbetterbytellingthemthegoalyouhavesetyourselfistohavemarkswithinatwoweeks’termbe-causeyoucan’tdemandmorefromyourteachers.
4.3.13 Comparableexamplesmaybegivenonmanymoresatisfactionstatements,reasontohandlethemwithacertaincare.
4.4 summary and conclusions
4.4.1 Inthischapterwehavebeendiscussingthequestionwhatexactlytellsyouaboutquality.Wemadeadivisionintwoseparatekindsofperformanceindicators:ontheonehand“hardfacts”,ontheotherhand“satisfactionstatements”.
4.4.2Hardfactscanbeeitherfinancialresultsoroperationalresults.Examplesoffinancialresultscouldbethetotalannualbudgetoftheconservatoireorthecostsperfte.Examplesofoperationalresultscouldbethenumberofenrolledstudentsorthepercentageofstudentsthatdropsoutduringthecourse.
18 SeePeterRenshaw,“thePlaceofMentoring”,http://www.lifelonglearninginmusic.org/BeheerContentPagina.asp?id=170.
26 27
4.4.3Satisfactionstatementsarestatementsmadeby(groupsof)individualsonthequalityofyourcon-servatoire.Apart fromyourself,manyotherpeoplecanmakesuchstatements.Wedistinguished students,alumni,theprofessionalfield(especiallypeersandemployers),staff,thepartyultimatelyresponsiblefortheconservatoire(mostlygovernment),andthegeneralpublic.
4.4.4Forbothhardfactsandsatisfactionstatementsitisimportanttorememberthattheamountofpossi-bledatatogatherisendless.Onehastochoosecarefulwhichperformanceindicatoronethinkstellsmostaboutthequalitygoalsonewantstoreach,defineameasuringinstrumentandameasuringunit,formulateatargetresult,gatherthedatatofindoutwhatyouractualresultis,comparetargetandactualresult,ifnecessarytakeactionandthenmeasureagaintofindoutwhetherornottheac-tionshaveledtomorepositiveresults.
4.4.5Finally,althoughhardfactsaswellassatisfactionstatementsaremoreorless“objective”datafromoutsidetheconservatoire,onehastorememberthattheyonlygetmeaningoncetheyhavebeenrelatedtoyourgoals,youranalysisandyourdecisions.Thatisthebasisofallinternalqualityassur-ance:itremainsdrivenfromtheinsideofheconservatoire.Thereisanuancetobemadeonthis,butthatwillbedoneinthenextchapter.
2928
5. settIng uP an Internal QualIty assuranCe system In your ConservatoIre
5.1 introduction
5.1.1 Ashasbeensaidbefore,almosteveryconservatoirehasatleastsomethingofanimplicitinternalqualityassurancesystemupandrunning.Itmaynotbeanintegralsystem,itmaynotbeastrictlydocumentedsystem,itmaynotevenbeaverysystematicsystem,butqualityis,bythenatureoftheconservatoire,anaturalconcerntoallworkinginit.
5.1.2 Inthischapter,however,wewillstartfromscratchandpretendweareworkinginanewconserva-toirethathastosetupanewsystemofinternalqualityassurance.Basically,wewouldinsuchacasetakeninesteps,asfollows:1. Stateyourqualitygoals:pickcarefullytheelementsonwhichyouwantinformationconsidering
thequalityyouareoffering;2. Pickcarefullyyourperformanceindicators:which“hardfacts”and“satisfactionstatements”you
thinkarenecessaryinordertogetinformationonthequality;3. Addexternallyformulatedqualitygoalsandperformanceindicators:whatisnecessarytodobe-
causeofpossibledemandsfromoutsidetheconservatoireconcerningyoursystemofinternalqualityassurance19;
4. Picktheinstrumentswithwhichyouwillgatherinformationanddefineyourmeasuringunits;5. Formulatetargetresults;6. Makeanimplementationplanforyoursystem;7. Describecarefullyforeveryactionthatwillbeimplementedwhoisresponsible,whatyoumeas-
ureetcetera;8. Startyourmeasurements,analysetheresultsandsuggestadaptationsifnecessary,implement
theadaptations,measureagainetcetera(makeyourPDCA-circleafullcircle!);9. Reviewyourinternalqualityassurancesystemregularly.
5.2 step 1: state your quality goals
5.2.1 Forconservatoireeducation,thereisno“logicallevel”perseforinternalqualityassurance.OnemaylookinvariousconservatoiresalloverEuropeandfindthatonestrictlystickstoproductmeasuringonly,anotheronestressesprocessesandmaintainsthatonceyouhaveguaranteedthequalityofthelearningandteachingprocesstheproductwillinevitablybeaqualitativegoodproduct,andathirdonewillfocusheavilyonorganisationalquality.
19 Theitemstwoandthreemaybereversedinordersothatonefirstinventarisestheexternaldemandstotheexternalqualityas-
surancesystemandthenaddswhatoneconsiderstobemissing.Wedo,however,prefertheoriginalorder,asitstartswiththe
intrinsicpreferencesoftheinstituteitselfandonlylateraddsthedemandsoftheoutsideworld.
29
5.2.2Whatonealsowill find isthat inmostof theconservatoireswithsomehistory informal internalqualityassurance,abalancedpositionistaken.Therewillbegeneralagreementonthefactthatinconservatoireeducation,strictmonitoringoftheproductqualitybypeersfromtheprofessionisthebasisoftheartisticlevelofthemusicprofession,buttherewillalsobegeneralagreementonthefactthatinordertomaintainthehighestqualityoftheproductoneneedstokeeptrackofthequalityofthemainprocessesinlearningandteaching,andthatforasoundqualityofprocessesandproductsatleastthemostbasicfeaturesoftheorganisationaldevelopmentmustbemonitoredaswell.Itisinfindingthebalancebetweenthesethreelevels,andinchoosingwherethefocuswillliewithineachlevel,thateachinstitutionmakesitsownchoicesandthusgivesitsinternalqualityassurancesystemapersonal“face”.
5.2.3Thefirststepinanewconservatoirewouldbetocarefullypickoutthoseelementsonwhichyouwantinformationconsideringthequalityyouoffer.Asdescribedinchapter3,youmaystarttoconsiderwhetheryouwanttomonitoryourqualityontheleveloftheproduct,theprocessortheorganisa-tion(monitoringthequalityofthequalityassurancesystemwouldbeanotheroptionbutasstatedinchapter3wewillleavethisoutofconsiderationintherestofthishandbook).
5.2.4Thechoicesherearenotexclusive,anditmightbewisetogoforamixofoptions.Agoodwaytodeterminethismixmightbetotakeagoodlookatthetotalityoftheproposedmodel,sketchinga“map”ofthepossibleelementsfor internalqualityassuranceandindicatingwhichelementsyouhavepickedout.Suchasketchcouldforexamplelooklikethis:
5.2.5Thesketchtellsyouthatyoutakeintoaccountelementsfromproduct,processandorganisation.Italsomakesyouawarethatyouhavedefinedintheproductwhatyouthinkisimportant:themusicalquality,theentrepreneurialqualityandthepresentationalquality.Intheprocessandtheorganisa-tion,itatoncemakesclearthatyouhaveonlyselectedafewitemstolookat:forexample,intheprocessyouhavedecidedthatteacheractivities,media/environmentandevaluationareespeciallyimportant,andthatyouleaveoutsuchaspectsasstartingsituationorlearningcontent.
30 31
5.2.6Elaboratingabituponthis,youmayendupwiththefollowingelements: Product:
- musicalqualityoffinalexamination;- entrepreneurialqualityoffinalexamination;- presentationalqualityoffinalexamination.
Process:- qualityofevaluations/assessments,specificallyinthemainsubjectareas;- qualityofteachingandcoaching;- qualityoflearningenvironment,specificallyclassrooms,rehearsalrooms,instruments,audio-
visualsetcetera.
Organisation:- qualityofartisticleadership;- qualityofteachingstaff.
5.3 step 2: Pick your performance indicators
5.3.1 Onceyouhavedeterminedyourqualitygoals,youdeterminewhichperformanceindicatorsyouneedinordertodeterminethequality.
5.3.2Youroverviewcouldthenlooklikethis:
Qualitygoals: Performanceindicators:
Product: Product:
-musicalqualityoffinalexamination -satisfactionofpeers
-entrepreneurialqualityoffinalexamination -satisfactionofpeers
-presentationalqualityoffinalexamination -satisfactionofaudiencepresent
Process: Process:
-qualityofevaluations/assessments -satisfactionofprofessionalfield
-qualityofteachingandcoaching -satisfactionofstudents
-qualityoflearningenvironment -satisfactionofstudents
Organisation: Organisation:
-qualityofartisticleadership -satisfactionofgeneralpublic
-qualityofteachingstaff -percentage of teaching staff with own professional
practice
30 31
5.4 step 3: add outside demands
5.4.1 Upuntilnowyouhavebeenconcernedwithyourowncrucialqualitygoalsandperformanceindica-torstomonitor.Ofcourse,internalqualityassuranceinmostinstitutesisnotentirelyselfdriven.Quiteoften“outsiders”willaskyoutomonitoryourownqualityaswell.Suchanoutsidermightbethegovernmentwho,whenaccreditingyou,willlookintoyourinternalqualityassurancesystemandmayspecificallyaskforcertainelements.Forconservatoireswithinbiggeruniversities,thegeneraluniversitylevelmayplayacomparablerole.
5.4.2Anexamplemightbethe“StandardsandGuidelinesforQualityAssuranceintheEuropeanHigherEducationArea”20,adoptedbytheEuropeanMinstersofEducationin2005.AlthoughthisdocumenthasonaEuropeanlevelthestatusnotofalawbutofarecommendation,onemayexpectthatincer-taincountriesindividualinstitutionswillbeexpectedtocomplywiththisdocument.Inthatcasetheymayamongstothersbeobligedtocoverthefollowingitems:- studentprogressionandsuccessrates;- employabilityofgraduates;- students’satisfactionwiththeirprogrammes;- effectivenessofteachers;- profileofthestudentpopulation;- learningresourcesavailableandtheircosts.
5.4.3Ittheniswisetoincorporatethisintheinternalqualityassurancesystem.Ifthatisnotdone,onemayeveryroundofaccreditationagainhavetoproduce,withmucheffort,thesamesortofinformation.
5.5 step 4: pick the instruments
Nowyouknowwhichhardfactsandsatisfactionstatementsyouneed,youhavetodeterminehowyouwillgathertheseandwhatyourmeasuringunitwillbe.Forexample:
Qualitygoals: Instruments: Measuringunits:
Product:
-musicalqualityoffinalexamination -countbyadministrativeunit -gradesgivenbypeersatexamination(scale1-10)
-entrepreneurialqualityoffinal
examination
-countbyadministrativeunit -gradesgivenbypeersatexamination(scale1-10)
-presentationalqualityoffinal
examination
-audiencequestionnaire -levelofsatisfactionaudience(scale1-10)
Process:
-qualityofevaluations/assessments -questionnaireprofessional
field
-percentageofsatisfactionprofessionalfield
-qualityofteachingandcoaching -questionnairestudents -percentageofsatisfactionstudents
20 FormoreinformationabouttheEuropeanStandardsandGuidelines,pleaseseeAppendixB.
32 33
-qualityoflearningenvironment -questionnairestudents -percentageofsatisfactionstudents
Organisation:
-qualityofartisticleadership -countbymarketingunit -totalnumberofpositiveitemsinregional/national
newspapers
-qualityofteachingstaff -countbystaffunit -percentageofteachingstaffwithownprofessional
practice
5.6 step 5: formulate target results
5.6.1 Finallyyouhavetoformulatetargetresultsforyourperformanceindicators.Thismaybeahardthingtodo.Sometimesyouwillquitepreciselyormoreonaveragenowwhatkindsofresultsyouwouldliketoobtain.Sometimesitismoreanintuitivefeeling,maybesecondedbyresultofcomparableinstitutesyouknowormeasuredagainstabenchmarkinstituteyouhighlyrespect.Inourexample,youmightcomeupwiththefollowing:
Qualitygoals: Measuringunits Targetresult
Product:
-musicalqualityoffinalexamination -gradesgivenbypeersatexamination(scale1-10) -average7
-entrepreneurialqualityoffinal
examination
-gradesgivenbypeersatexamination(scale1-10) -average7
-presentationalqualityoffinalexamination -levelofsatisfactionaudience(scale1-10) -average7
Process:
-qualityofevaluations/assessments -percentageofsatisfactionprofessionalfield -70%
-qualityofteachingandcoaching -percentageofsatisfactionstudents -80%
-qualityoflearningenvironment -percentageofsatisfactionstudents -80%
Organisation:
-qualityofartisticleadership -totalnumberofpositiveitemsinregional/
nationalnewspapersperyear
-regional:10;national:2
-qualityofteachingstaff -percentageofteachingstaffwithown
professionalpractice
-90%
5.7 step 6: make an implementation plan
Onceyouhavemadesurewhatyouwantinyourinternalqualityassurancesystemandwhatotherswantintoit,makeanimplementationplan.Donottrytoimplementacompletesystemfromscratchinoneyear. Ifyouhavemanythingstostart, takeafewyears ifpossible.Quiteoften it turnsoutthatonceyou’vebuildupsomeexperiencewithquestionnaires,interviewsorthegatheringof“hardfacts”,introductionofnewelementstakeslesstime;whereasacompleteintroductionatoncemaycauseyoutodrawninaswampofdata.
32 33
5.8 step 7: describe
5.8.1 Onceyouhavecomethisfar,itiswisetotakeastepyoumighttendtoforgetbutwhichcanbeofenormousvaluefurtheronintheprocess.Make,foreveryelementofyourinternalqualityassur-ancesystem,amethodicaldescription:whatareyourgoals,whatareyourperformanceindicators,whichinstrumentdoyouuse,whatisthetargetresult,howoftenwillyoumeasure,whoisrespon-sibleformeasuring,whatistheprocedureyoufollowinordertomakethePDCA-circleafullcircle.
5.8.2Adescriptionofthiskindmightlooklikethis:
goal musicalqualityoffinalexamination
performanceindicator(s) satisfactionofpeers
instrument/measuringunitused gradingofexternalmembersofexaminationcommittee,scale1-10
targetresult average7
howoftenmeasurement twiceayear,aftertheexaminationroundsinJanuaryandJune
responsibleformeasuring administrativeunit,onthebasisoffilledoutexaminationprotocols
PDCAfullcircle aftermeasurement:feedbacktoprincipalandheadofstudies;resultsandmeasurestobe
takenonthebasisoftheresultstobeannouncedintheTeachersNewsletter
5.9 step 8: carry out
5.9.1 Yoursystemisnowfinishedanddocumented.Carryoutwhatyouplanned.Gatheractualresultsbymeasuring,comparethemtoyourtargetresults,analysethediscrepanciesandsearchforpos-siblereasons,makeadaptationsinyourcurriculumoryourorganisation.Thengathernewresults,analysetheminordertofindifyouhavemadesatisfactoryprogress.Ifnot,analyseagaincarefully;ifyouhavemadeprogress,monitorwhetheryouhavekeptyourstandardsup.
5.10 step 9: review the system regularly
5.10.1 Finally,asystemisnotmeantforeternity.Whatyouwilloftenfind,isthatasystemstartsverybasic,butwillgrowquitefastbecausequiteoftentheanalysisofyourresultsmakesyoufeelyouneedotherkindsofresultsaswellinordertomaketherightdecisions.Youmayendupwithanenormoussystemofdata-gatheringwhichintheendmaymakeithard,orevenvirtuallyimpossible,tokeepyouranalysisofgoodqualityandthereforetotakeanysounddecisionsatall.
5.10.2 Thatmaybethepointtomakesuretogetridofsomeofthemechanismswhichstillatfirstsightmayseemtobeusefulbutstrictlyspeakinghavenocrucialstrategicmeaninganymore.Youmaydecideeithertodividesuchanenormousinternalqualityassurancesystemintwoparts:thebasisisagreatsetofmeasurementsyoudo,buttherealanalysistakesintoaccountonlythoseelementsyouconsiderofstrategicimportance.Theotherpossibilityistostopmeasuringatallallthoseele-mentsyoudonotconsiderascrucialanymore.
34 35
5.11 summary and conclusions
5.11.1 Inthischapterwehave,onthebasisoftheprecedingchapters,proposedawayofdevisinganinter-nalqualityassurancesystem.Wesketchedanine-stepprocedurewhichenablesyoutoformulatequalitygoalsandperformanceindicators,choosemethodsandunitsofmeasuringandformulatetargetresults,plantheimplementationanddescribethesystem,carryoutthesystemandeventu-allyrevisethesystemifnecessary.
5.11.2 Tworemarksaretobemade.Oneisthatthisnine-steppathtoaninternalqualityassurancesystemmaydifferfrommuchoftheliteratureyouwillencounter.QuiteoftenliteraturewillstressthevirtuesofwhatiscalledTotalQualityManagement–systemswhichpretendto(andmaybeindeeddo)covercompleteorganisations.Wehavenothingagainstsuchelaboratesystemsandifyouconsiderim-plementingtheminyourconservatoire,donothesitate(butbeawareoftheamountofbureaucracythatmaybeattachedtoit).Wehave,however,chosenthestandpointthatformanyitwillbeeasiertostartbottomup,withfewbutcrucialquestionsoneasksoneself.
5.11.3 Theotherremarkisthatwehavedeliberatelynotpresentedtwoorthree“complete”systemsofinternalqualityassuranceinconservatoireshere.Itisourfirmbeliefthatagoodinternalqualityassurancesystemhasthelocalasbasis,andthesectoral/national,Europeanandglobalascontext.Becausesuchasystemissodependentoncontext,itishardlyeverpossibletointegrallytransportitfromoneinstitutetotheother,evenwithincountries.Wedohavehowever,inAppendixA,compiledasetofinstrumentsusedininternalqualityassuranceinordertogiveyouanideaofthepossibili-ties.
34 35
aPPendICes
3736
37
aPPendIx a. Internal Quality assurance in Conservatoires – some examples
a.1 Introduction
A.1.1 Inthisappendixwepresentelementsofthesystemsofinternalqualityassuranceoftwoconserva-toires:theRoyalCollegeofMusicinLondon,UK,andthePrinceClausConservatoire,partoftheHanzeUniversityofAppliedSciences inGroningen,theNetherlands.TheexampleswereathandastheauthorisconnectedtothePrinceClausConservatoireandthechairofthePolifoniaTuningworkinggroupistotheRoyalCollegeofMusic.Intermsofqualityassuranceproceduresbothareinterestingandbotharedifferentfromeachother.
A.1.2 TheRoyalCollegeofMusicisanindependentcollege(slightlyover600students)withitsownde-gree-awardingpower,offeringonlymusicprogrammes.Ithascarefullydeviseditsqualityassurancesystemsforthepurposeofrigorousqualitycontrolcompletelygearedtowardsthespiritofmusiceducation.
A.1.3 ThePrinceClausConservatoireisoneoftheeighteenschoolsofabigmultisectoraluniversityofap-pliessciences(about22.000students)withaseparateandstrongcentralqualityassurancedepart-mentworkingforalltheacademies.TheinternalqualityassurancesystemisexplicitlydevisedintermsofaformofTotalQualityManagement,withavarietyofgeneralinstrumentstobeusedwhichcanpartlybegearedtowardstheneedsofthevariousacademies.
a.2 the Quality assurance system of the royal College of music, london (uk)
A.2.1 TheRoyalCollegeofMusichas itssystemofqualityassurance laiddown inaQualityAssuranceHandbookofapproximately25pages1.Thehandbooknotonlygivesanoverviewofthequalityassur-anceproceduresoftheRCMbutstatestheprinciplesbehindthemaswell.Whatfollowsbelowisatextconsistingofquotesfromthehandbook.ThetextdoesnotgivethecompletepictureofthequalityassurancesystemoftheRoyalCollegeofMusicbuthighlightsthegeneralprinciplesaswellassomeverymusic-specificelements.
general features
A.2.2“Theobjectivesofqualityassuranceat theRoyalCollegeofMusicare: toassuretheprovisionofthehighestqualitymusicconservatoireprogrammespossible,withinavailableresources,ofanin-ternationalstandardrelevanttothemusicprofessionandtoenhancethequalityoflearningandteach-ingbyprovidinganenvironmentwhichsupportstheirdevelopment.
1 Seewww.rcm.ac.uk/cache/fl0000907.pdf.
38
A.2.3The quality assurance processes encompass: programmes, including teaching and performanceactivitiesandadmissionsprocesses;learningandteachingsupportservicesandresources;assess-mentandstandardsofoutcomes,includingperformancestandards.(…)
A.2.4IndeliveringtheseobjectivestheRCM’squalityassurancesystemshavefivearms:- programmedevelopment,approval,reviewandmonitoring;- externalexaminers,externalspecialists,internalexaminers(includingauditionsforentry);- studentengagement;- reviewofperformancestandards;- professionaldevelopment,appraisalandpeerobservation.(…)
A.2.5PeriodicallytheCollegeevaluatestheeffectivenessofitsapproachtoqualityassurance.Incommonwithotherhighereducationinstitutions,theCollegeissubjecttoinstitutionalauditbythenationalqualityassurancebody(currentlytheQualityAssuranceAgencyforHigherEducation).Inpractice,therefore,forreasonsofefficiency,itconductsreviewsofitsapproachtoqualityassurancewiththecycleofQAAaudit,seeingtheQAAaudititselfastheculminationofthereviewprocess.TheCollegewelcomesthisexternalprocessasanimportantpartofitsownreviewprocess,providingavaluableexternalperspective.”(p.2-3)
programme approval and programme review
A.2.6“Initialprogrammeapprovalandsubsequentprogrammereviewbothinvolveadevelopmentprocessleadingtoanextended,usuallyfullday,meetingofapanelofexternalacademicsandmusicprofes-sionalsandinternalpeerswhichrecommendseithertheapprovalorotherwiseoftheprogrammeandanyrecommendationsorconditions,followingdiscussionswithprofessorialstaff,administra-tors,representativesofthelearningresources,andstudents.AreportisproducedbythepanelandsubmittedtotheBoardofProfessorsforapproval.Therelevantprogrammescommitteeconsidersandrespondstoanyconditionsand/orrecommendations.Programmereviewisdesignedtobeakeyqualityenhancementprocess.”(p.3)
A.2.7“Programmereviewisaprocessofreflectivereviewconductedeveryfive-sixyears.Itdrawsonan-nual monitoring and provides an opportunity for making significant structural changes to a pro-gramme.”(p.5)
criteria for programme approval and review
A.2.8“Theissuesbelowshouldbeaddressedbyprogrammedeveloperswhenpreparingforinitialapprov-alandreview.Initialapprovalandreviewpanelswillwishtosatisfythemselvesthattheyhavebeenadequatelyaddressedeitherbythedocumentationoraspartoftheirdiscussionswithprogrammedevelopmentteams.Ineffect,thesearethecriteriaforinitialorcontinuingprogrammeapproval.
Admission
- Dotheadmissionrequirementsrelatetotheaimsandobjectivesandcontentoftheprogramme?- DotheadmissionrequirementsandcriteriarelatetotheCollege’sequalopportunitiespolicyand
admissionspolicy?- Whatisthescopeforadmissionwithacademiccreditfromanotherinstitution?
Learning outcomes
- Whataretheintendedlearningoutcomesfortheprogramme?- Howdotheyrelatetoexternalreferencepointsincludingthemusicsubjectbenchmarkingstate-
ment(inthecaseoftheBachelorofMusic)andthenationalqualificationsframework?- Howdotheyrelatetotheoverallaimsoftheprogramme?- Aretheyappropriatetotheaims?- Howdoesthedesignandorganisationofthecurriculumpromotestudentlearningandachieve-
mentoftheintendedlearningoutcomes?- Howaretheintendedoutcomesoftheprogrammecommunicatedtostaff,studentsandexternal
examiners?- Dothestudentsknowwhatisexpectedofthem?
Achievement of the intended learning outcomes
- Howdoestheprogrammeencourageachievementoftheintendedlearningoutcomesintermsofknowledgeandunderstanding,specificpracticalskills,transferableskills,progressiontoca-reersasperformersorcomposersand/ortofurtherstudy,andpersonaldevelopment?
- Istheprogrammecontentanddesigninformedbyrecentdevelopmentsintechniquesofteachingandlearning,bycurrentresearchandscholarship,andbychangesinthemusicprofession?
Assessment process and standards
- Doestheassessmentprocessenablestudentstodemonstrateachievementoftheintendedout-comes?
- Aretheresatisfactorycriteriatoenableinternalandexternalexaminerstodistinguishbetweendifferentcategoriesofachievement?
- Cantherebefullconfidenceinthesecurityandintegrityofassessmentprocedures?- Doestheassessmentstrategyhaveanadequateformativefunctionindevelopingstudentabili-
ties?- Whatevidenceistherethatthestandardsachievedbystudentsmeettheminimumexpectations
fortheaward,asmeasuredagainstrelevantsubjectbenchmarksandthequalificationsframe-work?
- Howdoestheprogrammedevelopmentteamreviewandseektoenhancestandards?
Learning opportunities and effective teaching
- Howeffectiveisteachinginrelationtocurriculumcontentandprogrammeaims?- How effectively do professors draw upon their performance practice, composition, research,
scholarshiporotherprofessionalactivitytoinformtheirteaching?
39
40 41
- Isthereeffectiveengagementwithandparticipationbystudents?- Isthequalityofteachingmaintainedandenhancedthrougheffectiveprofessionaldevelopment,
peerappraisalofteaching,effectiveinductionofnewstaff?- Arestudentworkloadsmanageable?
Student progression, academic support and equal opportunities
- Howeffectivelyislearningfacilitatedbyacademicguidance,feedbackandsupervisoryarrange-ments?
- Arethearrangementsforacademictutorialsupportclearandgenerallyunderstoodbystaffandstudents?
- Whatarrangementsareinplacetosupportstudentswithdisabilities?- Howdoestheprogrammetakeaccountofthedutytopromoteequalopportunities/diversity,in-
cludingracialequality?
Learning resources
- Isthecollectiveexpertiseoftheprofessorialstaffsuitableandavailableforeffectivedeliveryoftheprogramme,fortheoverallteaching,learningandassessmentstrategyandfortheachieve-mentoftheintendedlearningoutcomes?
- Areappropriateprofessionaldevelopmentopportunitiesavailable?- Isappropriateadministrativesupportavailable?- HoweffectivelyislearningpromotedbytheCollege’slearningresources?- Issuitableteachingandlearningaccommodationavailable?- Arethelibrarystocksandservicesappropriateandaccessible?- AresuitablespecialistequipmentandappropriateITfacilitiesavailabletostudents?- Isthecareerguidanceserviceadequateandappropriatefortheprogramme?
Programme management
- Whatarethearrangementsforprogrammemanagement?- Dotheyensuretheeffectivedivisionofresponsibilitiesfordifferentaspectsoftheprogramme’s
operation?- Wheretheproposalinvolvesliaisonwithotherinstitutions,whatarethemechanismsformain-
tainingandmonitoringtheserelationships?
Resources
- Are theresources identified in theresourcestatementadequateandappropriate for thepro-gramme?”(p.9-11)
programme monitoring
A.2.9 “TheCollege’sannualprogrammemonitoringsystemisaprocessofqualitycontrol,whichfeedsintoperiodicprogrammereviews.Itisintendedtobeaconcisemechanismwhichreflectsonthepreviousyear’sstandardsofachievement,addressespointsinexternalexaminerreports,andsets
40 41
anactionlistforthecomingacademicyear.Likeprogrammereview,annualmonitoringisdesignedtobeakeyqualityenhancementprocess.Therelevantheadofprogrammescompilesthereport,whichissubmittedtotherelevantprogrammescommittee.”(p.3)
student participation
A.2.10 “Avarietyofstudentengagementandconsultationmethodsareused:- questionnaire-basedfeedback- consultationsessionswithstudents(yearorpathwaygroupsorrepresentatives,forexampleby
externalexaminers)- studentrepresentationonacademiccommittees- sessionswithstudentsaspartofinitialapprovalandreviewevents- regularliaisonwiththeStudents’Association,viatheStudentServicesManager- Director’sweeklyopensurgeriesforstaffandstudents.”(p.16)
external examiners
A.2.11 “The College has procedures for the appointment of external examiners and a published policystatementabout theirdutiesandresponsibilities.Externalexaminernominationsoriginate fromtheprogrammescommittee,andaresubjecttotheapprovaloftheBoardofProfessors.ExternalexaminersarerequiredtosubmitannualreportstotheDirectorofAcademic&AdministrativeAf-fairs,whocirculatesthemtotheDirector,Dean,therelevantheadofprogrammes,andtherelevantregistryadministrator.Therelevantheadofprogrammesisresponsibleforrespondingtoreports,usuallyinthecontextoftheannualprogrammemonitoringreport,acopyofwhichisprovidedtotheexternalexamineronceithasbeenfinalisedbytherelevantprogrammescommittee.
A.2.12 Externalspecialistsareincludedonallpanelsforgraduationorfinalrecitalassessments.Theyarenominatedbyheadsoffaculty.Internalexaminersforallpracticalexaminationsandforentranceauditionsarenominatedbyheadsoffaculty.DetailednotesfortheguidanceofexaminersandforthoseconductingauditionsarereviewedeachyearbytheAAMG[ArtisticandAcademicManage-mentGroup]andprovidedforallexaminers.Copiesofthecriteriatobeappliedaremadeavailableforstudentstoensurethattheyarefullyawareoftheparametersagainstwhichtheywillbeas-sessed.”(p.4)
review of performance standards
A.2.13 “Highperformancestandardsareacornerstoneofworldclassconservatoireeducation.Forthisreason,evidenceofthelevelofstandardsachievediscarefullymonitoredandreviewedonperiodicbasis.Theevidencebaseusedcomprisesreportsrequestedfromvisitingconductors,thosegivingmasterclassesorworkshops,prizecompetitionadjudicators,andexternalspecialistexaminers.
42 43
Annualoverviewreports,which includeananalysisof thequantitativeandqualitativedata fromreports,togetherwithexternalpressreportsofRCMperformancesandarecordofkeystudentsuc-cessesinexternalcompetitionsandotherperformance-relatedactivities,arecompiledbytheDeanandaresubmittedtotheBoardofProfessorsforapproval.”(p.4)
staff
A.2.14“Theknowledge,experienceandskillofmembersofstaff isoneof theCollege’smostvaluableassets.Tofulfilitsresponsibilitiestostudents,theCollegemustensurethatthequalityofthisre-sourceiscontinuallyenhanced.TheCollegeprofessionaldevelopmentpolicyincludesprovisionforthesupportofindividualandgroupproject-basedstaffprofessionaldevelopmentactivities.Profes-sionaldevelopmentisacorepartofstaffappraisalsystems.Monitoringandevaluationreportsonprofessional development are considered annually by the Board of Professors. The College alsosupportspeerobservationthroughitsLearningandTeachingStrategy.”(p.4)
a.3 the quality assurance system of the Prince Claus Conservatoire, Hanze university of applied sciences, groningen (nl)
A.3.1 ThePrinceClausConservatoirehasitssystemofinternalqualityassurancelaiddowninanannualQualityAssurancePlan(approx.35pages)2.TheQualityAssurancePlaniswrittenconformingtoauniversity-wideformat,whichlinkstheplantoanarrayofotherdocuments,proceduresandinstru-ments,bothuniversity-wideasacademy-specific.
2 HanzeUniversityofAppliedScience,PrinceClausConservatoire:“Kwaliteitszorgplanenprocedurehandboek2007-2008”,
version0.2.
42 43
A.3.2TheHanzeUniversityofAppliedSciencehasdefinedqualityassuranceasthepermanent,system-aticandcyclicalattentionformeasuring,monitoringandfosteringquality.IthasbaseditssystemofqualityassuranceonthePDCA-circle,representedvisuallyasfollows:
3 ManagementDashboard:aninstrumentshowinginrealtimetherelationbetweenactualresultsandtargetresultsforavariety
ofperformanceindicators
UniversityStrategicPlan
AcademyStrategicPlan
AcademyYearPlan
Managementcontract
Academyspecific
measurements
Academyreports Managementdashboard3
Prestation-Evaluation
Satisfaction
statements
Results
(hardfacts)
adapt
check
Prestation measurements
do
plan
Goals Performanceindicators
Goals Performanceindicators
Goals Performanceindicators
Activities
Goals Performanceindicators
44 45
A.3.3 Crucial inthisschemeistheconnectionbetweenthephaseofplanningandthephasesofcheckandadapt.Intheplanningphase,fortheuniversityasawholeaswellasforitseighteenacademiesgoalsandperformanceindicatorsareformulatedinvariousdocuments,rangingfromtheuniversitystrategytotheyearlymanagementcontractsbetweentheuniversityboardandthedeanofeachoftheacademies.
A.3.4 Inthecheckphase,satisfactionstatementsaswellasresults(“hardfacts”)aregatheredrelatedtothevariousgoalsandperformanceindicators.Thisispartlydonecentrallyforthewholeuniversitywithgenericinstruments,andpartlyhastobedonebytheacademiesthemselves.Satisfactionstate-mentsandresultsarepresentedinthegeneralmanagementdashboardandinthevariousreportsonacademylevel.Thephaseofadaptconsistsofstudyanddiscussionofthesedocumentsinordertodecideonfutureactionswhicharethanfedbackintotheplanningphase.
A.3.5 Thiswholecycleisforeveryacademy,soalsoforthePrinceClausConservatoire,laiddownintheQualityAssurancePlan.TheQualityAssurancePlancontains:- ageneralintroductiontothesystemofinternalqualityassurance;- thequalitygoalsandperformanceindicatorsfortheyearconcerned- anoverviewofthepersonsplayingaroleinqualityassurance,ofthedecisionmakingprocesses,
oftherelevantdocumentsandofthecommunicationplanconnectedtoqualityassuranceactivi-ties
- atimepathforthequalityassuranceactivities- schemesforeveryindividualqualityassuranceactivity:
Instrument
Responsibleperson
Goaloftheactivity
Normvalue
Reportby
Reportedto
Reportedbymeansof
Frequencyanddeadline(s)
Shortdescriptionofactivity(plan/do) Date Activity Executedby
Procedureleadingtoqualityimprove-
ment(check/adapt)
Remarks
44 45
aPPendIx B the enQa 2005 standards and guidelines for Quality assurance in the european Higher education area (esg) – summary version
ThissummarylistofEuropeanstandardsforqualityassuranceinhighereducationisdrawnfromChapter2of the full reportand isplacedhere foreaseofreference. Itomits theaccompanyingguidelines.Thestandardsareinthreepartscoveringinternalqualityassuranceofhighereducationinstitutions,externalqualityassuranceofhighereducation,andqualityassuranceofexternalqual-ityassuranceagencies.
Part 1: european standards and guidelines for internal quality assurance within higher education institutions
B.1.1 Policyandproceduresforqualityassurance:Institutionsshouldhaveapolicyandassociatedproce-duresfortheassuranceofthequalityandstandardsoftheirprogrammesandawards.Theyshouldalsocommitthemselvesexplicitlytothedevelopmentofaculturewhichrecognisestheimportanceofquality,andqualityassurance,intheirwork.Toachievethis,institutionsshoulddevelopandim-plementastrategyforthecontinuousenhancementofquality.Thestrategy,policyandproceduresshouldhaveaformalstatusandbepubliclyavailable.Theyshouldalsoincludearoleforstudentsandotherstakeholders.
B.1.2 Approval, monitoring and periodic review of programmes and awards: Institutions should haveformal mechanisms for the approval, periodic review and monitoring of their programmes andawards.
B.1.3 Assessment of students: Students should be assessed using published criteria, regulations andprocedureswhichareappliedconsistently.
B.1.4 Qualityassuranceofteachingstaff:Institutionsshouldhavewaysofsatisfyingthemselvesthatstaffinvolvedwiththeteachingofstudentsisqualifiedandcompetenttodoso.Theyshouldbeavailabletothoseundertakingexternalreviews,andcommenteduponinreports.
B.1.5 Learningresourcesandstudentsupport:Institutionsshouldensurethattheresourcesavailableforthesupportofstudentlearningareadequateandappropriateforeachprogrammeoffered.
B.1.6 Informationsystems:Institutionsshouldensurethattheycollect,analyseanduserelevantinforma-tionfortheeffectivemanagementoftheirprogrammesofstudyandotheractivities.
B.1.7 Publicinformation:Institutionsshouldregularlypublishuptodate,impartialandobjectiveinforma-tion,bothquantitativeandqualitative,abouttheprogrammesandawardstheyareoffering.
46 47
Part 2: european standards for the external quality assurance of higher edu- cation
B.2.1 Useofinternalqualityassuranceprocedures:ExternalqualityassuranceproceduresshouldtakeintoaccounttheeffectivenessoftheinternalqualityassuranceprocessesdescribedinPart1oftheEuropeanStandardsandGuidelines.
B.2.2Developmentofexternalqualityassuranceprocesses:Theaimsandobjectivesofqualityassuranceprocessesshouldbedeterminedbeforetheprocessesthemselvesaredeveloped,byallthosere-sponsible(includinghighereducationinstitutions)andshouldbepublishedwithadescriptionoftheprocedurestobeused.
B.2.3Criteriafordecisions:Anyformaldecisionsmadeasaresultofanexternalqualityassuranceactivityshouldbebasedonexplicitpublishedcriteriathatareappliedconsistently.
B.2.4Processesfitforpurpose:Allexternalqualityassuranceprocessesshouldbedesignedspecificallytoensuretheirfitnesstoachievetheaimsandobjectivessetforthem.
B.2.5Reporting:Reportsshouldbepublishedandshouldbewritteninastyle,whichisclearandread-ilyaccessibletoitsintendedreadership.Anydecisions,commendationsorrecommendationscon-tainedinreportsshouldbeeasyforareadertofind.
B.2.6Follow-upprocedures:Qualityassuranceprocesseswhichcontainrecommendationsforactionorwhichrequireasubsequentactionplan,shouldhaveapredeterminedfollow-upprocedurewhichisimplementedconsistently.
B.2.7Periodicreviews:Externalqualityassuranceofinstitutionsand/orprogrammesshouldbeunder-takenonacyclicalbasis.Thelengthofthecycleandthereviewprocedurestobeusedshouldbeclearlydefinedandpublishedinadvance.
B.2.8System-wideanalyses:Qualityassuranceagenciesshouldproducefromtimetotimesummaryre-portsdescribingandanalysingthegeneralfindingsoftheirreviews,evaluations,assessmentsetc.
Part 3: european standards for external quality assurance agencies
B.3.1 Useofexternalqualityassuranceproceduresforhighereducation:Theexternalqualityassuranceofagenciesshouldtakeintoaccountthepresenceandeffectivenessoftheexternalqualityassur-anceprocessesdescribedinPart2oftheEuropeanStandardsandGuidelines.
46 47
B.3.2Officialstatus:AgenciesshouldbeformallyrecognisedbycompetentpublicauthoritiesintheEu-ropeanHigherEducationAreaasagencieswithresponsibilitiesforexternalqualityassuranceandshouldhaveanestablishedlegalbasis.Theyshouldcomplywithanyrequirementsofthelegislativejurisdictionswithinwhichtheyoperate.
B.3.3Activities:Agenciesshouldundertakeexternalqualityassuranceactivities(atinstitutionalorpro-grammelevel)onaregularbasis.
B.3.4Resources: Agencies should have adequate and proportional resources, both human and finan-cial,toenablethemtoorganiseandruntheirexternalqualityassuranceprocess(es)inaneffectiveandefficientmanner,withappropriateprovisionforthedevelopmentoftheirprocessesandproce-dures.
B.3.5Missionstatement:Agenciesshouldhaveclearandexplicitgoalsandobjectivesfortheirwork,con-tainedinapubliclyavailablestatement.
B.3.6Independence:Agenciesshouldbeindependenttotheextentboththattheyhaveautonomousre-sponsibilityfortheiroperationsandthattheconclusionsandrecommendationsmadeintheirre-portscannotbeinfluencedbythirdpartiessuchashighereducationinstitutions,ministriesorotherstakeholders.
B.3.7Externalqualityassurancecriteriaandprocessesusedbytheagencies:Theprocesses,criteriaandproceduresusedbyagenciesshouldbepre-definedandpubliclyavailable.Theseprocesseswillnormallybeexpectedtoinclude:- Aself-assessmentorequivalentprocedurebythesubjectofthequalityassuranceprocess;- Anexternalassessmentbyagroupofexperts,including,asappropriate,(a)studentmember(s),
andsitevisitsasdecidedbytheagency;- Publicationofareport,includinganydecisions,recommendationsorotherformaloutcomes;- Afollow-upproceduretoreviewactionstakenbythesubjectofthequalityassuranceprocessin
thelightofanyrecommendationscontainedinthereport.
B.3.8Accountabilityprocedures:Agenciesshouldhaveinplaceproceduresfortheirownaccountability.
48
aPPendIx C. list of abbreviations used
AEC AssociationEuropéennedesConservatoires,AcadémiesdeMusiqueetMusikhochschulen
EFQM EuropeanFoundationforQualityManagement
ENQAEuropeanAssociationforQualityAssuranceinHigherEducation
EUA EuropeanUniversityAssociation
PDCA Plan-Do-Check-Adapt
QAA QualityAssuranceAgency
RCM RoyalCollegeofMusic
Association Européenne des Conservatoires,Académies de Musique et Musikhochschulen (AEC)PO Box 805 NL-3500AV Utrecht The Netherlands
Tel +31.302361242 Fax +31.302361290Email aecinfo@aecinfo.org Website www.aecinfo.org
top related