oslo summer school: evaluation - universitetet i oslo · the oss has three sources of income: (a) a...

TheOsloSummerSchoolcelebratedits25thyearin2017.Howcanitgetevenbetter,attractmoreUiOPhDcandidates

andreduceitsfinanciallosses?

Oslo Summer School: Evaluation REPORT

November,2017

1

Executive Summary ThisisanevaluationreportoftheOsloSummerSchoolinComparativeSocialScienceStudies(OSS)developedbythe internationalevaluationpanelconsistingofresearchersfromtheCzechRepublic,GermanyandSweden.OSSofferscoursesonthePhDlevelacrossmanysocialsciencedisciplines.TheOSSattractsstudents,mostlyPhDcandidates,fromallovertheworld.In2017,theOSScelebratedits25thanniversary.

ThegoalofthisreportistosupportthefurtherdevelopmentoftheOSSbyaddressingthefollowingquestions:

1. HowtoattractmorePhDcandidatesfromtheUniversityofOslo?2. HowtoreducethefacultyinvestmenttotheorganizationoftheOSS?

GeneralrecommendationstomakeOSScoursesbetter:

Our analysis of survey data shows that UiO PhD candidates are not very different from otherparticipants in theirexpectationsandpreferences.Slightdifferencesbetweenthesetwogroupsdonotjustifydifferentrecommendations.Therefore,wefirstlistgeneralrecommendationtomakeOSScoursesevenbetterandproduceevenmoresatisfiedparticipantswhowill spread thewordaboutOSSandattractmoreparticipants,includingUiOPhDcandidates:

• Students do not just expect to learn things. They also expect to acquire a lot of relevantliteraturerecommendationsanddrawmotivationforworkontheirPhDthesis.Therefore:

o moreemphasisshouldbeplacedongreatqualityreading lists.Theyshould includewell-selected relevant literature, be distributed long enough in advance, so thatstudents have enough time to prepare, and very importantly, they should beprioritized(showingwhichliteratureismostimportanttoreadforeachsession).

o The OSS could include some motivational programmes or features. TheirdevelopmentcouldbediscussedwithHRspecialists.ParticipationintheOSSshouldalsoserveasaboostinmotivationforthePhDcandidates.

• Inaddition,studentsconsideritveryimportanttolearnanewskillsormethods,andtogethands-onexperienceorlearnpracticalexamples.Therefore,lecturersshouldbeencouragedtopreparepracticalcourses.Theoreticalcoursescanalsobepracticalinthesensethattheyshowexamplesofhowcertaintheorycanbeusedandhowitcan informresolvingspecificacademicquestions.

• Methodologicalcoursespersistentlyshowhighpopularity.Theyaremorelikelytogeneratehighnumbersofapplications,andhencepreventsteepdropsinparticipationbetweenyears(i.e. prevent financial instability). The OSS could consider expanding its offer onmethodologicalcourses.Apartfromacourseoncasestudyresearchandacourseonmixedmethods, which are already taught, we have identified further courses in potentially highdemand: courses on software for qualitative data analysis (such as NVivo or Atlas.ti), andcourses on impact assessment. (Note that we have surveyed ex-participants. Non-participantsmightpreferdifferentcourses.AsurveyamongallPhDcandidatesattheFacultyonthismattershouldbeconsidered.)

• Keepthecoursesize limited.Toomanyparticipantsreducetheopportunities forquestionsandotherinteractions.

• Consider preparing class facilitation guideline. Lecturers at the OSS are mostly seniorlecturerswithprofoundexperience inteaching,buttheirexperiencemaybefromdifferentcontexts (different group sizes, different types of students). Sometimes, students wouldwelcome more structured sessions with a better management of interactive tasks andstudents’presentations.Afacilitationguidelineforlecturerscouldhelptomeettheseneeds.

2

Institutionalsteps:LinkingthesummerschooltoPhDeducation:

ApartfromcontinuousimprovementoftheOSScourseoffering,UiOPhDcandidatescouldalsobeattractedbyforgingastrongerlinkbetweenOSSandPhDeducationattheFacultyofSocialSciences.Wehaveidentifiedthefollowingpossiblestepstowardsthisgoal:

• Considermakingthepersonresponsible forPhDeducation inthedifferentdepartmentsexofficioalsothepersonwhorepresentsthedepartmentinthecooperationwiththesummerschool. This would improve the flow of information from the different tracks of the PhDprogramtotheplanningoftheSummerSchool.

• The evaluation of the PhD program last year identified a lack of access to more generic(transferable)skillssuchasacademicwritinginEnglish,gettingpublished,timemanagement,communicationwith themediaetc. If suchcoursesareofferedunder theOSSumbrellaasshortcoursesjustbeforethemainOSSprogram,internalPhDcandidatescouldnotonlygainmoreaccess to learnsuchskills,but theymightalso feel incentivized toconsider themainOSScoursesaswell.

• To promote inter-disciplinary understanding and communication at the faculty level, OSScould develop specific inter-disciplinary courses (e.g. economic, political and sociologicalapproachestoclimatechange).Suchacoursecouldserveasabridgebetweenthedifferenttracksof thePhDeducationat theFaculty and contribute to inter-disciplinary goalsof thefaculty.

Reducingfinancialloses

TheOSShasthreesourcesofincome:(a)agrantfromtheFacultyofSocialSciences,(2)grantsfromdepartmentsthatbuyadditionalcourses,and(3)coursefeespaidbystudents.Thisincomehasnotbeensufficient,andtheOSShasgeneratedlosses.TheFacultyofSocialSciencesislookingforwaystoreduce its financial involvement inthesummerschoolwithoutendangeringtheexistenceoftheinstitution.Herearesomesuggestionsfollowingtheseideas:

• Increase participation fees. Current fees are comparatively low. However, this should bedonewithcaution.Ourreferencesaremethodologicalsummerschools(e.g.ECPRandEssex),whiletheOSS is toa largedegreefocusedonmoretopic-basedandspecializedcourses. Inaddition, participants have to factor in the higher living costs in Oslo, so there is somejustificationforthecoursefeestobelowerinOslothanatthecompetingsummerschools.

• LinktheOSSmoreclosely toPhDeducation in theFacultyofSocialScience,andrelatedtothis,shiftsomeoftheOSScoststothedepartments.

• Considerchargingforlunchesseparatelyfromthecoursefee.• Consider making costs more predictable by increasing the general fee for lecturers and

abolishingthelesspredictableadditionalpaymentforthegradingofpapers.

Additionalrecommendations:

• For further advertising purposes aimed at international students (if there is ever such aneed), the OSS administration could consider investing into networking with otherinstitutionsand internationalgroups,whichwouldadvertise theOSSamong their studentson behalf of OSS. Participants perceive recommendations from their own university orinternationalnetworksasmosteffectivewaysofadvertising.

• The OSS holds a unique competitive advantage: its topic-based specialized courses asopposed to more frequently offered methodological courses. The OSS should considerexpandingitsofferonmethodologicalcoursesformultiplereasonsdiscussedinthisreport,butitshouldnotloseitscompetitiveadvantageintheprocess.

3

Thisreportisstructuredintothreeparts:

• Part1focusesontheparticipantsandprimarilyprovidesrecommendationshowtoattractmorePhDcandidatesfromtheUniversityofOsloaswellashowtoimprovetheOSSingeneral.

• Part2focusesonfundingsourcesandthepossibilitiestoreducefacultycontributiontotheSummerSchoolbudget.

• Part3focusesontheoverallacademicstandingoftheOSS.

Themainpartofthisreporthas19pagesincludingtheExecutivesummaryandCoverpage.Inaddition,thereare10pagesofappendicesthatprovidemoredetailedinformation.

4

Table of Contents

Executive Summary.................................................................................................................................1

Table of Contents....................................................................................................................................4

Part 1: Participants..................................................................................................................................5

Advertising the OSS.............................................................................................................................5

What courses do participants want?...................................................................................................6

How to make courses better.............................................................................................................10

How to attract more PhD candidates from UiO.................................................................................10

LinkingtheSummerSchooltothePhDeducationattheFacultyofSocialSciences........................12

OSS’s competitors.............................................................................................................................13

Part2:Funding......................................................................................................................................15

Income..............................................................................................................................................15

Expenses............................................................................................................................................16

ProfitsandLosses..............................................................................................................................16

Part 3: Academic standing of the summer school.................................................................................18

Appendices............................................................................................................................................21

Appendix 1 Number of participants over years grouped by course theme.......................................21

Appendix 2 Follow-up survey sample composition............................................................................23

Appendix 3 Additional charts – follow-up survey answers grouped by where participants are from26

Appendix 4 Additional charts – follow-up survey answers grouped by year of participation............27

5

Part 1: Participants Inthispart,weworkwiththreedifferentdatasources:

1. Materials provided by the Faculty of Social Sciences at UiO including course syllabi,participantlists,overviewsheetsandstudentevaluationsummaries.

2. Anewly collected follow-up survey conductedwith anonlinequestionnairedistributed viathefacultytosummerschoolparticipantsfrom2010-2017(n=308,n=305afterexcludingthose who say they did not visit OSS between 2010 and 2017). The sample is notrepresentativeandself-selectionmayplaysomeroleintheresponses.

3. Desk research of data available online, especially regarding information about competingsummerschools.

AtthispointwewouldliketothanktheofficeoftheOSSandespeciallyTronTornebyaswellastheadministrationoftheFacultyofSocialSciencesfortheirsupport inprovidinguswiththenecessaryinformationforthisevaluation.Withouttheirhelpwewouldnothavebeenabletofinishthisreport.

Amongthe308participantswhotookpart inthefollow-upsurvey,moreresponseswerecollectedfrommorerecentyears.(Thisisnotduetoacomparableincreaseinthenumberofstudents.Infact,thenumberofparticipantshasnotchangedmuchovertheyears,seeAppendix1formoredetails.)Thismeansthatanalysesonaggregateddataaremoreinfluencedbyevaluationsfromstudentsfrommorerecentyears,especially2017.SeeAppendix2forthefollow-upsurveysamplecomposition.Itisalso important torememberthatwehaveonlysurveyedparticipantsof theOSS.Non-participants,for example non-participating PhD candidates at UiO, might have different views and it could beusefultoaddresstheminfutureresearchtolearnmoreaboutreasonsfortheirnon-participation.

In this part of the report, focusing on participants, we first show how students learn about thesummerschoolandwhatareeffectivewaystoadvertisetheOSS.WeshowthatUiOPhDcandidatesareprobablymostofteninformeddirectlybytheirprofessorsandcolleagues.AsUiOPhDcandidatesarecloseto“thesource”,theycanlearnalotabouttheOSSandtheirdecisionsaboutparticipationlargelydependontheOSS’squality.Therefore,wefurtherfocusonpossiblewaysof improvingtheOSS’squality.First,welookatwhatkindofcoursesstudentswant.Second,welookatwhatstudentsexpect from a good summer school and what features should be strengthened tomake the OSScoursesevenbetter.

Advertising the OSS

To inform more efficient advertising of the OSS, we have first conducted a review of students’answers to thequestion “how they learnedaboutOSS”. It turnsout that there are three relevantsourceswhichfaroutweightheothers:learningabouttheOSS“throughmyuniversity,professororsupervisor”,“throughinternet(suchasGoogle/blogs)”,and“fromcolleaguesorfellowstudents”.Weassume that participants from UiO are the ones who usually learn about the OSS from theirprofessorsorcolleagues,whiletheparticipantsfromoutsideofUiOusually learnabout it fromtheinternet. Direct information from professors or via colleagues seems the most natural and trulyfunctioningwaytoattractparticipantsfromUiOandshouldbefurtherencouraged.

In addition, we have also addressed this question in the follow-up questionnaire to measurestudents’ perception of advertising efficiency of different advertising channels (Figure 1). Directrecommendationsfromtrustedactors(colleagues,owndepartmentorfaculty)areperceivedasmosteffective.Ifparticipants,andUiOPhDcandidatesinparticular,usuallylearnabouttheOSSfromtheir

6

colleaguesandfromthefacultyandiftheyalsoconsiderthesesourcesmosteffective,itislikelythattheattendanceofPhDcandidatesatUiOreflectstheirpreferencesandOSS’squality.ThebestwaytoattractmorePhDcandidatesfromUiOisthereforefocusingonacontinuousimprovementofquality,whichwewilladdressinthenexttwosections.

Figure1“WhatdoyouthinkareeffectivewaystoadvertisetheOSStopeoplelikeyou?”(theformulationsunderchartsaresimplified,seeAppendix3forfullformulations)(n=308)

Note:“sns”standsforsocialnetworkingsitessuchasFacebook,Twitteretc.

What courses do participants want?

To gainmore insight about theOSS’s participants and their view of the summer school, we haveanalysed students’ evaluation provided to us by the faculty (i.e. evaluation from 2014, 2015, and2016 aggregated by individual courses), and other coursematerials (these were available for theyears2010,2012,2014,and2017)suchassyllabi,participantlistsandoverviewsheets.

7

TheOSS has offered courses from across the social science academic disciplines. Our analysis hasshown,thatthenumberofparticipantsinmethodscourseshasalwaysbeenhighandincreasedovertime. In 2010, the OSS offered three courseswhich could be consideredmethodological.1 Two ofthemwere the first and second in terms of number of participants. In 2012, there was only onemethodologicalcourse (thepopularCaseStudyResearchMethods)andasharpdrop innumberofapplications from 320 to 227. We cannot say that this implies causation, but when anothermethodological course was introduced in 2014 (Mixed Methods), it also attracted a lot ofparticipants.Inthelastyear,2017,thealreadypopularcourseonCaseStudyResearchMethodsbyprofessorAndrewBennett,shotupinthenumberofparticipantsto46people(twiceasmanyaswasthe average number of students per course). The other two methodological courses (MixedMethods,andCollectingandAnalysingBigData)werethenextmostpopularones.

DespitetheundeniableattractionofprofessorAndrewBennett’scourse,theavailabledatasuggestthatmethodcourses ingeneralare inhighdemand.Toprovidefurtherbackingforthishypothesis,wehave askedparticipants in the follow-up survey about their expectations froma good summerschool.Figure1showsthat learninganewmethodorskill, indeed,belongstothemost importantparticipants'expectations.Otherthanthat,participantsalsoexpecttoreceiverelevantliteratureandmotivationforprogressing(andpossiblyredesigning)theirPhDthesis.Significantly lessparticipantsexpecttogetinputfortheirownteaching,butitisstillmorethanathirdofthemwhoconsiderthisabsolutelyorveryimportantaswell.

Whenwenextaskedtherespondents if there is“anythingelse [they]expect fromagoodsummerschoolandconsiderimportantwhatwasnotmentionedabove?”,werepeatedlyreceivedtheanswerthattheyalsoexpecthighqualityteaching,nicesocialeventsandgettingtoknowtheotherpeople,inspiration andnewperspectiveswhich go across disciplines, feedback on theirwork, networking,newtheoreticalinsights,andup-to-datefocus.

1TopicsinAppliedMicroeconometrics,CaseStudyResearchMethods,andEventHistoryAnalysisandtheLifeCourse.2ThiswouldalsobepossiblewithregardtootherFacultiesthatcouldhaveaninterestinbuyingcoursesinthe

8

Figure2“Wewouldliketoknowwhatyouexpectfromagoodsummerschool.Please,indicateforeachitembelow:Howimportant it is for a summer school that…” (the formulations under charts are simplified, see Appendix 3 for fullformulations)(n=308)

Our additional questions in the follow-up survey addressed students’ preferences for selectedmethodological courses (Figure 3). The selection comes from our desk research of other summerschools which focus on methods and cannot be considered comprehensive. In addition, we onlysurveyOSSpastparticipants.Non-participantscouldhavedifferentpreferences.However,theresultscould still be inspirational in the case that the OSS faculty decides to expand its offer onmethodologicalcourses.

The most popular courses among our respondents were Case study design, Mixed methods, andSoftware for qualitative data analysis. This could reflect the fact that OSS has mostly offeredqualitative methodological courses and its past participants are more likely to be qualitativeresearchers in general. The results are still interesting and imply some recommendations: Open acourse on software for qualitative analysis (such as NVivo) to attracted past participants to comeagain.Considercourseson“policyevaluationand impactassessment”or“Automatedtextanalysis(textmining,computer-assistedcontentanalysis)”forthesamepurpose.

9

However,theoretical/substance-oriented(orthematic)coursesareauniquepropositionbytheOSS.It may be a good idea to consider the strong and growing interest in method courses amongparticipantsandpossiblyexpandtheOSSbyoneortwoofthesecourses(oratleastmakesurethatthereremainssomeformofcourseoffering),buttheOSSshouldnotbetooquicktotransformintoamethodologicalsummerschoolandgiveuponitscompetitiveadvantageanduniqueness.Methodscoursesmayattractmorestudents,but it isanopenquestionfortheOSSfacultytodebatewheretheirprioritieslie.

Figure3“Ourdatashowsthatincreasingnumberofstudentstodaychoosemethodologicalcourses.Shouldyougotoasummerschoolforamethodologicalcourse,howlikelywouldyouconsiderthefollowingoptions?”(theformulationsunderchartsaresimplified,seeAppendix3forfullformulations)(n=308)

10

How to make courses better In thissection,wesummarizethemain insights fromtheopen-endedquestions fromthestandardevaluationsurveysfrom2014,2015,2016and2017.

Prioritizethereadinglistanddistributeitearly.

This seems to be the most resounding requirement. Many students across courses gave thesuggestiontoprioritizethereadinglist.Someofthemalsosuggestshorteningit,butsinceothersliketoreceivealotoftipsaboutliteraturesources,prioritizingseemsthebetterwaytogo.Sometimes,participantscomplainaboutotherstudentsnotdoingtheirreadingbeforetheclass.Thisslowsdowntheclassandbothersthosewhocomeprepared.Ifthereadinglistisprioritized,appealsbyteachersfor students todo their readingmaybemoreeffective and result inbetter classes for everybody.Somestudentsalsosuggestedthatreadinglistsshouldoffermorevarietywithinthecoursetopicorthatteachersshouldleaveoutoutdatedoroff-topicitems.Itisalsoimportantthatreadinglistsaredistributedearlyandshouldnotchangeshortlybeforethecoursestarts.

Givestudentshands-onexperience,interaction,andpracticalexamples.

Studentswouldoften appreciatemore interactions, such as discussions in small groups. Especiallyparticipantsofmethodologicalcourseswanttogainmorepracticalexperienceandseeapplicationsandexamplesofwhattheyarelearning.Theycallformorepracticeandlesstheory.

Keepthenumberofparticipantslimited.

Forthereasonsabove,keepthenumberofparticipantsineachcourselimited.Groupsofover30oreven40participantsseemtoobig.

Sessionsshouldhaveaclearstructure.

Somestudentswouldappreciatemorestructuredcourseandindividualsessions.Theysuggestusingkeywords, improving time management, and keeping the planned breaks. Teachers could alsoprovidesessionoutlinesatthestartsothateverybodyknowswhattoexpect.

Managestudents’presentations.

There are somewhat contradicting recommendations concerning student’s presentations. Somestudents would like to havemore opportunities to present their work and discuss it with others.Otherscomplainedabout toomuch timededicated to longstudentpresentations.This isprobablycoursedependent,butageneral recommendationcouldbe togivestudentsaclear (andrelativelystrict) time limit for theirpresentationandenforce it. Teachers couldalso instruct studentsnot topresenttheirworkbroadly,butselectparticularproblemstheywanttodiscusswithothersandonlyfocus on those. Students’ presentations in courses should not be about giving students space topresenttheirwork,buttogiveeverybodyanopportunitytolearnnewthingsbydiscussingparticularproblems.

How to attract more PhD candidates from UiO

The key question behind this aim is, whether the PhD candidates at UiO and their needs andexpectationsaredifferentfromthoseofotherparticipants?Toaddressthisquestion,wegroupthedatabytheuniversitywherestudentswerebasedatthetimeoftheirattendanceoftheOSS.Itturnsoutthattherearealmostnodifferencesbetweenthemandotherparticipants.Figure4showsthat

11

moreorlessthesameadvertisingchannelsareperceivedaseffectivebytheUiOPhDcandidatesasbyeverybody.

Figure4Figure3“WhatdoyouthinkareeffectivewaystoadvertisetheOSStopeoplelikeyou?”GROUPEDBYWHERETHEYAREBASED(n=297asthesmallgroupofparticipantnotbasedatanyuniversitywasexcludedfromtheanalysis)

Similarly, UiO PhD candidates have very similar expectations from a good summer school aseverybodyelse(theymightbeevenslightlymoreinterestedingettingtipsforrelevantliteratureandslightly less interested ingetting input for theirown teaching,but thedifferenceseemsnegligible)(Figure5).

Figure5“Wewouldliketoknowwhatyouexpectfromagoodsummerschool.“GROUPEDBYWHERETHEYAREBASED(n=297asthesmallgroupofparticipantnotbasedatanyuniversitywasexcludedfromtheanalysis)

Finally, thegeneral recommendationsaboutpotentiallypopularmethodologicalcourses toopen inthefuturealsoapplytothesubgroupofUiOPhDcandidates(Figure6).

12

Figure6"Shouldyougotoasummerschoolforamethodologicalcourse,howlikelywouldyouconsiderthefollowingoptions?”GROUPEDBYWHERETHEYAREBASED(n=297asthesmallgroupofparticipantnotbasedatanyuniversitywasexcludedfromtheanalysis)

(SeesimilarchartsgroupedbycountryoforigininAppendix4andbyyearofparticipationinAppendix5.)

WecanconcludethatUiOPhDcandidateshaveapproximatelythesameneedsandexpectationsasotherparticipantsof theOSS.Attractingmoreof themdoesnot imply takingdecisions thatwouldalienateexternalparticipants.Therecommendationsabout(a)howtoeffectivelyadvertisetheOSS,(b)whatcoursestoadd,andpossiblymostimportant(c)whatgeneralfeaturestofocusoninordertoimprovecoursequality,allseemgeneralenoughtoapplyforbothUiOandexternalparticipantsofthesummerschool.

However, in addition to these recommendations, more refined steps could be taken to attractspecificallyUiOPhDcandidates.Thesestepsareinferred inthenextsectionfromtheevaluationofthePhDprogramattheFacultyofSocialSciences,whichtookplacelastyearanddemandedcloserlinkingbetweentheOSSandthePhDeducationattheFaculty.

Linking the Summer School to the PhD education at the Faculty of Social Sciences

TheFacultyofSocialSciencesundertookanevaluationofitsPhDprogramlastyear.Inthecontextofthisevaluationone issuewas thecoursecomponent included in thePhDeducationat theFaculty.TheevaluationreportspecificallyreferstotheOSSandstatesthat:

“ThecommitteenotedthattheOsloSummerSchoolprovidesimportantcoursesforanumberofthetracks,anddiscussedwhetherthekindsofcross-disciplinarysynergiesthataredevelopedtherecouldbe reproduced in a common Faculty-wide induction event (possibly modelled on the Faculty ofMedicine’s induction) that brings together all PhDs to participate in a series of problem-basedactivities.”

Besides this recommendation the structure of the OSS also has the potential to address otherrecommendations made by the evaluation panel and through this foster cooperation among the

13

tracksofthePhDeducationaswellasincreasethenumberofinternalPhDcandidatesthattakepartintheOSS.

OnekeysuggestionfromtheevaluationofthePhDprogramwasthatthereisaneedformoreinter-disciplinarycoursesbetweenthedifferenttracksoftheprogram.HeretheOSSwithitsmoregeneralmethodscoursesclearlyoffersonevenuewherethiscouldbeintensified.Inaddition,itwouldalsobe a possibility to develop specific courses that foster inter-disciplinary understanding,communication, and cooperation around a specific topic (e.g. economic, political and sociologicalapproachestoclimatechange).Itwould,forexample,bepossibletohaveonesuchcourseeachyearthat is dedicated to inter-disciplinary exchange,which also acts as a bridge between the differenttracksof thePhDeducationat the Facultybye.g. being compulsory forPhD candidates thathavestartedattheFacultyinthepreviousyear.

Another option to better link the OSS with the PhD education at the Faculty would be to createsynergiesbymakingthepersonresponsibleforPhDeducationinthedifferentdepartmentsexofficioalso thepersonwho represents thedepartment in thecooperationwith the summer school.2ThiswouldimprovetheflowofinformationfromthedifferenttracksofthePhDprogramtotheplanningoftheOSS.Throughtheselinkagesitwouldalsobeeasiertoidentifymethodscoursesthatcouldberelevant formultiple tracks of the PhD education program at the Faculty and possibly offer 1 – 2additionalcourseseachyearthatcanbecountedaspartofthecompulsorycoursesofseveraltracksofthePhDprogram.

Finally,one issue that theevaluationof thePhDprogramalsoaddressedwas the lackofaccessofPhDstocoursesthatteachgenericskills:

“[thereisa]lackofaccesstomoregeneric/transferableskillstraining,forexample:academicwritingin English, getting published, time management, project management, financial management,workingwithyoursupervisor,teamworkskills,grantapplication,andcommunication–presentationskills,communicationwiththemedia,andmanagementofsocialmediaaswellasother‘soft’skills”

Thus, it could be an option for the OSS to offer, besides the regular courses, additional shortercoursesthatfocusontheseskills.Suchcoursescould,forexample,takeplaceshortlybeforeoraftertheregularcourses (maybeonaSaturdaybeforeorafter theregularcourses),andcouldgenerateadditionalincomeasitwouldbepossibletochargeextrafeesforthem.Finally,theywouldmaketheOSSalsomoreattractivebothforinternalaswellasexternalPhDcandidates,andlinkitbettertotheFaculty’sPhDeducation.

OSS’s competitors

The biggest OSS’s competitor by far, as perceived by surveyed participants, seems to be theEuropeanConsortiumforPoliticalResearch(ECPR)summerschool.Itwasmentionedby22oftheca.150 respondents who decided to answer this voluntary open question (“From your perspective,whichsummerschoolsaremostdirectcompetitionfortheOSS?”).Othermentionswhichappearedat least twicewere Essex (6mentions), CEU in Budapest (2mentions), and the London School ofEconomics(2mentions).Thevastmajorityofanswerstothisquestionwereanswerssuchas“Idon’tknow”,“noidea”etc.,whicharenotverydifferentfromnotansweringatall.Inotherwords,onlyarelativelysmallminorityofstudentswasabletonameacompetitor.Whatismoreinteresting,some

2ThiswouldalsobepossiblewithregardtootherFacultiesthatcouldhaveaninterestinbuyingcoursesinthesummer school. Thus, having a stronger link to UV’s or HF’s PhD education through bringing in theirrepresentatives forPhDtraining intheplanningof theOSScouldattractmorePhDstudents frominsideUiObutoutsideofSV.

14

respondents reflected that there is no or limited competition that they know of with focus onthematiccourses.SomeofthemmentionthatECPRisonlycompetitionformethodcourses,butnototherwise.Examplesofsuchanswersarefollowing:

• “Incaseofmethodstraining-ECPRsummerandwinterschools.Whenitcomestotheoretical/empiricalfocus-therearesummerschoolsthatsomeresearchcommunitiesorganize(e.g.ECPRStandingGrouponInterestGroups)buttheOSShasthebroadestoffer.”

• “Aboutmethods,theECPRmethodssummerschoolsarequitecompetitive.”• “OSSoffersthematiccourses(e.g.onDemocracyorInequality)andthismakesitdifferent

fromEssexorLjubljanaschoolswhichfocusexclusivelyonmethods.”• “None-formeitwasallabouttheprofessor.”• “Idon´tknowofanyothersummerschoolthatoffersthesehighleveltheorycourses(for

methods:ECPR,Nuffield,Essex)”• “specificallyonmethods-ECPRsummerschoolonmethods.”• “Uhh...there'sacompetition?Ifso,Everyoneshoulduptheirgameofadvertisingsummer

schools.Ionlyeverfoundinformationbymyownintensesearchesonlineandthroughseverallistservs.Ionlyknowofafewbigones,mainlylocatedintheUK.Perhapscompetitionshouldnotbethefocusbutratherfocusonofferingqualityregardless.”

• “atthetimethereweren'tanyothersofferingsomethingcomparable”• “ECPR.Butnotthatmuchastheyofferfewthematiccourses.”

This feedback suggest that theOSS holds a unique competitive advantage, at least in the eyes ofparticipatingstudents.Apartfromaddingoneortwopopularmethodologicalcourses(orinsteadofit),theOSScouldalsoleveragethisuniqueadvantageanduseitinitsadvertisingtostandoutamongotherinternationalsummerschools.ThiswillbeachallengefortheOSS’smarketingstrategy.Ontheonehand,marketingmethodologicalcourses,whichgenerallytendtohavemoreparticipants,couldattractextrastudents,butalsoeclipsetheuniquenessthattheOSSoffers.Ontheotherhand, it isnot clear if theuniqueofferonmore thematic courses iswhatmany studentswhogo to summerschoolsreallywant.Itisnotpossibletoconclusivelyanswerthisdilemmawithinthissmallstudy.

15

Part 2: Funding

Income

TheOSShasthreemainsourcesofincome(1)agrantfromtheFacultyofSocialSciences,(2)grantsfromdepartmentsthatbuyadditionalcourses,and(3)coursefeespaidbystudents.Thegrantfromthe facultyaswell as the student feeseachmakeupapproximately44%of the income,while thegrants fromdepartmentsmakeup the rest.This year’s summer school saw thehighestnumberofparticipatingstudentssince2005(theearliestdatapointavailableforthisreport).

GiventheaimstatedbytheFacultyofSocialSciences,thattheyarelookingforwaystoreducetheirfinancialinvolvementintheOSSinthelongrunwithoutendangeringtheexistenceoftheinstitution,itisnecessarytoevaluatedifferentwaystoalterthedistributionofincomebetweenthethreemainsources. A first obvious optionwould be an increase in the course fees. At themoment, theOSScharges 3.500NOK (ca. 367 EUR) for one course or 6.000NOK (ca. 630 EUR) for two consecutivecourses(week1+week2).Forthisfeestudentsreceiveadditionalservicesontopofthecoursetheytake,which amount to approximately 2.000NOK (ca. 210 EUR) of the fee (lunches, social events,printingetc.).

This fee is comparatively lowwhen lookingatother social science summer schools inEurope. Thewinter as well as summer methods school organized by the European Consortium for PoliticalResearch (ECPR), for example, charge between 500 EUR and 900 EUR course fees for one weekcoursesorbetween1.000EURand1.800EUR for twoweekcourses. TheEssexSummerSchool inSocialScienceDataAnalysischargesaround1.100GBP(ca.1250EUR)forcoursesinvolving35hoursofteaching.

Bothcomparisonsclearly indicatethatthere isroomto increasetheparticipationfeesfortheOSS.However,threepointsshouldbetakenintoconsiderationwhenconsideringthenewlevelofcoursefees.First,theothersummerschoolswhichhavebeenusedasareferencearefocusingexplicitlyonmethodscourses.ThesetypesofcoursesrunasmallerriskofnotcreatingenoughdemandastheyaregeneralenoughtoberelevantforPhDstudentswithverydifferentprojects.OneofthestrengthsoftheOSSisthatitalsooffersmoretopicalandspecializedcoursesthatarerelevanttoanarrowergroupofPhD studentsworking ina specific subjectarea. Increasing the course fees could causeadropinstudentnumbers,whichcouldbeespeciallyproblematicforthesetypesofcourses,whilee.g.thecourseonCaseStudyMethodsmostprobablywon’texperiencesuchaproblem.Second,costsfor accommodation and food in Oslo are comparatively high, thus creating a certain costdisadvantagefortheOSSasPhDstudentshavetobeabletocatertothesecostsinadditiontothecourse fees. The ECPR, for example, holds its summer and winter schools in Budapest (Ljubljanabeforethat)andBambergrespectively,allthesecitieshavelowercostsofaccommodation,food,andleisure. Finally, shiftingpartsof the income from the facultygrant to course feespaidby studentswhileparallel to that aiming to increase thenumberaswell aspercentageof internalparticipants(i.e.PhDstudentsfromtheFacultyofSocialSciencesortheUniversityofOsloingeneral),willmeanthatmorecostswillbeindirectlycoveredbythedifferentdepartmentsasmostinternalPhDsreceivedepartmental support to cover the course fees. Thus, theremight be a certain trade-off betweenincreasing the number of internal participants and increasing the income through course fees.Another option to shift the distribution of income for theOSSwould be to increase the costs fordepartmentsand institutestobuyadditionalcourses.Atthemoment,eachadditionalcoursecosts50.000NOK.However,giventhatonaveragethereareonlythreeadditionalcoursesthetotalgainofincreasingthesecostsisrathersmallrelativetothecoursefees.

16

Expenses

The expenses of theOSS amounted to approximately 1.5MioNOK per year for the years 2015 –2017.Theycanroughlybedividedintwocategories.Around700.000NOKperyearare“fixedcosts”thatcovertheexpensesfortheSummerSchoolSecretariatincludingitsadministrator,thechairmanetc.Around450.000NOKperyeararespenttofinancethelecturersofthedifferentcourses,while325.000 NOK are budgeted for lunches and social events for participants and lecturers. Finally,printingofcoursematerialsamountsto50.000NOKperyear.

WhenanalyzingtheexpensesbasedontheaimtoreducethefinancialinvolvementoftheFacultyinfinancingtheOSS,afirstoptiontoreducecostswouldbetonotprovidefreelunchesanymore.Thiswould reduce expenses by approximately 100.000 NOK per year. However, from a practicalperspectivemakingitanindividualresponsibilitytogetlunchcouldcreatelogisticproblems(e.g.longlinesinthecafeteriathatmightdelaycoursesfromstartingagainafterlunch).Analternativeoptioncouldbetochargelunchasanextraexpensetothecoursefeestogenerateadditionalincomeandthen use a similar system as was used e.g. for the ECPR conference this year to incentivizeparticipantstopre-orderandpre-paylunchestoreducewaitingtimesetc.Expensesforothersocialevents (reception etc.) amount to 225.000 NOK yearly and could also be an area where there ispotentialforcostcutting.Finally,thelecturersattheOSSarepaidbasedontwofactors,a)ageneralfeeforteachingthecourseandb)additionalfeesforgradingthepapersthatdependonthenumberofpapers thathavebeen submitted. Tomake the costsmorepredictable it couldbeanoption toslightlyincreasethegeneralfeeforthelecturersandabolishtheadditionalpaymentforthegradingofpapers.

ProfitsandLosses

Basedonthedatathathasbeenprovidedfortheevaluation,theOSS,withtheexceptionoftheyear2013, regularly makes a loss. It should be mentioned though that from 2011 until 2016 the losssteadilytendedtodecrease.In2016thelossstillamountedtoapproximately46.136NOK.For2017thenumbersarestillpreliminaryasthebookshavenotbeenclosedyetandthereisnofinalaccountof this year’s activities. Another reason to handle some of the calculations carefully is that thereseemstobesomediscrepancybetweentheaccountsthatthefinancialofficersoftheFacultyprovideandthecalculationsoftheSummerSchoolOfficeanditisnotalwaysclearwhattheoverallincomeorexpensesareinspecificcategories.

ThetablebelowgivesabriefoverviewofthedevelopmentoftheOSS’sfinancesaswellasstudentnumbers and acceptance rates3 over the last eight years. Thenumbers in the table aswell as theoverallprofits/losseshighlightthatitisnecessarytosolidifytheincomebaseoftheSummerSchooltoensureitslong-termstability.

As the table shows both the number of applicants as well as the number of accepted studentsfluctuatesovertheyearswith2017beinganespeciallysuccessfulyearwithregardtotheadmittedstudents. The income from course fees fluctuates similarly to the student numbers, the slightdifferences can most probably be linked to the number of students taking two courses in a rowinsteadofjustone.Theaveragenetincomeperstudentiscomparativelylowalsobecausethedirectcourse costs increase with the number of students due to expenses for lunch, printing etc. As

3ItshouldbekeptinmindthatthosewhoapplybutdonotacceptaplaceatthesummerschoolincludebothpeoplebeingsortedoutbytheSummerSchoolAdministrationaswellasapplicantswhorefusetoacceptaplaceinacoursethathasbeenofferedtothem.

17

indicatedinthefootnotetothetable,thefiscalnumbersfor2017arestillsomewhatpreliminary,butiftherearenomajordiscrepanciesintheprovidedstatisticsandtheyearwiththehighestnumberofparticipatingstudentssince2010willalsobeoneoftheyearswiththehighestfiscal loss,thenthisindicatesthatthereisaneedforfiscalreadjustment.

2010 2011 2012 2013*** 2014 2015 2016 2017**Applicants 320 296 227 290 285 260 307 303Studentnumber 221 181 187 202 216 188 166 228Acceptancerate 69,1% 61,1% 82,4% 69,7% 75,8% 72,3% 54,1% 75,2%Incomecoursefees****

821.265NOK

744.918NOK

734.992NOK

859.629NOK

817.800NOK

792.000NOK

728.500NOK

825.994NOK

Directcoursecosts***** --

606.690NOK

562.179NOK

603.347NOK

529.019NOK

682.504NOK

490.838NOK

779.400NOK

Averageincome/student

3.716NOK

4.116NOK

3.930NOK

4.256NOK

3.786NOK

4.213NOK

4.389NOK

3.623NOK

Netincome/student -- 764NOK 924NOK

1.269NOK

1.337NOK 582NOK

1.432NOK 204NOK

Win/Losssummerschool****

-1.697NOK

-264.473NOK

-154.694NOK

105.488NOK

-68.189NOK

-79.090NOK

-46.136NOK

-240.857NOK

*teachingstaff,travel,printing&lunch**Asthebooksarenotyetclosed,thenumbersfor2017arenotfinalandcanstillfluctuate***In2013thecostsforotherpersonnelhavebeenaccountedasanincome,forthesakeofthiscalculationtheyhavehoweverbeenregardedasanexpense****DatabasedonthecalculationprovidedbytheFinanceDepartmentoftheFaculty

TakingintoconsiderationtheaimoftheFacultytoreducetheirfinancialinvolvementintheOSSandtheavailablealternativesourcesof income,whichde factomeanscourse feespaidbystudents, inrelationtothestructureofexpensessomecaveatsneedtobehighlighted.First,dependingtoomuchon student course fees as a source of income for the OSS might be problematic due to thesubstantive amount of “fixed” costs that occur independently of the number of studentsparticipating in theSummerSchool.While thenumberof students thatparticipate in theSummerSchool has increased over time, there is a still significant fluctuation visible, for example, in thereductionofstudentsfrom2014to2016.Atthesametime,the“netperstudentincome”4hasneverbeenhigherthan1.450NOK,meaningtocover“fixedcosts”withoutfacultyordepartmentsupporteven with the highest “net per student income” would demand the participation of around 500students.Thisshowsthatthere isaneedtofindawaytosupportat least largepartsofthe“fixedcosts” fromother sources to createa situation inwhich theOSS couldbe sustainablewithout theFaculty’ssupport.IftheFacultycanfindanarrangementwiththeDepartmentstocoveralargepartof the “fixed costs” -whichwouldalsoallow theFaculty todecreasepartsof its 44%provisionofincomeforthesummerschool - thefinancialsecurityof thesummerschoolcouldbeensuredalsowithfluctuatingstudentnumbersandonlymodest increases incoursefees.Thiscouldbepartofalargerarrangement thatwouldalsogive theOSSabigger role in thePhDeducationof theFacultyandincreasethenumberofinternalPhDcandidatesthattakepartintheSummerSchool.

4Forthisreport„Netperstudentincome“isdefinedasthedifferencebetweentheincomefromcoursefeesandthedirectcostsassociatedwithstudentsparticipatingincourses(e.g.feesforlecturers,printingcosts,lunch).

18

Part 3: Academic standing of the summer school Inthissection,weassesstheacademicqualityoftheteaching,andtheeffectofthesummerschoolonthefaculty’sreputation.

Summer schools have a long and venerable tradition in academic life. In addition to serving as anopportunity for particularlymotivated students tomeet and engagewith leading academics, theyafford thesameacademicsopportunities towork ina focusedyet relaxedenvironment topresentand discuss topical issues and methods. Summer schools in themselves also function as informalnetworksbetweenhostsandvisitors,andalsoamongthevisitorsthemselvesformingandsustaining“invisiblecolleges”andincreatingintellectualnoveltyandvariation.Indoingso,theyalsocontributetothelong-termviabilityofthehostenvironments,enrichingalsosettingslocatedsomewhatoutsidethescientificcentreofgravity.

Summerschools thushaveapotential thatgoes farbeyondtheconfinesofa fewweeksofcoursegiving. But if a summer school is to achieve these laudable goals, it needs a clear identity, and astrongnetwork,butalsoleadershipwiththecapacityandwillingnesstorejuvenateandperhapseventransformitsoffering,tokeepitfromstagnating.

HowwellhastheOSSdoneintheserespects?Eversinceitwasinceptedin1993,theOSShasofferedcourses that reflect several of the most topical issues of the social sciences, and cutting edgemethodologicalapproaches.Manyof the invited lecturersbelongto the leaders in their respectivefields, and come from renowned institutions. It is also clear that the school has provided leadingscholars the opportunity to present on-going research and publications that have been recentlypublished orwhich are in the process of finalization. This has included, over the years, issues likestudies of case study methods, comparative methodologies, political protest, cultural sociology,technological change, scientific risks, life course analysis, just to mention a few. The quality andtopicalityoftheissueschosenareundeniable.

OSS’s activities are firmly rooted in the competitive advantage of the Nordic countries in socialscience, with a focus on issues like labour markets, welfare systems, and higher educationorganization.While this,wisely, reflects thehistorical strengthsanddirectionsofNorwegian socialscience, the schoolhas graduallywidened thenet and incorporate issues like globalization, justiceandcitizenshipwhicharemoregenericinnatureandwhicharenotspecificallytiedtotheprofileofNorwegiansocialscienceortotheNorwegiansociety.Thecorecontentremains,however,locatedinmacrosociology, comparative politics, and political economy, with adjacent methodologicalapproaches (quantitative, historical, mixed methods). In this sense, the OSS has contributed toreinforcing the strengths of Norwegian social science and aligning it with leading internationalscholarship.Itisanotherissueifthefacultyhasbeenabletofullycapitalizeonthefirst-ratecapacitythat the school has afforded, but it seems abundantly clear that the school’s offering is excellent.Simplyput,theOSSanditsactivitiesreflectscholarshipattheaveryhighinternational levelwithinthebroadframeworkofcomparativesocialsciences.

Whenitcomestoleadershipandthecapacityforrejuvenationovertime,theOSShasalwaysbeenlabelledaschoolofcomparativesocialsciences,whichreasonablyreflectstheoveralldirectionoftheschool.Thefocusis,asmentionedabove,onaspecificformofcomparativesocialsciencestudies,inparticularthosethatarticulatewiththelegacyofcomparativestudiesattheintersectionofpolitics,society andmarkets. The OSS has extended its reach. This we also take as an indication that theschoolaims toconnectwith thesocial science facultyasawhole,alsowhen thismeansstretchingthenotionof“comparativesocialscience”.

19

Leadershiphasbeenexercised in thesensethat theschool’sarticulationwiththe facultyhasbeenretainedovertimeasthefaculty’sprofilehaswidened.Leadership,itmustbeadded,hasalsobeenproven in the consistently high level of the school’s course offering, at a level which would beenviable to any social science faculty in the world. A summer school is highly dependent onintellectual and organizational leadership that monitor and reflect themoving frontlines of socialscience research, and Oslo has certainly benefitted from the depth and breadth of LarsMjøset’snetworkandownengagements.Itisvitalfortheschoolthatleadershipatthislevelismaintained.

Inaddition,theoutstandingworkprovidedbytheOSS’sadministration,especiallythelongstandingadministratorTronTorneby,shouldbehighlightedasanadditionalfactorthatensuredthesuccessoftheOSSinthelastyears.Throughhisinitiatives,suchascreatingandadministeringaFacebookgroupthatbringstogetheralumnioftheOSS,heprovidesalinktotheOSS’salumniandactsasthemaincontactpointforpotentialaswellasformerparticipantsofthesummerschool.HisworkthroughouttheyearandduringtheOSSwithstudentsaswellasincominglecturerslaysthefoundationforthesmoothoperationofthetwoweeksofcourses.

Does the school add to the value of the Faculty of Social Sciences at UiO? If judging from thecompositionof the courses, both contentwise and regarding recruited lecturers, it iswithout anydoubtthattheschoolshowcasestheOslofacultytotheworld,butalsoconcomitantlythatitexposesOslo to theworld. As has beenmentioned elsewhere in this report, and also pointed out by theevaluationofthePhDprogrammein2016,thereseemstoberoomforfurtherimprovementsinthePhDprogramme,improvementsinwhichtheschool’sfocusedandbroadprofilecouldcontribute.

Thecompositionisclearlystructuredaccordingtothefoundationalcompositionanddirectionoftheschool,a trajectorythathasbeenreproducedovertime inahighlyconsistentmanner.Thismeansthat students and lecturers who seek to elucidate the specific political, social and economicconditions that underpin welfare states and their labour markets, and who are engaged inmethodologicaldevelopments tounderstand theseprocesses,will findOsloacongenialhome.Wesee,however,fewerindicationsofthiscoreofthesummerschoolbeingalignedmoresystematicallywith areas that were sparsely represented in the early years of the schools, like psychology,anthropology or economics, or finding manners in which these can be incorporated under theumbrella of “comparative social sciences”. The growing engagement with science and technologystudieshasalso largelybeenconductedwithoutanyspecificarticulationwiththesummerschool’score.

ThispointstotheriskofacertainmarginalizationoftheOSSinrelationtotheexpandingforeofthefaculty.This isnotnecessarilyaprobleminitselfastheschool isanappendixofthefaculty,notitscentre, butwebelieve that the faculty shouldmakemoreuseof the school’s capacity as a globalintellectualmeeting space, and that it should engage the facultymore in and around the school’sactivities, for instance by engaging lecturers in departmental activities, but also by creatingintegrative settings where the different lecturers engage in interdisciplinary talks and meetings,perhaps in the form of aworkshop or symposium at the end of the summer school’s period.Werealize that the timingof schoolactivities (summer term) isnot ideal for suchalignments,but inafaculty thataimstobea leadingNordicsocialsciencefacultywithastrong internationalpresence,theschoolisdefinitelyaresource.

Conclusionsandrecommendations:

• TheOSSoperatesandhasconsistentlyoperatedonaveryhighinternationallevel.• Thethemeschosenaretopicalandcuttingedge.• Lecturers recruited are among the leaders in their respective fields and have taken the

opportunitytoshareanddiscusstheiron-goingresearch.• The alignment between core and periphery of the OSS offering could be addressed, for

instance,bybetterintegratingeconomicsandpsychology.

20

• Leadership has been outstanding – the OSS’s offerings and network could not have beenachievedwithoutthis.

• The faculty should takeadvantageof thenetworks that the lecturersbringwith themandworktoimprovethealignmentbetweentheOSSandtheFacultyofSocialSciences,toavoidcompartmentalizationandtosecureitsambitiontobeaninternationallyleadingfaculty.

• Such an alignment could be secured through faculty workshops or symposia, which couldaddress issues that go beyond and over the demarcations of each course. This could alsostrengthentheschool’sandfaculty’sambitiontoworkinterdisciplinary.

21

Appendices

Appendix 1 Number of participants over years grouped by course theme

(Note:Onlydatafortheyears2010,2012,2014and2017wereprovidedforthisanalysis)

Themesofthecoursesofferedin2010 No.ofcourses No.ofParticipants No.ofPhDstudents

Economics 1 41 39

Psychology 1 34 27

PoliticalScience/Sociology 4 55 40

EnvironmentalPolicy 1 25 16

Anthropology/Sociology 2 27 21

ResearchMethodology 1 39 37

TOTAL 10 221 180


Economics 0 0 0

Psychology 0 0 0





TOTAL 11 187 142

22


Economics 1 22 19

Psychology 1 29 18





TOTAL 9 216 188


Economics 0 0 0

Psychology 1 21 14





Innovationstudies/Sociology 2 52 32

Education/Sociology 1 11 6

TOTAL 10 228 182

23

Appendix 2 Follow-up survey sample composition

Respondentsofthenewquestionnairebyyear

Note:Majorityofrespondents(279,i.e.91%)onlyvisitedthesummerschoolinoneyear,only22respondents(7%)visitedittwice,and4(1%)threetimes.Threerespondentssaiditwasnoneoftheyearsoffered.

26

Appendix 3 Additional charts – fol low-up survey answers grouped by where participants are from

27

Appendix 4 Additional charts – fol low-up survey answers grouped by year of participation

Whatareparticipantsexpectingfromagoodsummerschool-groupedtheyearofparticipation

28

Note:thedatacomesfromthe2017surveycoveringparticipantsfromdifferentyears.

Whichmethodswouldparticipantsconsider-groupedtheyearofparticipation


Whichwaysofadvertisingdoparticipantsconsidereffective-groupedtheyearofparticipation


oslo summer school: evaluation - universitetet i oslo · the oss has three sources of income: (a) a...

Documents