to impact from ssh research - fteval.at · 23345678coccnte c5prf 3 content 5 preface heinz...

PROCEEDINGS OF THE CONFERENCE “Impact of Social Sciences

and Humanities for a European Research Agenda

Valuation of SSH in mission-oriented research”VIENNA 2018

to impact from SSH research

c/o ZSI – Centre for Social Innovation GmbHLinke Wienzeile 246, A-1150 Vienna

T +43 1 495 04 42 - 79F +43 1 495 04 42 - 40E [email protected] www.fteval.at

ZVR-Zahl: 937261837ISSN-Nr. 1726-6629© Vienna 2019

The fteval Journal for Research and Technology Policy Evaluation is a forum to discuss current evaluation practices in the field of research, technology and innovation policy.

2

EDITORIAL BOARDRebecca Allinson, Technopolis UK; Balázs Borsi, Eszterházy Károly College; Elke Dall, Centre for Social Innovation; Michael Dinges, Austrian Institute of Technology; Leonid Gokhberg, National Research University Higher School of Economics; Wolgang Polt, Joanneum Research; Andreas Reinstaller, WIFO; Klaus Schuch, fteval (chief editor); Michael Stampfer, WWTF; Lena Tsipouri, University of Athens

AUSTRIAN PLATFORM FOR RESEARCH AND TECHNOLOGY POLICY EVALUATION (fteval)c/o ZSI – Centre for Social Innovation GmbHLinke Wienzeile 246, A-1150 ViennaT +43 1 495 04 42 - 79F +43 1 495 04 42 - 40E [email protected] www.fteval.at

DESIGNW carotte.atE [email protected]

PRINTagensketterl Druckerei, Bad Vöslau

Gedruckt auf PEFCTM-zertifiziertem Papier.

AUSTRIAN PLATFORM FOR RESEARCH AND TECHNOLOGY POLICY EVALUATIONPlatform fteval members:

Austrian Federal Ministry of Education, Science and Research (bmbwf), Austrian Federal Ministry for Digital and Economic Affairs (bmdw), Austrian Federal Ministry for Transport, Innovation and Technology (bmvit), Austrian Cooperative Research (ACR), Austrian Council for Research and Technology Development, Austrian Institute of Technology (AIT), AQ Austria – Agency for Quality Assurance and Accreditation Austria, Austria Wirtschaftsservice (AWS), Christian Doppler Research Association (CDG), convelop Cooperative Knowledge Design GmbH, Austrian Research Promotion Agency (FFG), Austrian Science Fund (FWF), Institute for Advanced Studies (IHS), Industriewissenschaftliches Institut (IWI), Joanneum Research Forschungsgesellschaft mbH, Austrian Institute for SME Research (KMFA), Ludwig Boltzmann Society (LBG), Austrian Academy of Science (OEAW), Technopolis Group Austria, Vienna Business Agency – A service offered by the City of Vienna, Austrian Institute of Economic Research (WIFO), WPZ Research GmbH, Vienna Science and Technology Fund (WWTF), Centre for Social Innovation (ZSI)

WIFO – AUSTRIAN INSTITUTE OF ECONOMIC RESEARCHArsenal, Objekt 20Postfach 91, 1103 ViennaDr. Jürgen JangerE: [email protected]. Andreas ReinstallerE: [email protected]

OEAW – AUSTRIAN ACADEMY OF SCIENCEDr. Ignaz Seipel-Platz 2, 1010 ViennaNikolaus Göth, MScE: [email protected]

AUSTRIAN COUNCIL FOR RESEARCH AND TECHNOLOGY DEVELOPMENTPestalozzigasse 4/DG 1, 1010 Vienna Dr. Johannes GadnerE: [email protected]

AQ AUSTRIA – AGENCY FOR QUALITY ASSURANCE AND ACCREDITATION AUSTRIAFranz-Klein-Gasse 5, 1190 ViennaDr.in Elisabeth Froschauer-Neuhauser E: [email protected] Eva Maria FreibergerE: [email protected]

AIT – AUSTRIAN INSTITUTE OF TECHNOLOGYGiefinggasse 4, 1210 Vienna Mag. Michael DingesE: [email protected] Barbara Heller-Schuh, MASE: [email protected]

AWS – AUSTRIA WIRTSCHAFTSSERVICE GMBHWalcherstraße 11A, 1020 Vienna Mag.a Marlis BaurechtE: [email protected] Mag. Norbert KnollE: [email protected]

CDG – CHRISTIAN DOPPLER RESEARCH ASSOCIATIONBoltzmanngasse 20, 1090 ViennaDIin Maga. Brigitte MüllerE: [email protected]

WWTF – VIENNA SCIENCE AND TECHNOLOGY FUNDSchlickgasse 3/12, 1090 ViennaDr. Michael StampferE: [email protected]. Michael StrassnigE: [email protected]

VIENNA BUSINESS AGENCY. A SERVICE OFFERED BY THE CITY OF VIENNA.Mariahilfer Straße 20, 1070 ViennaRobert Mayer-UnterholzerE: [email protected]

ZSI – CENTRE FOR SOCIAL INNOVATIONLinke Wienzeile 246, 1150 Vienna Dr. Klaus SchuchE: [email protected]

LUDWIG BOLTZMANN GESELLSCHAFTNußdorfer Straße 64, 1090 ViennaMag. Patrick LehnerE: [email protected]

TECHNOLPOLIS GROUP AUSTRIARudolfsplatz 12/11, 1010 ViennaMag.a Katharina WartaE: [email protected]

WPZ RESEARCH GMBHMariahilfer Straße 115/16, 1060 ViennaDr.in Brigitte EckerE: [email protected]

JOANNEUM RESEARCH FORSCHUNGSGESELLSCHAFT MBHHaus der ForschungSensengasse 1, 1090 ViennaMag. Wolfgang PoltE: [email protected]. Jürgen StreicherE: [email protected]

KMU FORSCHUNG AUSTRIA –AUSTRIAN INSTITUTE FOR SME RESEARCH Gusshausstraße 8, 1040 Vienna Mag.a Iris FischlE: [email protected]. Peter KaufmannE: [email protected]

FEDERAL MINISTRY OF EDUCATION, SCIENCE AND RESEARCHMinoritenplatz 5, 1014 Vienna Mag.a Irene DanlerE: [email protected] Simone MesnerE: [email protected]

FEDERAL MINISTRY FOR DIGITAL AND ECONOMIC AFFAIRSStubenring 1, 1014 Vienna Mag.a Sabine Pohoryles-DrexelE: [email protected]

FEDERAL MINISTRY FOR TRANSPORT, INNOVATION AND TECHNOLOGYRadetzkystraße 2, 1030 Vienna Dr. Rupert PichlerE: [email protected]. Mario SteyerE: [email protected]

FFG – AUSTRIAN RESEARCH PROMOTION AGENCYHaus der Forschung,Sensengasse 1, 1090 ViennaDIin Dr.in Sabine MayerE: [email protected]. Leonhard JörgE: [email protected]

FWF – AUSTRIAN SCIENCE FUNDHaus der ForschungSensengasse 1, 1090 ViennaDr. Falk Reckling E: [email protected] Dr. Thomas VölkerE: [email protected]

IHS – INSTITUTE FOR ADVANCED STUDIESJosefstädter Straße 39, 1080 Vienna Dr.in Angela WroblewskiE: [email protected]. Richard Sellner E: [email protected]

CONVELOP – COOPERATIVE KNOWLEDGE DESIGN GMBHBürgergasse 8-10/I, 8010 Graz DIin Drin. Karin GrasenickE: [email protected]

Erdbergstraße 82/4, 1030 WienMag. Thomas JudE: [email protected]

ACR – AUSTRIAN COOPERATIVE RESEARCHSensengasse 1, 1010 Vienna Dr.in Sonja Sheikh E: [email protected]

INDUSTRIEWISSENSCHAFTLICHES INSTITUT – IWIMittersteig 10, 1050 WienFH-Hon.Prof. Dr. Dr. Herwig W. SchneiderE: [email protected]. Philipp BrunnerE: [email protected]

ISSUE 48 | JULY 2019 3

CONTENT 5 PREFACE

HEINZFASSMANN,AUSTRIANFEDERALMINSTEROFEDUCATION,SCIENCEANDRESEARCH

6 FOREWORD MATTHIASREITER-PÁZMÁNDY

8 IMPACTRE-LOADED THOMASKÖNIG,HELGANOWOTNYANDKLAUSSCHUCH

10 SSH-IMPACTPATHWAYSANDSSH-INTEGRATIONINEU RESEARCHFRAMEWORKPROGRAMMES.

THOMASKÖNIG

26 SOCIALSCIENCESANDHUMANITIESRESEARCHMATTERS. GUIDELINESONHOWTOSUCCESSFULLYDESIGN,AND IMPLEMENT,MISSION-ORIENTEDRESEARCHPROGRAMMES

THOMASKÖNIG

29 THEPUBLICVALUEOFTHESOCIALSCIENCES JOHND.BREWER

33 BRINGINGCONCEPTSTOGETHER:INTERDISCIPLINARITY, TRANSDISCIPLINARITY,ANDSSHINTEGRATION

JOËLGRAF

37 NEWPATHWAYSTOSOCIALCHANGE– CREATINGIMPACTTHROUGHSOCIALINNOVATIONRESEARCH

JÜRGENHOWALDT

49 THEIMPORTANCEOFSSHRESEARCHINHORIZONEUROPE JOHNSTEPHENBELL

53 INNOVATIONFORTHEREFLECTIVESOCIETY RICCARDOPOZZO

56 STRIVINGFORREFLEXIVESCIENCE ROGERSTRAND

ISSUE 48 | JULY 20194

62 SSHA-DRIVENKNOWLEDGETRANSFERWITHINTHETHIRD MISSIONOFUNIVERSITIES

GEORGRUSSEGGER

66 PANELCOMPOSITIONASPATHWAYTOIMPACT: DOWENEEDSTAKEHOLDEREXPERTISETOSELECTRELEVANT

MISSION-ORIENTEDPROJECTS? PETERVANDENBESSELAARANDULFSANDSTRÖM

72 SENIORACADEMICSASKEYNEGOTIATORSINTHEIMPLEMEN TATIONOFIMPACTPOLICIESINTHESOCIALSCIENCESAND HUMANITIES

MARCVANHOLSBEECK,THEODOSIADEMETRIOU,AGNEGIRKONTAITE,ANDREJAISTENIC STARCIC,VILLEKEISKI,EMANUELKULCZYCKI,ELENAPAPANASTASIOU,JANNEPÖLÖNEN, HULDAPROPPEANDMAJAVEHOVEC

80 INCLUSIVEFUNDINGFORENHANCEDIMPACTOFSOCIAL SCIENCESANDHUMANITIES

MALINLINDBERG,SVETLANAGROSS,MILDARÖNN,LISSANORDIN,JANSANDRED,LARS WÄRNGÅRDANDCATHARINANORBERG

86 THEOULUWAYOFSTRENGTHENINGSOCIALIMPACTOFSSH SCIENCES:FROMLINEARMODELSOFIMPACTTOPRODUCTIVE

INTERACTIONSANDBEYOND JUHATUUNAINEN,RAULISVENTO,PENTTIHADDINGTON,KIRSIOJUTKANGASANDSIRPA AALTO

91 QRIH:TOWARDSAFITTINGSYSTEMFORHUMANITIES RESEARCHEVALUATION

ADPRINS,JACKSPAAPEN,THEDVANLEEUWENANDNELLEKEVANDENBROEK-HONINGH

99 ETAPROGRAMME ARTS,SOCIALSCIENCESANDHUMANITIESASANINTEGRAL

PARTOFTHEINNOVATIONECOSYSTEMOFTHE21STCENTURY MARCELKRAUS


105 THE“6IRESEARCHMODEL”:EVOLUTIONOFANINNOVATIVE INSTITUTIONALSTIPOLICYFRAMEWORKATTHEUNIVERSITY OFDEUSTO

ANTONIACARO-GONZALEZ

114 EVIDENCE-BASEDPRACTICEANDPOLICESFORIMPACTON MENTALHEALTHOFCHILDRENANDADOLESCENTS RAPHAELAE.KAISLERANDJEANL.PAUL

120 IMPACTOFSOCIALSCIENCESANDHUMANITIESFORA EUROPEANRESEARCHAGENDA–VALUATIONOFSSHIN

MISSION-ORIENTEDRESEARCH BETTINAUHRIG

125 THECHANGINGPATTERNOFSOCIALSCIENCESAND HUMANITIESINTHEEUFRAMEWORKPROGRAMMES RETHINKINGSOCIETALIMPACT–COLLABORATIONWITH

STAKEHOLDERS ÜLLEMUST

129 SPECIFICSSHRESEARCHANDGENERALRESEARCHPOLICY INSTRUMENTS.THECASEOFTHENATIONALRESEARCH PROGRAMMESAND“NEEDDRIVENSSHRESEARCH”IN LITHUANIA

RŪTAPETRAUSKAITĖ

133 GHENTUNIVERSITY’SINTERDISCIPLINARYSSH-CONSORTIA– ASTRATEGYTOENHANCETHESOCIETALIMPACTOF RESEARCH

NOËLKLIMA,STEFANMEYSMAN,JULIECARLIER,ALEXISDEWAELEANDESTHERDESMET EVALUATIONOFSOCIETALIMPACTINNORWEGIANSSH

139 EVALUATIONSOFSOCIETALIMPACTINNORWEGIAN SSHEVALUATIONS

JONHOLMANDANETTEASKEDAL


143 STAKEHOLDERS’ROLETOPRODUCEIMPACTFROMSOCIAL SCIENCERESEARCH:WHATLESSONSFOREVALUATION?

EMANUELAREALE,SERENAFABRIZIOANDLUCIOMORETTINI

152 ASSESSINGTHEIMPACTOFSSH–RRIAPPROACHONICT RESEARCH&INNOVATION:THEHUBITPROJECT

TALSOFFER,RUTHZUZOVSKY,OLENANEDOZHOGINAANDEMANUELEBARDONE

159 RETHINKINGRESEARCHIMPACTASSESSMENT: AMULTIDIMENSIONALAPPROACH

SERGIOMANRIQUE,MARTANATALIAWRÓBLEWSKAANDBRADLEYGOOD

176 EVALUATIVEINQUIRY:ENGAGINGRESEARCHEVALUATION ANALYTICALLYANDSTRATEGICALLY

SARAHDERIJCKE,TJITSKEHOLTROP,WOLFGANGKALTENBRUNNER,JOCHEMZUIJDERWIJK, ANNEBEAULIEU,THOMASFRANSSEN,THEDVANLEEUWEN,PHILIPPEMONGEON,CLIFFORD TATUM,GOVERTVALKENBURGANDPAULWOUTERS

183 VALUATIONOFSSHRESEARCHFORATRANSFORMATIVE EUROPEANRESEARCHAGENDA–AFEWCLOSINGWORDS

KLAUSSCHUCH

188 SCIENTIFICCOMMITTEEOFTHECONFERENCE

189 POSTERS


HEINZFASSMANN,AUSTRIANFEDERALMINSTEROFEDUCATION,SCIENCEANDRESEARCH

PREFACE

LADIES AND GENTLEMEN,

Theconferenceproceedingsyouareabouttoreadaretheresultoftheconference“ImpactofSocialSciencesandHumanitiesforaEuropeanResearchAgenda”duringtheAustrianPresidency

oftheEuropeanCouncilinthesecondhalfof2018.TheconferencewaspartoftheAustrianpresidency’sactivitiesinthefieldofResearch&In-novationpolicyandservedasaforumforpolicymakersandresearcherstodiscussmattersofsocialsciencesandhumanities(SSH)researchinthe context of the evolving European research framework programmeHorizonEurope.

Thepapersintheseproceedingsreflecttherichandbroaddebateattheconference.Theygiveamostvaluableoverviewaboutissuesoftheintegrationof social sciencesandhumanities in researchprogrammesandabouttheimpactofSSH.

AsMinister responsible for scienceand researchpolicy letmeun-derlinethenecessityofthesocialsciencesandhumanitiesfromapolicyperspective.SSH-researchisofhighimportanceforEuropeandofhighimportanceforEuropeanresearchframeworkprogrammes.SSH-discip-linesproducefundamentalknowledgeaboutusasindividualsandasasociety. The social sciences and humanities show us where we comefromandhelpusbetterunderstandwherewearegoing.Expertise inthefieldsofhistory,society, law,languages,artsandculture,tonameonlyafew,provideanimportantcontributiontoeconomicandsocietalprogressinEurope.Thechallengesofourtime–climatechange,susta-inability, violent conflicts, an ageing society, artificial intelligence anditsconsequencesforsociety–cannotbesolvedbycontributionsfromthenaturalsciencesandengineeringalone.WeneedcontributionsfromSSHtobetterunderstandhumanbehaviorandtofindsolutionsfornewtechnologies,socialinnovationsorpoliticaldecisionmaking.

Socialsciencesandhumanitiesarescientificfieldsthathaveastrongtraitofself-reflexivity.Theyhaveintensivedebatesabouttheirtheoriesandmethods.Theircriticalperspectiveisconstantlychallengingestablis-hedpatternsofthinkingandbehaving.Thisperspectiveshouldbeusedfor research and innovation in general in a productive way. SSH alsohaveaspecificrole,whenitcomestothequestionofimpact.Theylookatthemselvesandtheirownimpact.Buttheyarealsodeeplyinvolvedindevelopingtheconceptof impact ingeneral.Thesedisciplineshavemuchcontributedtothedebateofre-definingimpactthathasdevelopedfromlookingmerelyatresearchimpactmeasuredbyh-factors,citations,andtheamountofpublicationstowardslookingmorebroadlyatsocietalimpact.

Another importantfeatureofthesocialsciencesandhumanities istheirroleincontributingtotheimplementationofscientificresultsintoreality.Onepathwaytobringscientificresults intoreality is,bycontri-butingtosectoralpolicies.Researchandinnovationplayanevermore

important role in sectoral policies. R&I policy should reflect this morestrongly.Thechallengesofourtimeneedmoderngovernance, that is,morecooperationbetweenallpolicyareas.SSH-researchplaysan im-portant role in facilitating this exchange between research policy andsectoralpolicies.

InHorizonEuropesocialsciencesandhumanitieswillplayastrongrole.Allthreepoliticalinstitutionsinvolvedinitsnegotiation,theEuro-peanCommission,theEuropeanParliamentandtheEuropeanMemberStatesagreethatSSHwillbekeytoturnHorizonEuropeintoasuccess.

SocialsciencesandhumanitieswillplayanimportantroleinthefirstpillarofHorizonEurope,intheEuropeanResearchCouncil.InthatpartoftheprogrammeSSHhasalwaysbeenastrongcomponent.SSHwillalsoplayaveryimportantroleinpillartwo,GlobalChallengesandIndustrialCompetitiveness.Thechallengesofourtimecannotbesolvedbycontri-butionsfromthenaturalsciencesandengineeringalone.Ineachclusterofthesecondpillar,SSH-integrationwillbringmuchneededexpertisetoadvisorygroups,callsandsubsequentresearchprojects.Cluster2willhaveaparticularfocusonresearchquestionsinthefieldsofdemocracy,culturalheritageandcreativityaswellassocietaltransformation.AllofthemarehighlyrelevantforEuropeanditsfutureandallofthesefieldswillgenerateSSH-intensiveresearch.Lastbutcertainlynot least,SSHwillalsobeessentialforthenewandpromisinginstrumentofMissionsinHorizonEurope.

TheViennaconference focusedonSSH-integrationandthe impactofSSH-researchinHorizonEuropeandbeyond.Theresearchframeworkprogramme,nowHorizon2020,verysoonHorizonEurope, isuniqueintheworld.JustalikeSSH-Integrationasabroad,cross-cuttingissueanda genuine strategy in such a large programme is a unique feature. ItreallyshowstheimportanceSSHhasinEuropeansocietyandEuropeanscience.

Iwishyoumuchinspiration,butmaybeevenmoreimportantstimu-lusforactionwhenreadingtheconferenceproceedings.

HeinzFassmannAustrianFederalMinsterofEducation,ScienceandResearch


1 In the current result of negotiations as a Proposal for a Regulation of the European Parliament and of the Council establishing Horizon Europe - theFrameworkProgrammeforResearchandInnovation,layingdownitsrulesforparticipationanddissemination,22.03.2019,2018/0224(COD),https://www.consilium.europa.eu/media/38902/st07942-en19.pdf

2 InthecurrentresultofnegotiationsasaProposalforaDecisionoftheCouncilonestablishingthespecificprogrammeimplementingHorizonEurope-theFrameworkProgrammeforResearchandInnovation-PartialGeneralApproach,15.04.2019,2018/0225(COD),https://data.consilium.europa.eu/doc/docu-ment/ST-8550-2019-INIT/en/pdf

LADIES AND GENTLEMEN, DEAR COLLEAGUES, DEAR FRIENDS,

Thisisthemomenttolookbackonaverysuccessfulconferenceontheimpactofsocialsciencesandhumanitiesandtheirinteg-rationinresearchframeworkprogrammes.

Theconferencehad30sessions,130speakers,20papersandonthetwodays340people(55%women)fromover30countriesattendedtheevent.Theconferencedidnotonlyinducereallifedebates,butalsolivelyinteractionsonline.DuringtheconferencethededicatedTwitterhashtag#SSHimpactwasahugetrendgeneratingmorethan1.900tweets.Theconference video and the livestreamed sessions were accessed morethan200timeseach.Thephotosandvideosabouttheconferenceweredownloadedseveralhundredtimes.

For me the conference was a very impressing experience. I like tothinkbacktotheveryintriguingstartwithpointedstatementsbyKlausSchuchandUlrikeFelt.InhisopeningwordsAustrianFederalMinisterforEducation,ScienceandResearchHeinzFassmannputgreatempha-sisontheimportanceofSSHforresearchingeneralandtheEuropeanResearchAreainparticular.DeputyDirectorGeneralWolfgangBurtscherlinedouthowtheEuropeanCommissionattachesgreat importancetoSSHfortheirtransformationalpowerandthepowertohelpsolvingthesocialchallengesofourtime.ThekeynotespeechbyJohnBrewerputforththenecessitythatthesocialscienceandhumanitiesengageintheimpact discussion because they are already very good at impact. TheveryspiritedafternoonkeynotefromInekeSluiterdiscussedtheroleofthe humanities in innovation, the humanities’ impact in teaching andshe provided the audience with insights about the roots of particularinnovationsinancientGreece.Anafternoonfullofhands-oninteractionwith policy officers from a number of Directorate Generals of the Eu-ropeanCommission,DGResearch&Innovation,DGMigration&HomeAffairs,DGHealthandDGConnect, showed thatSSHexpertsalreadyengagenowwithallscientificandpolicyfieldswheresociety isfacedwithchallenges.Buttheyalsoshowedthatthereisstillmuchpotenti-al tobroadenanddeepenthisexchangeacrossdisciplinesandacrosssectors.Iparticularlyliketorememberthelateafternoonsessionaboutartificial intelligence and SSH together with industry representatives

MATTHIASREITER-PÁZMÁNDYDOI:10.22163/fteval.2019.360

FOREWORD

thatresulted indemandsformore interdisciplinarycurriculaandmorepracticalguidelineinethicsandAI.TheseconddaystartedwithJamesWilsdon givingabroad overview about impactdebatesandSSH. Themainfocusofthatdaywasonthepapersessions.Youfindtheiroutputlargelyreflectedinthepaperspublishedinthisissue.Ilistedsomeofmypersonalhighlights,butofcoursethereweremanymore.TheconferenceendedwithanemphaticcontributionfromHeadofUnitHaraldHartungfromtheEuropeanCommission,DGResearch&InnovationandtheclearinvitationtowardstheSSH-communitytoworkmorecloselytogether.

TheeventjoinedthechoirofpositionsthatledtotheroleofSSHinthecurrentversionoftheHorizonEuroperegulation.SSHwillbemuchstrongerrootedintheregulationforHorizonEurope1thanitwasintheregulationforHorizon2020.Article4ontheprogrammestructureregu-latesthatsocialsciencesandhumanities(SSH)shallplayanimportantroleacrossallclustersofHorizonEurope.Article6aontheprinciplesoftheprogramme regulates thatHorizonEuropeshall ensureamultidis-ciplinaryapproachaswellastheintegrationofSSHacrossallclustersandactivitiesdevelopedundertheprogramme.Article7ontheMissionsdefinesthatMissionsshallstimulateactivityacrossdisciplinesincludingSSH.ThespecificprogrammementionsSSHalsoseveraltimesexplicitlyandregulatestheresearchareasinamoredetailedway.2Allthesecon-crete references are a substantial improvement and upgrading of therole of SSH in the legal basis of the future framework programme incomparisontoHorizon2020.

Theconferencewasnotonlydesignedasaforumforexchangebet-ween researchers and policy makers, but also as an undertaking thatproducestangibleoutcomessupportingresearchersandpolicymakersintheirwork.Thefirstpublicationwasafour-pagerwiththetitle“So-cialSciencesandHumanitiesResearchMatters.Guidelinesonhowtosuccessfullydesign,andimplement,mission-orientedresearchprogram-mes”,inshort“SSH-guidelines”.Thecontentwasintensivelydiscussed,in a dedicated workshop during the conference, with experts fromresearch and policy making, including the European Commission. Theaimofthehands-ondocumentistoprovidepolicymakerswhodesignresearchprogrammes,butalsoresearchmanagers,reviewersandevalu-atorswithrecommendationsfortheirwork.EversinceitwaspublishedinFebruary2019ithassparkeddebateandactiononEuropean,butalsoonnationallevel.


TheSSH-guidelineswerefollowedbytheWorkingPaper“SSH-ImpactPathwaysandSSH-IntegrationinEUResearchFrameworkProgrammes”byThomasKönigthatwillbroadentheperspectiveof thepractical re-commendationsinscientificandpolicydebates.ItbuildsonthescientificdiscourseonvaluationofSSHresearchaswellasSSH-integrationinEUframeworkprogrammes.Itdiscussesrecenttrendsinresearchfunding,providesabriefhistoricaloverviewoftheeffortsofintegratingSSHintotheEUresearchframeworkprogrammesandconcludeswithsuggesti-onsforSSH-researchers.TheWorkingPaperisincludedinthisissue.

One of the specific recommendations is to organize workshops forSSHresearchersandpolicymakersonthenationallevel.InAustriawestartedwithapilotworkshopinMarch2019.SSHresearchersfromdiffe-rentdisciplinesandresearchfields,fromuniversitiesandnon-universityresearchinstitutionsmetupwithnationalProgrammeCommitteedele-gatesfromdifferentsectoralMinistriesaswellastheNationalContactPoints(NCPs)inAustria.TheworkshopwasagreatstartofaninitiativethatneedstogrowfurtherasHorizonEuropewillcomecloser.

Iwould like to thankonceagainallwhocontributed to theconfe-rence.Firstofall,athankyougoestotheLocalAdvisoryBoardoftheconferencewhodiscussed theearliestdraftsof theconcept:TaraAn-drews, Christian Fleck, Michaela Glanz, Barbara Horejs, Monika Mok-re,StephanieRammel,andFrankWelz.AparticularthankyougoestoThomasKönigandKatjaMayerwhoworkedcloselywithusthroughoutthepreparatoryphase.IwouldliketothanktheinternationalScientificCommitteethatdidallthehardworkofreviewingthepapersoftheCalltothisconference:PaulBenneworth,OlivierBouin,UlrikeFelt,YvesGin-gras, Poul Holm, Jürgen Howaldt, Carina Keskitalo, Kirsten Langkilde,StefaniaMilan,AndreaPető,ClaudioRadaelli,EmanuelaReale,Sarahde Rijcke, Evelyn Ruppert, Marta Soler, Jack Spaapen, Tereza Stöcke-lova,JohannesVogel, andMilenaZic-Fuchs. I alsowant to thank thecommittee’schairHelgaNowotny,whowasgivingusthemostvaluableinputearlyonanddraftedthe“ImpactRe-loaded”inputpaper.Iwouldalso like to thanktheEuropeanCommissionfor their financialsupportandendorsement.AspecialthankyougoestothestaffattheUnitB6inDGResearch& Innovation,whowereextremelyhelpful throughoutthepreparation:HeadofUnitHaraldHartungandProjectOfficerMoni-caMenapace.AveryspecialthankyougoestoProjectOfficerBasudebChaudhari,whowasputtingthatextraeffortinourcommonpreparatorywork.IalsowouldliketothankforthesupportwithinmyownMinistry,especially from our Director General Barbara Weitgruber, the Head ofEU-DepartmentChristianNaczinskyandmyHeadofDepartmentUrsulaBrustmann.Ialsohavetothankallthoseinvaluablehelpinghandsthatcreated such a good environment at the conference: Philipp Brugner,Patrik Cunat, Alexander Degelsegger-Márquez, Isabell Duscher, TanjaFeiler,RobertFrühstückl,MaximilianJäger,DorisKaiserreiner,ElisabethKoller, Alexandra Krammer, Silvia Kraml, Martina Lindorfer, GottfriedPrinz,PeterSeitz,andGorazdWeiss.Averyspecial thankyougoestoElkeDall,andanextraspecial thanks toBettinaGlaserandBerenickeEckerwhowentthemetaphoricextramile–andattheconferencetheydidsointherealsenseofthesaying.Lastbutnotleast,aspecialthankyougoestoKlausSchuch,thedirectoroftheZSI,CentreforSocialInno-vation,whodidabeautifuljobbothontheorganisationalaswellasontheintellectualleveloftheconferencepreparations.

Theconferenceisover,thepublicationsarepublished,buttheworkwillcontinue.WeneedtocooperatetounfoldthefullpotentialofSSHinHorizonEurope, itsClustersandMissions.Moreexchangebetweenpolicymakersandresearchersisstillneeded.SSHresearchersneednottoshyawayfromapproachingtheirpolicymakers.Policymakersshouldseek toexchangemorewithSSH researchers, theirprojectsand theirfindings.Thisconferenceaimedatprovidingaforumforthisexchange.Idowishthattheconferencepublications–theSSH-Guidelines,theWor-kingPaperandtheseproceedings–willhelptocarryonthisexchange.Haveaninterestingread!

MatthiasReiter-PázmándyAustrianFederalMinistryofEducation,ScienceandResearch


concept. The reason is that they study impact, they reflectimpact,andtheyassessimpact.Itisnotbychancethatasocialscientisthaselevatedthenotionof“unintendedconsequences”to prominence and that assessing these consequences hasbecomeoneof themainrationalesofappliedresearch in thesocialsciencesandhumanities.

b.The social sciences and humanities have made tremendousprogressinthepasttwodecades,intermsofexpandingtheirmethodology and conceptual approaches. While there is stillmuchdisagreementamongdisciplines,schoolsofthought,andepistemiccommunities,muchofthisisduetothelogicofhowacademia is organised. Here, “impact” may offer a powerfulleveragetoaddressinconsistenciesandtocomeupwithamorecollaborativeunderstandingofwhatisatstake,therebyironingoutmanyoftheratherfrustratinginternalacademicstruggles.

c. Finally, the rise and productivity of the social sciences andhumanitieshavebeenstronglyconnectedandinevitablyshapedby the process of modernity. While this interdependence hasbeen acknowledged, the repercussions have not fully beenabsorbed. Transformative science must be transformative inadoublesense:wantingtoexert influenceinsocietybutalsoopentobeinfluencedbysocietyanditsneeds.Inotherwords,only if open two-way mutual communication channels areestablished,societalneeds,regardlessoftheiroriginsandtheways in which they are expressed, the transformative powerinherent inSSH researchcanunfold in society.Amongother,this necessitates greater openness and readiness as well ascapability to involve heterogeneous groups in society, ratherthanremainingfixatedonpolicy-makers.

Theconferenceoffersauniquesettingtotakeonthesevariousas-pects,andtorethinkthevitalrolethesocialsciencesandhumanitiescanplayinfacingmanyofthechallengesEuropeansocietiesareconfrontedwith.Policyissuesrange:fromenvironmentalissuesandbehaviour,ag-riculturalpolicyandconsumption,technologyandinnovation,security,foreignanddefence,publicfinances,cultureandmedia,health,judici-ary,totransportandeconomicsustainability.Tostimulateaprocessofre-loading, we invite for papers from different perspectives of impact.Inparticular,wewouldliketomove“impact”fromamostlydefensive,albeitpolicy-relevantinstrumenttosomethingthatwillbecomeatrans-formativeelementtowardsamoreinclusivesociety.

There are various attempts to circumscribe and catch the me-aning of “impact” related to and resulting from scholarly re-search from thesocial sciencesandhumanities.1Forall their

commendable efforts, these definitions cannot remove the impressionthat the initialneedtocomeupwithadefinition isdrivenbypoliticalmotives.Asaresult,theuseoftheterm“impact”hasoftenacquiredadefensivetone.Thepoliticalmotivesspringlargelyfromincreasingde-mandsforaccountability;andthedefensivenesscanbedetectedintheway“impact”issetuptoprovetherelevancetosociety.

We argue that time has come to move beyond a purely defensivestance on the part of the social sciences and humanities. There is amoresubstantial issue involved,namely, to re-thinkthetransformativerelationship between science and society. Scientific research is abouttransformation–howtoenableit,orhowtoavoidit.Itisaboutthetrans-formationthatsocietyisundergoingasmuchasaboutthetransforma-tivepowerinherentinknowledgeandpoliciesbasedonsocialscienceknowledge.Thesocialsciencesandhumanitiesaredeeply involved intheprocessesthatusescientificandscholarlyapproachestobringaboutabettersociety,difficultasitmaybetodefineit.Arguably,theirsocie-talandpoliticalrelevancehasalwaysbeenmorepresentinthepoliticalarenathanthatof thenaturalsciences.Thisshouldbeacknowledgedandnotdenied.

Socialsciencesandhumanitieshavetolookat“impact”inadifferentway–thetermneedstobe“re-loaded”witharenewedsenseofres-ponsibilityandreflectingadifferentself-imageoftheirroleandpositionin society. Instead of using “impact” solely as a defensive instrumentto preserve the status quo of the social sciences and humanities, thecontemporaryfocuson“impact”offersauniquewindowofopportuni-ty for thesocialsciencesandhumanities to reflectuponand redefinetheir role and redefine their societal relevance. This understanding of“impact”isnotlimitedtotheinstrumental“usevalue”thatSSHresearchmayprovideforcertainusergroups,butiswide-rangingthroughtheim-plicitembeddednessofSSHwithinsociety,providedthatitremainsopentosociety,anditspowertoanalyseandexplainsocialphenomenaandto contribute to overcoming societal drawbacks through a diversity ofdiscourseandexchangelevelsandformats.Theseaspectscanbedealtwithdistinctively,albeittheyareinterrelated.

a.With “impact” becoming the driving force for assessingrelevance of scientific endeavors the social sciences andhumanities are in a position to contribute to, and shape the

THOMASKÖNIG,HELGANOWOTNYANDKLAUSSCHUCHDOI:10.22163/fteval.2019.361

IMPACTRE-LOADED

1 Forthesakeoforientation,Reale,E.etal.provideadefinitionofscientific,socialandpoliticalimpact,statingthat“SSHresearchgeneratesscientificimpactwhenitinfluencestheproductionoffurtherresearchoutputsfollowingnewapproachesforanalysisorbasedonnewresults.Changesrelatedtosocialimpactaffectthecultural,economic,andsociallifeofindividuals,organizations,andinstitutions.Politicalimpactincorporatesthecontentsofresearchintopoliticaldecisions,andmotivationsandrationalesforpoliticalactionandprioritysetting.”Reale,E.etal.(2017):Areviewofliteratureonevaluatingthescientific,socialandpoliticalimpactofsocialsciencesandhumanitiesresearch.ResearchEvaluation2017,1-11,doi:10.1093/reseval/rvx025.

ISSUE 48 | JULY 2019 11

AUTHORSTHOMAS KÖNIGInstitut für Höhere Studien - Institute for Advanced Studies (IHS)JosefstädterStraße39,Vienna,1080(Austria)E:[email protected]

HELGA NOWOTNYWiener Wissenschafts-, Forschungs- und Technologiefonds (WWTF), Vienna Science and Technology FundSchlickgasse3/8,Vienna,1090(Austria)E:[email protected]

KLAUS SCHUCHZentrum für Soziale Innovation GmbH - Centre for Social Innovation (ZSI)LinkeWienzeile246,Vienna,1150(Austria)E:[email protected]


1. INTRODUCTION

This Working Paper reflects the current status of research in thesocialsciencesandhumanities(inthefollowing:SSHresearch)inthe context of European research policy.1 It examines three see-

minglyseparateissues:therecentdevelopmentofresearchpolicy,bothin termsofactual fundingaswellas its rhetoric; theactualhistoryofSSHresearchwithintheEuropeanUnionresearchfundinginstruments;andtheepistemologicalcharacteristicsofSSHresearch.Tyingtheseis-sues togetherwill provideabetterunderstandingofwhere the socialsciencesandhumanitiesstand,whattheircapacitiesare,andwhattheycanprovide to tacklesocietalchallenges thatwe,ashumankind, facetoday.Basedonthisbackground,theambitionofthisWorkingPaperistodiscusshowtoenhancetheroleofSSHresearchincurrentandfutureresearchfundingpolicies.

ThisWorkingPaperfollowsupontwoshorterpolicypapers.Thefirst,called“impactre-loaded”,waswritteninspring2018bythreeco-orga-nisersoftheSSHimpactconferenceinViennainNovemberofthesameyear,makingthecasetotheircolleaguesintheSSHcommunityto“re-flectuponandredefinetheirroleandredefinetheirsocietalrelevance”.Specifically,thepaperwantedtoachievethreethings:“tocontributeto,andshapetheconcept”ofimpact;toshedawayacademicstrugglesinorder“tocomeupwithamorecollaborativeunderstandingofwhat isatstake”;andfinally,while“wantingtoexertinfluenceinsociety”,alsobeing“opentobeinfluencedbysocietyanditsneeds.”(König,Nowotny,andSchuch2018)WhilethisWorkingPaperhopestoprovideadditionalinsights intoall threeof theseaspects, it isclearly focusedonthese-condaspect,thatis,tocontributetoenhancingtheconditionsforSSHtoproviderobust,andlasting,contributionstosolvingsocietalchallenges.Theotherpaper,shortlySSHGuidelines,summarisesrecommendationsfor R&D programme authorities, reviewers and programme evaluators(König2019).Sincethissecondpolicypapercouldonlymakeclaims,theWorkingPaperalsoaimsatsubstantiatingtherole,andcharacteristics,ofSSHresearch(formoredetailsontheSSHGuidelines,seesection5).

GiventheperspectiveandsupplementalroleofthisWorkingPaper,thereare three important restrictions toannounce rightaway. (1) Theambitionofthisdocument isnotto layout indetailwhatkindofSSH

IN REMEMBRANCE OF PHILIPPE KERAUDREN (1963-2017)

ABSTRACT

ThisWorkingPaperbuildsonthescientificdiscourseonvaluati-onofSSHresearchaswellasSSH-integrationinEUframeworkprogrammesandaimsatsummarisingthekeyfindingsfromthe

November 2018 Austrian EU Presidency Conference “Impact of SocialSciencesandHumanitiesforaEuropeanResearchAgenda–Valuationof SSH in mission-oriented research”. It deals with the topic in threeinstalments.First, itwilldiscussrecenttrendsinresearchfunding.Se-cond,itprovidesabriefhistoricaloverviewoftheeffortsofintegratingSSH into the EU Research Framework Programme. It then adds someobservationsaboutcontinuedchallengesinSSH.Finally,itwillconcludewithsomesuggestionsforSSHscholars,basedonthediscussionsfromtheconference.InthatregardtheWorkingPaperisalsoadocumentforfurtherreadingforthosewhohavereadearlier,shortertextsthatwerepublishedinpreparationofthatconference.

TABLE OF CONTENTSSH-ImpactPathwaysandSSH-Integration inEUResearchFrameworkProgrammes. 10Abstract 101. Introduction 102. Recentdevelopmentsinresearchfunding 113. HistoricalassessmentofintegratingSSH 134. ChallengesinSSH 175. Impactre-loadedinHorizonEurope 18

Annex:SummaryofMeeting“SocialSciencesandHumanitiesinHorizonEurope” 20Bibliography 22Keywords 25

THOMASKÖNIGDOI:10.22163/fteval.2019.362

SSH-IMPACTPATHWAYSANDSSH-INTEGRATIONINEURESEARCHFRAMEWORKPROGRAMMES.

1 ThisWorkingPaperhasreceivedprojectfundingfromtheEuropeanUnion’sHorizon2020researchandinnovationprogrammeundergrantagreementNo814729.IamgratefultoRafaelSchögler,ChristianFleck,UlrikeFelt,HelgaNowotnyandKatjaMayerforinitialdiscussionsonthematter,aswellastoMat-thiasReiter-PázmándyandKlausSchuchforcommentsonapreliminarydraft.

ISSUE 48 | JULY 2019 13

2. RECENT DEVELOPMENTS IN RESEARCH FUNDING

Fromascholarlyperspective,“science”canbedescribedasaself-regulatory,multi-faceted,highlyspecialisedinstitutionwhosenumerousfactionsanddivisionsnonethelesssharesomeimportantinformalnorms(Merton 1957, 537–61), and research (or, in economical terms: know-ledgeproduction)isoneofthekeycomponentsofthisinstitution.4Yetscienceisalsoregulatedbypolicy,andmoneyhasemergedasadefiningingredientinthisrelationship(Greenberg2001).Historically,publicpolicyattemptstoguaranteepublicbenefitswhilemaintainingscientificinde-pendencecanbedifferentiatedintoperiods.5Againstthisbackdrop,therelationshipof“scientificresearch”andpublicpolicyhasbeencoinedbythreeinterlacedtrendsoverthelasttwodecades.Thefirstistheinno-vationpolicynarrative;thesecondisaboutinterdisciplinarycooperation;thethirdisaboutimpact.AllthreehaveconsequencesforSSHresearchintheEuropeanresearchfundinglandscapeatlarge,andinthemission-oriented research fundingpartsof theFPspecifically. In the followingthreeinstalments,acloserlookateachofthosetrendsisprovided.

THE INNOVATION POLICY NARRATIVE

Thenarrativeofinnovationpolicystressestheimportanceofscien-tificresearchforinnovation,andthus,forthewell-beingofindividualsandoursocieties.Ifeconomicgrowthisthebedrockofdemocracy,theninnovationisthebestguaranteeforeconomicgrowth.Butbecausein-vestmentinscientificresearchisbroadlyacceptedtobeacommongood(Stephan2012),innovationmustbestimulatedthroughpublicspendinginresearchanddevelopment(R&D).IntheEuropeanUnion,thisnarrati-veemergedinthe1990s(Ulnicane2015),solidifiedintoanew,additio-nalEuropean“governancearchitecture”(BorrásandRadaelli2011),and,with its flagship “innovation union” (European Commission 2010) hasbecomeoneof the latesthopefuldriving forces for further integrationamidstanEUthatotherwiseisoftendescribedasbeingincrisis.6ThecurrentdebateaboutthefutureEU-Budget,thenextmultiannualfinan-cialframework(MFF)from2021-2027,vividlycontinuesthisnarrative.

Theinnovationpolicynarrative(seeFigure1)sharessomesimilaritieswithwhat isusuallyknownas the linearmodelof innovation, theas-sumptionthatthereisasequenceofstepsfrom“basicresearch”throughappliedresearchtodevelopmentandmarketisationofnewproducts.Ashasbeenconvincinglyargued,whilethelinearmodelofinnovationisof-tenthoughtofastoosimplebyexperts7,itremainsa“socialfact”partly

research is relevant for cooperative, interdisciplinary research tacklingsocietalchallenges,andtowhatend.Otherreportshavealreadyprovi-dedsubstantialinputtothis,andinterestedreadersareexplicitlyinvitedtoreadthemwithgreatattention(seeAtkinsonet al.2009;Drotner2013;Dastonet al.2018).Rather,thisWorkingPaperistodiscussthecontext,constraints,andpotentialsofSSHresearch.Itismuchmoreconcernedwithquestionsrelatedtosciencepolicyand,morespecifically,researchpolicy.

(2) For the remainder of this Working Paper, SSH research meansprimarilyresearchcarriedoutalongproject-basedfunding.AtEuropeanlevel,thisismostlydoneundertheEUResearchandInnovationFrame-workProgramme(akaFP,currently in itseighthedition,calledHorizon2020andfrom2021onwardsinitsnintheditioncalledHorizonEurope).Obviously,thereisawidearrayofcontributionsofsocialsciencesandhumanitiesinotherareasoftheEuropeanpolity–providingcrucialso-cialandeconomicdata(likeEUROSTAT),buildinguptransnationalinfra-structure(suchasCESSDA,CLARIN,DARIAH,ESS,SHARE),orprovidingintellectualreflectionandindependentanalysisoftheEuropeanintegra-tionprocess(byinstitutessuchasEUI,butalsoinacademicconferences,etc.).2

(3)Even the focuson theFPand itssprawlingsetof funding inst-ruments requires further restriction, as thispaper is interestedmainlyinprogrammesdealingwithmission-orientedresearchfunding.Again,thereareotherfundingopportunitieswithintheFPthatenableresear-chersfromSSHtoconductacademicresearch.3Therestrictionis justi-fiedbecausethequestionathandisaboutthepotentialroleofSSHincontributingtoproducingnewknowledgespecificallytosolveproblemsthataregenerallyperceivedtobeworryingriskstoindividuals,peoples,societiesandhumanity.Theseproblemsarenotdefinedinapurelyscien-tificmanner,albeitscientificresearchmayhavecontributedtotheirexis-tenceinthefirstplaceandusuallyalsoprovidesthetoolkitstorecogniseandunderstandthem.Forexample,thelistof“SustainableDevelopmentGoals” (SDG), as adopted by the United Nations General Assembly in2015,consistsofanumberofproblemsthathavebeenidentified,ack-nowledged,andalsonegotiatedinanintricatepolicyprocessinvolvingallUNmemberstates.

Whatevertheirdenominationinthespecificpolicycontext:theem-phasison“challenges”,goals”and“missions”recognisesthatthereareproblemssowickedthatwerequireparticulareffortstocopewiththem.Obviously,science–andnewscientificknowledge–iskeytounderstan-dingthoseproblems,toalleviatingthemandalsotopreparingforpoten-tialfallouts.Atthesametime,thisaddedanewlayertotheambitionsof research funding policy. It has also renewed the quest to increasecooperationbetweendifferentfieldsofscienceandscholarship,andhasreinforcedthegrowingdemandfor“impact”.

2 ForusefulreflectionsoftherelationshipbetweensocialsciencesandEuropeanintegration,seeRosamond(2007),alsoAnderson(2009).3 Mostprominently,thisistheEuropeanResearchCouncil(ERC),whichoffersgenerousfundingtoindividualresearchersinahighlycompetitivemanner

(König2016).4 Othercomponentsaretraininginscientificmethodsandteachingoftheories,anddisseminationofresearchresults.Merton,inthebookreferredto,also

pointsoutthat“scienceisadeceptivelyinclusiveword”,andrestrictshisownanalysisto“scienceasaninstitution”(551).Thisistruealsoforthewaythetermisusedhere,exceptthatitexplicitlyincludesthesocialsciencesandhumanities.

5 Themostbasic,andbestknown,differentiationistheonebetween“Mode1”and“Mode2”periods;cf.Gibbonsetal.(1994;seealsoBraun2003forfurtherelaboration).Elzinga(2012)suggestsaperiodisationthatbetteralignstohistoricaldevelopmentssincetheendofWWII;hedistinguishesbetweentheconsecutiveperiodsof“legitimation”,“professionalisation”,and“accountability”.

6 See,fortheEuropeanUnion,ashortanalysisinKönig(2017,123–27)7 Alternativeapproachesincludethe“Mode2”andvarious“helix”models;foragoodoverview,cf.HesselsandvanLente(2008).


contractualcooperation–withalltheproblemsandopportunitiesthatgo along with it (Lyall et al. 2013; König and Gorman 2016). But it isnecessarytothinkofdifferent“modes”ofinterdisciplinarycooperationthatresultfromthespecificquestionstobetackledasmuchasfromthebroadercircumstancesthatdriveresearch.Indeed,onecandistinguishbetweenanintegrative,asubordinate,andanagonisticmodeofinter-disciplinarycooperation(Barry,Born,andWeszkalnys2008,28–29).ItiseasytoseehowthisisofparticularimportanceforSSHresearch:ontheonehand,thetendencytobringscholarlyresearchinthesocialsciencesand humanities under an all-encompassing funding regime, togetherwiththenaturalandlifesciencesandengineering,isanopportunitytomakebetteruseofSSHresearchandtoopenupthefield.Yetthereisalsoaconsiderablepressuretoalignresearchonintricateandcomplexrelationsofsocietalailmentstotheformalrequirementsofthosetem-porarycombinationsofresearchers.Also,thereisatendencytodelegatecertainaspects (likeparticipation,communicationorethics)ofa largecooperativeresearchprojecttopartnersfromSSHfields,whichdoesnotnecessarilydojusticetothepotentialinputthatcouldbeprovided.

IMPACT ASSESSMENT

Publicfundingbodieshaveestablishedandfine-tunedadministrativeprocedurestomakesurethattaxpayers’moneyiswell-used.Asinmanyotherareasofperformancemeasurement(Muller2018),theNewPublicManagementstylehasfounditsexpressionina“metrictide”atuniver-sities(Wilsdonet al.2015)withtheattempttoassessinput,output,andimpactofscientificresearch(deRijckeandRushforth2015).Whileinthelate1990sandearly2000s,themainfocusofassessingthequalityofresearchandscientificconducthasbeenonacademicrelevance(oftenundertheterm“excellence”),recentlytherehasbeenashifttowardsthebroadernotionofimpact.10

Impactofresearchmeansatleastthreedifferentdimensionsofnew-lyproducedknowledge;besidesacademicimpactitalsoincludesimpactonthepoliticalrealmandonthepublic,orsociety,byandlarge.Acade-mic impactof knowledgeproduction is rathereasilyassessed,usually

becauseitissoentrenchedinstatistics(Godin2009,27).Similarly,whilethereare reasonabledoubtsabout theunderlyingassumptionsof thenarrative (Wladawsky-Berger2018),andattempts tocomeupwithal-ternatives(Nowotny2016),itseemsfairtosaythattheinnovationpolicynarrativeremainsconvincingforpolicymakersthusfar.

Whythispersistence?Theriseoftheinnovationpolicynarrativeisnotpurelyadiscursivephenomenon,asithasincreasedattentiontowardscreating opportunities for new knowledge (Flink and Kaldewey 2018).Policymakersandthepublichavebeenwillingtopourmoremoneyintoscientificresearchwiththeexpectationofincreasedsocialbenefit.Butthisiscomingwithstringsattached,andpotentialramificationsforSSHresearch inparticular.Oneconsequence isthat“innovation” isusuallythoughtofinanarrowsense:everythingthatleadstocommodification,marketisationofproducts.8Suchexpectationsarealsosomewhatprede-terminingthetypeofresearchthatistobesupportedinthefirstplace.Also,thereseemstobeapreferenceforspendingadditionalmeansintheformofcompetitiveproject-basedresearchfunding.Finally,thereisanincreaseddemandtoprovethevalueofresearchfundedbypublicdo-main,the(perceived)pressureonpolicymakerstoshowaccountabilitytothetaxpayer,andtoobjectivelycontroltheusageofpublicfundinginanewbureaucraticfashion(“auditculture”).

INTERDISCIPLINARY COOPERATION

Debate about the illnesses of academically organised, disciplinaryresearchisongoing.9Oneoftheoldesttropesofsciencepolicyhasbeenthenotionof“interdisciplinarity”–theideaofovercomingthe“episte-micrent-seeking”ofscientificdisciplines(Fuller2016)byintegratingthestrengthsof variousdisciplines towardsone researchgoal (Frodeman,Klein,andPacheco2017).Inthewordsofoneoftheleadingscholarsonthetopic,thenotionofinterdisciplinarityismoreabout“expressingourdissatisfactionwithcurrentmodesofknowledgeproduction”thanpro-vidingaconceptofwhatitactuallyis(orcouldbe)(Frodeman2010).Atthesametime,thiscombinationofemptinessandpromisemighteasilybeonemajorreasonforitscontinuedsuccess.

Thisdoesnotmeanthatinterdisciplinaryresearchisnottakingplace.Yettheinnovationpolicynarrativeanditsaforementionedramificationsforresearch(andSSHresearchspecifically)bringanewdynamictotheage-oldideaofinterdisciplinarity.Theincreaseofproject-basedresearchfunding and the new emphasis on tackling societal challenges meanthat interdisciplinaryresearch isoftenexpected intermsoftemporary,

8 Ahistoriographicalanalysishasrevealedthecomplexhistoryoftheterm“innovation”,seeGodin(2015)9 Forapowerful,recentargumentinthatcontextseetheessaybyDanSarewitz(2016).Agoodsummaryof“malfunctions”ofscienceisprovidedbyFischer

(2008).10 Forarecent,powerfulcritiqueonthenotionofexcellenceinresearchseeMooreetal.(2017).

Fig. 1:Schematicdepictionoftheinnovationpolicynarrative(Drawn by the author)

ISSUE 48 | JULY 2019 15

3. HISTORICAL ASSESSMENT OF INTEGRATING SSH

Againstthebackdropofthegeneralcontextofrecentdevelopmentsin research funding mentioned in the previous section, it is now ne-cessarytoassessthedevelopmentofdealingwithsocialsciencesandhumanitiesunderthelatesteditionsoftheEUResearchFrameworkPro-gramme(FP).The2009Lisbon“TreatyonEuropeanUnion”putscienceandresearchofficiallyattheEuropeanstage(TEUArt3(3),andTFEUArt179-190),butasamatteroffacts,researchpolicyhadbeenthereforalongtimealready(Banchoff2002,7–8;alsoGuzzetti2000,2009).SocialScienceshadtheirowntargetedprogrammefrom1994on,immediatelyfollowingtheMaastrichtTreaty(Kastrinos2010,300).Sincethesixthedi-tionoftheFP,thehumanitieswerealsoofficiallyincluded(Smith2003).ForSSHresearchasawhole, therefore, theroleofEuropeanfunding,andtheEuropeanCommission’sFPspecifically(Schögler2013;Schöglerand König 2017), has increased over the past two decades, and withregardstotwoaspects.

InhisanalysisofSSH inEurope from2010,NikosKastrinos (2010)foundthat,despitetheemphasisofresearchprioritiesandthematicori-entations,Europeanresearchfundingthenwasmovingmoreandmoretowards a “diffusion-oriented model”, emphasising capacity buildingoverfulfillingadistinctmission(301).Thiswouldalsoremainthecasewith the eighth edition of the Framework Programme, Horizon 2020,eventhoughthemissions-approachwouldsoonmakeacomeback.ThesecondobservationwasthattheEUresearchprogrammeshademergedaspoints-of-referencefor thememberstates,both intermsof themes(suchasthechallenges)andintheorientation(diffusioninsteadofmis-sion);13insomerespecttheyhadevenoutpacedfundingopportunitiesatnationallevel.Thethirdobservationreferredtothefactthat,despiteofitslimitedsizewithintheoverallFPbudget,anddespiteseveralnationalfundingschemestargetingresearchinthesocialsciencesandhumani-ties, “in comparative terms” the FP’s own dedicated research fundingforSSH“hasbeenthelargesttargetedprogrammeinEurope”thatwasavailableforresearchinsocialsciencesandhumanities(304).

RUNNING UP TO HORIZON 2020

Kastrinosarticle summarised the stateofdevelopment forSSH re-searchshortlybeforenegotiationsoftheeightheditionoftheFP(Horizon2020,whichwas scheduled tobeginwith2014), and the roleofSSHresearchinit,started.However,tounderstandthedebatethatfollowed,it is important to also take into consideration the broader context ofthat time. The diffusion-oriented approach of defining broad thematicchallenges, thegrowing importanceofcoordinationof researchpolicyat European level, and the fact that the latest editions of the FP alsoincludedlargeprogrammesfundingSSHresearchalreadyputpressuretofit inonthosecommunitiesthatperceivethemselvesaspartof thelabel“SSH”.Thisonlyintensifiedin2008andthefollowingyears,when

throughcitations;itreliesonadecade-oldfieldcalled“scientometrics”(MingersandLeydesdorff2015;Gingras2016).Broadeningthemeaningofimpacthasopenedthedoortoawidervarietyoftoolsofassessment,someofwhichrelyonexcitingnewtechniques;11yetitalsobroughtinconsiderabledifficulties,orambiguities.12Tostartwith,therearediffe-rent types of impact along two dimensions (expected vs. unexpected,and intendedvsunintended) (Realeet al.2014,37).Also, thereareatleastfourproblemswhenassessing,ormeasuring,impactofresearch:theproblemofcausality,theproblemofattribution,theproblemofinter-nationality,andtheproblemoftheobservationperiod(FeltandFochler2018,9–10).Thesedifficultiesapplynotsolely toSSHresearch.Giventhedifficultiesthatcomealongwithit,thebroadeningoftheconceptofimpacthasspecificramificationsforthesocialsciencesandhumanities.(Realeet al.2017)

WHAT THESE TRENDS MEAN FOR SSH RESEARCH

Basedonthistourdeforce,wecanbrieflysummarisetheconstraintsthat current trends in research fundingposeonSSH research specifi-cally. One is that the narrow concept of innovation seems to excludebroadernotionsofsocietalinnovation.Anotheristheurgetocollaboratetemporarilyandthetendencyofbeingdelegatedaspecificrole intheinterdisciplinarymachinery.Andyetanotheronerelatestotheinherentdifficultiesofprovingitsvalueunderthecurrentauditcultureregime.Atthesametime,onemustalsoemphasisetheopportunitiesthatarecrea-tedhereforSSHresearchtoactuallyplayamoreimportantroleintheproductionofknowledgethatisrelevantforsociety.Wecanseewithinthecontinuedparadigmof innovationpolicythatadualshift is takingplace.Ontheonehand,thisshift ismovingawayfromtheexcellencerhetoricthatwasbehindthedrivetoreinvigoratetheEuropeanResearchArea,aimingatbroader impact;on theotherhand, thenew focusontacklingsocietalchallengesthroughmission-orientedresearchfundinginstrumentsalsomeansthatthenarrowunderstandingof“innovation”maybepronetosomeconceptualadaptation.

Acritical issueof this summary is thatmuchdependson theSSHcommunitiesthemselves: it isuptothemtoget involvedandtomakesuretheirconsiderableamountofexpertiseisbetterheard.Thiscallforactive involvement isnotnew.Thenextsectionaimsto takea lookatthehistoryofSSH involvementand theachievements thathavebeenmadesofar.

11 See,forexample,thetopicanalysisintheUKreportonartsandhumanitiesbyDrauxandSzomszor(2017)12 ForasomewhatdifferentdebateonimpactassessmentintheUS,seeKamenetzky(2013).13 Similarly,LebeauandPapatsiba(2016).


theHorizon2020programmewouldstandthebriskausterityambitionsofEuropeanUnionmemberstates.Besidesthefactthatitwasforeseentosubstantiallyincreasethebudgetforthisprogramme,itscreatorsper-ceivedHorizon2020as“acleardeparturefrombusinessasusual”,astheCommissionerstatedinanearlierspeech(Geoghegan-Quinn2011a).ItisthereforetemptingtoassumethattheCommissiondidnotwanttohaveadditional political disturbances in getting their ambitious programmethrough.ItaimedatnothavingtooverthrowtheconceptionbehindtheHorizon2020programme,andthereforeremainedconciliatorybutfirm.

Thisapproachhadseveral consequences thatwoulddominate thesecondpartofthediscursivecontroversy,mostlyconstitutedthroughre-portsandstatementsbyinterestgroups(vandenDoel2012;ScienceEu-rope2013):First,theoverallstructureofHorizon2020wasnottouched;instead,anotherchallengewasadded.Thediscussionnowfocusedonhowthisnew(additional)challengeshouldbenamed,andhowmuchresources itwouldget.Second, it reluctantlybroadened thenotionof“innovation”thatisthecoreoftheCommission’spoliticalagenda(Euro-peanCommission2009;Paraskevopoulou2012).Thediscussionfocusedonwhat“socialinnovation”actuallyshouldbe,andwhetherthismeantan“instrumentalisation”ofSSHoritsusefulapplication.Third,itsoughttoencourageSSHresearcherstothinkoutoftheboxandtocooperatewithcolleaguesfromthenaturalsciences.Thus,theprosandconsof“in-terdisciplinarity”and“integration”wereatthecentreofthediscussion,andhowSSHwouldfarewithintheremainingsixchallenges.

ThiswasalsothecontextoftheVilniusConferencethatmarkedthefinalphaseofnegotiatingthestructureoftheHorizon2020programmeanditsunderlyingprinciples,andtransferredthediscussionintotheope-rationaldetailsofWorkingProgrammes,membershipinAdvisoryGroupsandsoon.TheconferenceinVilniusundertheLithuanianPresidencyinthesecondhalfof2013(Mayer,König,andNowotny2013)crystallisedinto an important one-time event in which the Commission would beabletoshowitsgood-willwhilemembersoftheSSHcommunitiescouldexpresstheirhopeforabetterfuturewhileventingtheirfrustrationswiththecurrentsetup.

RESULTS OF THE INTEGRATION EFFORTS UNDER HO-RIZON 2020

Overall,theeffortsintheearlyyearsofthe2010sresultedinagoodcompromise.Ontheonehand,oneSocietalChallenge(SC)wasdedica-ted,as inpreviouseditionsof theFrameworkProgramme, to topicsattheheartofresearchfromsocialsciencesandhumanities(theso-calledSC6,named“Inclusive,InnovativeandReflectiveSocieties”).Whilethe-rewaslessfundingreservedfortheSSH-labelled“challenge”thaninthepreviouseditionsoftheFP(inshare),14atleasttheveryissuehasbeensuccessfullyretained.15Ontheotherhand,theideaofintegratingSSHintootherparts (“challenges”) of thepolicy-oriented research fundingpartofthenexteditionoftheFPallowedforsomevaguepromisethatsomenewformsofcooperativeresearchmightemerge.

Thecrucialquestion,ofcourse,ishowwellthisplayedout.TheEuro-peanCommissionholdssignificantswayintheimplementationofpoli-cies.Thereshouldbenodoubtthat,onceformallyputinthelegaltextofHorizon2020(EuropeanParliamentandCounciloftheEuropeanUnion

researchersanduniversitiesalikeexperiencedthat,innumerousmem-berstates,nationalbudgetswereconcentratedandcutduetofinancialconstraints.When,in2010,thedirectoratededicatedtosocialsciencesandhumanitiesresearchintheDirectorateGeneralforResearchandIn-novationwasabolished,thisexperiencewasnowalsoprojectedontotheEUresearchframework.

Inresponsetothis,membersoftheSSHcommunitiesbegantorally.InDecember2010,researchersfromHUBerlinmobilisedagainstwhattheyperceivedasthe“thematicandfinancial”“downsizingofSocialSci-encesintheEU”.(Börzel,Risse,andSprungk2010)ThiswasfollowedbyanOpenLettertotheEuropeanCommissionbythenewlycreated“Eu-ropeanAllianceforSocialSciencesandHumanities”(EASH2011;Klein2011). In thoseandothercommentsand interventions, thecoreargu-mentscanbeextrapolatedinthefollowingway:(1)Toexpressfearaboutthe “downsizing” of SSH in Horizon 2020. (2) To emphasise the needfor specific topicsand“SocialSciencesandHumanities (SSH)-centredchallenges”(EASH2011)thatservethepurposeoftheSSHcommunity.(3)Toquestion the reasoningbehind thesocietal challenges,pointingoutthenarrowdefinitionof“innovation”.(4)TohighlighttheimportanceofSSHresearchforEurope,andmorespecifically,forfulfilmentofthesuccessfulsolutionoftheSocietalChallenges.

While this spray of arguments hardly represented a stringent lob-bying campaign, it represented the various concerns and beliefs fromwithinthewiderSSHcommunities.TheinitiativewassuccessfulinsofarastheOpenLetterwassignedbyalmost26,000people,andtheEUre-searchministersweresuccessfullymobilisedtoexpresstheirconcerns“whethertheroleofsocialscienceandhumanitieswillbeadequatelyreflected in the tackling of the grand societal challenges” (Myklebust2012).Inresponse,theEuropeanCommissionlaunchedaninformationcampaignonitsown.ThethenCommissionerMáireGeoghegan-Quinnand the leading management of the Directorate General for Researchand Innovation, headed by Robert Jan Smits, went long distances topresent the Commission’s ideas of Horizon 2020 to associations, lear-nedsocieties,andsoon.ToalleviatetheconcernsexpressedbytheSSHcommunities, theysettledontwoarguments:onewasthat, intheso-calledfirstpillarofHorizon2020,SSHwouldbecontinuedtobeservedbyfundingfromthemoreacademicallydriveninstruments,suchastheERCandtheMarieSkłodowska-CurieActions;thesecondwasthat, inthesecondpillardealingwith“GrandChallenges”,SSHwouldhavetobemeaningfullyintegrated.

In November 2011, Geoghegan-Quinn addressed the issue at agatheringattheBritishAcademy.Shereassuredtheparticipants“thatfuturefundingattheEuropeanlevelwillprovidesignificantspaceforso-cialsciencesandhumanitiesresearch”.Thisshouldbereachedthroughaddinganotherdedicatedchallenge,andthrough“embedding”thesoci-alsciencesandhumanitiesintoallsocietalchallenges“toworkbeyondthe ‘silos’ of different disciplines” (Geoghegan-Quinn 2011b; see alsoYoung2015).Inotherwords,theCommissioneracceptedtheinstalmentof an additional “challenge” which was perceived as the one dedica-ted toSSH.Shealsocontinuedtoargue forabroadeningof the term“innovation”andemphasisingthecrucialroleofSSHtothesuccessfulcompletionofall(nowseven)challenges.

ThestrategyoftheCommissioner–toembracethecritics–isunder-standableonlyifseeninthecontext.Atthattime,itwasallbutclearif

14 Forthenumbers,seeSchöglerandKönig(2017).15 Forareflection,seeReiter-Pázmándy(2017).

ISSUE 48 | JULY 2019 17

called“SpecificProgramme”(whichis infactasub-programmewithintheoverallFrameworkProgramme;hencethenameofthelatter),annualorbi-annualWorkProgrammesdefinethecallsthatwillbeannounced.TheWorkProgrammesthemselvesaredraftedbytheEuropeanCommis-sion,basedon input fromtheadvisorygroupsconsistingofexperts inthefield.ThedraftWorkProgrammeisamendedalonginputfromtheso-calledProgrammeCommittee,thatis,agatheringofrepresentativesfromallEUmemberstates(typically,thoserepresentativesareministryofficials).17 Research proposals, submitted on funding calls, are evalu-ated along evaluation criteria by independent reviewers; the fundingdecisionisthenmadebytherespectiveCommissionservicetaskedwithcarryingoutthefundingcall.

(1)Advisorypanelsplayacrucialrole intheSocietalChallengesofHorizon 2020 insofar as they consist of experts that suggest fields ofresearchand thereforeoftenhelp shaping theWorkProgrammesandfundingcalls.TheadvisorygroupsareputtogetherbytheCommissionservicesandmeetonaverage two to three timesevery year. The sizeofeachpanelvaries,andinsomegroupstherearenotonly individualexperts but also public entities represented. While the mechanism ofselectingmembersisnotdisclosed,andoverallcompositionmaychangeoverthecourseoftheeditionoftheFP,itseemsclearthateachgroupisexpectedtofollowsomebasicrulesconcerningdiversity intermsofgender,countryoforigin,andalsodisciplinarybackground(asseenrele-vantfortherespectiveSC).Thelatterisinterestingtoourcase;ascanbeseenfromFigure2,whileeachgroupholdsatleastonerepresentativefromSSH,theshareisquitesmall,and,notably,consistingprimarilyofeconomists.

2013), theCommission–as theexecutivearmof theEuropeanUnion– took the task of integration very seriously. SSH integration becameoneofseveral “cross-cutting issues” runningacross theentireFP.TheCommissionsetupmeasuresforbetterintegratingSSHintotheothersixSocietalChallengesaswellasintootherpartsofHorizon2020,meaningthat its routinesandprocedureswereamended inawaythat fundingcalls could require participation of SSH partners. Such calls would be“flagged” and participation of one (or more) SSH partners would berewarded throughbetterevaluationscores.16 TheCommission’seffortsalsoresultedinsubstantialannualanalysesoftheextenttowhichthein-tegrationexercisewassuccessful(Hetel,Møller,andStamm2015;Birn-baumet al.2017;Stromet al.2018;Swinnen,Lemaire,andKania2019).

Given those efforts, it is therefore worth assessing briefly to whatdegreetheCommission’seffortsborefruit.TheVilniusDeclarationfrom2013 (Mayer, König, and Nowotny 2013) defined four “conditions forthesuccessfulintegrationofSocialSciencesandHumanitiesinHorizon2020”: “recognising knowledge diversity”; “collaborating effectively”;“fosteringinterdisciplinarytrainingandresearch”;and“connectingso-cialvaluesandresearchevaluation”.Itisdifficulttoidentifyindicatorsforeachoftheseconditions;however,somedatacanbegatheredtoas-sesstheinterimresults.OneindicatoristhecompositionoftheadvisoryboardsestablishedforeachSocietalChallenge(1).Anotheristheshareof topicsactually flaggedforSSH integration (2),andyetanotheroneconcernstheactualoveralldistributiontoSSHresearch(3).

Tounderstandthesignificanceandcontextofthoseindicators,itisimportant tobriefly reiterate theprocesses fromdevelopinga fundingcall for research to theactual fundingdecision. Typically,withina so-

16 Fordetails,seehttps://ec.europa.eu/research/participants/docs/h2020-funding-guide/cross-cutting-issues/ssh_en.htm(lastaccessed:2019-03-01)17 Forameticulousprocessoverviewandanalysisofhowworkprogrammesaredevelopedandadopted,seeSchögler(2013,74–106)18 Thegroupshavebeenidentifiedthroughthe“RegisterofCommissionexpertgroups”http://ec.europa.eu/transparency/regexpert/(lastaccessed:2018-

08-15).SocietalChallenge1seemstohavetwobodiesadvisingontheWorkProgramme.NoentrycouldbefoundforSocietalChallenge7.Thenumberofexpertsforeachgrouprefersexclusivelytothe“individualexpertsappointedinpersonalcapacity”.

Group Title Experts SSH representatives

E02942 AdvisorygroupforHealth,demographicchangeandwellbeing(SC1)

26 1economist

E03279 ScientificPanelforHealth(SPH) 25 2economists

E02939 AdvisoryGroupforFoodSecurity,SustainableAgriculture,MarineandMaritimeResearchandtheBioeconomy(SC2)

18 4economists,2socialscientists,1humanist

E02981 AdvisoryGrouponEnergy(SC3) 23 5economists,3socialscientists

E02969 AdvisoryGroupforSmart,greenandintegratedtransport(SC4) 23 2economists,1socialscientist

E02924 AdvisoryGroupforClimateAction,Environment,ResourceEfficiencyandRawMaterials(SC5)

10 3economists,1socialscientist

Fig. 2:AnalysisofHorizon2020advisorypanelsofsixchallenges18

(Puttogetherbytheauthor)


available,itmaybemoreinterestingtolookattheactualaccumulatedfundingshareflaggedforSSHintegrationperSC,andtherefore,essen-tially,availabletoSSHresearch.BasedontheCommission’sannualSSHintegrationreports,thedataindicatethattheshareisdifferentineachSocietalChallengeprogramme,asdepictedinFigure3,andthattherearesubstantialannualfluctuations.Puttogether,thesharehasimprovedovertime,36percentin2014(€902Millionoutof€2.515Million)to47percentin2017(€960Millionoutof€2.060Million).

(2)FlaggingoftopicsistakingplaceduringtheprocessofwritingtheWorkProgramme. It isobviouslyan importantprerequisite foractuallyintegratingSSHresearch;hencetheinterestingquestionis,howmanytopicsperSChavebeenflagged?Thenumberoftopicsvarieswidelybet-weentheSocietalChallenges,andalsobetweenyears(WorkProgram-mes),from15to50.Between2014and2017,theshareoftopicsflaggedforSSHintegrationhasnotbeenlowerthan20percent,andnothigh-erthan55percent.However,giventhattopicshavedifferentbudgets

0

10

20

30

40

50

60

70

80

90

2014 2015 2016 2017

SC1 SC2 SC3 SC4 SC5 SC7 overall

Fig. 3:AnnualshareofbudgetperSocietalChallengeflaggedforSSHintegration19

(Puttogetherbytheauthor)

19 BasedondataintheannualSSHreports(Hetel,Møller,andStamm2015,9;Birnbaumetal.2017,17;Strometal.2018,15;Swinnen,Lemaire,andKania2019,17).NumbersinthisFigure,aswellasinthecorrespondingparagraph,aresolelyonSocietalChallenges1-5and7.SocietalChallenge6isnotconsid-ered,becauseitisthedesignated“SSH”programme,andthereforenotsubjectoftheintegrationexercise.Itshouldbeaddedthat,intermsoffunding,SC6isalsobyfarthesmallestprogrammeofallSocietalChallenges,with€114Millionin2014,€127Millionin2015,€93Millionin2016,and€126Millionin2017.

20 Itisimportanttomentionthat,forthefirstthreecriteriaofthecompositeindicator,thereportactuallydefinestwothresholds:onebeing10percent,asmentionedabove;theotherbeing20percent.Ifthelatterthresholdisapplied,theshareofprojectsachieving“good”SSHintegrationfallsto41percent.AmethodologicaldifficultyconcernsthefactthattheCommissionalsoincludesprojectsfromtheSC6programme,whichcentrearoundSSHresearchbydesign.

(3)WhilethepreviousparagraphwasconcernedwiththequestiontowhatextentSSHintegrationhasbeenenabledbyapplyingtheoppor-tunityof“flagging”ofspecific topics (and, thereby,dedicated fundingbudgets), it is yetanotherstoryhowmuch fundingactuallyendedupinprojectsthathadatleastoneSSHpartneronboard.Tothatend,theannualCommissionreportshavedevelopedausefulcompositeindicator,which allows to better judge the actual SSH integration of each pro-ject. The indicator consistsof four criteria: the shareofSSHpartners;thebudgetgoingtoSSH;andtheperson-monthsbySSHpartnersalltobeabovethethresholdof10percent. Inaddition,thefourthcriterion

isaboutwhethercontributions intheprojectarecomingfromat leasttwoSSHdisciplines.AgoodintegrationofSSHisachievedwhenallfourcriteria are met; with three criteria met, it is “fair”; “weak” with two;and“none”withzero.AccordingtotheCommission’sownassessment(thefine-tunedanalysisonprojectlevelcannotbereproducedwiththeavailabledata),theshareofprojectsfromwithintheflaggedtopicswithgoodSSHintegrationhasrisenfrom2014,with40percent,to56percent, in2017 (Swinnen,Lemaire,andKania2019,6–7).20However,21percenthavenoSSHresearchcomponentwhatsoever.

ISSUE 48 | JULY 2019 19

hasalsobeenexemplarilybeenrealisedbyother,moretechnology-fo-cusedfundinginstrumentsintheHorizon2020portfolio(seeforexampleLangeret al.2016) Importantquestionsremainopen,however.Wedonotknowtheamountofmoneythatwillbespent.GiventhereluctanceofnationalpolicymakerstospendmoremoneyatEuropeanlevel,andthefactthatthepiewillnotgrowsubstantially,powerfullobbieswilldotheirbesttoincreasetheirshare,whichwillleavetheSSHcommunityinperils.

4. CHALLENGES IN SSHSeen from a historical perspective, the social sciences and huma-

nitiesaredeeplyentangledwiththehistoryofthenationstateanditsagencies,withmodernityanditsculturalachievements(Wittrock2000;PorterandRoss2003;Wagner2007;Raphael2012).Asagroup,socialsciences and humanities have proven to be useful by providing tech-niquesandconceptsthathelptoanalyse,understand,andimpactthesocialworld.Withtheprofessionalisationandextensivegrowthofscien-tific(andscholarly)institutions,disciplinesassociatedwithsocialscien-cesandhumanitieshavealwaysalsobeenpartoftheacademicpeckingorder–andhavebeendrawninto,ortakenabackfrom,beingcountedasasocialscientificdiscipline,orahumanitiesdiscipline.

Along thesame line, thehistoryofsocialsciencesandhumanitiesisfullofattemptstodescribetherelationshipwithintheirownepiste-miccommunities,aswellastheirrelationshiptoscience,intermssuchas“nomothetic”vs.“ideographic”,“descriptive”vs.“analytical”,twoorthreeworlds,etc.(Kagan2009;Sala2013).Today,theumbrellatermSSHhasbeenestablished,butwhilethismay(ormaynot)helptoovercomeinfightsbetweendisciplinesandschools,italsodisguisesthedifferen-ces–andresultingfromthis,differentchallenges–thatthenumerousdisciplines,fieldsandschoolsarefacingunderneath.

However:onechallengeremainsthesame,andthatisthefactthat,today,socialsciencesandhumanitiesareincreasinglytreatedthesamewaytheSTEMfieldsare.Thatthisisthecasemaybearguednormatively(to treatall thesameway),but itdoesnotnecessarilymakesense intermsofefficiency–sincethesocialsciencesandhumanitiesarguablyhaveamorecomplexrelationshiptotruth,power,andknowledgethantheir siblings from the sciences. It may well have been useful to finddifferent regimes of funding for different purposes; but this does noteasily comply with fairness, and audits. Interestingly, SSH are treateddifferentlyinsomerespectswhenitcomestocurricula,andapplicationoftheirmethods,concepts,andtheories.Savetoassume,however,thattwocomplementaryforceswereatwork.Availablefunds,andattachedreputationisanattraction.Representativesfromthesocialsciencesandhumanitiesquicklyfelttheurgetoparticipate.Atthesametime,itwasmoreconvenientforpolicy-makerstosetupfundinginawaythatmimi-

Lessons to be learntWhatcanwelearnfromtheseassessments?Certainly,theCommis-

sionhasputalotofeffortsintoenabling,andachieving,integrationofSSH research into theSCprogrammes ofHorizon 2020 (and this is inaddition to the funding for SSH research provided through other inst-rumentsofthiseditionoftheFP).Onapracticallevel,itseemstohavebeenexecuted ina rathermechanisticway.Giventhe immenseappa-ratusthathasbeensetuptoassurethatthemoneyspentthroughtheFrameworkProgrammeislegally,financially,andpoliticallyaccountableandlegitimate,thismaynotbesurprising.Inanycase,itcomeswiththedangerofreifyingsomeofthetraditionalrolesthatSSHhavebeenascri-bed to–mostnotably the tendencyofdelegating thepublic relationsaspectsofacooperativeprojecttoSSHpartners.21AsforthebalanceofSSHdisciplinesandfields,itisobviousnotonlythateconomicsismuchbetterrepresentedintheadvisorygroupsthantheothersocialsciences,whilehumanitiesarebarelyinplaceatall,butalsothatthepredominantshareoffundingfromtheSCprogrammesgoestosocialsciences,na-melyeconomics,politicalscience,publicadministrationandlaw,aswellaseducationandcommunication.Together,thesefewfieldsaccountedfor71per centofall fundinggoing toSSH researchpartners in2017(Swinnen,Lemaire,andKania2019,25).22

However,moresubstantialisthefactthattheexistingarrangementhasmostlypreservedfrompreviouseditionsoftheFPtheoverallfundingthatisactuallygoingtoSSH.Also,thediscussionaboutintegrationofSSHhasenabledimportantresearchprojectsthatdealwiththeSSHatEuropean(thatis,transnational,comparative)level,providingthusmuchnew insightand transnationalexpertiseaswellasnetworks ina fieldthatis,byhistoricaldefinition,ratherdrawntothenationalcontext(anissuethatwillbediscussedfurtherinthenextsection).23

WiththedebateonthenexteditionoftheFP,HorizonEurope,thereisgeneralagreementthatintegrationisreallytakenfromtheheart,andconsideringallcircles.Thishasalsobeenemphasisedbyadvisorydocu-ments,mostnotablytheLamyReport(Lamyet al.2017).Anotherimpor-tantaspectisthattherepresentativesoftheSSHcommunitiesbynowseemtohavemoreexperience,inthesensethattheynowknowbetterwhothepeopleare toaddress,knowhowtheFrameworkProgrammemachineryisrunninginprincipleandthushaveabetterunderstandingwhen,andwhere,tointervene;andalsoknowbetterhowtoarguewithpolicymakers, shiftingaway fromcomplaining tomakingconstructivesuggestions.

Most importantly, the efforts of learning from the past have cometofruition–amongotherinitiatives,thisholdstruetothefactthatthe-re was another Conference (this time under the Austrian Presidency,inNovember2018)dedicatedtodiscussingtheroleofSSH inHorizonEurope,24in a reinvigorated joint platform (now slightly rebranded asEASSH),25andinthecontinuedeffortsbythenetworkofNationalCon-tact Points Net4Society.26 The importance of integrating SSH research

21 Italsocontinuestobeinthemind-setevenofthoseCommissionofficialsthataresympathetictotheideaofSSHintegration.Forexample,thesecondlastassessmentreportstatesthat“althoughresearchintechnologiescanprovidetechnicalsolutionstomajorchallenges,SocialSciencesandHumanities(SSH)canhelpmakingthemaccepted,understoodandappropriatedbythegeneralpublic.”(Strometal.2018,6)

22 Again,notethattheCommissionreportincludesprojectsfromSC6,whichcannotbesubtractedoutduetolackofdata.23 ThoseprojectsareACCOMPLISH,DANDELION,and,asaCOSTaction,onecouldaddtheENRESSHnetwork.24 Seetheprogrammeoftheconference“ImpactofSocialSciencesandHumanitiesforaEuropeanResearchAgenda–ValuationofSSHinmission-oriented

research”underhttps://www.ssh-impact.eu/programme25 SeethewebsiteoftheEuropeanAllianceforsocialSciencesandHumanities,https://www.eassh.eu26 Seethewebsitehttps://www.net4society.eu


Another,finalimportantobservationfromthefieldhasbeenthedy-namicwithinthesocialsciencesandhumanitiestowards“fractaldistinc-tion”(Abbott2001,2015).Becauseofitscomplexity,thereisaninherenttendencywithinthefieldsanalysingthesocialworld(oroneof itsas-pects)tocreateevernewapproaches,questions,focalpoints.WhatmaybecalledparadigmatictheoryaccordingtoThomasKuhn(Kuhn1970)is,inmanybranchesofthesocialsciencesandhumanities,mostoftenonlyshort-lived and quickly disputed internally. Instead of being desperateaboutthis,thisshouldbetakenasafeatureanddealtwithassuch.ItdoesnotmakesensetotrytostyliseSSHinthemannerofotherbran-chesofthescientificenterprise,butratherembrace,acknowledgetheabove-mentionedspecificitiesandbuildonthat.Also,itisimportanttounderstandthatdespitethefactthatSSHcommunitiesoftenresortonthe lower endof thepecking order,SSHbring alongexpertise that isurgentlyneededspecificallyforthetaskoftacklingsocietalchallenges.

5. IMPACT RE-LOADED IN HORIZON EUROPE

Facing the overall ambition of Horizon Europe towards impact ge-neration, an argument for stronger cooperation with and within SSHismadeheretoshiftthefocusawayfrommarginalisationexperiencesandlamentofthepast.Itwasnotbychancethatthescopepaperforthe conference in 2018 was called “impact re-loaded” (König, Nowot-ny,andSchuch2018).Similarly,theconferenceaimedatpracticalSSHGuidelinesdirectedatthosewhodealwithresearchfundingprogram-mes,andspecificallyprogrammesthataimattacklingasocietalproblemthroughthemeansandopportunitiesprovidedbyscientificandscholarlyresearch.

ABOUT THE SSH GUIDELINES

TheconferenceandthisWorkingPaper,togetherwiththeSSHGuide-lines“SocialSciencesandHumanitiesResearchMatters.Guidelinesonhowtosuccessfullydesign,and implement,mission-oriented researchprogrammes”(König2019),intendtobuildonthisposition,andtopushfurtherformoreandbetterintegrationinHorizonEurope.Thisalsome-ansthattherehastobeasubstantialunderstandingwhatSSHresearchisabout,andhowit isproperlytreatedandvalued.Todoso,theSSHGuidelinesconcentrateentirelyonmission-oriented researchprogram-mes. Itdistinguishes foursteps in the life-cycleofsuchaprogramme,namelydesign,implementation,evaluationanddecision-making;anditaddressesallthosepersonswhoplayaroleineitherofthosesteps.

TheideaoftheSSHGuidelinesistoprovideacomprehensive,quick-to-readsetofargumentsforwhySSHshouldbecentralformission-ori-entedresearchprogrammes,andhowtovaluethemproperlyateachofthestepsoftheprogramme’slifecycle.ItprovidesanumberofpracticaltipsforbringingSSH-expertisetothedesignandimplementationofR&I-programmes.Itbuilds,andextends,theextremelyusefulleafletprodu-cedbyNet4Societythathasasimilarambition,albeititwasdirectedat

ckedtheestablishedpathsofsciences.Theresultisthatsocialsciencesandhumanitieshavebeenincreasinglycaughtupinreceivingproject-basedfunding.

Thisisoftenseenasaproblem,andatthelevelofindividualresearchquestions,thismightbejustified.However,SSHresearchfundamentallysharesthesamevaluesasresearchfromotherfields,thatis,toprodu-cerobustknowledgeandtoenhancehumankind;andthatisalsotruewhen it comes to thesocialcontributionsof research.Given this fact,itmaybewellworthtoreassessbrieflyexisting,highlyinstructiveandreflective literatureonthenatureofsocialsciencesandhumanitiestogive credit to the diversity of SSH. By doing so this section also aimsat establishing an argument why and how this diversity can serve asstrength,ratherthanaweakness,forcooperativeresearchthatistaskedtocontributetosolvingsocietalchallenges.

Methods,terms,andconceptshavepermeatedtheacademicworldandchangedthewaypeoplelookattheirlives,societies,andpolities.Fromthispointofview,socialsciencesandhumanitieshavebeenspec-tacularlysuccessfulatleastattwolevels.Oneis,thatthesetechniqueshavebecomestandard requirements forcivil servantsaswellasaspi-ringmembersoftheelite.Andthattheknowledgeproducedbythesetechniques and theoretical presumptions is critical for states, for bu-reaucracies,togovern.Demography,forexample,enablesgovernmentstoassesstheirpopulaceandtoperformoneoftheirmostbasictasks,namelyredistribution(Desrosières1998).WolfgangStreeckhasrecentlyrenewedthisargument,namelythat“thedescriptiveanalysisofsocialrealitybycounting,measuring,observingmightbeofsignificantpracti-calandsocietaluse”(Streeck2011,8).

Justlikethenaturalandlifesciences,aswellasinengineering,thesocial sciences and humanities have considerably contributed to thewaysweunderstandandlookatoursocialworld.Ifitistruethatwhatthenaturaland life sciencesandengineeringhavecontributed toourmodernsocietieshasbecomeinvisible(Shapin2016),thisisevenmoretrue for the social sciences and the humanities, simply because theyhaveamuchcloseranddirect relationship to society (Felt 2000).Dueto the thematicorientationofSSHonmattersofsocial relevance, theboundariesbetweenacademiaandtherestof theworld isevenmoreblurred, which iswhy theacademic social sciences inparticular haveestablishedawayofabstracttheorisingthatisnotonlyoftenhidingabanality,butisalsoperceivedashermetic.27

Thecurrentepistemologicaldebatesaboutsocialsciencesandhu-manities cannot be addressed in full detail here. But it is possible topointtothefollowingissues.Asmentionedbefore,SSHdealswithcon-textualisedknowledge,andisnotsomuchaboutdiscoveringuniversallawsor functionalanalysis,but ratherabout “intentionalexplanation”(Elster1983).Notonlydosocialsciencesandhumanitieshavea“per-formativity”onsocietyoftheirown(MacKenzie,Muniesa,andSiu2008;Boltanski,Esquerre,andMuniesa2015),theyalsoareinseparablefrompoliticalgoals,anditisoftendifficult,albeitimportantasanexerciseinself-reflection, toseparateanalysis fromvalue judgment (Weber1968;Ringer1997).SSHplayan important role inwhatcanbecalled“newknowledge relations” within the scientific disciplines, that is betweentheSSHandtechnosciences,butalsoregardingtherelationoftraditi-onalactorsintheinnovationchainandsocietalactors(Felt2014,394).

27 Thishasbeentreatedwithscornbymanyauthors;exemplarily,seeBillig(2013).

ISSUE 48 | JULY 2019 21

ferently.Threeaspectsshouldbeemphasisedhere:One, itsometimesisworthtotakethestepandsubmitaproposal,eventhoughthismayappeartoberisky.Alongthesameline,itisalsoimportanttobeready,andtomustercapacity,totakeovertheconsortiumcoordination,inor-dertostrongerinfluencethetoneforaproject.Onarelatedmatter, itis crucial to ask for local support infrastructure. SSH sometimes havethedisadvantageofnotbeingsupportedthesamewayastheirSTEMcolleaguesare.

A PRACTICAL WAY FORWARD – FOR SCHOLARS AND POLICY MAKERS

Asapracticalnext step, and takingup themany suggestionsandideasbroughtforwardinvariousmeetingssuchasthe2018conferenceinVienna,representativesofSSHresearchcouldsetupmeetingsatna-tionallevelwiththerespectiveDelegatesinProgrammeCommitteesandNationalContactPoints(NCPs).Asapilot,suchameetingwasorganisedinAustriainMarch2019,withgreatsuccess(seeAnnex).Despitethoseefforts,thenecessaryrequirementstoenableSSHscholarsengaginginthosetwoactivities–designingfundingcallsandparticipatinginprojectapplications–arestillfarfrombeingfullyachieved.Yetbyaddressingtheneedandplayingamorepro-activerole,furtherimprovementistobeexpected,particularlygiventhepositivedevelopmentsatEuropeanlevelinpreparationof“HorizonEurope”.

theintegrationofSSHunderHorizon2020.28Indeed,theSSHGuidelinesintendtomakesuretheeffectivenessofthe ideabehindthe leaflet istakenon,andmadeuseof,basedonanelaborateconsultationprocess,whichstartedseveralmonthsbeforetheconference,resultinginafirstdraftversion,whichwasthensubjecttofurtherdiscussion,andscrutiny,ataspecificallydedicatedondraftingtheSSHGuidelines.Twoadditionalcyclesofconsultationwithnumerousexpertsresultedinthefinalversionofthepolicypaperinmid-January2019.29

SUGGESTIONS FOR SSH SCHOLARS

Complementary to theSSHGuidelines, thissection isdedicated tosomekeysuggestionsforSSHscholarswhosetouttoimprovethedepthandrangeofcooperationinthemission-orientedpartsofHorizonEuro-pe,aswellasotherresearchfundinginstrumentsatEuropean,national,orlocallevel.TheSSHGuidelinesdefinefourspecificstrengthsofSSHindesigningresearchfundingprogrammes:

• theexpertisetocalibratemissions• the capacities of translating between academic disciplines,

policymakers,anddifferentpublics• theexpertise inplacingspecificproblems inbroadercontexts

(combininglocalandglobalperspectives)• andthecapacityofmethodologicalreflexivity.30

Whilethesestrengthsaimatsettingthetoneforpolicymakersandmanagers,italsoprovidesagoodintroductiontotheconcludingsectionofthisWorkingPaper.Itdiscussessomeideasforscholarsandresear-chersfromSSHcommunitiesinordertoadvancetheroleofSSHinPillar2(withthetitle“GlobalChallengesandIndustrialCompetitiveness”)ofthenextFrameworkProgramme,aswellasother(national)researchfun-dingprogrammesthatarededicatedtofundmission-orientedresearchprojects.

Themostimportant,yetoftenoverlookedaspectconcernstheparti-cipationintheprocessofdesigningaresearchfundingprogramme,orresearchfundinginstrument.Aswehaveseen,thenumberofSSHscho-larsintheadvisoryboardsofvariousSocietalChallengesinHorizon2020hasbeenlow.Thisisarealproblem:it is inthisrealmthattheoverallgoalsoftheprogramme,orinstrument,aredefined;hencebringingSSHscholarstothetable iscrucial if interdisciplinarycooperationbetweenSSHandSTEMisreallyexpectedtoleadtonew,relevantknowledge.

FundingcallssometimesrequireSSHresearcherstobecreativeand,when it comes to finding funding opportunities, to look at things dif-

28 https://www.net4society.eu/_media/170110_Factsheet_Expert%20meeting_INTEGRATION_def.pdf(lastaccessed:2018-08-14)Theafore-mentionedpolicydocumentbytheFETAdvisoryGroupalsoprovidessomeimportantsuggestions(Langeretal.2016).

29 Forfeedbackandcommentsduringtheproductiveconsultationprocess,IamgratefultoPaulBenneworth,BasudebChaudhuri,AliceDijkstra,MartinaKa-dunc,AngelaLiberatore,GabiLombardo,StephanieRammel,AngelaSchindler-Daniels,MarcVanholsbeeck,inadditiontothecolleaguesalreadymentionedinfootnote1,aswellasmanyothers.ThesuggestionsintheSSHGuidelineshavebeeninspiredbyvariousdocumentsthat,inrecentyears,startedtotakeacriticalviewonthemetricscraze(Muller2018),highlightingthe“patinaofprecision”(Gingras2016),the“differenttypesofimpact”(Realeetal.2014)the“gatekeepersofhighimpact”(Hicksetal.2015),the“ubiquityofexcellencerhetoric”(Mooreetal.2017).

30 ThissectionispartlyquotedfromtheSSHGuidelines(König2019).


societalchallenges.Otherpresentationswereaboutprovidingkeystati-sticson integrationofSSHintotheClusters (“SocietalChallenges”)of“Horizon2020”,informationonthestateofnegotiationsregarding“Ho-rizonEurope”,andresultsfromtheNovemberConference.Forthelatter,ThomasKönigpointedtowardsthebooklet“SocialSciencesandHuma-nitiesResearchMatters”,acomprehensivesetofguidelinesaddressing“researchprogrammes thatsetoutaspecificgoal to tackleasocietalproblemthroughthemeansandopportunitiesprovidedbyscientificandscholarlyresearch–bothfromSSHandSTEM”.34AllpresentersagreedthatSSHshouldbefurtherintegratedinfutureEUresearchfunding.

InterdisciplinaryandespeciallySSH-aspectshavetobecontributedthroughout the whole development of a framework programme, saidMatthiasReiter-Pázmándy,fromtheinitialnegotiations,totheStrategicPlanningandtheyearlyWorkProgrammes.SpecialattentionhastobepaidtoincludeSSH-researchersinthevariousAdvisoryGroups,inparti-cularintheMissionBoards,butalsointheevaluationpanelsof“Horizon2020”and“HorizonEurope”.ResearchersfromSSHalsoshouldregistertobeavailableasevaluatorsinordertoprovideenoughchoiceforthosewhoconvenethepanels. Inadditiontothat, it is importanttoprovidefora,whereresearchersandpolicymakerscanmeetandexchangeac-rosstheboundariesofdisciplinesandthevarioussectoralpolicyareas.Thiseventdidexactlythat.

Thekeyelementofthemeeting,however,concernedtheremaining90minuteswhichprovidedspacefordiscussionamongparticipants.Tothatend,participantswereseatedononeofsixtables,eachofwhichwasdedicatedtooneof the (prospective) thematicclusters inHorizonEurope.35Theintentionwastobringpolicymakers(theNationalDelega-testothespecificprogrammecommitteesin“Horizon2020”andintheupcoming“HorizonEurope”),supporters(theNCPs)andSSHresearcherstogetheranddiscusshowtobettertakeadvantageofSSHexpertiseindesigningandshapingtherespectivethematiccluster.

SUMMARY OF CLUSTER TABLES

Beforeajointlunchbuffetwasserved,thediscussionsweresumma-risedandpresentedtothefullaudience,alongtwosetsofquestions:

a.In which of the cluster’s topics is specific SSH expertiserequired?

b.What concrete measures can help Delegates and NCPs tofacilitate integration of SSH in the cluster? Here are thesummariesofeachofthediscussiontables:

HEALTH

a.All topics in this clusterare relevant forSSHexpertise;muchdepends on the actual design. “Health systems” might be afocalpointthatworksasa“catchall”.

b.AtEUlevel,moreemphasishastobeonevaluationcriteria,and

ANNEX: SUMMARY OF MEETING “SOCIAL SCIENCES AND HUMANITIES IN HORIZON EUROPE”(byThomasKönig,StephanieRammel,MatthiasReiter-Pázmándy,KlausSchuch,JohannesStarkbaum)

On Friday, March 8, the meeting “Social Sciences and HumanitiesinHorizonEurope” tookplaceon thepremisesof the Institute forAd-vanced Studies (IHS), Vienna. It was a follow-up of the Austrian EUPresidencyConference “ImpactofSocialSciencesandHumanities fora European Research Agenda – Valuation of SSH in mission-orientedresearch”,31whichhadtakenplaceinViennaon28-29November2018.As the current EU Research Funding Programme, “Horizon 2020”,32 iscomingtoanend,anddiscussionsforthenextedition,called“HorizonEurope”,33haveintensified,thereistheneedandopportunitytoengagepolicymakersandSSHrepresentativesatthenationallevel,inordertoopenupspacefordiscussiononhowtobetterinvolveSSHexpertiseinthedraftingprocessof the thematicclustersof“HorizonEurope”.Theclustersaregatheredundertheparamounttitle“GlobalChallengesandIndustrialCompetitiveness”.

The follow-upmeetingwasorganisedbyThomasKönig (IHS),Ste-phanieRammel (FFG),MatthiasReiter-Pázmándy (BMBWF),andKlausSchuch (ZSI). It brought together about fifty people – representativesfromsocialsciencesandhumanitiesinAustria,NationalContactPoints(NCPs)forthedifferentthematicareasaswellaspolicymakersandmi-nistryofficials.

PURPOSE OF THE MEETING

Theinitiativetothemeetingwasdrivenbytwoinsights.Oneisthat,asStephanieRammelmadeclearinherpresentation,integrationofSSHintothethematicresearchfundinginstrumentsofthecurrent“Horizon2020”isanambitiousattempt,butstillfarfrombeingsatisfying.AnotheristhatrepresentativesfromSSHrepeatedlycomplainedthattheyarenotinvolvedintheshaping,anddesigning,offundingcallsandworkpro-grammes.Oncetheremitofacallisdecidedupon,itisdifficulttobringspecificSSHknowledgein–unless,maybe,asanadd-on.GiventhefactthattheFrameworkProgrammeshaveincreasinglybecomealsotempla-tesforresearchfundingprogrammesinthememberstates,onecannotunderestimate the role–bothdirectlyand indirectly– in shaping thestatus,andinvolvement,ofSSHinEuropeanresearchfundinggenerally.

ThemeetingkickedoffwithakeynotebyProf.UlrikeFeltwhopro-videdfoodforthoughtbytalkingabouttheroleofSSHincopingwith

31 https://www.ssh-impact.eu32 https://ec.europa.eu/programmes/horizon2020/en33 https://ec.europa.eu/info/designing-next-research-and-innovation-framework-programme/what-shapes-next-framework-programme_en34 https://www.ssh-impact.eu/guidelines-on-how-to-successfully-design-and-implement-mission-oriented-research-programmes35 Theseventhcluster,called“‘CultureandInclusiveSociety’”,isdealingwithSSH-specifictopics,whichiswhyitwasnotincluded.

ISSUE 48 | JULY 2019 23

areas”)–therearesocialimpacts,conflictpotentials,andmoregenerally,apoliticaleconomytobeanalysed.

b.Technological “solutionism” approaches may not beenough;39real problem solving requires integration of SSH inproblem framingandanalysis.Amoreholistic approach fromstrategytocallsisrequired!Thisalsoimpliesaculturalchange,i.e.inthelanguageusedtodescribeaproblem.

NEXT STEPS

Themeetingwasanexperimentinsofarasnothingsimilarhaseverhappened.Albeittherewaslittletimeforanexhaustiveexchange,deba-teswereinitiatedandthemeetingwasthuswidelyseenasagreatsuc-cess.Theremaybethreereasonsforthat.Oneisthatmission-orientedresearchfundingdemandsexchangeofSSHrepresentativeswithpolicymakersinordertoaligncalls,proposalsandresearchtowardsmissions.AnotheristhatAustrianDelegatesandNCPshaveaninterestinincre-asing theshareof funding that flows fromtheEU level toAustria.Soeveniftheyrepresentclustersthattraditionallystandforamoretechno-scienceorientation,theysharethecoreinterestofSSHrepresentatives.Finally,allthishappensinthecontextofamorepositiveattitudetowardsSSHingeneral,40whichprovidesthebackgroundforthisinitiative.Ha-vingsaidallthis,thereisstillmuchtodo,atnationallevelaswellasatEuropeanlevel.

AT NATIONAL LEVEL

OnewayforwardwouldbeforSSHexperts,nationaldelegates,andNCPstomeetregularlyforfurtherexchange.Thiswouldcertainlysup-port theuptakeofSSHexpertiseononehand,andawareness-raisingand re-orientation on the side of SSH researchers on the other hand,whichallowssettingconcreteaction.ItisnowuptotherespectiveAus-trianinstitutesintheirfieldstotaketheleadandcontinuetheworkthatwasinitiatedinthismeeting.

AT EUROPEAN LEVEL

It is important to highlight this meeting to SSH representatives inotherEUmemberstates,sothattheycanorganisesimilarevents.Also,asharedmeetinginBrusselsonpresentingtheSSH-GuidelineslaterinsummerwouldprovideagoodopportunitytoreportabouttheprogressmadeinAustria.

theparticipantportalhastobemadeuseoftoidentifypotentialpartners.AtAustrianlevel,thematicplatformsshouldbemadeuseofforcooperation(e.g.,ÖPPM36,NetzwerkAltern37),policymakersshouldbestrongeradvocatesforSSH,andexchangeattheleveloftheclustershouldbeintensified.

CIVIL SECURITY FOR SOCIETY

a.SSH is crucial for topics such as radicalisation, terrorism,prevention,andresilience.

b.Since topics are mostly identified by governments, SSHrepresentativesshouldgetincontactwithNCPsandministries.Also,with thenational security researchprogrammeKIRAS,38

thereisalreadyanationalmodelavailableforintegratingSSH.

DIGITAL, INDUSTRY AND SPACE

a.AlltopicswereconsideredimportantforSSHexpertise;thisisparticularlythecaseforAI,BigData,NextGenerationInternet,andDigitalSkills.

b.The Evaluation process is critical, both in terms of skills ofreviewersandtheevaluationcriteria, thesameistruefortheworkprogrammes,andthedeliverablesinthegrantagreement.Inrelationtothe“digitalskills”topic,asortof“meta-SSH”wasemphasised, acting as a support-mechanism for a number ofdifferentresearchprojectsanddealingwiththeirsocialimpact,assessingalsodiscriminatoryaspects,exclusionandfears.

CLIMATE AND ENERGY; MOBILITY

a.AlltopicsacrossthisclusterarerelevantforSSH.b.Evaluationhastobeorganisedinaninterdisciplinarymanner;

move away from techno-economic, sector-specific solutions,towards integrating behavioural insights and socioculturalpractices. SSH can serve as guidance for sectoral policies toimplement R&D-based solutions. Researchers and sectoralpolicymakersshouldstepoutoftheirbubblesandgettogethermoreoften.

BIOECONOMY, FOOD, NATURAL RESOURCES & ENVI-RONMENT

a.There are “areas of connectivity” (“bio economy”, “foodsystems”) and also areas that would require a strongerinvolvementofSSH(demandandsupplyprobleminareassuchas“environmentalobservation”,“agriculture,forestry,andrural

36 https://www.personalized-medicine.at37 http://www.netzwerk-altern.at38 https://www.kiras.at39 Cf.E.Morozov,“ToSaveEverything,ClickHere:Technology,Solutionism,andtheUrgetoFixProblemsthatDon’tExist”,London2013)40 SeeLamyetal.(2017)aswellasMazzucato(2018).


ReportoftheToSecureKnowledgeTaskForceoftheSSRC.NewYork:SocialScienceResearchCouncil.https://www.ssrc.org/to-secure-know-ledge.

Desrosières, A. (1998).The Politics of Large Numbers: A History of Stati-stical Reasoning.HarvardUniversityPress.

Doel, W. (2012).“SocialSciencesandHumanities:EssentialFieldsforEu-ropeanResearchandinHorizon2020.”11.AdvicePaper.Brussels:LERU.

Draux, H. and Szomszor, M. (2017).“TopicModellingofResearchintheArtsandHumanities.AnAnalysisofAHRCGrantApplications.”DigitalResearchReports.ArtsandHumanitiesResearchCouncil (AHRC).htt-ps://doi.org/10.6084/m9.figshare.5621260.v1.

Drotner, K.(2013).“HumanitiesintheSocietalChallenges.12Compel-lingCasesforPolicymakers.”Brussels:ScienceEurope.

EASH.(2011).“Horizon2020:SocialSciencesandHumanitiesResearchProvidesVitalInsightsfortheFutureofEurope.”EASH.October28,2011.http://www.eash.eu/openletter2011/index.php?file=openletter.htm.

Elster, J.(1983).Explaining Technical Change: A Case Study in the Philo-sophy of Science.CambridgeUniversityPress.

Elzinga, A. (2012).“FeaturesoftheCurrentSciencePolicyRegime:Vie-wedinHistoricalPerspective.”Science and Public Policy39(4):416–28.https://doi.org/10.1093/scipol/scs046.

European Commission. (2009).Communication on ‘Reviewing Commu-nity Innovation Policy in a Changing World.’O.J.Vol.C26/12.

European Commission..2010.Europe 2020 Flagship Initiative Innovation Union.O.J.Vol.C121/53.

European Parliament and Council of the European Union. (2013).Regulation (EU) Establishing Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and Repealing Decision No 1982/2006/EC.O.J.Vol.L347/104.

Felt, U. (2000). “Die ‘unsichtbaren’ Sozialwissenschaften:ZurProble-matik der Positionierung sozialwissenschaftlichen Wissens im öffentli-chenRaum.”InSoziologische und historische Analysen der Sozialwissen- schaften,editedbyC.Fleck,177–212.ÖsterreichischeZeitschriftfürSo-ziologie,Sonderband5.Opladen:WestdeutscherVerlag.

Felt, U.(2014).“Within,AcrossandBeyond:ReconsideringtheRoleofSocial Sciences andHumanities in Europe.”Science as Culture 23 (3):384–96.https://doi.org/10.1080/09505431.2014.926146.

Felt, U. and Fochler, M. (2018).“DergesellschaftlicheImpactsozial-wissenschaftlichen Wissens in Österreich: Wirkungswege, Messung,Potentiale.EineexplorativeStudie.”Vienna:RatfürForschungundTech-nologieentwicklung.

Fischer, K. (2008).“ScienceandItsMalfunctions.”Human Architecture: Journal of the Sociology of Self-Knowledge6(2):1–22.

BIBLIOGRAPHY

Abbott, A. (2001).Chaos of Disciplines.Chicago:UniversityofChicagoPress.

Abbott, A.(2015).“NachdemChaos:SelbstähnlichkeitenindenSozi-alwissenschaften.”InSoziologiegeschichte: Wege und Ziele,editedbyC.DayeandS.Moebius,284–307.Berlin:Suhrkamp.

Anderson, P.(2009).The New Old World.London;NewYork:Verso.

Atkinson, T., Berkhout, F., De Grauwe, P., Floud, R., Hooimeijer, P., Honkapohja, S., Höpfinger, S. et al.(2009).“VitalQuestions.TheContri-butionofEuropeanSocialScienc.”PositionPaper.Strasbourg:EuropeanScienceFoundation.

Banchoff, T.(2002).“Institutions,InertiaandEuropeanUnionResearchPolicy.” Journal of Common Market Studies 40 (1): 1–21. https://doi.org/10.1111/1468-5965.00341.

Barry, A., Born, G. and Weszkalnys, G. (2008). “Logics of Inter-disciplinarity.” Economy and Society 37 (1): 20–49. https://doi.org/10.1080/03085140701760841.

Billig, M. (2013). Learn to Write Badly: How to Succeed in the Social Sciences.Cambridge:CambridgeUniversityPress.

Birnbaum, B. I., Keraudren, P., Strom, T. and Vavikis, T. (2017).“Inte-grationofSocialSciencesandHumanitiesinHorizon2020:Participants,BudgetandDisciplines.2ndMonitoringReportonSSH-FlaggedProjectsFunded in 2015 under the Societal Challenges and Industrial Leader-ship Priorities.” Brussels: European Commission. http://ec.europa.eu/research/social-sciences/index.cfm?pg=library&lib=other_pub.

Boltanski, L., Esquerre, A. and Muniesa, F. (2015). “Grappling withtheEconomyofEnrichment.”Valuation Studies3(1):75–83.https://doi.org/10.3384/VS.2001-5592.153175.

Borrás, S. and Radaelli, C. M. (2011).“ThePoliticsofGovernanceAr-chitectures: Creation, Change and Effects of the EU Lisbon Strategy.”Journal of European Public Policy18(4):463–84.https://doi.org/10.1080/13501763.2011.560069.

Börzel, T. A., Risse, T. and Sprungk, C.(2010).“AgainsttheDownsizingof Social Sciences in the EU.” December 9, 2010. http://www.polsoz.fu-berlin.de/en/v/transformeurope/news/allgemeines/2010_Against_Downsizing_Social_Sciences.html.

Braun, D.(2003).“LastingTensionsinResearchPolicy-Making—aDe-legationProblem.”Science and Public Policy30(5):309–21.https://doi.org/10.3152/147154303781780353.

Daston, L., Gray-Little, B., Holt, R., Katznelson, I., King, G., Marrett, C. B., Nelson, A., Prewitt, K., Reed, J. S. and Zegart, A.(2018).“ToSecureKnowledge.SocialSciencePartnershipsfortheCommonGood.”

ISSUE 48 | JULY 2019 25

MuldurandL.Soete,64–77.Cheltenham:EdwardElgarPublishing.

Hessels, Laurens K. and van Lente, H. (2008).“Re-ThinkingNewKnow-ledgeProduction:ALiteratureReviewandaResearchAgenda.”Research Policy37(4):740–60.https://doi.org/10.1016/j.respol.2008.01.008.

Hetel, L., Møller, T-E. and Stamm, J.(2015).“IntegrationofSocialSci-encesandHumanitiesinHorizon2020:Participants,BudgetandDiscip-lines.MonitoringReportonSSH-FlaggedProjectsFundedin2014undertheSocietalChallengesandIndustrialLeadership.”Brussels:EuropeanCommission. https://ec.europa.eu/programmes/horizon2020/en/news/integration-social-sciences-and-humanities-horizon-2020-participants-budget-and-disciplines.

Hicks, D., Wouters, P., Waltman, L., de Rijcke, S. and Rafols, I. (2015).“Bibliometrics: The Leiden Manifesto for Research Metrics.” Nature News520(7548):429.https://doi.org/10.1038/520429a.

Kagan, J. (2009).The Three Cultures: Natural Sciences, Social Sciences, and the Humanities in the 21st Century.CambridgeUniversityPress.

Kamenetzky, J. R.(2013).“OpportunitiesforImpact:StatisticalAnalysisoftheNationalScienceFoundation’sBroaderImpactsCriterion.”Science and Public Policy40(1):72–84.https://doi.org/10.1093/scipol/scs059.

Kastrinos, N.(2010).“PoliciesforCo-OrdinationintheEuropeanResearchArea:AViewfromtheSocialSciencesandHumanities.”Science and Pu-blic Policy37(4):297–310.https://doi.org/10.3152/030234210X496646.

Klein, R.(2011).“UnderstandingEuropeinaGlobalContext:Transitionstowards Innovation Societies.” Non-paper. Brussels. http://www.eash.eu/openletter2011/docs/SSH_GrandChallenge_Draft_140411_olweb.pdf.

König, T.(2016).“PeerReviewintheSocialSciencesandHumanitiesattheEuropeanLevel:TheExperiencesoftheEuropeanResearchCouncil.”InResearch Assessment in the Humanities,editedbyM.Ochsner,S.E.HugandH-D.Daniel,151–63.Cham:SpringerInternationalPublishing.https://doi.org/10.1007/978-3-319-29016-4_12.

König, T.(2017).The European Research Council.Cambridge:PolityPress.

König, T. (2019). “Social Sciences and Humanities Research Matters-GuidelinesonHow toSuccessfullyDesign,and Implement,Mission-Oriented Research Programmes.” Vienna: ZSI. DOI: 10.22163/ fte-val.2019.305.

König, T. and Gorman, M. E. (2016).“TheChallengeofFundingInter-disciplinaryResearch:ALookinsidePublicResearchFundingAgencies.”InThe Oxford Handbook of Interdisciplinarity,editedbyR.Frodeman,J.ThompsonKleinandR.Pacheco,2nded., 513–24.OxfordHandbooks.Oxford:OxfordUniversityPress.

König, T., Nowotny, H. and Schuch, K. (2018). “Impact Re-Loaded.”Non-paper.Vienna.https://www.ssh-impact.eu/impact-re-loaded/.Kuhn, T. S.(1970).The Structure of Scientific Revolutions.2nded.Chica-go:UniversityofChicagoPress.

Flink, T. and Kaldewey, D.(2018).“TheLanguageofSciencePolicyintheTwenty-FirstCentury.WhatComesafterBasicandAppliedResearch?”InBasic and Applied Research: The Language of Science Policy in the Twen-tieth Century,editedbyD.KaldeweyandD.Schauz,251–84.EuropeanConceptualHistory.NewYork:Berghahn.https://www.berghahnbooks.com/downloads/OpenAccess/KaldeweyBasic/9781785338113_OA.pdf.

Frodeman, R.(2010).“Introduction.”InThe Oxford Handbook of Interdis-ciplinarity,editedbyR.Frodeman,J.ThompsonKlein,andC.Mitcham,xxix–xxxix.Oxford:OxfordUniversityPress.

Frodeman, R., Thompson Klein, J. and Pacheco, R. C. S. eds.(2017).The Oxford Handbook of Interdisciplinarity.SecondEdition.OxfordHand-books.Oxford,NewYork:OxfordUniversityPress.

Fuller, S. (2016).“What Is theProblemforWhich Interdisciplinarity IstheSolution?”Journal. Items. Insights fromtheSocialSciences.June7,2016.http://items.ssrc.org/what-is-the-problem-for-which-interdiscip-linarity-is-the-solution/.

Geoghegan-Quinn, M.2011a).“TheFutureofEU-FundedResearchandInnovationProgrammes:AnEmergingConsensus….andaNewName.”Speech SPEECH/11/432. Brussels. EU Press releases database. http://europa.eu/rapid/press-release_SPEECH-11-432_en.htm.

Geoghegan-Quinn, M.(2011b).“TheFutureofSocialSciencesandHu-manitiesinHorizon2020.”SpeechSPEECH/11/741.London.EUPressre-leasesdatabase.http://europa.eu/rapid/press-release_SPEECH-11-741_en.htm.

Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P. and Trow, M.(1994).The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies.London;ThousandOaks,Calif:SAGEPublicationsLtd.

Gingras, Y.(2016).Bibliometrics and Research Evaluation: Uses and Ab-uses.

Godin, B.(2009).The Making of Science, Technology and Innovation Po-licy: Conceptual Frameworks as Narratives, 1945-2005.Montréal:CentreUrbanisation Culture Société de l’INRS. http://www.csiic.ca/PDF/The-MakingOfScience.pdf.

Godin, B. (2015). Innovation Contested: The Idea of Innovation over the Centuries.1.Ed.RoutledgeStudies inSocialandPoliticalThought98.NewYork,London:Routledge,TaylorandFrancisGroup.

Greenberg, D. S.(2001).Science, Money, and Politics: Political Triumph and Ethical Erosion.Chicago:UniversityofChicagoPress.

Guzzetti, L., ed.(2000).Science and Power: The Historical Foundations of Research Policies in Europe.Luxembourg:OfficeforOfficialPublicationsoftheEuropeanCommunities.

Guzzetti, L., ed.(2009).“The‘EuropeanResearchArea’IdeaintheHis-toryofCommunityPolicy-Making.”InEuropean Science and Technology Policy: Towards Integration Or Fragmentation?,editedbyH.Delanghe,U.


Paraskevopoulou, E.(2012).“Non-TechnologicalRegulatoryEffects:Im-plicationsfor InnovationandInnovationPolicy.”Research Policy41(6):1058–71.https://doi.org/10.1016/j.respol.2012.03.018.

Porter, T. M. and Ross, D. eds.(2003).The Modern Social Sciences.TheCambridgeHistoryofScience7.Cambridge;NewYork;Melbourne[etc.]:CambridgeUniversityPress.

Raphael, L. (2012). “Embedding the Human and Social Sciences inWesternSocieties,1880-1980:ReflectionsonTrendsandMethodsofCurrentResearch.”InEngineering Society: The Role of the Human and Social Sciences in Modern Societies, 1880-1980,editedbyK.Brückweh,D.Schumann,R.F.Wetzell,andB.Ziemann,41–56.PalgraveMacmil-lan.

Reale, E., Avramov, D., Canhial, K., Donovan, C., Flecha, R., Holm, P., Larkin, C. et al. (2017). “AReviewof LiteratureonEvaluating theScientific, Social and Political Impact of Social Sciences and Humani-tiesResearch.”Research Evaluation,no.SpecialIssue:1–11.https://doi.org/10.1093/reseval/rvx025.

Reale, E., Nedeva, M., Thomas, D. A. and Primeri, E.(2014).“Evalua-tionthroughImpact:ADifferentViewpoint.”Fteval Journal for Research Technology Policy Evaluation,no.39:36–41.

Reiter-Pázmándy, M. (2017).“APumpingHeartforEuropeanResearch.”PolicyDocument.Vienna.https://era.gv.at/object/document/3300.

Rijcke, S. de, and Rushforth, A. (2015).“ToInterveneorNottoInter-vene;IsThattheQuestion?OntheRoleofScientometrics inResearchEvaluation.”Journal of the Association for Information Science and Tech-nology66(9):1954–58.https://doi.org/10.1002/asi.23382.

Ringer, F. K. (1997).Max Weber’s Methodology: The Unification of the Cultural and Social Sciences.Cambridge:HarvardUniversityPress.

Rosamond, B. (2007).“EuropeanIntegrationandtheSocialScienceofEUStudies:TheDisciplinaryPoliticsofaSubfield.” International Affairs83(1):231–52.

Sala, R.(2013).“One,Two,orThreeCultures?HumanitiesVersustheNaturalandSocialSciencesinModernGermany.”Journal of the Know-ledge Economy4(1):83–97.https://doi.org/10.1007/s13132-012-0124-5.

Sarewitz, D.(2016).“SavingScience.”The New Atlantis49:1–40.

Schögler, R. Y.(2013).“EuropeanUnionResearchFunding.PrioritySet-tingintheSocialSciencesandHumanities.”Graz:UniversityGraz.

Schögler, R. Y., and König, T.(2017).“ThematicResearchFundingintheEuropeanUnion:WhatIsExpectedfromSocialScientificKnowledge-Ma-king?”Serendipities2(1):107.https://doi.org/10.25364/11.2:2017.1.7.

Science Europe. (2013).“EmbeddingSocialSciencesandHumanitiesintheHorizon2020SocietalChallenges.”ScienceEuropePositionState-

Lamy, P., Brudermüller, M., Ferguson, M., Friis, L., Garmendia, C., Gray, I., Gulliksen, J. et al.(2017).“LAB–FAB–APP.InvestingintheEu-ropeanFutureWeWant.”ReportoftheindependentHighLevelGroupon maximising the impact of EU Research & Innovation Programmes.Brussels:EuropeanCommission.doi:10.2777/477357.

Langer, J. M., Makarow, M., Teicher, M., Neves Amoroso, A. C., De Gelder, B., Diamond, D., Floreano, D. et al.(2016).“TheNeedtoInte-gratetheSocialSciencesandHumanitieswithScienceandEngineeringinHorizon2020andBeyond.”PolicyDocument.Brussels:EuropeanCom-mission.

Lebeau, Y. and Papatsiba, V.(2016).“ConceptionsandExpectationsofResearchCollaboration in theEuropeanSocialSciences:ResearchPo-licies, InstitutionalContextsand theAutonomyof theScientificField.”European Educational Research Journal 15 (4): 377–94. https://doi.org/10.1177/1474904116642777.

Lyall, C., Bruce, A., Marsden, W. and Meagher, L. (2013).“TheRoleofFundingAgenciesinCreatingInterdisciplinaryKnowledge.”Science and Public Policy40(1):62–71.https://doi.org/10.1093/scipol/scs121.

MacKenzie, D. A., Muniesa, F. Siu, L.(2008).Do Economists Make Mar-kets?: On the Performativity of Economics. Princeton, N.J.; Woodstock:PrincetonUniversityPress.

Mayer, K., König, T. and Nowotny, H.(2013).“HorizonsforSocialSci-ences and Humanities.” Conference Report. Vilnius: Mykolas RomerisUniversity. http://horizons.mruni.eu/wp-content/uploads/2014/02/ssh_mru_conference_report_final.pdf.

Mazzucato, M. (2018). “Mission-OrientedResearchand Innovation intheEuropeanUnion.AProblem-SolvingApproachtoFuelInnovation-LedGrowth.”Brussels:EuropeanCommission.doi:10.2777/360325.

Merton, R. K.(1957).Social Theory and Social Structure; toward the Co-dification of Theory and Research.Revisedandenlargededition.Glencoe,Ill.:FreePress.

Mingers, J. and Leydesdorff, L.(2015).“AReviewofTheoryandPracti-ceinScientometrics.”European Journal of Operational Research246(1):1–19.https://doi.org/10.1016/j.ejor.2015.04.002.

Moore, S., Neylon, C., Eve, M. P., O’Donnell, D. P. and Pattinson, D. (2017). “‘Excellence R Us’: University Research and the Fetishisati-on of Excellence.” Palgrave Communications 3 (January). https://doi.org/10.1057/palcomms.2016.105.

Muller, J. Z.(2018).The Tyranny of Metrics.

Myklebust, J. P. (2012).“ResearchMinistersDemandKeyRoleforSoci-alSciencesinHorizon2020.”University World News,February26,2012,sec.GlobalEditionIssue210.http://www.universityworldnews.com/ar-ticle.php?story=20120223201123111&mode=print.Nowotny, H. (2016).The Cunning of Uncertainty.Cambridge,UK;Mal-den,Mass.:PolityPress.

ISSUE 48 | JULY 2019 27

ment.Brussels:ScienceEurope.

Shapin, S. (2016). “Invisible Science.” The Hedgehog Review 18 (3).http://www.iasc-culture.org/THR/THR_article_2016_Fall_Shapin.php.

Smith, J.(2003).“ImplementationoftheEuropeanResearchAreaintheSocialandHumanSciences.”DiscussionPaperEUR20601.RTDInfo.Eu-ropean Commission. http://ec.europa.eu/research/social-sciences/pdf/study-era-shs-03_en.pdf.

Stephan, P. E. (2012). How Economics Shapes Science. Cambridge,Mass.:HarvardUniversityPress.

Streeck, W.(2011).“ManWeißEsNichtGenau:VomNutzenDerSozi-alwissenschaften Für Die Politik.” MPIfG Working Paper 09/11. http://www.mpifg.de/pu/workpap/wp09-11.pdf.

Strom, T., Lemaire, C., Zacna, J., Montanez, J. A., and Birnbaum, B. I.(2018).“IntegrationofSocialSciencesandHumanitiesinHorizon2020:Participants, Budget and Disciplines. 3rd Monitoring Report on SSHFlaggedProjectsFundedin2016undertheSocietalChallengesandIn-dustrialLeadershipPriorities.”Brussels:EuropeanCommission.https://publications.europa.eu/en/publication-detail/-/publication/4365f75a-5efe-11e8-ab9c-01aa75ed71a1/language-en.

Swinnen, L., Lemaire, C. and Kania, K. (2019). “IntegrationofSoci-alSciencesandHumanities inHorizon2020.Participants,BudgetandDisciplines:4thMonitoringReportonSSHFlaggedProjectsFundedin2017undertheSocietalChallengesandIndustrialLeadershipPriorities.”Brussels: European Commission. https://publications.europa.eu/en/pu-blication-detail/-/publication/4365f75a-5efe-11e8-ab9c-01aa75ed71a1/language-en.

Ulnicane, I. (2015).“BroadeningAimsandBuildingSupportinScience,Technologyand InnovationPolicy:TheCaseof theEuropeanResearchArea.”Journal of Contemporary European Research11(1):31–49.

Wagner, P. (2007). “Public Policy, Social Sciences, and the State: AnHistorical Perspective.” In Handbook of Public Policy Analysis: Theory, Politics, and Methods,editedbyF.FischerandG.J.Miller,29–40.BocaRaton:CRCPress.

Weber, M. (1968). “Die ‘Objektivität’ sozialwissenschaftlicher und so-zialpolitischer Erkenntnis (1904).” In Gesammelte Aufsätze zur Wissen-schaftslehre, edited by J. Winckelmann, 3rd ed., 146–214. Tübingen:MohrSiebeck.

Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., Jones, R. et al. (2015).“TheMetricTide:ReportoftheIndependentRe-viewoftheRoleofMetricsinResearchAssessmentandManagement.”Reportfromanexpertgroup.London:HEFCE.http://www.hefce.ac.uk/pubs/rereports/Year/2015/metrictide/Title,104463,en.html.

Wittrock, B. (2000).“Modernity:One,None,orMany?EuropeanOriginsandModernityasaGlobalCondition.”Daedalus129(1):31–60.Wladawsky-Berger, I. (2018).“InanEraofTechInnovation,WhispersofDecliningResearchProductivity.”Wall Street Journal,July13,2018,sec.

CIO Journal. https://blogs.wsj.com/cio/2018/07/13/in-an-era-of-tech-innovation-whispers-of-declining-research-productivity/.

Young, M. (2015).“ShiftingPolicyNarrativesinHorizon2020.”Journal of Contemporary European Research11(1):16–30.

AUTHORTHOAMS KÖNIG

Institut für Höhere Studien - Institute for Advanced Studies (IHS)JosefstädterStraße39,Vienna,1080(Austria)E:[email protected]

KEYWORDS: SSHresearch;socialsciences;humanities;researchpolicy;Horizon

2020;HorizonEurope;Europeanintegration


traditionsand lifestyles,andhistoric trajectories.Tocorrectlyidentifyandaddresstheproblem,thosedeep-runningconnec-tionshavetobeanalysedandunderstood.Onecanfindaspectsofpoliticaleconomyandadiversityofsocialandculturaldimen-sions ineach researchcluster,be ithealth,energy transition,climatechange,bio-economy,ortransport.

Similarly,ifscientificresearchisexpectedtoproviderealsolutionsforailmentsofhumansorsocietiesatlarge,it is importanttostudythe–intendedaswellasunintended–impactofinnovationsandtheirpoten-tialreboundeffectsthoroughly.Innovationisanythingnewthatcreatessome form of value – often economic, but not always. Value creationalsohappensbyadopting innovations,which isbasicallyasocialpro-cesswithvarious societal implications. Innovation is thusnot just thebusinessofbusiness,butalsothebusinessofsociety,and,thus,alsoalineofactionforSSH.

Bynow, theuniquesetofexpertise,knowledge,andcapacity thatSSHholds for research in tacklingsocietalchallenges is fullyacknow-ledged:betterunderstandingofthesocialdimensiontothechallengeswefaceneedstobetackledatthesametimeasweseektousetechno-logicaladvancementtosolveproblems.Europehasrealisedtheuntap-pedresourceofSSHresearchandhastheambitiontobecomeaglobalpioneerof “integrating”SSHacross its research fundingprogrammes.TheLamyReporton“HorizonEurope”(starting2021)emphaticallystatesthat“Missions…will,bydesign,fullyintegratesocialsciencesandhu-manities(SSH).”1TheCompetitivenessCounciloftheEuropeanCouncilagreed“thatsocialsciencesandhumanities(SSH)shallplayanimpor-tantroleacrossallclusters”.2Similarly,SSHcommunitieshaveworkedtirelesslyinrecentyearstomakethemselvesusableforaddressingthesocietalchallenges.3

Despite tremendousprogress thatour societieshavemade inrecentdecades,equally challenging tasks remain. Theseso-cietal challenges directly concern the way we interact with

eachotherandourenvironment,thewayweproduceandconsume,andthewayinwhichweconstructandperceivemeaninginouractionsorchangeourbehaviour.

Scientific research is an important driver for economic and socialwell-being. Itprovidesanalyticalcapacityand laysthegroundworkforcreatingrelevantandevidence-basedpolicysolutions.Itisthusnotsur-prisingthatmanyresearchfundingprogrammesaimatputtingvalueinexcellentresearchfortacklingsocietalchallenges.

Cooperation across and beyond different disciplinary backgroundsand with different (methodological, technological, theoretical) know-ledgeprovidesnuanced,multi-layeredanalysesandenablesmitigationofgrandchallenges.That’swhyresearchfundingprogrammesoftenaskspecifically for interdisciplinary approaches, and for experts to look atproblemsfromdifferentperspectives.

SSH RESEARCH IS CRUCIAL FOR SUCCESS OF PROGRAMMES

Infundamentalaspects,researchinsocialsciencesandhumanities(SSH)plays a crucial role in the successof any research fundingpro-grammethataimsattacklingsocietalchallenges:

• Almost all of our current societal problems are influenced bydifferent aspects of politics, social and cultural norms, ethicsand legal frameworks, production and consumption patterns,

THOMASKÖNIGDOI:10.22163/fteval.2019.305

SOCIALSCIENCESANDHUMANITIESRESEARCHMATTERS.GUIDELINESONHOWTOSUCCESSFULLYDESIGN,ANDIMPLEMENT,MISSION-ORIENTEDRESEARCHPROGRAMMES

1 PascalLamyetal.,“LAB–FAB–APP.InvestingintheEuropeanFutureWeWant,”ReportoftheindependentHighLevelGrouponmaximisingtheimpactofEUResearch&InnovationProgrammes(Brussels:EuropeanCommission,July2017),16,doi:10.2777/477357.

2 Seehttp://data.consilium.europa.eu/doc/document/ST-15102-2018-INIT/en/pdf,103 Cf.Net4Society,„KeystosuccessfulintegrationofSocialSciencesandHumanitiesinH2020“,https://www.net4society.eu/_media/170110_Factsheet_Ex-

pert%20meeting_INTEGRATION_def.pdf

ISSUE 48 | JULY 2019 29

FROM “INTEGRATION” TO COOPERATION

Yet so far, existing programmes have not entirely managed to un-leash the full potential of interdisciplinary cooperation between SSHresearchandresearchfromthenaturalsciences, technology,enginee-ring,andmathematics(STEM).Monitoringoffundedprojectsaimingat“integration” provides evidence of mixed results. Serious efforts mustbestrengthenedtocreateabasiswhereSSHandSTEMaddressglobalchallengestogetherandonanequalfooting.Thusfar,SSHisoftenonlybroughtinoncetherespectiveresearchtaskhasalreadybeenframedorevenonlyadded-onattheendofaproject–asifitwereaconsultancyservicetomakepublicslovethetechnologiesthatarebeingdeveloped.However, framingaspecificproblemormissionomitting insights fromSSHmayprovedetrimental,thusintegrationfromtheverybeginningisessential.

Aprogrammewhichidentifiestheconnectednatureoftechnologicalandhumanandsocialdimensionswillhavegonea longway toover-coming the hurdle for being successful and effective. For researchersfromSSHbeingenabledtotrulycooperatewiththeirSTEMcolleagues,theeffortsforachievinganequalfootinghavetobeincreased–attwolevels. One concerns the level of implementing research programmes,ashasbeendonealready.Atthislevel,muchinsightcanbedrawnfromrecentexperiences.Theotherconcernsthelevelofdesigningresearchprogrammes–andthisiswherelittleexperienceexiststhusfarandwhe-remoreactiveinvolvementofSSHcommunitiesisneeded.

Thissuggests twonecessaryavenues forSSH research tohave thebestchanceofmaximisingitscontributiontotacklingsocietalchallenges.

1. STIMULATING AND ENABLING COOPERATION WHEN DESIGNING A RESEARCH FUNDING PROGRAMME

When designing a new research funding programme, or when re-furbishinganexistingone, it isofutmost importance toco-determineagendasandprioritieswithinsightsandexpertisefromSSHresearchersinanatmosphereofmutual respect.Thissection is intendedtospeakdirectlytopolicymakersandmanagersoffundingbodieswho–togetherwithexternalstakeholders–usuallydefinetheoverallgoalsoffundingprogrammes,andwhosetasidebudgetsforfundingresearchtoachievethedesiredgoals.Thosemanagerssettingupsuchprogrammeshavetorecognisethedifferencesofthefieldsandtheirownpotentials,andthattheycanmakeuseofpracticalguidanceforachievingtermsunderwhichsuccessfulcooperationwilloccurandincrease.

WhatdoesSSHresearchbringtothetable?“SSH” covers a broad field of academic disciplines and scientific

areas. Because SSH research is as diverse as our societies, cultures,andeconomiesare, it framesandco-shapestransformativeaspectsofresearchandcontributestointegratingcomplexcross-domainperspecti-vesandstandpoints,includingthosefromotherscientificdisciplinesand

non-academicactors. From thisbreadthanddiversity,wecan identifythemulti-dimensionalstrengthsof research in thesocialsciencesandhumanities:

Theexpertise to calibratemissions,highlightingpriority aspects tofocuson“whatmatters”

Thecapacitiesof translatingbetweenacademicdisciplines,policy-makersanddifferentpublics

• Theexpertiseinplacingspecificproblemsinbroadercontexts,integratingbothlocalandglobalperspectives

• Thelong-standingtraditionofmethodologicalreflexivity,recog-nisingsocialandculturalinfluencesonresearchitself

Practicaltipsforunleashingthefullpotentialofinterdisciplinaryco-operationtotacklesocietalchallenges

• Bring members of different scientific fields to your advisorybodies,andspecificallythosefromSSHresearchfields,toco-determinethegoalsoftheresearchfundingprogrammeyouareabouttoestablish.

• RegardSSHresearchnotasacriticaladd-on,butasavitalcon-tributiontocorrectlyunderstandingtheproblemathand,andforimplementingtheresultingsolutionappropriately.Thisway,cooperationwithSSHresearchwillautomaticallyshiftfrombe-ing“mandatory”tobeingobviousandfruitful.

• Grant respect equally to scientists and researchers from SSHas fromSTEM; trust thediscursivepowersof interdisciplinarynegotiationsandtheexpertiseofSSHresearchinprocessesofco-creation.

• Begenerouswithstipulationsconcerning interdisciplinaryco-operation, as it requires time and space for researchers fromdifferentbackgroundstobecomeacquainted.

2. FOSTERING COOPERATION WHILE IMPLEMENTING A RESEARCH FUNDING PROGRAMME

Oncearesearchfundingprogrammeisimplemented,itismandatorytomakesurethatSSHresearchistakenintoaccount.Thus,thissectionisdirectedspecificallyat

• officers and managers within funding agencies establishingandexecutingfundingprogrammes

• panellistsandreviewersprovidingexpertiseandjudgementfordecision-makingintheexecutionoffundingprogrammes

• independentevaluatorsofthosefundingprogrammes

Ifyouareaprogrammeofficer• Makefundingcallsinclusive!Throughoutthetextofacall,ex-

plainthatthesocialdimensionsofaspecificchallengeneedtobeaddressedalongsideotheraspects.

• Definecriteriathatencouragejurypanellistsandreviewerstoidentify the rightpeople–notnecessarily thosewith the for-mally best track and publication record. Metrics, rankings, or


indicatorsmayserveasmeansfordecision-making,butshouldnotserveasauniversalpanacea.4

• Increase variation! Involve experts (plural!) from SSH in theevaluationproceduresofyourcalls.

• EncourageSSHresearchersproactivelytocompeteforfunding,andtoleadprojectsandconsortia.

• Ifyouareapanellist,orareviewer• Takecontextintoaccount!Localand/orcontextualisedexpertise

fromSSHmaybemorevaluableforaprojectthan“global”rec-ognitionofanyscholar.Don’tfallfortheubiquityofexcellencerhetoric!

• Allowfororiginalproposalsthatinclude,orareledby,SSHre-searchers.RespecttheautonomyofSSHresearcherstobringintheirownwaysofworkingtoprojectstheyareleading.

• Academic disciplines have different sizes and express theirhierarchiesdifferently.Donotbelieve thepatinaofprecision,projectedbymetrics,astheyoftensuggestimpactwherethereisnone.

• Lookbeyondpotential scientific impactandconsideralsopo-tentialtransformativesocietal,economic,political,ecologicalorculturalimpact.

Ifyouareanindependentprogrammeevaluator• Lookoutfordifference!Therearedifferenttypesofimpact,and

that they may be long-term as well as immediate. Ideas andconceptstaketimetorippleoutfrominitialacademiccommuni-tiesintosociety.

• Make sure the programme scope and call texts consider thesocialdimensionsofthesocietalchallengetobetackled,andcomparewiththecallwinningteamsandtheircomposition.

• Accountforthereflexivedimensionoftheprogramme,andlookoutforwhatsocialvaluesareinscribedintotheprogramme.

• Checkthetypesofcooperationthatareprojectedandactuallytakeplace,andtowhatdegreeparticipationandcommunica-tionacrossandbeyonddisciplinesaremadepossible.

TO WHOM ARE THESE GUIDELINES ADDRESSED?

This document is directed at all people who deal – in one way oranother–withresearchfundingprogrammes.Specifically(butnotex-clusively),theseguidelinesaddressresearchprogrammesthatsetoutaspecificgoaltotackleasocietalproblemthroughthemeansandoppor-tunitiesprovidedbyscientificandscholarly research–both fromSSHandSTEM.Theexpectedresearchisoftendescribedas“mission-orien-ted”,albeitusageofthistermdiffers.

Theseguidelineshavebeendrafted,consultedandcompiledbyTho-masKönigonbehalfoftheorganisersoftheAustrianPresidencyoftheCounciloftheEuropeanUnionConferenceon‘Impacts of Social Sciences and Humanities for a European Research Agenda – Valuation of SSH in Mission-Oriented Research’. The conference was supported by projectfundingfromtheEuropeanUnion’sHorizon2020researchandinnova-tionprogrammeundergrantagreementNo814729.Theauthorlikestoacknowledgewithmuchappreciationtheinputoftheconsultedexpertswhocontributedtotheseguidelines.

Citation:König,T. (2019).SocialSciencesandHumanities researchmatters-Guidelinesonhowtosuccessfullydesign,andimplement,mis-sion-orientedresearchprogrammes.AustrianPresidencyoftheCounciloftheEuropeanUnionConferenceon‘ImpactsofSocialSciencesandHumanities for a European Research Agenda – Valuation of SSH inMission-OrientedResearch’organisedbyCentre forSocial Innovation,Vienna.

AUTHORTHOAMS KÖNIG

Institut für Höhere Studien - Institute for Advanced Studies (IHS)JosefstädterStraße39,Vienna,1080(Austria)E:[email protected]

4 Itisunderstoodthatpracticaladviceforreviewingwillbemadeavailable,suchastheSanFranciscoDeclarationonResearchAssessment(https://sfdora.org/read/)andthe“LeidenManifesto”http://www.leidenmanifesto.org/

ISSUE 48 | JULY 2019 31

assessment(seeFigure1).Byprocessismeantthemethodofdeliveringimpact,byassessment itsmeasurement.Theprocessof impactcanbesimplifiedbyreducingittothreequestionswhichallsocialscientistscanaskthemselvesabouttheirresearch,evenwhereitistheoretical:Whoaretheusersofourresearch?HowdoIengagewiththem?Whathasbeen/couldbetheeffectsofthisengagement?Theassessmentofimpactrevol-vesaroundonequestion.Whatistheevidenceoftheseeffects?

Answerstothefourthquestion,whichdefineitsmeasurement,aremoredifficulttoconjure,especiallyevidenceofeffectswhichareinde-pendentof theeffects themselves rather thanduplicatesof them.Aneffectoftheresearchistheintendedorunintendedchange,duedirectlyor indirectlytoanintervention,whereasimpact isperceivedasthein-tended or unintended effects on beneficiaries of the intervention, theimpactonwhichismeasuredbyitseffects.Thisiscircularargumentanditisparticularlytrickytoaccuratelyconnecttheresearch,itseffectsandtheevidenceoftheseeffects.Thisrepeatstheobservationthatmeasu-rement is themostproblematicpartof impact.Whensystems imposethemeasurementofimpact,impactcangetreducedtotheeffectsoftheresearch,andwhenthereisnoindependentevidenceofimpactseparatefromtheseeffects,impactisitsmeasures.

Thisproducesoneofthemajorparadoxesofthecurrentimpactde-bate: themeaningof impact isbroadand inclusivetoenhance itspo-pularity, but its measurement is narrow and exclusive. The process ofimpactanditsassessmentoperateinoppositiontooneanother,withtheinclusivenessofitsmeaningnotresolvingthecomplicationsofitsmea-surement.Asystemthat insists on itsassessment thusendsupbeingheavilycriticisedandpractitionerslosesightofthefeasibilityanddesi-rabilityofdealingwiththeprocessofimpact.Thebabyhasbeenthrownoutwiththebathwater;socialscientistshaverejectedimpactbecauseofthedifficultiesofitsmeasurement.

Ofcourse,theneo-liberalcontextinwhichimpacthasemergedcon-tributes to the suspicion amongst social scientists that it is a wolf insheep’sclothing,deceptive,dangerousanddevouring.Themarketisati-onofsocialscientificknowledge,viaideasof‘impact’,‘use’,‘knowledgetransfer’and‘benefit’,combinewiththeprivatisationofpublicuniversityeducationandenhancedstateregulationofuniversitiesthroughtheau-ditculture,toreinforcemutualsuspicionbetweengovernments,highereducationmanagersandsocialscientists.

Ibelievethedebatethereforeneedstomoveonfromthepublicim-pactofsocialsciencetoitspublicvalue.Publicvalueisavocabularyea-sieraroundwhichtodevelopacommonconversationinordertoconductreasoneddebate.Thus,myargumentisnotoneinsupportofthenarrowimpact agenda that is currently dominating social science and highereducationmanagers.Iwanttobroadenthedebate.

Publicvalueisintegraltotheverynatureofthesocialsciences,sincetheyemergedasseparatedisciplinesoutofmoralphiloso-phyintheeighteenthcenturypreciselyinordertobetterdiagno-

seandimprovethesocialcondition.Engagementwithsocialandhumanprogressandwithimprovementandbettermentmarkssocialscienceasapublicgood.Incidentally,Iwouldsaythesameaboutthehumanities.

Two contemporary threats exist to social science, however, which,again,applyequallytothehumanities.Thefirstistheglobaluniversitycrisis,epitomisedbytheintensificationoftheauditcultureandmarketi-sationinhighereducation.Withrespecttothesocialsciences,Isuggestwe see this threat simultaneously as an opportunity to empower thesocialsciencesinanewformof“publicsocialscience”.Thesecondth-reatistheimpactagenda,whichislinkedtothefirstbuthasdevelopeddynamicsofitsown.Isuggestsocialsciencecanengagepositivelywiththeimpactagendasincetheprocessofimpactiseasytodemonstrateforthesocialsciences.

However,impactisalsoadeeplyflawedapproachtoassessthepublicvalueofsocialscienceresearch.Therearediverseviewsonthemeaning,itisverydifficulttomeasure,evenwithinthepolicyevaluationtraditionforwhichtheideaofimpactslipseasilyoffthepen,andthehostilityge-neratedbytheimpactagenda,associatedasitnegativelywiththeauditculture,hasturnedthedebategangrenousandruledoutthepossibilityofreasonedargument.Difficultiesover itsmeasurementhaveresultedinprioritisingcertainformsofimpactbecausetheycanbemoreeasilymeasured,suchthatmeasurementdrivesthedebate.Impactcanthusbediscriminatory.Thereisaninevitable–almostinherent–biastowardsfavouringresearchwhoseimpactismorereadilydemonstrable;andthismostlybecauseofitsdirectpolicybenefitoruserengagement.

Furthermore, impact is reducible toactivitiesnotdirectlyconnectedtothequalityoftheresearch,forimpactismediatedbyalargenumberofprocesses independentof its findingsand theirquality. These inclu-dethesocialnetworksresearchersareembeddedinforcommunicatingtheirresultsandforengagingwithusers,especiallypowerfulgroups,re-searchers’communicationskillsandtheirpriorrelationshipswiththosewho takeup the results, likepolicymakers, themediaandotherusers,theextenttowhichthefieldisonewherepolicydebateissettledorstilllive,andhowsensitisedusersalreadyaretothepotentialbenefitoftheresearchfindings.Reducingimpacttometrics,likecitationcounts,furtherreinforcestheself-referentialandcoincidentalnatureofimpact.Impactisserendipitous,conditional,involvinghugeelementsofchanceandluck.

Giventhisargument,itmayseemparadoxicalformetosayherethatimpactisasheepinwolves’clothing;itismuchmorewarmandcuddlyandmuchlessdangerousthanitappears.Twodimensionsofimpactmustbedistinguished inorder todemystify it: theprocess of impactand its

JOHND.BREWERDOI:10.22163/fteval.2019.363

THEPUBLICVALUEOFTHESOCIALSCIENCES


Iadvancefiveclaimswithrespecttoimpact:• socialscienceiswellequippedandreadilycapableofdemon-

stratingtheimpactofsocialscienceresearch;• impact, however, is a deeply flawed way of approaching the

publicvalueofsocialscience;• it isnecessarytoshiftthetermsofthedebateawayfromthe

publicimpactofsocialsciencetoitspublicvalue;• valuecanbedeconstructedintoseveraltypeswhichshowthe

diversewaysinwhichthesocialscienceshavevalue;• itispossibletodevelopadefinitionofpublicvaluethatdemon-

stratessocialsciencetobeapublicgood.Thisinvolvesdeconstructionoftheterm‘value’(seeFigure2).There

areatleastthreedifferentmeaningstotheterm:valueasusefulnessandutility;valueasqualityandworth;andvalueasjudgementandevalua-tion.Thefirstwemightcallusevalue,thesecondpricevalue,thethirdnormativevalue.Theypromptfurtherdeconstruction.Usevaluecanbedirect or indirect, price value intrinsic or added (giving us the phrase‘valueadded’)andnormativevaluecanbeprivateorpublic.

Directusevaluedescribesthelevelofusefulnessofanitemunmedi-atedbyotherthings,indirectistheutilityaccordedwhenusedincombi-nationwithotherthings.Usevaluedoesnotnecessarilydiminishwhenitisindirect.Asingleuniversityhasdirectusevaluebutitsindirectuseva-luecanbeenhancedwhensetinrelationtoallotheruniversitieswithinhighereducationasawhole.Intrinsicpricevalueistheworthoftheiteminherentuntoitself,suchasthecostofrunningthesingleuniversityoralltheothersinthehighereducationsystem.Addedpricevaluedescri-bes theworthof thingswhenput touse indirectly, suchas thepricevalueattributedtoastudent’seducationthatutilisestheuniversityortouniversities’contributiontothelocaleconomy,allofwhichuniversitiesnow feverishly estimate to head off criticism. Private normative valuereferstothequalityattributedtoanitembyanindividualintermsofthestatustothemderivedfrompossessingit,publicnormativevaluetothequalityattributedtoitmorewidely,suchasitssocialstatusandculturalsignificance. Personal sentiment can attach immense normative valuetoanitemwhichisoflittlemeaningandstatustootherindividualsorcollectively,andviceversa.

Elementsofuse,priceandnormativevalueareruntogetherincur-rentdebatesaboutimpact,where‘impact’isoftennarrowlyreducedtouse-valueandwhereargumentsaboutthedefiningpurposesofsubjectsis often related exclusively to their economic benefits. The neo-liberalhabitofattachingapricetoeverythingineffectreducesvaluetoprice-value–whatitcosts.Bydevelopinganappropriatesenseofthepurposeofthesocialsciences,itispossibletoestablishadefinitionoftheirvaluethatbroadensitfromeconomicusefulnessandcosts.

Thisconceptualvocabularymeansthatwehavetoassesstheva-lue of the social sciences across different dimensions of value, andthat the assessment of their worth varies accordingly. For example,thisconceptualdeconstructionallowsustoarguethatthevalueofthesocialsciencesisnottobefoundsolelyindirectusevalue(say,eco-nomicusefulness),asifthiscanbeassessedinisolationfromindirectusevalue(say,theireconomicusefulnesswhenassessedinrelationtoother things,suchas theeconomicusefulnessofsocialsciencegra-duates across their working lives, or the indirect use value of socialscienceresearchincombinationwithotherscientificresearch,intheform of medical-social science research, biological and social scien-cesresearch,andclimatechangescienceandthesociologyofclimatechange,andsoon).

Wecanfurtherarguethatthepricevalueofthesocialsciences(theircost to thepublicbudgetsetagainstwhat they realiseby theirdirectusevalue)isaverypoormeasureofvalue.Ifthefocusisonpricevalue,weshouldproperly calculateboth the indirectuse valueof the socialsciences and their “valueadded”price value– theprice valueof thesocialscienceswhenmeasuredbywhattheyaddtotheuse,priceandnormativevalueofotherthings.Thepricevalueofthesocialsciences,forexample,shouldbesetinthecontextofwhattheyaddtothepriceva-luederivedfrom,say,studentexchanges,intellectualtourismandsocialandculturalevents,ortheimpactofsocialscienceresearchontransportpolicy,housing, thewelfarestate, ‘race’ relations,betterhospitalcareforthedying,crimerates,andsoon,andwhataddedpricevalueaccru-esfromhavingpeopleeducatedinthesocialsciences(intermsof,say,socially-informedcitizenry,workforces,communitiesandthelike).Socialscienceresearchoninter-culturalandinter-ethnicrelations,ageingandpopulationdemographics,sport,heritageandsooncanbestressedaspartoftheiraddedpricevalue.

Thismultidimensionalviewofvaluealsomeansthatthenormativevalueofthesocialsciencesisanimportantdimensionequaltotheiruseandpricevalue.Thisisnotjustmeantinthenarrowsenseofwhattheyaddtothequalityoflifeandstatusofindividualseducatedinthesocialsciences or to the lives of people affected by social science research,importantastheseareameasureofprivatenormativevalue.It isthatthevalueofthesocialsciencescanbeassessedbytheircontributiontothesocialvaluestheyhelpgarneranddisseminateinculture,themarketandthestate.

The public normative value of the social sciences, therefore, givesthesocial sciences twoqualitiesagainstwhich their statusshouldbeevaluated:theynotonlygenerateinformationaboutsociety,theyareamediumforsociety’sreproduction.Theyarethewayinwhichsocietycanfindoutaboutitselfandinsodoinggeneratetheideaofsocietyitself.Ifitisthoughtthatthissortofvalueisincalculable,itisnomoresothantheproperenumerationoftheuseandpricevalueofthesocialsciences.

The language of ‘public value’, as distinct from ‘public impact’, ischallengingpreciselybecauseitisnotreducibletomonetarycalculationinthesamewaypriceandusevalueare,whichiswhyestablishingthepublic value of social science is so important for rescuing the debatebackfromthemarketeerswhoreduceeverythingtouseandpricevalue.

Myargumentisthussimpleandclearcut:makingpeopleawareofthemselvesascomprisingasocietyhelpsinthedevelopmentanddisse-minationofkeysocialvaluesthatmakesocietypossible–culturalvaluesliketrust,empathy,altruism,tolerance,compromise,socialsolidarityandsensesofbelonging.Theseeverydayvirtuesassistinsociety’songoingbetterment and improvement. The social sciences help us understandthe conditions which both promote and undermine these values andidentifythesortsofpublicpolicies,behavioursandrelationshipsthatareneededinculture,themarketandthestatetoamelioratetheirabsenceandrestoreandrepairthem.Itisforthesereasonsthatsocialscienceisapublicgood.

ThepublicnormativevalueofthesocialsciencesliesintheirdirectengagementwiththeDNAofsociety–individuals,groups,socialrela-tions, civil society, culture, law, legal governance, the market and thestate.Theyaremodesforunderstandingthemechanismsthroughwhichwelivesociallyandassuchareessentialformakingsociallifepossible.SocialsciencesdissecttheDNAofsocietyandtheinformationthisdis-closeshelpsthemimprovethequalityofsociallife.Assuch,thesocialsciencesexistwithinamoralandethicalframeworkandsimultaneously

ISSUE 48 | JULY 2019 33

helptoconsolidateitastheframeworkwithinwhicheveryoneexistsassocialbeings.

Thisisnottheonlyformofvalue,however.Peoplewhodeclarethesocialsciencesasapublicgoodalsoneedtorecognisethatthenotionofpublicvalueintowhichitfitsismulti-dimensional.‘Economicbenefits’havetoformpartofthevaluenarrativeanduseandpricevalues(?)are

partofthedebateaboutthepublicvalueofthesocialsciences.Thisme-ansarticulatingthatthesocialandculturalrelevanceofsocialscienceresearchhaseconomicutilityinadditiontoitsotherbenefits.Myargu-mentisthatthesocialscienceshavebotheconomicvalueandconstituteapublicgood.Thankyou.

THE TWIN DIMENSIONS OF IMPACT.

THE PROCESS OF IMPACT ________________________________________________________________________________________

Who are the users of my research?

CultureNGOs,civilsociety(nationalandglobal),educatedcitizenry,culturalconsumers,librarians,archivists,schools,media,publicbodies,privateorganizations,charities,individuals,families,etc.

ThestateGovernments(local,devolved,nationalandregional),politicalparties,politicians,policymakers,civilservants,nationalandinterna-tionalstrategists,etc.

Themarketbusiness,industry,tradeunions,consumers,workers,etc.

________________________________________________________________________________________How do I engage with them?

Culturemailinglists,newsletters,website,socialmedia,publictalks,seminars,publications,popularwritingsandjournalism,radio,televi-sion,posters,brochures,conferencesandpresentations,etc.

Thestatepublications,briefingpapersandreports,workshops,talks,popularwriting,presentations,etc.

Themarketsameastheabove

________________________________________________________________________________________What has been/could be the effects of this engagement?

Culturebehaviourandpursuits,understanding,civicandhumanitarianvalues,publicdebate,publicbenefits,sharedbeliefs,healthandwell-being,healthpromotion,schoolperformance,familyrelations,etc.

Thestateevidence-basedpolicy,managementanduseofpublicresources,decision-making,strategicthinking,etc.

Themarketknowledge transfer, spinoff companies,productdevelopment,evidence-basedmarketbehaviourandstrategy,decision-making,managementofeconomicandhumanresources,industrialrelations,consumerbehaviourandchoice,disputemanagement,etc.

THE ASSESSMENT OF IMPACT________________________________________________________________________________________

What is the evidence of these effects?

Culturetake-upofresearch,influenceonbehaviours,beliefs,valuesandcivicpractice,etc.

Thestatepolicy,practice,evaluations,improvedpublicscrutinyandaccountability,etc.

ThemarketKnowledge transfer,policyandpractice inbusinessand industry, strategic thinking, industrial relations, conflictpreventionanddisputemanagement,consumerevaluations,etc.

Figure 1.Thetwindimensionsofimpact.


Figure 2.Typesofvalue.

AUTHORJOHN D. BREWERSenator George J Mitchell Institute for Global peace, Security and Justice, Queen’s University BelfastUniversityRoad,BelfastBT71NN(UK)E:[email protected]

ISSUE 48 | JULY 2019 35

isfarfromtrivialandrequiressoundmethodologicaltools.Thequestionof“how”tointegrateSSHdisciplinesintoconsortiahasnotbeeninthefocusof theEuropeanCommission (EC) so far and remainsmainlyupto thecommitmentandcompetenceof the individualapplicant.2Neit-herhastherebeenabroadreflectiononhowSSHintegrationrelatestootherconceptssuchasinter-andtransdisciplinarity.3Againstthisback-ground,thepresentarticlehighlightsthepotentialofusingtheoreticalandmethodologicalexpertiseinthefieldsofinter-andtransdisciplinarityinordertoimprovetheimpactgeneratingprocessesinSSHresearch.

INTERDISCIPLINARITY, TRANSDISCIPLINARITY AND SSH INTEGRATION

When looking into Horizon 2020 topics one finds references to abroad rangeof collaborative researchapproaches, suchasSSH integ-ration,transdisciplinarity,interdisciplinarity,multidisciplinarity,crossdis-ciplinarity, multisectorality, co-creation and co-design.4 For a commongroundofdiscussion, it iscrucialtodefinethosetermsandclarifytherelationshipsbetweeneachother’s. Inwhat follows, thiswillbedonewithinterdisciplinarity,transdisciplinarityandSSHintegration.

The FAQ section of the participant portal defines interdisciplinarityas“the integration of information, data, techniques, tools, perspectives,

ABSTRACT

The European Commission has made a strong commitment tointegratetheSocialSciencesandHumanities(SSH)acrossHo-rizon2020.Theaimistoenhancetheimpactofactivitiestack-

lingsocietalchallenges.However, thequestiononhowsuchan“SSHintegration”shouldhappen inpracticestill offers room fordiscussion.Therefore,thisarticlefocusesonthemethodologicalchallengesofbrin-ging SSH into collaborative Horizon 2020 projects. It emphasises thatSSHintegrationisaspecialcaseof inter-andtransdisciplinarity,sincedifferent scientific disciplines as well as non-academic stakeholdersareinvolvedintheresearchprocess.Takinginter-andtransdisciplinaryexpertisemoresystematicallyintoaccountmaythuscontributetobothbetterproposalsandimprovedprojectimplementationforHorizon2020andtheupcoming“FrameworkProgrammeHorizonEurope”.

INTRODUCTIONTheideaofSSHintegrationinHorizon2020goesbacktothedecision

oftheEuropeanParliamentandtheCouncilstatingthat“social sciences and humanities will be mainstreamed as an essential element of the ac-tivities needed to tackle each of the societal challenges to enhance their impact.”1Thisstatementsoundsconvincingandseemsintuitivelyplausi-ble.However,thepastyearshaveshownthatsuchan“SSHintegration”

JOËLGRAFDOI:10.22163/fteval.2019.364

BRINGINGCONCEPTSTOGETHER:INTERDISCIPLINARITY,TRANSDISCIPLINARITY,ANDSSHINTEGRATION

1 EuropeanParliamentandCouncil(2013).Regulation(EU)No1291/2013of11December2013establishingHorizon2020,AnnexI,121(URL:https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:347:0104:0173:EN:PDF,lastaccess31.10.2018).

2 ThededicatedmonitoringreportsoftheEuropeanCommissionarelimitingtheirassessmentonhowmanySSHpartnersconsortiainvolveandhowmuchbudgetisdedicatedtoSSHresearchers.Thesefiguresareofhighrelevance.However,itwouldbealsoimportanttoassessthequestiononhowSSHin-tegrationhappened(commonproblemframing,workshopsetc.).Thiswouldrequireaqualitativeratherthanaquantitativeapproach.B.I.Birnbaumetal.(Ed.).(2017).2ndMonitoringreportonSSH-flaggedprojects.Luxembourg:PublicationOfficeoftheEuropeanUnion(URL:https://publications.europa.eu/en/publication-detail/-/publication/acac40f5-e84b-11e6-ad7c-01aa75ed71a1,lastaccess31.10.2018).

3 Itissignificantthata2018topicintheSocietalChallenge6asksfor“lessonsfromthepracticesofinterdisciplinarity”butexplicitlyexcludesdiscussionsonthe“epistemologyofinterdisciplinarity”.

EuropeanCommission2018).WorkProgramme2018–2020:Europe inachangingworld– Inclusive, innovativeand reflectivesocieties, topicGOVERN-ANCE-15-2018:TakinglessonsfromthepracticesofinterdisciplinarityinEurope,54–55(versionof24.07.2018.URL:http://ec.europa.eu/research/partici-pants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-societies_en.pdf,lastaccess31.10.2018).

4 AnexampleforthebroadmixofconceptsistobefoundintheintroductionoftheSocietalChallenge5WorkProgramme.Itasksfor:“achallenge-driven,solutions-oriented,trans-disciplinaryperspectivethatintegratestechnology,businessmodelsandeconomicorganisation,finance,governanceandregula-tionaswellasskillsandsocialinnovation,andinvolvesco-creationofknowledgeandco-deliveryofoutcomeswitheconomic,industrialandresearchactors,publicauthoritiesand/orcivilsociety.”(Myitalics,JG).EuropeanCommission(2018).WorkProgramme2018–2020:Climateaction,environment,resourceefficiencyand rawmaterial, introduction,6 (versionof24.07.2018.URL:http://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-climate_en.pdf,lastaccess31.10.2018).


concepts or theories from two or more disciplines. Disciplines may be from the natural sciences, technology, engineering, economics, social sciences and humanities.”5Thisdefinitionfocusesontheintegrationofknowledgefromdifferentacademicdisciplines.6

EventhoughexplicitreferencestotransdisciplinarityaretobefoundinvariousHorizon2020topictexts,itisquitedifficulttofindanofficialdefinitionbytheEC.Aguidancedocumentforevaluatorspublishedin2014states that “trans-disciplinarity […] refers to approaches and me-thodologies that integrate as necessary (a) theories, concepts, knowledge, data, and techniques from two or more scientific disciplines, and (b) non-academic and non-formalized knowledge. In this way, trans-disciplinarity contributes to advancing fundamental understanding or solving complex problems while fostering multi-actor engagement in the research and innovation process.”7Thisdefinitiongoesbeyondthecollaborationbet-weenscientificdisciplinesbyincludingtheknowledgeofnon-academicstakeholders.8

TheprocessofSSH integrationisexplainedontheparticipantportalasfollows: “Social Sciences and Humanities (SSH) are needed to tackle many of the complex societal challenges addressed in H2020, and contri-butions from one or more of these disciplines are frequently necessary for a successful proposal. These contributions are usually part of an interdisci-plinary approach, involving either:

• collaboration between SSH disciplines and/or,• collaboration between SSH disciplines and non-SSH disciplines

such as natural sciences, medicine and technology.”9

SSH integration can therefore be considered as a specific form ofinterdisciplinarity.10Atthesametime,SSHflaggedtopicsareoftenre-quiringtheinvolvementofnon-academicstakeholders.Inpractice,inte-gratingSSHresearchersthusalsotouchesissuesoftransdisciplinarity.

STATE OF THE ART AND INVOLVEMENT OF EXPERTS

SSHintegrationcanonlycontributetoexcellentscienceiftheinteg-ration process itselfismeetingstateoftheartprinciples.Therefore,exis-tingexpertiseoninter-andtransdisciplinarymethodologiesshouldtobetakenintoaccountbyallstakeholders,i.e.theEC,applicants,“NationalContactPoints”(NCPs),andevaluators.

Suchexpertise is tobe foundatmanydifferent levels.Switzerlandmayserveasanexample.TheSwissAcademiesofArtsandSciencesarehostingtd-net,adedicatedcontactpointforresearchersandfundersinthefieldofinter-andtransdisciplinaryresearchandteaching.11Amongstothers,td-netprovidesaplatformwithconcretetoolsfortheimplemen-tationof inter-and transdisciplinarity.Another institutiondedicated tocollaborativeresearchistheDepartmentofEnvironmentalSystemsSci-enceattheETHZurich,whichincludesthetransdisciplinarylaboratoryTdLab.TdLabaimsat“integrating knowledge and values from different scientific perspectives, as well as from other societal actors”.12

Europehasalotofexpertsoninter-andtransdisciplinarity.13BringingthemtogetherwithHorizon2020applicantscouldleadtoimprovedpro-posalsandbetterprojectimplementation.

4 AnexampleforthebroadmixofconceptsistobefoundintheintroductionoftheSocietalChallenge5WorkProgramme.Itasksfor:“achallenge-driven,solutions-oriented,trans-disciplinaryperspectivethatintegratestechnology,businessmodelsandeconomicorganisation,finance,governanceandregula-tionaswellasskillsandsocialinnovation,andinvolvesco-creationofknowledgeandco-deliveryofoutcomeswitheconomic,industrialandresearchactors,publicauthoritiesand/orcivilsociety.”(Myitalics,JG).EuropeanCommission(2018).WorkProgramme2018–2020:Climateaction,environment,resourceefficiencyand rawmaterial, introduction,6 (versionof24.07.2018.URL:http://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-climate_en.pdf,lastaccess31.10.2018).

5 “HowshouldinterdisciplinarityandstakeholderknowledgebeaddressedandevaluatedinHorizon2020proposals?”.EuropeanCommission(2016).FAQPar-ticipantPortal,ID935(09-02-2016)(URL:https://ec.europa.eu/research/participants/portal/desktop/en/support/faqs/faq-935.html,lastaccess31.10.2018).

6 Suchanunderstandingofinterdisciplinaritycorrespondswiththestateoftheartoftherelevantresearchliterature.G.HirschHadornetal.(2008):TheEmergenceofTransdisciplinarityasaFormofResearch.InG.HirschHadornetal.(Ed),HandbookofTransdisciplinaryResearch(Dordrecht:Springer),19–39,here28.

7 EuropeanCommission(2014).“Howshouldtrans-disciplinaritybeaddressedandevaluatedinproposals?”.GuidanceforevaluatorsofHorizon2020pro-posals: 6 (version 1.0, 15.07.2014, URL: http://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/pse/h2020-evaluation-faq_en.pdf, lastaccess31.10.2018).

8 Thisisinlinewiththestateoftheartofthededicatedresearchcommunity.G.HirschHadornetal.(2008):TheEmergenceofTransdisciplinarityasaFormofResearch.InG.HirschHadornetal.(Ed),HandbookofTransdisciplinaryResearch(Dordrecht:Springer),19–39,here29.

9 EuropeanCommission(2018).“HowshouldSocialSciencesandHumanities(SSH)beaddressedandevaluatedinH2020proposals?”.FAQParticipantPortal,ID938(26.01.2018)(URL:https://ec.europa.eu/research/participants/portal/desktop/en/support/faqs/faq-938.html,lastaccess26.01.2018).

10 SeealsoC.Schmaltz (2016).Multi-andTransdisciplinaryResearch inHorizon2020.Presentationgivenat theNationalNetworkand InformationEvent2016,NCPLifeSciences,Cologne,01June2016,here7.(URL:https://www.healthncp.net/sites/default/files/downloads/Plenar01_Health.pdf,lastaccess31.10.2018).Alsonotethattransdisciplinarityisheredefinedas“creatingaunityofintellectualframeworksbeyondthedisciplinaryperspectives”(6).Thisunderstandingoftheconceptdiffersfromtheonereferredtoabove(Guidanceforevaluators”(2014)).

11 td-net:NetworkforTransdisciplinaryResearch(URL:http://www.transdisciplinarity.ch,lastaccess31.10.2018).12 USYSTdLab(URL:http://www.tdlab.usys.ethz.ch,lastaccess31.10.2018).13 SeealsotheHorizon2020projectACCOMPLISSH(URL:https://www.accomplissh.eu,lastaccess04.12.2018).

ISSUE 48 | JULY 2019 37

EURESEARCH PILOT EVENT ON TRANSDISCIPLINARITY IN HORIZON 2020

InApril2018,Euresearch,theSwissadvisorynetworkon“EuropeanResearchand Innovation”,organisedaneventon transdisciplinarity inHorizon 2020.14 Applicants, coordinators, evaluators and experts dis-cussedonhowtheoreticalandmethodologicalexpertiseintransdiscipli-naritymaycontributetobothbetterproposalsandimprovedimplemen-tationofprojects.

Expertsontransdisciplinarymethodologiesassessedthegeneralde-signofcollaborativeprojectsinHorizon2020.Theyespeciallymentionedtheimportanceofthecommonproblemframingbytheconsortiummem-bers.15Againstthisbackdrop,oneofthemainchallengesconcernstheimplementationoftransdisciplinaryprocesseswithintop-downcalls,asthelatterareoftenstronglypre-definingthescopeoftheprojects.

Ina secondstep,evaluatorsandcoordinatorsofHorizon2020dis-cussedtheconcretepotentialoftransdisciplinaritywithinproposalsandprojectimplementation.Itbecameobviousthatincertainresearchfields(suchas“PublicHealthandSustainableDevelopment”)theuseoftrans-disciplinarytoolsisdailybusiness.Inotherfields,transdisciplinarityhap-pensratherimplicitlyandbylearningbydoing.Asfortheevaluation,allpanellistsagreedonthatanexplicitconsiderationoftransdisciplinaritywouldmakeproposalsmorecredible.However,theinvolvementoftrans-disciplinaryexpertscouldalsoleadtoconflictofaimsassuchexpertsge-nerateadditionalcostsfortheconsortium.Thegoalwouldthereforebetoconvinceevaluatorsthattheincorporationoftransdisciplinaryexpertsinconsortiaisanintegralpartofcollaborativeresearchandinnovation.

HORIZON EUROPE AND MISSION-ORIENTED RESEARCH

InJune2018, theECpublished itsproposalon the “9thEuropeanFrameworkProgrammeHorizonEurope”.Oneofthemainrecommenda-tionsisthepreservationofthethreepillarapproachofHorizon2020.Forthepresentarticle,thepillar“GlobalChallengesandIndustrialCompe-titiveness”isofspecialinterest.Itshould“encouragecross-disciplinary,

cross-sectoral,cross-policyandcross-bordercollaboration inpursuitofthe UN SDGs and the competitiveness of the Union’s industries ther-ein.”16On topof the regular call forproposals “a limited set ofhighlyvisible missions will be introduced. […] Missions, with ambitious buttime-boundandachievablegoals,shouldspeaktothepublicandengageit where relevant. They will be co-designed with Member States, theEuropeanParliament,stakeholdersandcitizens.”17Onthisbasis,itseemsverylikelythatSSHintegration,aswellasinter-andtransdisciplinarity,willplayanimportantroleintheupcoming“FrameworkProgramme”.

However,thereremainsomeopenquestionswhichhavetobecon-sidered.TheroleofSSHisnotexplicitlyaddressedinthedocument.ThemostimportantprogrammeforSSHIntegrationinHorizon2020wasthe“SocietalChallenge”6on“InclusiveSocieties”.Whiletherewillbeasi-milarclusterinHorizonEurope,thebudgetwillbecomparativelylow.18Itremains tobe seenhowmuchbudget from theother clusterswill bededicated toSSH research.As for thequestionof inter- and transdis-ciplinarity, the ideaof co-designing thenewly introducedmissions to-getherwithcitizens is certainlya laudable initiative. There is, though,anobviousareaof tensionbetween the top-downapproachofhighlyprescriptivetopictextsand/ormissionsontheonehandandinter-andtransdisciplinaryprocessesontheotherhand,asthelatterrequirecer-tainopenness.19

CONCLUSION AND RECOMMENDATIONS

From a methodological point of view, SSH integration is a specialcase of interdisciplinarity. As SSH flagged topics often involve non-academicstakeholders,transdisciplinarityisalsoofhighrelevance.TherelationbetweenSSHintegration,interdisciplinarityandtransdisciplina-rityshouldthereforebethoroughlydiscussedandtheresultsshouldbemadeavailablefortheHorizon2020stakeholders.

The EC should reconsider the methodological terminology for col-laborativeprojects,especiallyregardingthetopictexts.Onepossibilitywouldbetoincludebasicconceptsintheglossaryofthe“FundingandTenders Portal”. Somedefinitions are already provided on theHorizon2020FAQsection.Theyare,however,somewhathiddenandincomplete(e.g.noreferencetotransdisciplinarityisgiven).Asforthe“9thEuropeanFrameworkProgrammeHorizonEurope”,theECshouldincludeexpertsoninter-andtransdisciplinarity.Thelattercouldgivevaluableinputson

14 Euresearcheventon“TransdisciplinarityinHorizon2020.ChallengesandApproaches”.Bern,24April2018(URL:https://www.euresearch.ch/en/events/event-detail/showUid/746/,lastaccess31.10.2018).

15 C.Pohletal.(2017).TenReflectiveStepsforRenderingResearchSocietallyRelevant.GAIA26/1(2017),43–51.16 EuropeanCommission(2018).Proposal foraRegulationof theEuropeanParliamentandof theCouncilEstablishingHorizonEurope(COM(2018)435fi-

nal,2018/0224(COD)),17(URL:https://ec.europa.eu/commission/sites/beta-political/files/budget-may2018-horizon-europe-regulation_en.pdf,lastaccess31.10.2018).

17 EuropeanCommission(2018).Proposal foraRegulationof theEuropeanParliamentandof theCouncilEstablishingHorizonEurope(COM(2018)435fi-nal,2018/0224(COD)),10(URL:https://ec.europa.eu/commission/sites/beta-political/files/budget-may2018-horizon-europe-regulation_en.pdf,lastaccess31.10.2018).

18 Theproposalsuggestsaround3billionEuroforaclustercalledInclusiveandSecureSocieties.EuropeanCommission(2018).ProposalforaRegulationoftheEuropeanParliamentandoftheCouncilEstablishingHorizonEurope(COM(2018)435final,2018/0224(COD)),32(URL:https://ec.europa.eu/commission/sites/beta-political/files/budget-may2018-horizon-europe-regulation_en.pdf,lastaccess31.10.2018).

19 C.Pohletal.(2017).TenReflectiveStepsforRenderingResearchSocietallyRelevant.GAIA26/1(2017),43–51;L.vanDroogeandJ.Spaapen(2017).Evalu-ationandMonitoringofTransdisciplinaryCollaborations.TheJournalofTechnologyTransfer(2017)(URL:https://doi.org/10.1007/s10961-017-9607-7,lastaccess31.10.2018).


do’sanddon’tsregardingtheframingofcollaborativeprojects(e.g.re-garding the top-down approach, common problem framing, co-designetc.).

CollaborativeHorizon2020projectsofteninvolvedifferentacademicdisciplinesandnon-academicstakeholders. Inthesecases,applicantsshouldexplicitlyaddressmethodologicalissuesregardingtheintegrationofknowledgeandtheelaborationofcommonresearchandinnovationresults.Theyshouldtake intoaccountthestateoftheartresearchoncorrespondingconceptsand– ifnecessary– involveexpertson inter-andtransdisciplinaritybothfortheproposalwritingandtheprojectim-plementation.

EvaluatorsshouldbethoroughlybriefedaboutSSHintegration,notonlyregardingthenumericalinvolvementofSSHresearchersbutalsore-gardingtheactualprocessofhowthespecificcompetencesareintegra-tedintotheconsortium.Ingeneral,evaluatorsshouldbesystematicallylookingatinter-andtransdisciplinarymethodologies.

“National Contact Points”dealingwithSSHintegrationshouldad-viseapplicantsaboutthebasicchallengesofintegratingtheknowledgeofdifferentstakeholderswithincollaborativeprojects.Dedicatedtools–suchaswebinars,factsheetsetc.–doalreadyexistinsomecases20andshouldfurtherbedeveloped,alsobytheNCPnetworks.Specificeventsastheonedescribedabovemayalsocontributetothediscussion.

REFERENCESBirnbaum, B. I. et al. (Ed.). (2017). 2nd Monitoring report on SSH-flaggedprojects.Luxembourg:PublicationOfficeoftheEuropeanUnion(URL: https://publications.europa.eu/en/publication-detail/-/publica-tion/acac40f5-e84b-11e6-ad7c-01aa75ed71a1,lastaccess31.10.2018).

European Commission (2018). Horizon 2020 Work Programme 2018–2020:Climateaction,environment,resourceefficiencyandrawmaterial,introduction(versionof24.07.2018.URL:http://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-clima-te_en.pdf,lastaccess31.10.2018).

European Commission (2018). Horizon 2020 Work Programme 2018–2020: Europe in a changing world – Inclusive, innovative and reflecti-vesocieties (versionof24.07.2018.URL:http://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-socie-ties_en.pdf,lastaccess31.10.2018).

European Commission (2016). “How should interdisciplinarity andstakeholder knowledge be addressed and evaluated in Horizon 2020proposals?”. FAQ Participant Portal, ID 935 (09-02-2016) (URL: https://ec.europa.eu/research/participants/portal/desktop/en/support/faqs/faq-935.html,lastaccess31.10.2018).

European Commission(2018).“HowshouldSocialSciencesandHuma-nities(SSH)beaddressedandevaluatedinH2020proposals?”.FAQPar-ticipantPortal,ID938(26.01.2018)(URL:https://ec.europa.eu/research/participants/portal/desktop/en/support/faqs/faq-938.html, last access

26.01.2018).

European Commission (2014). “How should trans-disciplinarity beaddressedandevaluatedinproposals?”InGuidanceforevaluatorsofHo-rizon2020proposals:6(version1.0,15.07.2014,URL:http://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/pse/h2020-evaluation-faq_en.pdf,lastaccess31.10.2018).

European Commission(2018).ProposalforaRegulationoftheEuropeanParliamentandoftheCouncilEstablishingHorizonEurope(COM(2018)435 final, 2018/0224 (COD)) (URL: https://ec.europa.eu/commission/sites/beta-political/files/budget-may2018-horizon-europe-regulation_en.pdf,lastaccess31.10.2018).

European Parliament and Council(2013).Regulation(EU)No1291/2013of11December2013establishingHorizon2020,AnnexI(URL:https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:347:0104:0173:EN:PDF,lastaccess31.10.2018).

Hirsch Hadorn, G. et al. (2008).TheEmergenceofTransdisciplinarityasaFormofResearch.InG.HirschHadornetal.(Ed),HandbookofTrans-disciplinaryResearch(Dordrecht:Springer),19–39.

Net4Society4 (n.d.). Factsheet on “Keys to successful integration ofSocial Sciences and Humanities in Horizon 2020” (URL: https://www.net4society.eu/_media/170110_Factsheet_Expert%20meeting_INTEG-RATION_def.pdf,lastaccess31.10.2018).

Pohl, C. et al.(2017).TenReflectiveStepsforRenderingResearchSo-cietallyRelevant.GAIA26/1(2017),43–51.

Schmaltz, C. (2016). Multi- and Transdisciplinary Research in Horizon2020.PresentationgivenattheNationalNetworkandInformationEvent2016,NCPLifeSciences,Cologne,01June2016

van Drooge, L. and Spaapen, J. (2017). Evaluation and MonitoringofTransdisciplinaryCollaborations.TheJournalofTechnologyTransfer(2017) (URL: https://doi.org/10.1007/s10961-017-9607-7, last access31.10.2018).

AUTHORJOËL GRAF

EuresearchEffingerstrasse19,Bern,3008(Switzerland)E:[email protected]

KEYWORDS: Interdisciplinarity,Transdisciplinarity,SSHIntegration

20 See,forinstanceNet4Society4(n.d.).Factsheeton“KeystosuccessfulintegrationofSocialSciencesandHumanitiesinHorizon2020”(URL:https://www.net4society.eu/_media/170110_Factsheet_Expert%20meeting_INTEGRATION_def.pdf,lastaccess31.10.2018).

ISSUE 48 | JULY 2019 39

1. INTRODUCTION Theimportanceofsocialinnovationinsuccessfullyaddressingsocial,

economic,politicalandenvironmentalchallengesofthe21stcenturyisrecognisednotonlywithintheEurope2020strategy,butalsoonaglobalscale.Asanovelapproachtoaddresscomplexproblemsinglobalhealth,socialcare,education,energy,andenvironmentalpolicies,socialinnova-tionhasbeenembracedbystakeholdersandcommunitiesonthelocal,regional and even national level (Franz et al. 2012; Hochgerner 2013.Moulaertetal.2013;Nichollsetal.2015).

Theterm“SocialInnovation”canbetracedbacktotheearly19thcen-tury, longbefore technological-economic connotationsdetermined thecommonunderstandingofinnovation(Godin2012,pp.21).Nevertheless,thereisnosharedunderstandingofsocialinnovationinthesenseofacleardifferentiationfromotherconceptssuchassocialentrepreneurshiporbusinessinnovationbasedonnewtechnologies,organisationalfea-turesandmarketingmodels.Likewise,there isnointegrationofsocialinnovationinacomprehensiveinnovationpolicy(Howaldtetal.2014).

Wedefinesocialinnovationasanewcombinationand/ornewcon-figurationofsocialpracticesincertainareasofactionorsocialcontexts,promptedbycertainactorsorconstellationsofactorsinanintentionaltargetedmannerwiththegoalofbettersatisfyingoransweringneedsandproblemsthanispossibleonthebasisofestablishedpractices.Aninnovationisthereforesocialtotheextentthatit,conveyedbythemar-ketor“non/withoutprofit”,issociallyacceptedanddiffusedthroughoutsocietyorincertainsocietalsub-areas,maybecometransformeddepen-dingonsocietalcircumstances(context)andultimatelyinstitutionalisedas a new social practice or made routine. As every other innovation,“new”doesnotrequireabsoluteorgenuinenovelty:Most innovationsarenewinrelativeterms,i.e.transferredordisseminatedtoanotherre-gion,cityorsocialgrouping,othersectorsandpolicyfields.Moreover,aninnovationtermedsocialinnovationdoesnotnecessarilyprovideimpactthatis“good”forallor“sociallydesirable”inanextensiveandnormativesense.Accordingly,theactors’practicalrationale,socialattributionsforsocialinnovationsaregenerallyuncertain(Howaldt/Schwarz2010).

Witha focusonsocialpractices, their reproductionandchangeasthe central element of sociality, “Social Practice Theories” (SPT) allowfor identifyingthesocialdynamicsofchangeprocesses.Thismodifiedunderstandingofthesocialassocialpracticesopenstheviewontheirreconfigurationasacoreelementofsocialinnovationandsocialchange(Shoveetal.2012).Thesocialworldisthereforecomposedofveryspe-

The social sciences and humanities are deeply involved in the processes that use scientific and scholarly approaches to bring about a better society, difficult as it may be to define it.(Königetal.2018)

ABSTRACT

The paper emphasises the crucial role of social innovation insuccessfullyaddressingsocial,economic,politicalandenviron-mentalchallengesofthe21stcentury.Inthiscontext,theglobal

mappingoftheinternationalresearchprojectSI-DRIVErevealsthecapa-citiesofsocialinnovationstomodifyorevenre-directsocialchangeandtoempowerpeople–i.e.toaddressawidevarietyofstakeholdergroups,aswellasthebroaderpublic,inordertoimprovesocialcohesionandtoallowforsmart,sustainableandinclusivegrowth.Liketechnologicalin-novations,successfulsocialinnovationsarebasedonnumerouspresup-positions and require appropriate infrastructures and resources. Thisincludesanewroleofpublicpolicyandgovernmentforcreatingsuitableframeworkandsupportstructures,theintegrationoftheeconomyandcivilsocietyaswellassupportingmeasuresbyscienceanduniversities(e.g.educationforsocialinnovationperformance,know-howtransfer).

Thisalsoraisesthequestionoftheroleofuniversitiesingeneralandofsocialsciencesinparticularinsocialinnovationprocesses.Itwillbeamajorchallengeforthedevelopmentofsocialinnovationtoensureamuchhigherinvolvementofresearchandeducationfacilities.Intheseprocessessocialscienceswillbechallengedtoredefinetheirfunctionswithregardtoinnovation.Inthepast,innovationresearchinthecontextofsocialscienceshascontributedtoexplainthesocialdimensions,thecomplexity and paradoxes of innovation processes. Henceforth, muchwilldependonrealigningtherangeofcompetenciesofsocialscienceaswellassocialscientistsbycontributingactivelytothedevelopmentofsocial innovation.Against thisbackgroundparticipatoryapproachesthatpromoteinvolvementandempowermentofcivil-societyactorsareindispensable.

JÜRGENHOWALDTDOI:10.22163/fteval.2019.365

NEWPATHWAYSTOSOCIALCHANGE–CREATINGIMPACTTHROUGHSOCIALINNOVATIONRESEARCH


cifically nameable, individual, although interdependent practices ofgovernanceandorganising;partnership;negotiations;self(cf.Reckwitz2003);comfort,cleanlinessandconvenience(Shove2003);workingandnurturing(Hargreavesetal.2013),and;consumption(Brand2010).

Therefore it does not surprise that in the context of the broad so-cialdebatesurroundingsustainabledevelopmentandnecessarysocialtransformationprocesses(Loorbach/Rotmans2010),thequestionoftherelationshipbetweensocialinnovationsandsocialchangebecomesim-portant(Avelinoetal.2014):Howcanprocessesofsocialchangebein-itiatedwhichgobeyondtheillusionofcentralistmanagementconceptstolinksocialinnovationsfromthemainstreamofsocietytotheintendedsocialtransformationprocesses(McGowan/Westley2015)?

Butwhataretheconditionsunderwhichsocialinnovationsflourishandcreate impact?Whoare the stakeholders?Howdo social innova-tionsdiffuseandleadtosocialchange?Againstthisbackground,anewgenerationofEU-fundedprojectsworkedonasoundtheoreticalunder-standingof social innovationand its relation to (transformative) socialchangetocontributetoabetterunderstandingoftheconditionsunderwhichsocialinnovationsdevelop,flourishandfinallyincreasetheirso-cietalimpact(chapter2).

Thisalsoraisesthequestionoftheroleofuniversitiesingeneralandofsocialsciencesinparticularinsocialinnovationprocesses.Itwillbeamajorchallengeforthedevelopmentofsocialinnovationtoensureamuchhigher involvementof researchandeducation facilities (chapter3).Intheseprocessessocialscienceswillbechallengedtoredefinetheirfunctionswithregardtoinnovation.Againstthatbackgroundparticipa-toryapproachesthatpromoteparticipationandempowermentofcivil-society actors are indispensable to increase impact (Howaldt/Schwarz2010)(chapter4).

2. THE GROWING IMPORTANCE OF SOCIAL INNOVATION – “A GLOBAL MAPPING OF SOCIAL INNOVATION INITIATIVES”

Recent years have witnessed the emergence of this new type ofinnovation, as an object of research and development appearing in avarietyofformsandinfluencingourlives.Thereisagrowingconsensusamong practitioners, policy makers and the research community thattechnologicalinnovationsalonearenotcapableofovercomingthesocialandeconomicchallengesmodernsocietiesarefacing.Thisiswhyavastnumberofsocial innovation initiatives indifferentworldregionsprovi-dingnewleversforsolvingproblemsandcontributingtosocialchange,canbeidentified.

Thefirstglobalmappingofsocial innovation initiatives,whichwasconducted in the SI-DRIVE (Social Innovation: Driving Force of SocialChange)project,revealedtheimportanceofsocialinnovationinaddres-singsocial,economic,politicalandenvironmentalchallengesofthe21stcenturyonaglobalscale.Itdemonstratestheneedforsocialinnovati-ontoovercomethe(policyfieldrelated)societalchallengesandsocialdemands. In many policy fields we find a variety of social innovationinitiatives(seefigure1).

Social innovations change the manner in which we live together(shared housing), work (telework), consume (car-sharing), distributewealth(unconditionalbasicincome)ordealwitheconomiccrises(shorttimeworkinsteadoftermination).Socialinnovationsprovidenewformsof collaboration between people (co-working spaces), organisations(private-public-partnerships) and states (agreement on the free move-mentoflabour).Socialinnovationscanemergewithindifferentsectors:in civil society (urban farming), politics (parental leave), and economy

Figure 1.Socialinnovationscrosspolicyfields.

ISSUE 48 | JULY 2019 41

Figure 2.Societalleveltheinitiativeisaddressing.

As figure2 illustrates,most initiativesdonotaddressone societallevel alone, but rather different combinations. At the same time, thesocietal level addressed by the initiatives is varying in the differentpolicy fields with a strong focus on social needs in most of the poli-cy fields, except for “Transport and Mobility” and “Energy Supply”,which both have a stronger orientation towards societal challenges.This result is also reflected in the feedback from policy workshopswhichhighlights thedominantpractice fields:cooperativesandwell-connectedneighbourhoodinitiativesinthefieldof“EnergySupply”aremostlyworkingonanagendawhichgoesbeyondconcreteandlocalso-cialdemands,andsodomobilityclustersofinclusiveness/accessdimen-sionandgreeningmobilityinthefieldof“TransportandMobility”.Globaldevelopmentssuchasoilprices,environmentalchangeandstandardoflivingareconsideredacentraldriverinbothpolicyfields.

Atthesametime,theglobalmappingrevealedthediversityofthechallengesmodernsocietiesarefacingandthecomplexityofinnovationprocesses. The mapping demonstrated that, like technological innova-tions, successful social innovationsarebasedonnumerouspresuppo-sitionsandrequireappropriateinfrastructuresandresources.Moreover,social innovations require specific conditions because they aim at ac-tivating, fostering, and utilising the innovation potential of the wholesociety (BEPA2010). Therefore,newwaysofdevelopinganddiffusingsocial innovations are necessary (e.g. design thinking, innovation labsetc.)aswellasadditionalfarreachingresources,inordertounlockthepotentialofsocialinnovationinsocietyandtoenableparticipationoftherelevantactorsandcivilsociety.

Thisisnotonlyamatterofappropriatefundingbutalsoofnewparti-cipationandcollaborationstructures,co-creationanduserinvolvement,empowermentandhumanresourcesdevelopment(seefigure3).Atten-tionhastobepaidtotheinventionanditsdevelopmentaswellasitsdiffusionandimitation.Fromthis innovationprocessanddevelopmentperspective,resources,capabilitiesandconstraints,driversandbarriersarenotonlyrelevantfortheinventionandimplementation,butalsoforscalinganddiffusionofsuccessfulinnovations.

(microcredits). Inshort:social innovationsinasenseofnewpracticesareomnipresentandcontributetosocialchange.Theestablishmentofnewsocialpracticesdoesplayaprominentroleinmakingmobilitymoreenvironmentallyfriendly,diseaseslessscaryortheenergyturnaroundmoresuccessful.Thehighdiversityofsocialneedsandsocietalchallen-gesaddressedby the initiativesarenot limited toonebutoftenworkacrossseveralpolicyfields.Socialinnovationhasbecomeaubiquitousconcept(Howaldtetal.2016).

Atthesametimetheglobalmappingdemonstratedthecapacitiesofsocialinnovationstomodifyorevenre-directsocialchangeandtoempo-werpeople–i.e.toaddressawidevarietyofstakeholdergroups,aswellasthebroaderpublic,inordertoimprovesocialcohesionandtoallowfor smart, sustainableand inclusivegrowth (Howaldtetal.2018).Themappingempiricallyshowsthatthesocietalandgovernancesystemsinwhichthesocialinnovationsareembeddedarecomplex,theproblemsaddressedaredeeplyrootedinestablishedpracticesandinstitutionsandthatmanyinitiativesaresmallinscale.Therefore,tobetterunderstandthisrelationshipbetweensocialinnovationandsocialchange,thesocialembeddednessofanyinnovationinadensenetworkofexistingpracti-ces,routines,institutionsandcontextconditions,ontheonehand,andinnovationstreams,ontheotherhand,hastobeanalysed.Anysocialinnovationresultsinanoutcomeforthoseinvolved,yettodisseminateanimpactfurtherintosocietydependsonspecificconditionsandmutualresonancebetweenvarioussocialinnovations.Growingsocialnumbersand the range of social innovations may be likely to affect pace andperhapsdirectionsofsocialchange.Thus,social innovation ingeneralhasanimpactonsocietaldevelopment,justasinnovationsinbusinessaremeanttohaveanimpactoneconomicdevelopmentandgrowth.Theimpactof social innovationsvaries (ineverycase) from raisingaware-ness,whichisessentialintheideationphaseandthestartingpointofinitiativestocreateandimplementaninnovation,uptotheformationofinstitutions(whichisnotnecessarilythesameasinstitutionalisationofnewinnovativepractices,butoftenrequiredtoensurethesustainabilityofsocialinnovation).Themappingshedlightonthegreatmany,oftennameless but still important, social innovations responding to specificand every-day social demands or incremental innovations (Howaldt etal.2016).

Socialinnovativeprojectsandinitiativesaimtoaddresssocialneedsandsocietalchallengesratherthanfocusingprimarilyoneconomicsuc-cessandprofit.ReferringtoadistinctionintroducedbyBEPA(“BureauofEuropeanPolicyAdvisers”)whosuggeststhat“the output dimension refers to the kind of value or output that social innovation is expected to deliver: a value that is less concerned with mere profit, and including multiple dimensions of output measurement”(2010,p.26)therearethreesocietal levelsonwhichoutputmaytakeplace. Inthisunderstanding,socialinnovations

• “respond to social demands that are traditionally not addressed by the market or existing institutions and are directed towards vulnerable groups in society […],

• tackle ‘societal challenges’ through new forms of relations be-tween social actors, […] respond to those societal challenges in which the boundary between social and economic blurs, and are directed towards society as a whole […],

• or contribute to the reform of society in the direction of a more participative arena where empowerment and learning are both sources and outcomes of well-being”(ibid.,p.29).


Toovercomesocietalchallenges,cross-sectorcollaborationiscrucial,activelyinvolvingpublic,economicandcivilsocietypartners–includingactiveuserorbeneficiaryinvolvementinalmosthalfofthesocialinno-vation initiatives. This shows that most of the initiatives develop newalliances,guaranteecross-sector fertilisationandmobilisecivil society(alsoprovedbythehighnumberofvolunteerssupportingtheinitiatives).

Suchcollaborationsarepickedupbyat least twodifferentheuris-ticmodels, thequadruplehelix (Wallin2010)on theonehand,wheregovernment, industry, academiaandcivil societywork together to co-create the futureanddrivespecific structural changes,and thesocialinnovationecosystem(Sgaragli2014)ontheotherhand(seefigure4),whichalsoasksforinteractionsbetweenthehelixactors,addstheno-tionof systemic complexity and looksat both the serendipity andab-sorptivecapacityofasystemasawhole.Academicknowledgeonsocialinnovationecosystemsisveryscarceandtheconceptisstillfuzzy.

Figure 4.Socialinnovationecosystem.

TheresultsoftheglobalmappingoftheSI-DRIVEprojectdemonstra-tedthatsocialinnovationprocessesandtheunderlyingresources,capa-bilitiesandconstraintsarerelatedtotheactorsofthedifferentsectorsofthesocialinnovationecosystem.Thisincludesanewroleofpublicpolicy

andgovernmentforcreatingsuitableframeworkandsupportstructures,theintegrationofresourcesoftheeconomyandcivilsocietyaswellassupportingmeasuresbyscienceanduniversities(e.g.educationforsoci-alinnovationperformance,know-howtransfer).

Whileprivatecompanies,publicbodiesandNon-GovernmentalOr-ganisations/Non-Profit-Organisations NGOs/NPOs are involved in themajorityofinitiatives,surprisingly,socialenterprisesareengagedonlyinminorpartsoftheinitiatives.Additionally,academiaisonlyapartnerinsomeofthesocialinnovationinitiatives(seefigure5).

Figure 5. Partnersinvolvedintheinitiative.

Themarginalengagementofresearchandeducationfacilities is instrongcontrasttotheiressentialroleasknowledgeprovidersinclassicalinnovation processes (Mowery/Sampay 2005) and as one actor of thetriplehelixmodel.

Figure 3. Cross-cuttingthemesaddressedbytheinitiative.

ISSUE 48 | JULY 2019 43

referto(social)innovations).Hence,aslongasthosewhoworkinthisareaandaimat introducingchangehavenoclearconceptandunder-standingofsocialinnovation,itwillbedifficulttosucceed.

Thisleadsmetoasecondchallenge.Thetopicofsocialinnovationshouldbeintegratedalongthethreemissions.Ontheonehand,socialinnovationisappearingonagrowingnumberofuniversities’agendas,sometimesevenbecominganimportantpartoftheirdevelopmentstra-tegies.Someuniversitiesofferclassesanddegrees,suchasMasterorBachelor. Others focus on research in social innovation. Probably themostcommonwayforuniversitiestoengageinthistopicthatwecanobserveisrelatedtomanifoldactivitieswithinwhatisusuallyreferredtoasthethirdmission(heremainlyunderstoodassocialresponsibility,out-reachandengagement).Ontheotherhand,wecanrarelyseeauniver-sitywheresocialinnovationismajorpartofthestrategyandintegratedinallthreemissions(McKelvey/Zaring2017).Therefore,thechallengeisnotonlytodevelopactivitiesinteaching,researchandthethirdmission.Itistheissueofintegratingsocialinnovationalongthethreemissionsinacomprehensiveway:theworkinevery“mission”needstobeconnec-tedtotheworkinothermissions,sothatitcanbenefitfromtheothers.

Third,therearetwointerrelated,fundamentalcharacteristicsofuni-versitysupportforsocialinnovationthatneedtochange:i)socialinno-vationsupportactivitiestendtobeadhocandlargelyaltruistic,ii)asaresult,whilecommercialinnovationisrecognisedandinstitutionallysup-portedbywell-establishedknowledgetransferoffices, there isnopro-fessionalsupportfunctionwithinuniversitiesforsupportingsocialinno-vation.Untilnow,neithertheinfrastructurenorthefundinghasexistedtomakethispossible,largelybecausegovernmentsandevenuniversityexecutiveshavebeenresistanttothenotionofsocialinnovationasaneffectivesocioeconomicinstrument.Theadoptionofsocialinnovationatapolicylevelbygovernmentsthroughouttheworldiscreatinganenvi-ronmentinwhichinstitutionalsupportforthisareabecomesincreasinglyprevalentwithfunderswillingtoinvestinprojects.

Fourth,thereisachallengeofintegratingboththetop-downandthebottom-upperspective.Usually,whenuniversitiesassumetheirroleassociallyresponsible institutionsregardingtheirenvironment,theystartdeveloping initiatives, which are supposed to favour different targetgroups(e.g.communities).However,suchinitiativestendtobedesignedand implemented from the university’s perspective, missing to involvethetargetgrouprightfromthestart. It isnotsurprisingthenthatpro-jects developed by HEIs do not necessarily respond to the needs, theideasandthevisionsofcommunitiesandothertargetgroups.HEIshaveto learnhowtoworkwith targetgroupsonequal footingandhowtointegratetheirownperspectivewiththelatter’sperspective(Andersonetal.2018).

4. PARTICIPATORY APPROACHES IN SOCIAL INNOVATION RESEARCH

WhilethefutureengagementofHEIsinsocialinnovationiscrucialwithregardtotheimpactofsocialinnovationforsocieties,particularat-tentionshouldbepaidtotheroleofsocialsciences.Itwillbenecessarytoovercomethetraditional“divisionoflabour”ininnovationprocessesbetweennaturalandengineeringsciences,ontheonehand,andsocial

3. THE MISSING LINK – THE ROLE OF “HIGHER EDUCATION INSTITUTES” (HEIS) IN SI-PROCESSES

Thisraisesthequestionoftheroleofuniversitiesingeneralandofsocialsciencesinparticularinsocialinnovationprocesses(seechapter4).Themarginalengagementofresearchandeducationfacilitiesshowninthemapping,isinstrongcontrasttotheiressentialroleasknowledgeprovidersinclassicalinnovationprocessesandasoneactorofthetriplehelixmodel.Thatmeansthatatthistimewefindanuncompletedeco-systemofsocial innovation (quadruplehelix)withone importantpillarmissing.Itwillbeamajorchallengeforthedevelopmentofsocialinno-vationtoensureamuchhigherinvolvementofresearchandeducationfacilities.

Theshiftinfocustowardssocialinnovationmeansmorethanjustta-kingneworotherphenomenaintoaccount.Totheextentthatsomethingnew occurs at the level of social practices and not in the medium oftechnicalartefact,afundamentalconceptualrealignmentininnovationresearch isnecessary. It relates“to living together in communities and society” and concretely means “new forms of participation and social in-tegration, of reconciling interest and social justice as well as individuality and solidarity”(Rammert2010,p.43).

AgainstthatbackgroundtherolethatHEIsareplayinginsocial in-novationhasevolvedinrecentyears.Besidesresearchingtransformati-onprocesses,moreapproachesinwhichscienceitselfisconsideredasanactiveparticipant inprocessesofsocial innovationare increasinglycoming to the fore. Concepts such as “Design Thinking” or “Transfor-mativeResearch”withfocusonactiveparticipationofstakeholdersarebecomingmoreimportantfortheworkofHEIswiththeirenvironments(Schneidewind/Singer-Brodowski 2013). Through transformative re-search,scienceseekstosolvesocietalproblemsbyactivatingprocessesofsocietalchange.Againstthisbackground,thecreationofappropriatestructures(“LivingLabs”andotherspacesforexplorationandlearning)thathelp todevelopknowledgebasedonexperience inorder toesta-blish new social practices has received growing attention and needstobe furtherpromoted.Onlybysensitisingpeopleaboutsocietalpro-blems and possible solutions, HEIs can advance the development ofsocialinnovationwithcommunitymembers.Throughconcepts,suchas“ServiceLearning”or“ExplorativeLearning”,knowledgeandexperienceofstudentsare takenonand linksbetweenacademiaandsocietyaredeveloped,withthelatterbecominganimportantpartnerinadditiontoeconomy.Thisalso includesthequestionofnewmodesofknowledgeproductionandscientificco-creationofknowledgeaimingataninteg-rationofpractitionersandsocialinnovatorsintheinnovationprocesses(Nowotnyetal.2001).

Nevertheless, thereare several challenges thatHEIsneed tomeetinordertoadvanceintheareaofsocialinnovation.First,theyneedtounderstandbetterwhatsocialinnovationis:whilemoreandmoreHEIsrecognisetheimportanceofsocialinnovationforsocietaldevelopmentandtheneedtoengageinthisarea,theydonotnecessarilyunderstandwhatsocialinnovationisexactlyabout(e.g.,itisoftenconfusedwiththeareaof“University’sSocialResponsibility”,whichdoesnotnecessarily


sciencesandthehumanities,ontheotherhand: “Natural and enginee-ring sciences are different from social sciences and the arts primarily in that the former produce innovations or the prerequisites for innovations while the latter reflects on the emergence, the implementation and the success of innovation or also seek to explain the process” (Blättl-Mink2006, p. 31). In the past, innovation research in the context of socialscienceshashelpedtoexplainthesocialdimensions,thecomplexityandparadoxesofinnovationprocesses.

Specificallyinitsanalyticalfunction,researchinthesocialsciencescontributed greatly to conceptually processing the social prerequisi-tesfor innovationandthesocialcharacterof innovationprocesses. Itsstrengthsrestintheanalysisofinnovationprocessesandtheircontex-tualcircumstances.Thefindingspickedupherehavepermeatedsocialconsciousnessdeeply,havedeterminedthethinkingandactionofsocialactorsandhavecontributedsignificantlytoestablishinganewsociologi-callyenlightenedinnovationparadigm.

Shiftingtheperspectiveoninnovationfromtechnicaltosocialinno-vationasanindependenttypeofinnovation,thepresentself-limitationof the social sciences to the concomitant research associated with areferencetothecomplexityandparadoxicallyloadednatureofinnova-tionproves tobe insufficient. For it ishere that the subjectmatterofinnovationitselfrestsimmediatelyinthedisciplinaryperspectiveandtheaffiliatedcapacityforactionandformation.

Intheclassicalprocessofsocialscienceproduction,researchtakesplaceinresearchinstitutionssocietybeinganexcursionformining,anempiricalsourceofdataandinformationbutnotapartner,inthebestcase,alsotheaddressoftransferactivitiesonceresearchisconcluded.Butsocialscienceproductioncanbeseenasasocialproductionofsci-ence. Social actors from the fields of social action relevant to the re-search theme or project participate in the whole process of research.Social scientists are social actors among others with the special taskand role of driving the process towards the production of knowledge,knowledgeachievingvaryingscopesofrelevance: from“simple”prob-lemsolvingwithandforindividualpartnerstogeneralproblemsolutionsinprocessesofsocietal transformation.Practitioners fromcivil society,companies and institutions – all these groups work together creatingnewknowledge.Sothedifferentformsofknowledgecreatedhavetobecombinedandtestedtoevolveintosociallyrobustknowledge(Nowotnyetal.2001).

Purelyanalyticalconceptsfallshortpreciselyinrelationtothespe-cific content of social innovations. After all, as mentioned previously,socialinnovations(incontrasttotechnologicalinnovations)areanaturalsubject of the social sciences in terms of content, and as such socialinnovationcanbenotonlyanalysedandindicatedfromalevelofcom-prehension,butalsobeengenderedand(co)shapedintermsofits(socialandsocietal)preconditions,repercussions,etc..Thus,itishardlysurpri-singthattheroleofthesocialsciencesinexaminingandshapingsocialinnovationisanimportantissueintheinternationalscientificdiscussiononsocialinnovationwithastrongfocusonparticipatoryapproachesthatpromoteparticipationandempowermentofcivil-societyactors(Howaldt/Schwarz2010)1.

4.1 THE CONCEPT OF WOLFGANG ZAPF

AlreadyWolfgangZapfconnectedtheanalysisofthemeaningandspecificsofsocialinnovationswiththequestionabouttheroleandpos-sibilitiesof thesocial sciences in researchingsocial innovations (ZAPF1989,p.182etseq.).Zapfemphasisesthatitispreciselytheapplication-oriented“tools for making decisions [delivered by the social sciences] – forecasts, incremental planning, social experiments, evaluation, practices for mobilisation and motivation – (…) that [can] indeed enhance the abi-lity of modern societies to solve problems and direct themselves” (ibid.,p.183).Zapfdistinguishespotentialcontributionsthesocialsciencescanmaketosocialinnovation:

• decision-makingsupport(surveyresearch,personalitytests,riskassessmentandtechnologyimpact,humanresourcesplanning,etc.),

• sourcesofsocialtechnologies(qualitymanagement,co-deter-minationmodel,grouptherapy),

• approachestogeneraltheoryinordertobetterunderstandin-novationandproductivity(1989.);

Thissortofunderstandingof innovationprocessesrequiresdevelo-ping appropriate forms of co-operation between science and practicethatarenotcentrallyfocusedonthetransferofexpertknowledgeintosocialpractice.Theaimoftheconceptionofco-operationistoorganisetheprocessofchangeitselfasalearningprocessthatfostersthedeve-lopmentandskillsofeveryactorinvolvedandenhancestheirabilitytodetermineandreflect.

4.2 SOCIAL INNOVATION IN LOCAL AND REGIONAL DEVELOPMENT

Oneofthemostprominentareasinwhichtheconceptofsocialinno-vationhasincreasinglybecomearesearchfocusinthesocialsciencesislocalandregionaldevelopment.Itistheurbancontextinwhichchallen-gessuchastheeffectsoftheeconomiccrisis,demographicorclimatechangebecomedirectlyvisibleaspressingsocialdemands.Anditisthecities where unlikely collaborations emerge to tackle problems whennewcompetencesarehandeddownfromnationalorregionallevelswit-houtcorrespondingbudgetallocations(Moulaertetal.2013).

InEurope,aseriesofresearchprojectsdeliveredimportantfindingsontheroleofthelocallevelforsocialinnovation;thelattermainlyview-edundertheperspectiveofthesocialeconomy.Forexample,theproject“Integrated Area Development” (IAD) dealt with challenges faced byneighbourhoodsandprovided“an alternative to the more prevalent forms of market-led economic development”(Moulaertetal.2013b,p.19).Ano-ther important project in order to better understand the role of socialinnovation in community building was SINGOCOM (Social Innovation,Governance and Community Building). Findings from SINGOCOM alsoessentially contributed to the understanding of governance processesonthelocallevel.Forexample,byfocusingonthegovernancestructuresofneighbourhoodmanagement,itwaspossibletodescribeandanalyse

1 Socialinnovationresearchcantherebybuilduponthelongtraditionofparticipatoryapproachesinsocialsciences(e.g.actionresearchetc.)(Gustavsen2012).

ISSUE 48 | JULY 2019 45

howadirectlinkbetweentheneedsanddemandsofexcludedgroupsand the resources to tackle them can be established (Moulaert et al.2005).Itshowedthatsocialinnovationsinvolvedifferentdimensions–suchas the relation toculture,socialconnectionand identity–goingbeyondmaterialandeconomicissues(Moulaertetal.2013b,p.9).

The“InternationalHandbookonSocial Innovation”,publishedbyagroup ledbyFrankMoulaert,presentsa researchperspectiveonsoci-alinnovationthathasbeendevelopedcooperativelyoverthelastthirtyyearsandwhichisintendedtobeacoherentmethodologicalperspecti-vethatdealsbothconceptuallyandpracticallywithstructural,politicaland cultural forces that generate social exclusion. Furthermore, it hasthe potential for social change and socially innovative initiatives, andcombinessocietalwell-beingwiththeshapingandorganisationofsoci-ety(Moulaertetal.2013).Theapproachcentresonathree-dimensionalframeofreferencethatconsistsofthemutuallyassociateddefiningcha-racteristicsofsocialinnovation:satisfyingneedsinthesenseofhumandevelopment,reconfigurationofsocialrelationships,andempowermentorpoliticalmobilisation.Atthesametime,theaimistodevelopandde-monstrateaspecifictypeofsocialinnovationresearchthatseekstofindtherightbalancebetween “research on action”,“action in research”and“research through and by action”(Moulaertetal.2013a,p.6),andthatillustrates the extraordinary importance of social innovation as a fieldbothofresearchandofactionandsocialchange(cf.ibid.,p.5).

Accordingtotheauthors,socialinnovationisaboutacompletelynewontology,whichhas todowithsocialisedchangepractices insteadoforganisationalefficiencyandanoptimiseduseofknowledge.Thisnotionofadifferentontologicalperspectiveandanorientationtowardsacons-titutive,performativeroleofsocialpracticesandtheirtransformativepo-tentialisaninterestingideawhichwouldbeworthfurtherdevelopment.

Inthissense,socialinnovationisanarenaforadeliberatingkindofdecision-makingwithatransformativepower,basedonpoliticalnegot-iationatlocal/regionallevelbypublicscreatedbythepoliticalpowerofsocial movements. In this arena, social innovation researchers can beactiveactors:Socialinnovationresearchbecomesaninteractiveprocessofresearchandaction,startingfromacollectivediscussionanddecisionbyatransdisciplinarygroupregardingtheproblemsofhumandevelop-mentthatshouldbeaddressedandwhichquestionsexplored,whatthecompositionoftheteamshouldbe,andwhatthemeta-theoreticalframeofreferenceshouldlooklike.

4.3 EMPOWERMENT AND DESIGN-THINKING

The BEPA report supports this view when emphasising that socialinnovationshavethefunctionofmobilisingcitizenstotakeanactivepartin innovationprocessesandtherebyenhancesociety’sgeneric innova-tive capacity (BEPA 2010). Here, new models of governance in favourof self-organisation and political participation are required, allowingunexpected results through the involvement of stakeholders.2 If socialinnovationalsohastodowithinnovationinsocialrelations(Moulaertet

al.2013b),thenitcanbeexpectedtobecomewhatformerEC-PresidentBarrosoreferredtoaspartofanewcultureofempowerment(Franzetal.2012).Thisnotionofculturebecomesimportantwhentheconditionsforsocialinnovationsarenotrestrictedtothelevelofactors,butunderstoodasanecosystem,a“complex environment in which social innovations are created, develop and flourish, on the one hand, and take effect or perish, on the other hand”(Eckhardtetal.2017,p.73).

Against this background different concepts of design thinking andrelatedapproacheshavegainedattentionoverthepastyearsinawiderangeofcontextsbeyondthecommunitiesofdesignersanddesignre-searchersincludingthediscussionofsocialinnovation. “The core idea is that the ways professional designers solve problems is useful in different contexts where individuals and groups in economy and society try to in-novate and make change happen. This section reviews the core ideas of the concept of design thinking with regard to social innovation and social change”(SchaperRinkel/Wagner-Luptacik2014,p.97).

In theCritical Literature Report of theSI-DRIVEproject the role ofdesignthinkingininnovationprocessessummarised:

“Designthinkinghasbecomeadominantissueincontemporarydesigndiscourseandrhetoric,especiallywiththedesignthin-kingpracticeofthedesignandinnovationfirmIDEO,andwiththeapplicationofitsconcepttodesigneducationatprestigiousd.school, the Institute of Design at Stanford University (Bjog-vinssonetal.,2012).Themaincharacteristicofdesignthinkingis its approach to think beyond the omnipotent designer andtoovercometheobsessionwithartefacts,products,andthings(Bjogvinssonetal.,2012).Thisisoneoftheinterfacesbetweendesign thinking and social innovation approaches. Designthinkingaspartofdesignstudies includesthecomplexsocialcontext of design to highlight the contradictionbetween uni-quenessofdesignanddesignerasbasisofbusinessmodelsintraditionaldesignandtheconceptoftransferablesolutionsasinsocialinnovationconcepts.Fromthisperspective,designthinkingiscloselyconnectedwithtraditionssuchas“participatorydesign”,“design forchange”(Bjogvinssonetal.,2012,p.101)andsociallyresponsibledesign(Mellesetal., 2011)”. (SchaperRinkel/Wagner-Luptacik2014,p.97)

AsDesertiandothersdemonstrated,differentapproachesofdesignthinkinghavebeendevelopedtopromoteprocessesofsocialinnovationby involvingstakeholders indifferentcontexts (Desertietal.2018,pp.66etseq.).

4.4 TRANSITION RESEARCH AND DESIGN

Social innovationresearchthataddressessystemtransformationorembracesatransitionperspectivelaysastrongfocusonthereorganisa-tionof societyviaparticipation,empowermentandsocial learning (cf.

2 Klein,Fontan,Harrisson,andLévesque(2013)describethedevelopmentoftheQuébecModelassocialinnovationlinkedtosocialtransformation.“Fromthisstandpoint,participativegovernance,co-productionofservicesoractivities,co-constructionofpublicpolicies,aswellasthepluralcharacteroftheeconomy[…]representimportantdimensionsofsocialinnovation”(Klein,Fontan,Harrisson,andLévesque,2013,p.382).Therebytheyidentifythe“economicturn”–“thefactthatsocialmovementshaveswitchedfrommerelydemandingactionsfromothertoproactiveactionsattheeconomiclevel”(Kleinetal.,2013,p.382)–asanimportantsourceforsocialinnovation(Kleinetal.,2013,p.371).


BEPA2010;Elsen/Lorenz2014).Awholeseriesofmoreorlesstheoreti-callyinformedapproachesthatconceptuallyandprogrammaticallyfocusontheshapingorshapeabilityoftransformationprocessesintermsofsustainableandhumandevelopmenthasbeendeveloped.Transforma-tivesocial changehere isnotunderstood tobea largelyuncontrolledoutcomeofgradualevolutionarydevelopments(cf.Osterhammel2011),butratherassomethingwhichcaninprinciplebeshapedbysociety,i.e.“bytheactorsandtheirinnovations”(Schneidewind2013,p.123).Thusheterogeneous,moreorlesstheoreticallyinformedapproaches(tosha-ping)changecometothefore,whichelevateinvestigatingandshapingthetransformationprocessitselfaswellastheincreasingimportanceofsocialinnovationsinthisconnectiontothestatusoftheactuallyrelevanttheme.

However,the “varied use of the term ‘transformation’”(Aderholdetal.2015,p.135)–ascanbeseeninapproachessuchastransitionmanage-ment, transition design, transformation design, social design, and theGreatTransformation–leads “to a conceptual uncertainty”(ibid.,p.135)rather than toa theoreticallygrounded,practicablemodelof transfor-

mation(cf.Howaldt/Schwarz2016,p.43etseq.).Giventheimportanceofsocialinnovationinthesediscourses,asmentionedearlier,ourviewisthatthelackofawell-developedandworkableconceptofsocialin-novationthatgoesbeyondametaphoricaldescriptionofcertainpheno-menaand initiatives isoneof themainreasonsfor thisunsatisfactorysituation3.

Onetransitionapproachwhichwithaviewtosustainabledevelop-mentdirectlyaimsattransformingsocialpracticesandatthesametimeexplicitlyaimstoincludeanddeveloptheoriesofchangeinordertobet-terunderstandthedynamicsofchangeinthesocialandnaturalworld,istransitiondesign(cf.Hopkins2008).Itaimstomobiliseexistingchangepotential in a collaborative process, and emphasises transdisciplinaryandreintegrationaswellastherecontextualisationofknowledge.Itislessabouthavingashapinginfluenceonsocialphenomena,andmoreaboutadeeperunderstandingofspecificenvironments(“ecosystems”),abouttherelationsbetweenitsdifferentparts,whatthespecificneedsare,whatworksandwhatdoesnot,andhowthingscoulddevelopinthefuture(seefigure6).

3 Thelackofasocial-theoryfoundationfortransformationdiscourseisalsoillustratedbythefactthat,withregardtosocialtransformationprocesses,recourseisoftenmadetothemulti-levelperspective(MLP)(cf.Geels2006;Geels/Schott2007)thatwasdevelopedinsocio-technicalinnovationresearch,andthegovernancemodeloftransitionmanagementthatbuildsonit(cf.Loorbach2007).Inthisperspective,systeminnovationsinsocialfunctionalareassuchastransportation,theenergysupply,food,housing,andcommunicationareconsidered(cf.Geels2005).Thesefunctionalareasarecharacterisedbyspecificsocio-technicalsystems.Systeminnovationsemergefrominterlinkeddevelopmentsondifferentlevels.Differentsocietalsectors,actors,practices,(learning)processes,routines,abilities,andrulesplayarolehere,butthisisalwayswithregardtothequestionoftheirinfluenceontheemergence,developmentandestablishmentofnewtechnologies,andsocio-technicalsystemsorregimesthatareshapedasaresult.

Despitevariouscriticisms,theapproachseemstohavelostlittleofitsattractivenessasatheoreticalmodeloftheshapingofsocialtransformationprocesses.However,fromthepointofviewofanunderstandingofsocialinnovationthatisgroundedinpracticetheory,itdoesnotofferanysuitablebasisforanappropriateunderstandingofsocialtransformationprocesses.Itsystematicallyignoresthechangedynamicsofsocialpracticesandisthereforeunabletocapturetheimportanceofsocialinnovationsintransformationprocesses(cf.Avelinoetal.2014).

Figure 6.“TheTransitionFramework”(Irwinetal.2015,p.7).

ISSUE 48 | JULY 2019 47

Oneprominentapplicationoftransitiondesignisthetransitiontownmovement (http://www.transition-initiativen.de) initiated by Rob Hop-kins,andtheembeddedtransitionresearchnetwork(http://www.tran-sitionresearchnetwork.org),whichaims tobring togetherandpromotetransitioninitiativesandtransitionresearch.Aroundtheworld,some500transitioninitiativesarenowregistered,andhaveinitiateddiversesocialinnovationsatlocallevel(https://www.transitionnetwork.org).

Thetransitiontownmovementcanbeinterpretedasaconcretisationofthepost-growtheconomyandeconomyforthecommongood(cf.Pufe2014,p.276).Hereitisnotaquestionoftheories,butofpracticewhichitself“is the intellectual equipment for the process of transforming soci-ety as a whole, for an economy and a society that is on its way into and through the 21st century”(ibid.,p.291).

In the German sustainability discussion, the concept of “transfor-mation design”hasgained importance in recentyears.Transformationdesignbeginswithsmalltransformationexamplesthataffectonlyali-mitednumberofpeopleasexercises inpath-changingand inspirationforsimilarpathchanges,andishereunderstoodasshapinganecessaryprocessoftransformationofthecapitalistgrowtheconomy,i.e.achangeprocessincludeschangingsocialstructurestogetherwiththecorrespon-dingpowerandcontrolstructures(Sommer/Welzer2014).

5. CONCLUSION – RETHINKING SOCIAL SCIENCE

In the increasingdiscussiononsocial innovationnewparticipatoryconceptsforsocialscienceresearchhavebeendeveloped.Whiletherearealotofdifferenceswithregardtothefieldofaction,theirobjectivesandtheaddressedproblemstheseapproachesarebasedontheideaofdevelopingresearchandinnovationprocesswithandforsociety.

Inthesebrieflyoutlinedapproachessocialsciencesarechallengedtoredefinetheirfunctionswithregardtoinnovationandsocietaltransforma-tion.Thisgoesfarbeyondabetterunderstandingofscienceornewcon-ceptsoftransfer,butdeeplyaffectsthetraditionalacademicwaysofknow-ledgeproduction.Newmodesoftheproductionofsocialscienceandthesocialproductionofsciencewillbecomenecessary.“Mode2“hasbeenthelabeltaggedtothisnewlyemergingtypeofknowledgeproductionbyNowotnyetal.(2001)mostlyreferringtonaturalorengineerialsciences.

Thereisalargegapbetweenthetraditionalunderstandingofsocialresearchandscienceand thenewmodeofgeneratingsocially robustknowledgeundertheframeworkconditionsaswehaveoutlinedthem.The new mode of knowledge production will definitely require a tho-roughreviewoftheclassicalqualitycriteriaofwhat isscientificalongwiththedevelopmentofnewconcepts,methods,proceduresandorga-nisationalstructures.Thediscussionaboutsuchaninnovativeapproachtotheproductionofsocialscienceasaprocessofsocialproductioncouldbeveryvaluableforunderstandingthespecificcontributionofthesocialsciencestoprocessesofinnovationandsocietaltransformation4.

Inthepast,innovationresearchinthecontextofsocialscienceshascontributedtoexplainthesocialdimensions,thecomplexityandparado-xesofinnovationprocesses.Henceforth,muchwilldependonrealigning

therangeofcompetenciesofsocialscienceaswellassocialscientistsbycontributingactivelytothedevelopmentandintegrationofinnovationsaswellasbydevelopingsocialinnovation.Thegreatchallengeforcon-temporaryinnovationresearchliesinanalysingitspotentialinthesearchfornewsocialpracticesthatenableustosecurethefutureandallowpeopletolive“a richer and more fulfilled human life”(Rorty2008,p.191).

Againstthatbackgroundparticipatoryapproachesthatpromotepar-ticipation and empowerment of civil-society actors are indispensable.Therequisiteknow-howisfoundnotonlyinthesociologyoftechnology,economicsociology,andorganisationalsociology(cf.Blättel-Mink2006)butalsointhedebateabouttheimportanceofstakeholderinvolvementtoincreasetheimpactofthesocialsciencesandhumanities(Spaapen/van Drooge 2011). This also includes the question of new modes ofknowledgeproductionandscientificco-creationofknowledge(Nowotnyetal.2001)aimingatanintegrationofpractitionersandsocialinnovatorsintheinnovationprocesses(SolerGallart2017).Thereisalotofevidencethatsocialinnovationresearchwillbecomeofgrowingimportancenotonlywithregardtosocial integrationandequalopportunitiesbutalsowithregardtopreservingandexpandingtheinnovativecapacityofso-cietyasawhole.

REFERENCESAderhold, J., Mann, C., Rückert-John, J. and Schäfer, M.(2015).Ex-perimentierraum Stadt: Good Governance für soziale Innovationen aufdem Weg zur Nachhaltigkeitstransformation. Umweltforschungsplandes Bundesministeriums für Umwelt, Naturschutz, Bau und Reaktor-sicherheit, TEXTE 04/2015. Retrieved April 8, 2018 from: https://www.umweltbundesamt.de/sites/default/files/medien/378/publikationen/texte_04_2015_experimentierraum_stadt_good_governance.pdf.

Anderson, M. M., Domanski D. and Howaldt, J. (2018).SocialInnova-tionasachanceandachallengeforHigherEducationInstitutions.InJ.Howaldt,C.Kaletka,A.SchröderandM.Zirngiebl(Ed.),Atlas of Social Innovation. New Practices for a Better Future(pp.50-53).Dortmund:Sozi-alforschungsstelle,TUDortmundUniversity.

Avelino, F., Wittmayer, J., Haxeltine, A., Kemp, R., O’Riordan, T., Weaver, P., Loorbach, D. and Rotmans, J.(2014).Game-changersandTransformativeSocial Innovation.TheCaseof theEconomicCrisisandtheNewEconomy.TRANSIT working paper.RetrievedApril8,2018from:http://www.transitsocialinnovation.eu/content/original/TRANSIT%20outputs/91%20Gamechangers_TSI_Avelino_etal_TRANSIT_workingpa-per_2014.pdf.

BEPA (Bureau of European Policy Advisers) (Ed.) (2010). Empowering People, Driving Change. Social Innovation in the European Union.Luxem-bourg:PublicationsOfficeoftheEuropeanUnion.RetrievedApril8,2018from:http://www.transitsocialinnovation.eu/resource-hub/empowering-people-driving-change-social-innovation-in-the-european-union.

Bjogvinsson, E., Ehn, P. and Hillgren, P.-A.(2012).DesignThingsand

4 AgainstthisbackgroundsocialinnovationresearchcanbeseenasimportantpartinthedebateonResponsibleResearchandInnovationinEurope(Euro-peanCommission2012;vonSchomberg2013).


DesignThinking:ContemporaryParticipatoryDesignChallenges.Design Issues,28(3),101-116.

Blättel-Mink, B.(2006).Kompendium der Innovationsforschung.Wiesba-den:VSVerlagfürSozialwissenschaften.

Brand, K.-W.(2010).SocialPracticesandSustainableConsumption:Be-nefitsandLimitationsofaNewTheoreticalApproach.InM.GrossandH.Heinrichs (Ed.),Environmental Sociology: European Perspectives and Interdisciplinary Challenges(pp.217-235).Dordrecht:Springer.

Deserti, A., Rizzo, F. and Cobanli, O.(2018).FromSocialDesigntoDe-sign for Social Innovation. In J. Howaldt, C. Kaletka, A. Schröder andM.Zirngiebl(Ed.),Atlas of Social Innovation. New Practices for a Better Future(pp.65-68).Dortmund:Sozialforschungsstelle,TUDortmundUni-versity.

Eckhardt, J., Kaletka, C. and Pelka, B. (2017).Inclusionthroughdigitalsocialinnovations.Modellinganecosystemofdriversandbarriers.InM.AntonaandC.Stephanidis(Ed.),Universal access in human - computer interaction, UAHCI 2017 Proceedings, Part I(pp.67-84).Springer.

Elsen, S. and Lorenz, W. (2014).Social innovation, participation and the development of society: Soziale Innovation, Partizipation und die Entwick-lung der Gesellschaft.Bolzano:UniversityPress.

European Commission (2012). Responsible Research and Innovati-on. Europe’s ability to respond to societal challenges. Retrieved Au-gust 10, 2018 from: https://ec.europa.eu/research/swafs/pdf/pub_rri/KI0214595ENC.pdf.

Franz, H.-W., Hochgerner, J. and Howaldt, J.(2012).Challenge Social Innovation: Potentials for Business, Social Entrepreneurship, Welfare and Civil Society.Berlin,NewYork:Springer.

Geels, F.W. and Schot, J.(2007).Typologyofsociotechnicaltransitionpathways.Research Policy,36(3),pp.399-417.

Geels, F.W.(2005).Processandpatternsintransitionsandsysteminno-vations:Refiningtheco-evolutionarymulti-levelperspective.Technologi-cal Forecasting and Social Change,72(6),pp.681-696.

Geels, F.W. (2006).Multi-levelPerspectiveonSystemInnovation.InX.OlsthornandA.J.Wieczorek(Ed.),Understanding Industrial Transforma-tion (pp.163-186).Dordrecht:Springer.

Giesecke, D.(2014).Postwachstum:Geschichten„zurSenkungderAr-beitsmoral“.WSI Mitteilungen7/2014,551-554.

Godin, B. (2012).Social Innovation:Utopiasof Innovationfromc.1830tothePresent.Project on the Intellectual History of Innovation. Working Paper No. 11.Montreal.

Groß, M., Hoffmann-Riem, H. and Krohn, W. (Ed.) (2005).Realexpe-rimente. Ökologische Gestaltungsprozesse in der Wissensgesellschaft.Bielefeld:Transcript.Gustavsen, B.(2012).SocialInnovationandActionResearch.InH.-W.

Franz,J.HochgernerandJ.Howaldt(Ed.),Challenge social innovation: Potentials for business, social entrepreneurship, welfare and civil society (pp.353-366).Berlin,NewYork:Springer.

Hargreaves, T., Haxeltine, A., Longhurst, N. and Seyfang, G.(2013).Up,Down,RoundandRound:ConnectingRegimesandPracticesinIn-novationforSustainability.Environment and Planning,45(2),402–420.

Hochgerner, J. (2013).Social InnovationandtheAdvancementoftheGeneralConceptofInnovation.InC.RuizViñalsandC.ParraRodríguez(Ed.),Social Innovation. New Forms of Organisation in Knowledge-based Societies (pp.12-28).Oxon:Routledge.

Hopkins, R.(2008).The Transition Handbook: from oil dependency to local resilience.WhiteRiver,VT:ChelseaGreen.

Howaldt, J. and Schwarz, M. (2010). Social Innovation: Concepts, Research Fields and International Trends. IMO International Monitoring.Dortmund: Sozialforschungsstelle, TU Dortmund University. RetrievedJune 12, 2018 from: http://www.sfs.tu-dortmund.de/odb/Repository/Publication/Doc/1289/IMO_Trendstudie_Howaldt_Schwarz_englische_Version.pdf.

Howaldt, J. and Schwarz, M. (2016).SI-DRIVE Report: Social Innovation and its Relationship to Social Change.Dortmund:Sozialforschungsstelle,TUDortmundUniversity.RetrievedOctober24,2018fromhttps://www.si-drive.eu/wp-content/uploads/2016/07/SI-DRIVE-D1-3-Social-Change-final-260416-2.pdf.

Howaldt, J., Butzin, A., Domanski, D. and Kaletka, C. (2014). The-oretical Approaches to Social Innovation: A Critical Literature Review.Dortmund: Sozialforschungsstelle, TU Dortmund University. RetrievedMarch 29, 2018: http://www.si-drive.eu/wp-content/uploads/2014/11/D1_1-Critical-Literature-Review_final.pdf.

Howaldt, J., Kaletka, C. and Schröder, A.(2018).SocialInnovationontheRise.ResultsoftheFirstGlobalMapping.InJ.Howaldt,C.Kaletka,A.SchröderandM.Zirngiebl(Ed.),Atlas of Social Innovation. New Practi-ces for a Better Future(pp.12-15).Dortmund:Sozialforschungsstelle,TUDortmundUniversity.

Howaldt, J., Schröder, A., Kaletka, C., Rehfeld, D. and Terstriep, J.(2016).Comparative Analysis (Mapping 1) - Mapping the World of Soci-al Innovation: A Global Comparative Analysis across Sectors and World Regions (D1.4).Dortmund:TUDortmundUniversity.RetrievedMay18,2018 from: https://www.si-drive.eu/wp-content/uploads/2016/07/SI-DRIVE-D1-4-Comparative-Analysis-2016-08-15-final.pdf.

Irwin, T., Kossoff, G., Tonkinwise, C. and Scupelli, P.(2015).Transiti-on Design 2015. A new area of design research, practice and study that proposes design-led societal transition toward more sustainable futures.Pittsburgh.RetrievedOctober24,2018from:http://design.cmu.edu/si-tes/default/files/Transition_Design_Monograph_final.pdf.

Klein, J. L., Fontan, J. M., Harrisson, D. and Lévesque, B.(2013).TheQue-becModel:asocialinnovationsystemfoundedoncooperationandconsen-susbuilding.InF.Moulaert,D.MacCallum,A.Mehmood,andA.Hamdouch

ISSUE 48 | JULY 2019 49

(Ed.),The International Handbook on Social Innovation: Collective Action, Soci-al Learning and Transdisciplinary Research(pp.371–383).Cheltenham:Elgar.

König, T., Nowotny, H. and Schuch, K. (2018). Impact re-loaded. Pa-thways to impact from SSH research.RetrievedOctober24,2018from:https://www.ssh-impact.eu/impact-re-loaded/.

Loorbach, D. and Rotmans, J. (2010).ThePracticeofTransitionMa-nagement:ExamplesandLessonsfromFourDistinctCases.Futures,42(3),pp.237–246.

Loorbach, D.(2007):Transition Management. New mode of governance for sustainable development.Utrecht:InternationalBooks.

McGowan, K. and Westley, F. (2015).AttheRootofChange:TheHis-toryofSocialInnovation.InA.Nicholls,J.SimonandM.Gabriel(Ed.),New Frontiers in Social Innovation Research(pp.52-68).London:PalgraveMacmillan.

McKelvey, M. and Zaring, O.(2017).Co-deliveryofsocialinnovations:exploringtheuniversity’sroleinacademicengagementwithsociety.In-dustry and Innovation, 25,6,pp.594-611.Abingdon:Routledge.

Melles, G., de Vere, I. and Misic, V.(2011).Sociallyresponsibledesign:Thinking beyond the triple bottom line to socially responsive and sus-tainableproductdesign.Codesign-International Journal of Cocreation in Design and the Arts,7(3-4),143-154.

Moulaert, F., MacCallum, D., Mehmood, A. and Hamdouch, A.(2013).The International Handbook on Social Innovation. Collective Action, Social Learning and Transdisciplinary Research. Cheltenham, Northhampton:EdwardElgar.

Moulaert, F., MacCallum, D., Mehmood, A. and Hamdouch, A.(2013a).General introduction: the returnof social innovation asa sci-entificconceptandasocialpractice. InF.Moulaert,D.MacCallum,A.MehmoodandA.Hamdouch(Ed.),The International Handbook on Social Innovation. Collective Action, Social Learning and Transdisciplinary Re-search(pp.1-6).Cheltenham,Northhampton:EdwardElgar.

Moulaert, F., MacCallum, D. and Hillier, J.(2013b).Socialinnovation:intuition,precept,concept,theoryandpractice.InF.Moulaert,D.Mac-Callum, A. Mehmood and A. Hamdouch (Ed.), The International Hand-book on Social Innovation. Collective Action, Social Learnig and Transdisci-plinary Research(pp.13-24).Cheltenham,Northhampton:EdwardElgar.

Moulaert, F., Martinelli, F., Swyngedouw, E. and González, S.(2005).TowardsAlternativeModel(s)ofLocalInnovation.Urban Studies,42(11),1969–1990.

Mowery, D. C. and Sampat, B. N. (2005).Universities inNational In-novationSystems.InJ.Fagerberg,D.C.MoweryandR.R.Nelson(Ed.),The Oxford Handbook of Innovation (pp.209-239).Oxford:UniversityPress.

Nicholls, A., Simon, J. and Gabriel, M. (Ed.)(2015).New Frontiers in Social Innovation Research.London:PalgraveMacmillan.Nowotny, H., Scott, P. and Gibbons, M. (2001). Re-thinking Science.

Knowledge and the Public in an Age of Uncertainty.Cambridge:PolityPress.

Osterhammel, J.(2011).Die Verwandlung der Welt. Eine Geschichte des 19. Jahrhunderts.München:C.H.Beck.

Pufe, I. (2014).Nachhaltigkeit.2.Ed.Konstanz:UVKVerlagsgesellschaftmbH.

Rammert, W.(2010).DieInnovationenderGesellschaft.InJ.HowaldtandH.Jacobsen(Ed.),Soziale Innovation. Auf dem Weg zu einem postin-dustriellen Innovationsparadigma (pp.21-52).Wiesbaden:VSVerlagfürSozialwissenschaften.

Reckwitz, A. (2003). Grundelemente einer Theorie sozialer Praktiken:Eine sozialtheoretische Perspektive. Zeitschrift für Soziologie, 32 (4),282–300.

Rorty, R. (2008).Philosophie als Kulturpolitik [Philosophy as Cultural Poli-tics].FrankfurtamMain:Suhrkamp.

Schaper-Rinkel, P. and Wagner-Luptacik, P. (2014).DesignThinking.InJ.Howaldt,A.Butzin,D.DomanskiandC.Kaletka(Ed.),Theoretical ap-proaches to social innovation: A critical literature review(pp.97-103).Dort-mund:Sozialforschungsstelle,TUDortmundUniversity.RetrievedMarch29, 2018 from: http://www.si-drive.eu/wp-content/uploads/2014/11/D1_1-Critical-Literature-Review_final.pdf.

Schneidewind, U. and Singer-Brodowski, M. (2013). Transformative Wissenschaft. Klimawandel im deutschen Wissenschafts- und Hochschul-system.Metropolis:Marburg.

Schneidewind, U. (2013).Wandelverstehen–AufdemWegzueiner„TransformativeLiteracy“.InH.WelzerandK.Wiegandt(Ed.),Wege aus der Wachstumsgesellschaft (pp. 115-140). Frankfurt am Main: FischerTaschenbuch.

Sgaragli, F. (2014).Enabling social innovation ecosystems for community-led territorial development.Rom:FondazioneGiacomoBrodolini.

Shove, E.(2003).Comfort, Cleanliness and Convenience: The Social Orga-nization of Normality.Oxford:Berg.

Shove, E., Pantzar, M. and Watson, M.(2012).The Dynamics of Social Practice: Everyday Life and How it Changes.LosAngeles:Sage.

Soler Gallart, M.(2017).Achieving Social Impact. Sociology in the Public Sphere.Cham:Springer.

Sommer, B. and Welzer, H.(2014).Transformationsdesign. Wege in eine zukunftsfähige Moderne.München:oekomverlag.

Spaapen, J. and van Drooge, L. (2011).Introducing‘ProductiveInterac-tions’inSocialImpactAssessment.Research Evaluation,20(3),211–218.

Tarde, G.(2009).Die sozialen Gesetze. Skizze einer Soziologie (1898).Mar-burg:MetropolisVerlag.Von Schomberg, R. (2011).ProspectsforTechnologyAssessment ina


frameworkofresponsibleresearchandinnovation.InM.DusseldorpandR. Beevroft (Ed.), Technikfolgen abschätzen lehren: Bildungspotenziale transdisziplinärer Methoden.Wiesbaden:VSVerlag.

Wallin, S.(2010).The co-evolvement in local development – From the trip-le to the quadruple helix model. Paper presented at Triple Helix VIII, Ma-drid.RetrievedOctober24,2018from:http://www.leydesdorff.net/th8/TRIPLE%20HELIX%20-%20VIII%20CONFERENCE/PROCEEDINGS/0110_Wallin_Sirkku_O-104/triple%20helix%20Wallin%20final.pdf.

Zapf, W.(1989).ÜbersozialeInnovation.Soziale Welt,40(1-2),170-183.

AUTHORJÜRGEN HOWALDT

Sozialforschungsstelle, TU Dortmund University, EvingerPlatz17,Dortmund,44339(Germany)E:[email protected]

KEYWORDS: SocialInnovation,SocietalImpact,SocietalTransformation,Empower-ment,ParticipatoryApproaches,CivilSociety

ISSUE 48 | JULY 2019 51

tingoutalternativesinrigorous,butnon-experimentalforms.Philosophyoffersthechancetothinkthroughhypotheticalalternatives,whilstlite-raturemakesuseofimaginationtoconceivealternativescenariosandtoexplorethem.Historiansandarchaeologistsusetheanalysisofthepastasaformoflaboratoryofdifferentworlds.Beingabletothinkthrough“whatif?”scenariosdeepenone’sunderstandingoftheworld.

Theseapproachesalsobringintoconsiderationthenon-materialfea-turesofourhumanexistence.Thequalityoflifedependsnotonhavingnewgadgetsornewproducts,butonbeingabletolivealifewhichhasvaluethatmaymakeuseofwhattechnologyhastoofferinavaluableway.Vision,beauty,style,andenjoymentareintegraltoavaluablehu-manlife.

Thesocialsciencesofferyetdifferentwaysofchallengingcontem-porary norms and traditions of doing things. Techniques such as mo-dellingenablealternativescenariostobebuiltandtestedinarigorouswaywithouttheabilitytorepeatexperimentsasinlaboratoryscience.Modellingoftenreducescomplexitybyfocusingonkeyfeaturesofasi-tuationandthenvaryingthem.

Quantificationistypicallyanapproachofeconomics,geographyandsociology.Otherapproacheslookatqualitativeanalysis,scalingupfromsamples. These social sciences enable us not just to gain informationabout what might change, but also identify and test our deep valuesagainstwhichtotestthesocialcontributionoftechnologicaladvances.

Inour view,anyworthwhile scienceprogramme for theEUhas toharnessthepotentialofallbranchesofscientificendeavourandtoen-couragethemtoworktogether.ThisperspectiveagreeswiththeviewoftheLamyReport:“Innovation is more than technology. EU innovation policy must be based on a definition of innovation that acknowledges and values all forms of new knowledge – technological, but also business mo-del, financing, governance, regulatory and social – which help generate value for the economy and society and drive systemic transformation.”2

“Innovation”shouldberedefinedandimplementedmoreholisticallyandopenlyinordertoachievetheaimstheEUwishestosupport.Inno-vationisnotlimitedtobusinessandeconomicopportunities,butitisalsofundamentallyabouttransformingthewayweliveandthethingswedo,sociallyandculturallyaswellaseconomically.Thehumanitiesandsocialscienceshaveaverystrongcontributiontomakesuchtransformationshappen.

Similarly, “impact”shouldbeconceivedintermsofhowitaffectsnotonlytheeconomyandgovernmentalpolicies,butalsothewaysocialin-

INTRODUCTION

Thispaper isa revisionof theproposals for the regulationandspecific programme of the forthcoming European FrameworkProgrammeforResearchandInnovationpresentedbytheEuro-

peanCommission(EC)on7June2018.ItpresentsideasonhowSocialSciencesandHumanities(SSH)researchcouldbebetterintegratedandputs forward suggestions for collaborative research and innovation asamainlineofengenderingchangeandsecuringcompetitiveness.It iscrucialforthefutureofthe“EuropeanResearchArea”torecognisethevalueand importanceoftheSSH, includingthroughcontinuedannualSSHMonitoringReportsthathaveuptonowillustratedthelackofpro-gressthattheEChasmadeinintegratingSSHinHorizon2020.

Byengagingwiththeconceptsofinnovationandimpact,thepaperpromotesanunderstandingofinnovationasafactortotransformsocietyandcalls foraconceptualisationof impact that is takingwidersocial,culturalandpoliticaldevelopmentsintoaccount.Lastbutnotleast,thisis followed by some practical suggestions for potential missions andwaysofimplementation.

THE CONCEPTS OF “INNOVATION” AND “IMPACT”

HorizonEuropebringstogethertheEuropeanUnion’s(EU)researchandinnovationactivitieslargelyunderoneFrameworkProgramme.Butthere isalwaysadanger that theemphasison thecontributionof re-searchtoeconomicgrowthfostersatechnocraticparadigminwhichthetranslation of fundamental research into innovative ‘products’ is seenas thebenchmarkof success. In thepast, theEChasunderstood therelationshipbetweenresearchandinnovationtoomuchintermsofanoverly simplistic, linear process in which research is expected to leadto ever higher Technology Readiness Levels (TRLs). The dominance ofthisparadigmbelittlesthecontributionofHumanitiesandtheSocialSci-ences.HumanitiesandSocialScienceshavedifferentperspectivesonproblems,buttheycontributetoaroundedapproach.

WhatisitthattheHumanitiescontributetoinnovation?Theyofferanabilitytochallengepresentwaysofapproachingsocialproblemsbytes-

JOHNSTEPHENBELLDOI:10.22163/fteval.2019.366

THEIMPORTANCEOFSSHRESEARCHINHORIZONEUROPE1

1 ThispaperdrawsheavilyontheALLEAPositionPaperDevelopingaVisionforFrameworkProgramme9(July2017).2 ReportoftheIndependentHighLevelGroup,Investing in the European Future we want(July2017),p.12.


teractions,cultureandwaysofthinkingareaffected.Therearegoodex-amplesofthisbeingrecognisedinSocietalChallenge6ofHorizon2020,buttheapproachisnotreflectedconsistentlyacrosstheotherChallen-ges.Adefinitionofimpactthatincorporatedthecontributionofresearchand innovation to thewellbeingof societywouldbeasignificantandimportantstepintherightdirection.

TheambitionofEuropeanresearchneedstolookathowtheindivi-dualandcollective livesof residents in theEUare improvedandhowEuropecancontributetothequalityoflifeofotherpartsoftheworld.Researchneedstolookatnotonlyindividualsituations,butalsothefea-turesofstructureswithinsociety–power,institutions,politicalpartici-pation,andnewactorsincivilsociety.Itexplorescohesionanddiversityin the way we are living together. It needs to investigate appropriatefoundations and ingredients of contemporary democracy to make oursocietiesmoresustainable,openandresilient;widenourknowledgeonthesocialandculturaldynamicsandeffectsof(democratic)governancestructuresaswetakeadvantageofchangesthatscienceandtechnologybringandthenewquestionstheyraise.Forinstance,fieldsofinquiryandmoreconcreteobjectivescouldincludethesocialapplicationofhistoricalstudies.

WhatisanappropriatebalancebetweenindividualityandsolidarityformodernEuropeansocieties?Socialandculturaldiversityarevalua-blefeaturesoflifeinEurope(asinmanyotherpartsoftheworld).Thisprovidesnotonlyacontextforresearchandpolicy,butitalsoprovidesawonderfulresource.Forexample,ifwelookattheartsandsociety,wemightaskwhetherliterarymodelscanenhancesocialcohesion?

Literatureandartofferusalaboratoryofthefuture,drawingonourdeepsenseofidentitiesinthepresentandrelatedtoourpast.Develo-pingstrategiestofostersocialaccesstoarthistoryandtocritique,andtoincreaseparticipationinculturalandartisticendeavourhavecreativepotentialtocontributetothetransformationofsocietyasmuchasanytechnologicalinnovation.

THE DESIGN OF THE EUROPEAN RESEARCH AGENDA

InordertotackletheglobalchallengesofthedecadestocomeandenableEuropeancitizensandsocietiesmaintainthepaceofinnovationandsocialtransformation,Europeneedstoharnessthecreativecapaci-tiesofallitsresearchersandsocialactors.Thiscanbeonlyachievedina joint endeavour, especiallyby intensified inter- and transdisciplinarycooperation.Inordertounderstandthehumandimensionofsocialandtechnologicaltransformation,itiscrucialtolookatdifferentperspectivesandusethepotentialofthehumanitiesandsocialsciencesinenablinginnovationandreflection.Eachbranchofsciencehasitsowncontribu-tiontoknowledgeandHorizonEuropeneedstodrawonthemall inaholisticway.

Thishasimplicationsforthedraftingofthe“GlobalChallenges”and“Missions”.Howdoweidentifytheproblemswhichtheseinstrumentsaredesignedtosolveandthemethodsappropriate for tackling them?ThechallengesEuropeanditscitizensfacetodayandinthedecadestocomearenotmerelyeconomic,technologicalandpolitical,theyarealso

social,cultural,legalandethical.Challengessuchasrisinginequalities,nationalism,radicalismandterrorismthreateninclusion,socialcohesionanddemocraticgovernancealloverEurope.Demographicchange,mig-rationanddigitisationcreateconstantchange.Thesechallengescallforaprofoundandinclusivedialoguebetweenallactorsinsociety.

Technological innovation is obviously necessary to improve manyfeaturesofthewaywelive,e.g.inmedicalinterventionsforhealthcare,insmartsystemstoimprovethequalityoflife,inwaysofreducingth-reatstotheclimateandinimprovedtransport.Buttechnologiesneedtobeembeddedinanunderstandingofhowwehumanbeingsmightusethem,howlivesmightadapttotheirpresence,andwhetherthiswouldimprovethequalityofour lives.Scientistsunderstandthiswell–theyare,afterall,membersofsocietywithahumaneinterest in livingandcontributingtogoodlives.Theyarekeentoinvolvedifferentbranchesofknowledgeinensuringthattheireffortsreallydotransformthelivesofpeopleinsociety.

Fromtheperspectiveofthehumanitiesandsocialsciencescommu-nities, these challenges require concerted efforts within and outsideEurope,cuttingacrossborders,cultures,languages,disciplines,sectorsandinstitutions.Thatiswhynotonlythe“ALLEAWorkingGroupHorizonEurope”arguesformoreinterdisciplinarityandabiggerandwell-definedroleoftheSSHindesignandevaluationoftheresearchwhichisfundedthroughHorizonEurope.Otherwisethesocietalchallengetobuildinclu-sive,innovativeandreflectivesocietiesrunsthedangerofbeingmargi-nalisedbyother,moretangiblematerialandtechnologicalchallenges.

MISSIONS Theambitionof“Missions”toachievetangibleresultswithinade-

finedtimeframeislaudableinmanyways.Nevertheless,again,itisne-cessary to guarantee interdisciplinarity and a bigger and well-definedroleoftheSSHindesignandevaluationofthemissions.Inastatementpublished incooperationwithotherstakeholders3, theALLEAWorkingGroupHorizonEuropecriticallyreflectsonthetypeandscopeofmissionsthatwouldadequatelyrespondtothesocietalchallengesEuropefacesintheyearsanddecadestocome.

The2030AgendaoftheUnitedNations(UN)shouldserveasaframe-workofinspirationwhentargetingthesechallengesthroughthedeve-lopmentofmissions.Such“Missions”haveastrongpotential tobringtogetherresearchersfrommanydisciplinesaswellaspolitical,cultural,economicandsocialactorsandcivilsocietyinacommonendeavourofensuringthatEuropeisattheforefrontofresearch,innovationandsmartimplementation–andhencewellequipped toanswerurgentsocietalquestions.Actions shouldbe inter-ormultidisciplinaryand involveor-ganisations in the cultural, economic or social sectors: Co-creation ofresearchquestionswillallowthe translationofsocietalneeds into re-searchand innovationandfacilitatethetranslationofresearchresultsintosmartapplicationsandsocietaluptake.

If it is to achieve worthwhile results, mission-oriented researchshouldthus

(1) betransformativeinthatitgeneratesnewknowledgeandun-derstanding,

(2) acknowledgethatinnovationismorethantechnology,

3 SeefurthertheStatementledbyALLEAandHERA,LivingTogether:MissionsforShapingtheFuture(2018).

ISSUE 48 | JULY 2019 53

(3) bebroadenoughandnottooconstrainedonspecificend-prod-uctsthroughaprematureidentificationofindicatorsofsuccessorfailure,

(4) beopentoresearcherstocomeupwithprojectsofallsizestoproduceinnovativeideas(bottom-upapproach),

(5) integrateall countriesand regions inorder tocounter the re-searchandinnovationdivideinthe“EuropeanResearchArea”and maintain openness towards collaboration with non-EUcountries.

ManyoftheseideasarereflectedintheMazzucatoreport4onwhichthe“DraftRegulation”nowdraws.TheMazzucatoreportsoughttofindawayinwhichresearchandeconomicgrowthcouldbesteered.Asaresult,itssecondcriterionisthatmissionsshouldbe“targeted,measu-rable”andtime-limited(nowArticle7(3)(c)ofthe“DraftRegulationonHorizonEurope”,p.14),anditsthirdcriterionisthattheyshouldbeam-bitious,butrealistic.Thatrequirescarefulthoughtintermsofthedesignofmissioncalls.Thisisatop-downactivitybyEUinstitutions.Ofcourse,thefinalcriterionformissions isthattheyshouldbeopentomultiple,bottom-upproposals(nowArticle7(3)(f)).Thatleavesanimportantscopefor the initiative of individual researchers and innovators or groups ofthem.Butthosebottom-upproposalswillcomewithinaframework.ThisfeaturedrawsoutamajorareaforthoughtaboutHorizonEurope–notjustthecontentandthebudget,buttheprocessbywhichitisimplemen-ted,particularlyattheleveloftheEC.

IMPLEMENTATION Call design:Weconsiderthatthedraftingof“WorkProgrammesfor

ChallengesandMissions”shoulddrawonresearchersindifferentdiscip-linesanddifferentmethodologiesaswellasonexpertsfromcivilsocietyandthecultureandeconomicssector.ThewordingofcallsshouldreflecttheneedtodrawonthefullrangeofresearchcapacitywithinEurope.ThathasnotalwaysbeenthecaseinHorizon2020(especiallyoutside“SocietalChallenge6”SC6)andthisisreflectedinthelowlevelofparti-cipationbySSHdisciplineswithinthoseotherchallenges.

Theideaofinvolvinghumanitiesandsocialsciencesinplanningistoenabletopicstobeidentifiedcorrectlyinthefirstplace.Takeacurrentexample:Inthe“WorkProgramme”ofSC4(Smart,GreenandIntegratedTransport) for2018-2020, there isacall in relation to“Harnessing and understanding the impacts of changes in urban mobility on policy making by city-led innovation for sustainable urban mobility” (LC-MG-1-3-2018).Thedetail of thechallengestates that“Urban mobility is in transition. This is a result of, for example, changing user needs; emerging transport technologies; new transport services using new business models; and new institutional and financing structures.” (LC-MG-1-3-2018,p.20).Furtheritstatesthat“Specialattentionshouldbepaidtotheneedsofvulnerablegroupsanduserswithdifferentculturalbackgroundstakingintoaccountgenderissues;andtothespecificcontextofareasthatareundergoingrapideconomicchange.”(LC-MG-1-3-2018,p.20).Bothoftheseclearlycallforacontributionbysocialscientistsandhumanitiesscholarsinor-dertounderstandthesocialneedsthattransporttechnologies,businessmodelsandfinancingstructuresarerequiredtoserve.

However, compared with the emphasis on data-driven planning,newbusinessmodelsandtechnology,notmuchthoughthasgoneintoidentifyingthesocialphenomenawhichresearchinthisareashouldbeaddressing.Itshouldalsobelookingatwhypeopleareusingvehicles,howchangesinworkpatterns(e.g.mobileandhomeworking)affectde-mandsforandtimingofvehicleuse,andwhetherthelocationofschools,leisureandshoppingvenuesmakeadifference. Inadocumentwhichrunstooverahundredpagesonallthecalls,thethoughtgiventothepo-tentialcontributionofhumanitiesisverylimited,andthecontributionofsocialscientists,suchassocialgeographers,isbadlyunder-developed.Therearewordswhichhavepotential,butincomparisonwiththedetailonothermatters,theygivetheimpressionofbeinganafter-thought.

HorizonEuropeshouldaimtodesignatebroadfieldsofenquirywhichleave substantial flexibility to accommodate the innovative, but unex-pectedproposal.Thismeansthat thedraftsofcallsshouldbefar lessdetailedthanthecurrentcallsfor“SocietalChallenges”withinHorizon2020.

Emerging priorities: Given the uncertainty about the future andthe rapid development of technology, the fields of research identifiedfor“Missions”shouldnotcoverthewholeofthe2021-2027period,butshouldinitiallybeshorter,withthepossibilityofcontinuationwheretheyprovefruitful.

Project design: Projectssubmittedshouldbebroadenoughtoinclu-de,whereappropriate,participatoryactions (co-design)bynon-resear-chers.Forexample,researchonmigrantsorelderlypeoplemightinvol-vethosegroupsinshapingthedesignofprojectsandinselectingthematerialstobeincludedaspartoftheresearch.Itisinthesewaysthat“citizen-ledscience”isbestunderstood.Suchprocessesofco-creationofknowledgeensurebetteracceptanceandimplementationinsocietyandtheeconomy.Inmanyresearchprojectsinthehumanities,itiscommontobringtogetherindividualsfromcommunitiesthatarebeingstudiedtohelpdesignandimplementtheresearchthroughwriting,oralhistoryorarticulatingfeaturesof theircommunitieswhichshapetheunderstan-dingsthatarenecessaryforeffectiveresearch.

Evaluation: The evaluation of proposals should include represen-tatives of a range of disciplines, including the humanities and socialsciences. The diversity of social science and humanities subjects (likethediversityofbiologicalsciences)requiresarangeofexpertevaluatorstobe involved to reflect thediversityofdisciplines (and theemergingnewfieldsgeneratedthroughinter-andtransdisciplinarycollaboration).“EthicsReviews”needinformedexperts.

Project size: Thereshouldbegreaterflexibilityinchoosingthesizeofaproject.Projectsinthehumanitiesandsocialsciencestypicallydonothavetheneedforexpensiveequipment.Therearesometimesgoodreasons for larger teams,butoften closeworkingby smaller teams isthemostproductivewayforward.ThepermittedsizeofbidsshouldbesmallerthaninHorizon2020.Itmaybeprudenttogivesmalleramountsofinitialfundinguntiltheproofofconceptstageisreachedorpotentialsocialorconceptualimpactisenvisaged.Thiscallsforfollow-upfundingforpromisingideas.

Monitoring: Reviewcriteriashouldrecognisethatresearch/scienceacceptadiversityofgoodsolutionsandacomplexityofcontextsinwhichsolutionsachieveresults.Newindicatorsforsocietalandculturalimpactneedtobedevelopedandused.Assessingtheperformanceofmissions

4 MarianaMazzucato,Mission-OrientedResearchandInnovationintheEuropeanUnion(2018)


cannotsimplybeintermsofsuccess(manonthemoon)orfailure(noman on the moon). Scholars from the humanities and social sciencesneedtobeintegraltothemonitoringifthepotentialforsteeringresearchandsocialdevelopment(notjusteconomicgrowth)istoberealised.

CONCLUSIONThe humanities and social sciences need to be deeply embedded

intheresearchagendaofHorizonEurope.Thisisbecausetheresearchendeavourneedstheminordertoachieveitstransformationalpotenti-al.Researchersinthesefieldsdohavedifferentresearchmethodsandambitions compared with engineers and scientists, but this offers thepotentialformutualenrichment.Ultimately,weareservingacommuni-tyofnationswithintheEUwhohavetheambitiontoworktogethertoimprovetheirownqualityofindividualandcommunitylivesaswellasthoseofthewiderworld.IfwefailtodothiseffectivelythroughHorizonEurope,weletdownourselvesasresearchersandthepeopleweserve.

REFERENCESALLEA (2017). Developing a Vision for Framework Programme 9. Ret-rieved October 18, 2018 from: https://www.allea.org/wp-content/up-loads/2017/07/ALLEA_Statement_FP9.pdf.

ALLEA et al. (2017).LivingTogether:MissionsforShapingtheFuture.Retrieved October 18, 2018 from: https://www.allea.org/wp-content/uploads/2017/12/Living_Together_Missions_for_Shaping_the_Fu-ture_2017.pdf

High Level Group on maximizing the impact of EU Research & Inno-vation Programmes(2017).LAB–FAB–APP–InvestingintheEuropeanfuturewewant,ReportoftheindependentHighLevelGrouponmaxi-misingtheimpactofEUResearch&InnovationProgrammes.RetrievedOctober 18, 2018 from: http://ec.europa.eu/research/evaluations/pdf/archive/other_reports_studies_and_documents/hlg_2017_report.pdf

LC-MG-1-3-2018,(2018).Harnessingandunderstandingtheimpactsofchanges in urban mobility on policy making by city-led innovation forsustainable urban mobility: http://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-transport_en.pdf

Mazzucato, M.(2018).Mission-OrientedResearchandInnovationintheEuropeanUnion.RetrievedOctober18,2018from:https://ec.europa.eu/info/sites/info/files/mazzucato_report_2018.pdf.

AUTHOR

JOHN STEPHEN BELLChair of the ALLEA Working Group Horizon Europe, All European Acade-mies (ALLEA)Jägerstraße22/23,Berlin,10117(Germany)E:[email protected]

ISSUE 48 | JULY 2019 55

2 REFLECTIVE SOCIETYThe“reflectivesociety”isalsoasyntagmathathasfoundwideusage

among researchers since a precise date, in this case the fall of 2013,whentheEuropeanCommissionintroduceditforpostingSocialScien-cesandHumanitiesrelatedcallswithinthesixthsocietalchallengeofHorizon2020,theoneabout“Inclusive,innovativeandreflectivesocie-ties”.Thelastadjectivereferstotheroleofdeliberativecommunicationofcitizens inamodernpublicsphereaimingatmutualunderstandingandgoesbacktoImmanuelKant(1790),G.W.F.Hegel(1812-13),JürgenHabermas(1973),JamesS.Fishkin(1993),UlrichBeck,AnthonyGiddensand Scott Lash (1996) as well as Alessandro Ferrara (1998). A closer scrutinyrevealsthatHabermashasappliedtosocietywhatHegelhadelaboratedas thepassage fromthesurfaceofbeing to thegroundofessence,apassagethattakesplace,literally,byreflectinginto the thing – likereflected light that illuminatessomethingpreviously invisible,orcreatesapatternnotpreviouslyexisting.Insistingonreflexivityhelpstoraiseawarenessfortheimportanceofframingissuesaroundengagingwith science and society, identifying problems and defining solutions.The“FaroFrameworkConventionontheValueofCulturalHeritageforSociety”ofUNESCO(2007)encouragesreflectionontheroleofcitizensin theprocessofdefining, creating,andmanaginga cultural environ-mentinwhichcommunitiesevolve.

It is true, innovation ispartofeconomics,because it isaboutmo-neygeneratingknowledge.Thereis,however,innovationinsocietyandin culture.Social andcultural innovation isa fact.Whileaccording tothe traditional – so-called Mode 1 – knowledge production, which ismotivatedbyscientificknowledgealone(fundamentalresearch)andisneitherbotheredbytheapplicabilityofitsfindingsnorbybridgingovertootherdisciplines,incontemporaryresearch,multidisciplinaryteams–so-calledMode2–arebroughttogetherforshortperiodsoftimetoworkon specific problems in the real world for knowledge production. Thismodecanbeexplainedbythewayresearchfundsaredistributedamongscientistsandhowscientistsfocusonobtainingthesefunds(Gibbons,Limoges,Nowotny,Schwartzmann,ScottandTrow1994).Relativelyre-centmodels,suchastriplehelixandopeninnovationhavestressedthatthe collaboration among different institutions is crucial for successfulinnovation.Butonlymarginally thesemodelshave taken intoaccounttheactualandpotentialrolethatcitizensinthereflectivesocietyhaveinshapingtheinnovationprocess(LeydesdorffandEtzkowitz1998;Etzko-witzandLeydesdorff2000;CarayannisandCampbell2009;Chesbrough2003).

ABSTRACT

“Culturalinnovation”soundslikeanoxymoron.Itisnot,though.Itissomethingrealthattopsupsocialandtechnologicalinnovation.Howcanwemeasure“cultural innovation”?Theanswer is,asa

resultofco-creation.Itemsofculturalinnovationare:contentproviderssuchasmuseums, science centresand libraries, aswell asprocessestriggered by issues such as cosmopolitanism, inclusiveness, mobility,migration,heritageandcreativity.Valuatingtheimpactisfundamentaltoimprovesocietalacceptanceofpublicinvestmentinculture,becausethesemeasurementsmayprovideabasisforaligningresearchandin-novationwith thevalues,needsandexpectationsofsociety. In recentyears, ithasbecomeclear that co-creationplaysa central rolewithinopen innovation,becausea“specific innovation can no longer be seen as the result of predefined and isolated innovation activities but rather as the outcome of a complex co-creation process involving knowledge flows across the entire economic and social environment”(OpenScience,OpenInnovation,OpentotheWorld.EC2016,p.11).Thepaperoffersmigrati-on-relatecasestudiesforevaluatingtheimpactofculturalinnovationinsocietiesthataimatbeinginnovative,reflectiveandinclusive.

1 INTRODUCTION“SocialandCultural Innovation” isasyntagmathat is receiving in-

creasedusageamongresearchersafteritwaschosenbythe“EuropeanStrategy Forum Research Infrastructures” (2016) for the name of theworkinggroupthatdealswithresearchinfrastructuresprimarilyconnec-ted with Social Sciences and the Humanities (SSH). Innovation refersto the creation of new products and services by bringing a new ideatothemarket.Economicgrowthturnsoninfrastructures,whichprovideaccesstoservicesandknowledge,e.g.byovercomingthedigitaldivide.Globalisationhasmadeitclearthatamosturgentobjectiveistoworkoutpoliciesofsocialandculturalinnovationtotheadvantageofcitizens–policiesthataimatachievingchangesintheregulatoryenvironmentthatmakesocietiesbothinclusiveandreflective.ThinkingaheadofHori-zonEurope,thereissomefearthenotionof“culturalinnovation”mightsoundlikeanoxymoron,nodoubt.Itisnot,though.Culturalinnovationis something real that tops up social and technological innovation byprovidingthereflectivesocietywithspacesofexchangeinwhichcitizensengageintheprocessofsharingtheirexperienceswhileappropriatingcommongoodscontent.Wearetalkingofpublicspacessuchasuniver-sities,academies, libraries,museums,science-centres,butalsoofanyplaceinwhichco-creationactivitiesmayoccur,e.g.,researchinfrastruc-turessuchas“DARIAH-DigitalResearchInfrastructurefortheArtsandtheHumanities”.Atthislevel,socialinnovationbecomesreflectiveandgeneratesculturalinnovation.

RICCARDOPOZZODOI:10.22163/fteval.2019.367

INNOVATIONFORTHEREFLECTIVESOCIETY


3 CULTURAL INNOVATION

Whatissocialinnovationweknowabout:itoughttobethebackboneofallEuropeanresearchpolicies,asMarceloRebelodeSousa,PresidentoftheRepublicofPortugal,madeitclearintheconcludingremarkshegaveattheOpening up to an Era of Social Innovation ConferenceinLisbonon28October2017.Westillknowlittleaboutculturalinnovation,though.Thesyntagma ismentionedat times just tosay thatculture tooneedsinnovationandinfactproducesinnovation:museumstudiesfosterinno-vation in museography; archaeology fosters innovation in data sciencethatbecomesdatahumanities;musicandartfosterinnovationthroughsocialnetworks.Thereareevenstudiesinthe“philosophyofmuseums”,whose itemsarequestionssuchas“What typesofvaluedomuseumshave?Whatistheethicallycorrectstanceforamuseumtotaketowardsitspublic?Andtowardstheobjectsconstitutingitscollection?Shouldmu-seumexhibitsseektomakeaclaimtoobjectivity?“(Harrison,BergqvistandKemp2016).Insum,wecantalkof“heritage-ledinnovation”,whichmeansthatculturefosterstechnologicalinnovation.Digitisationisinitselfinnovation.Datasciencehasbecomedatahumanities.Westillneedagreat deal of reflection on digitisation. However, we can look at it theotherwayaround.Culturalinnovationtopsupsocialinnovation,whichonitsturnreliesontechnologicalinnovation.Innovationmustcometotermwithsocialinnovation,then,thisisaEuropeanrequirement.

Thequestioniswhatpartofsocialinnovationisculturalinnovationandwhatrightscanculturalinnovationclaimwithrespecttosociety(Ko-efoed2017)?Apreliminaryansweris:Europeanculturalheritagemarksour cultural identity, which is at the same time cultural diversity. The“EuropeanYearofCulturalHeritage”isaboutidentityanddiversity,saidJean-ClaudeJunckerinhisopeningspeechattheEuropean 2017 Culture Forum inMilanon7December2017.Buttoassesscultural innovationasthevalue-sensitiveintegrationtotechnologicalandsocialinnovationisthegreatchallengecontemporary“ScienceandTechnologyStudies”are confronted with and we need to look at it more closely. The new“missions”of thenext“EuropeanFrameworkProgrammeforResearchandInnovation”ofthemulti-annualfinancialperiod2021-2027willfos-terresearchonsystemicchangeinthenewgenerationsandcontributetothecreationofacross-borderandmulti-disciplinaryopeninnovationenvironmentsforresearchdata,knowledgeandserviceswithengagedstakeholdersandorganisations.Thecurrentmigrantcrisishasmadeitclear with extraordinary force that a most urgent objective is to worktowardsEuro-Mediterraneansocietiesthatareinclusive,reflective,andattentive to the impact thatmigration ishavingonsocialandculturalinnovation,securityandhealth,environmentandbiodiversity.

Thebiggestchallengeofthiscentury,whichismigration,asksforanewnarrativeof inclusionandreflection.Kantianphilosophy,e.g.,hasthebest chances toprovide it.WhatKanthaswrittenon the rightofvisit(das Recht eines Fremdlings), on hospitality (hospes) and sovereignty (hostis) is the key to shaping the narrative. The commentary to the third de-finite article of Perpetual Peace makes it clear that “originally no one has more right [Recht] than another to live on a particular place [Ort] on the earth” (Kant 1795, p. 41). Looking at late eighteenth-century colonialism, Kant envisaged a form of ius cosmopoliticum (Weltbürgerrecht), whose consequence is universal hospitality (allgemeine Hospitalität), which is to be acknowledged as the right of the foreigner (das Recht eines Fremd-lings), although hospitality does not entail the right of the foreigner to rob, exploit, and enslave (Kant 1797).Insum,philosophy,initshistorical

dimension, isable togranta sharednarrativeofwhathashappened,what ishappeningandwhatwillhappenwithmigration inourgloba-lisedworld.Onthebasisofacross-disciplinaryapproach,philosophersis to be trusted to achieve what Hanna Arendt (1963) did achieve fortheHolocaustandJürgenHabermas(1991)forcitizenship,bypushingforwardtherecentproposalofDonatellaDiCesare(2017)foraphiloso-phicalnarrativeofmigration.

Itisnowtimetoexaminetheroleofreflectionforrethinkingthewaysinwhichculturehasbeenenvisioned,particularlytovisualisethevariouswaysinwhichusersengagewithculturalprocessesinthepast,present,andfuture.Letmeproposea casestudy.Imagineasecond-generationdiaspora child (huaqiao 华桥) who attends a human sciences highschoolinItaly.Atacertainpoint,s/hemightbeaskedtoreadatextbyPlato, possibly the Apology of Socrates (Apologia Sokratous Ἀπολογία Σωκράτους),firstinItalian,thenperhapsintheGreekoriginalorintheRenaissanceLatinrenderingofMarsiliusFicinus.Studentstodaydelveeasilyintomultilayered,multilingualhypertexts,andtheydosoonthebasisofthereciprocalguidancemadepossiblebysocialreadingtools.OurstudentoughttoreadthesametextinmodernunifiedChineseaswell,sothats/hemightbeabletostartadiscussiononSocratesinitsChinese-speakingfamily.Inversely,schoolmatesmightappropriate,say,theAnalects (Lunyu 伦语) of Confuciusthroughtheconceptualreferen-cesindicatedbyourstudent.Togethertheymaystartthinkingonmove-ment(dong 动),rest(jing 静),humanbeing(renji人际),humaneness(ren仁),andeventuallycometograspkeytenetsofNeo-Confucianism,suchasthedictumthatrepresentstheunityofheavenandhumanorsu-pernalheavenandhumanity(tianrenheyi 天人合一),whichamountsto“restoring the Heavenly Principle and diminishing human desires”(Wang2005,p-320;NiPeimin2017).

4 CONCLUSIONRémi Brague (2004) has noted that the Arabic term for dictionary,

سوماق (qāmūs), is a translation of the name of the Titan of GreekmythologyὨκεανός (Okeanós), in theoriginal literal senseofa liquidextension that embraces all emerged lands, permitting navigation andhencecommunication.Leibnizhasusedtheoceanmetaphorforanen-cyclopaedia,whichistheverysameideaconcerninglanguagesthatthispapertriestodefend.WeexpectSSHresearchtotriggerachangeinthemind-setasregardslocatingcultureforinclusionandreflectionineduca-tion,life-longlearning,healthcare,urbandevelopmentandregeneration.Culturecannotbebutplural,changing,adaptable,constructed.Inclusionand reflection are constructed whenever we are in contact with otherhumanbeings,regardlesswheretheycomefrom.Thiswehavetolearn.

REFERENCESArendt, H. (1963).‘EichmanninJerusalem:Areportonthebanalityofevil’.The New Yorker,17MayandfollowingBeck,U.,GiddensA.andLashS.(1996).Reflexive Modernisierung.Frank-furt:Suhrkamp

Brague, R.(2004).‘Languesettraditionsconstitutivesdelaphilosophieen Europe’. In: Cassin, B. (ed.) Vocabulaire européen des philosophies: Dictionnaire des intraduisibles,editedby,pp.694-99,Paris:Seuil

ISSUE 48 | JULY 2019 57

Carayannis, E. and Campbell, D. (2009). ‘“Mode 3” and “QuadrupleHelix”:towarda21stcenturyfractalinnovationecosystem’,International Journal of Technology Management,46,201-34.

Chesbrough, H. W.(2003).Open innovation: The new imperative for crea-ting and profiting from technology.HarvardUniversityPress

DG-R&I (2016).Open science, open innovation, open to the world: A visi-on for Europe.Brussels:EC

Etzkowitz, E. and Leydesdorff, L. (2000).‘Thedynamicsofinnovation:Fromnationalsystemand“Mode2”toatriplehelixofuniversity-indust-ry-governmentrelations’,Research Policy,29,109-23

Ferrara, A. (1998).Reflective authenticity.London:Routledge

Fishkin, J. S. (1993). The dialogue of justice: Towards a self-reflective society.YaleUniversityPress

Gibbons, M., Limoges, C., Nowotny, H., Schwartzmann, S., Scott, P. and Trow, M.(1994).The New production of knowledge: The dynamics of science and research in contemporary societies.ThousandOaks:Sage

Habermas, J.(1973).‘Wahrheitstheorien’.In:FahrenbachH.(ed.)Wirk-lichkeit und Reflexion: Walter Schulz zum 60. Geburtstag, pp. 211-265.Pfüllingen:Neske

Habermas, J.(1991).Staatsbürgerschaft und nationale Identität.Frank-furt:Erker

Harrison, V. S., Bergqvist, A. and Kemp, G.(eds.2016).Philosophy and museums: Essays on the philosophy of museums.CambridgeUniversityPress

Hegel, G.W.F.(1812-13).Wissenschaft der Logik.Nürnberg:Schrag

Kant, I.(1790).Kritik der Urteilskraft.Königsberg:Nicolovius

Kant, I.(1795).Zum ewigen Frieden.Königsberg:Nicolovius

Kant, I.(1797).Metaphysik der Sitten.Königsberg:NicoloviusKoefoed, O.(2017).‘Culturalheritageandsocialinnovation:Amemoryofthefuture?’,Cartaditalia,1:417-44

Leydesdorff L. and Etzkowitz, E. (1998).‘Thetriplehelixasamodelforinnovationstudies’.InScience and Public Policy,25,3,195-203Ni, P. (2018). Understanding the Analects of Confucius: A new translation of Lunyu with annotations,Albany, N.Y.: SUNY PreSS

Pozzo, R. and Virgili, V. (2016). ‘Governingculturaldiversity:Commongoods,sharedexperiences,SpacesforExchange’, Economia della cultu-ra,26: 41-47,doi.org/10.1446/84035

Pozzo, R. and Virgili, V. (2017). ‘Social and cultural innovation: Re-searchinfrastructurestacklingmigration’,Diogenes: International Jour-nal of Human Sciences,64,doi.org/10.1177/0392192117739822

Wang, R. R. (2005). ‘Zhou Dunyi’s diagram of the supreme ultimateexplained(Taijitushuo):AconstructionoftheConfucianmetaphysics’, Journal of the History of Ideas 66 (2005) #3, PP. 307-323, doi.org/10.1353/jhi.2005.0047

AUTHORRICCARDO POZZODepartment of Human Sciences, University of VeronaViaSanFrancesco22,Verona,37129(Italy)E:[email protected]


“An important aim of ELSA is creating societally robust bio-, nano- and neurotechnologies. The programme will work to explore central challenges for governance, risk, regulation, culture, and values connected to these technologies. The programme should contribute to increase reflexivity and promote learning among ELSA researchers as well as scientists.” (Forskningsradet,s.a.)

Likewise, a frequently stated goal for responsible innovation/res-ponsible research and innovation (RRI), is that R&I processes becomereflexive/reflective–suchasinthe“R”oftheBritish“AREA”framework:“Anticipate,Reflect,EngageandAct”i.WhilethereisnoconsensusatallontheoverallgoalandpurposeofRRIintheEU–forsome,thegoalisreflexivepracticeandgovernanceofscienceandtechnology;forothersitis“better alignment” between civil society and the R&I sector – it is in-teresting to reflect upon the origin of the RRI concept. Except for sporadic and quite unrelated mentions, the term was introduced by philosopher and European Commission (EC) Directorate-General (DG) Research and Innovation (RTD) policy officer René von Schomberg in 2011. Interestingly, he did so with explicit reference to the potential of technology to have negative ethical and social implications:“[…] we are confronted with the Collingridge dilemma, implying that ethical issues could be easily addressed early on during technology de-sign and development whereas in this initial stage the development of the technology is difficult to predict. Once the social and ethical consequences become clearer, the development of technology is often far advanced and its trajectory is difficult to change.” (vonSchomberg2011,p.8)

Inthisregardtheso-calledCollingridgedilemmaistakentostandforthe following:Technologies (createdby researchand innovation)havenegativeside-effects(suchasrisksandhazards),butbythetimethesi-de-effectsareidentifiedandunderstood,thetechnologieshavebecomeentrenchedinsocietyandinfrastructureorotherwisedifficulttoremove.Neither existing modes of technology assessment, ethics procedures,risk assessment nor market mechanisms have been able to solve thisproblem.R&Ipracticeandgovernanceaccordinglyshouldbecomemoreanticipatory–betterable toanticipateandavoidR&I trajectories thatinstantiatethedilemma.

This narrative, as well as the accompanying idea that SSH know-ledgeandpracticecancontribute inthestriveforreflexivity,buildsonextensivescholarship–somewouldsaybacktoVico(Rommetveitetal.2013),others toHeideggerand theFrankfurterSchool,andyetotherswouldmakeamoreeasilydocumentedclaimthatitbuildsonlatterde-cades’“ScienceandTechnologyStudies”(STS),history,philosophyandsociologyofscienceandtechnology,andrelatedstrandsofscholarship.Indeed,sincethelate1960s,therehavebeenvariousmaximsofcriticalscience,radicalscience,thescienceandsocietymovement,technologyassessment,post-normalscience,sociallyrobustknowledgeandfinallyresponsibleresearchandinnovationthathadsimilarcontent(seeSardar

ABSTRACT

Thevalueofreflexivityhasrepeatedlybeenmobilisedinclaimsfor Social Sciences and Humanities (SSH) involvement in Sci-ence, Technology, Engineering and Mathematics (STEM) re-

search.Inshort(ifnotincaricature),thepolicynarrativegoeslikethis:Scientists, scientific practices, the governance of science and indeedmodern society should become more reflective/reflexive. This can beachievedbyinvolvingSSH,whichareinherentlyreflexive.

Inthispaper,Iwillfollowthisnarrativefromthe“Ethical,LegalandSocial Implications” (ELSI)/ “Ethical, Legal and Social Aspects of bio-,neuro-,andnanotechnology(ELSA)policiesandpracticesofthe1990sand 2000s, to the concept of “Responsible Research and Innovation”(RRI) and the “need to integrate” SSH with STEM to address societalchallenges in Horizon 2020. Drawing upon my experience as an SSHpractitionerinELSA,RRIandsocietalchallenges-focusedinterdisciplina-rycollaborations,Ishallproposetwolessonslearnt.Onekeylessonistheneedtogobeyondthesimplepolicynarrative“SSH makes science more reflexive”andthemanydisappointmentsthatitinvariablyproduces.TheotherkeylessonistheneedtogobeyondsimpledichotomiesbetweenSSHresearchandscholarshipononehandandnon-SSHresearchontheotherinordertolookformeaningfulcollaborations.

INTRODUCTION: THE REFLEXIVITY POLICY NARRATIVE

Thispaperdiscusses the roleandvalueof thesocial sciencesandthehumanities (hereafterabbreviatedasSSH) in researchendeavoursprimarily driven by the natural “Science, Technology/Technoscience,EngineeringandMathematics”(hereafterabbreviatedasSTEM).TotheconceptofSTEMwemayalsoincludethemainpartofmedicalscience,whichinitsmethodsandorientationisquitesimilartonaturalsciences.Intheabstractofthispaper,Iclaimedtheexistenceofapolicynarrativethatcanbesummarisedasfollows:STEMscientists,scientificpractices,the governance of science and indeed the modern, knowledge-basedsocietyshouldbecomemorereflective/reflexive.ThiscanbeachievedbyinvolvingSSH,whichisinherentlyreflexive.

Thenarrativeisneverexpressedexactlyassuch,orwithsuchblunt-ness, intheEuropeanUnionR&Ipolicydocuments,whichhavetoba-lancetheargumentforSSHwiththeappropriatetokensofrespectforSTEM.Inasmallcountrysuchasmyown(Norway),however,onecanfindmoredirectexpressions.Thefollowingquotestatesthemissionofthe(second)“ELSAfundingprogramme”oftheResearchCouncilofNor-way(2008-2014;ELSA=Ethical,LegalandSocialAspectsofbio-,neuro-andnanotechnology):

ROGERSTRANDDOI:10.22163/fteval.2019.368

STRIVINGFORREFLEXIVESCIENCE

ISSUE 48 | JULY 2019 59

and van Loon 2012 for an introduction to the history of this develop-ment).At times, the callwas simply for anawakeningof thepoliticalandethicalsensibilitiesof (natural)scientists–getoutof the labandengageinsociety!–whileoftenthiswasnotseenasenough:Therewasanimplicitdiagnosisofpoliticalignoranceandsocial,ethicalandepiste-mologicalnaivetywithinSTEMresearchculturesthatSSHinvolvementpresumptivelywouldcorrect. Indeed,thisresembleda“deficitmodel”,this time the scientists being the empty vessels that should be filledwithknowledgefromSSHinordertobecomereflexive.ItalsofollowedwhatwouldbethemostrelevantknowledgefromSSH:Aboveallhistory,philosophyandsociologyofscience,STS,Wissenschaftstheorie (in theGermaniclanguagearea),ethicsandphilosophyoftechnologyetc–thatis,thevariousstrandsofscholarshipthathavescienceandtechnologyastheirobjectofstudy.

IwriteasifIhaveironicdistancetothisnarrative.Ishouldimmedia-telyadmitthatIamamongitsmanynarrators.Forinstance,IwasamongthosewhostronglyarguedforthemandatorypresenceofELSAinbio-andnanotechnologyastheNorwegiangovernmentreviseditsresearchpoliciesinthelate2000s;andthegovernmentagreed.Thestriveforre-flexivesciencealsounderpinnedmyandothers’effortstogivecontenttoRRIintheEUcontext,althoughtheeffortsrarelyborethedesiredfruit(Rip2016).AndIhavekeptpublishingclaimstothatinverteddeficitmo-del,evenwithapaperentitled“NaivetyintheMolecularLifeSciences”(Strand2000).Theneedtohistoricisetheseclaimsdoesnotvoidthemoftruthvalue.Weshouldnote,however,thespeculativenatureoftheclaimforSSHasameanstomakeSTEMpracticeandgovernancerefle-xive.Totheextentthatthepolicynarrativehasbeenusedtolegitimiseaspace forSSH in fundingprogrammes, it shouldbeadmitted that itwasnot, andcouldnotbe, evidence-based.Rather, toapplyourownconcepts,itwasmoreofanimaginedfutureofadesirablesocial,scien-tificand technologicalorder inwhich theCollingridgedilemmawouldbesolved.Inthissense,thatis,inthesenseofJasanoffandKim(2009),ourpolicynarrativeofhowSSHwouldbeconducivetoreflexivescienceconstitutesasociotechnicalimaginary.

ELSI/ELSA AND RRI: OPPORTUNITIES AND DISAPPOINTMENTS

Themanypublishedself-reflectionessaysfromSSHscholarsconfirmthepersonalexperienceofmycolleaguesandIwhohavetakenpartinthevariousgenerationsofELSI/ELSAiiandRRI-labelled interdisciplina-rycollaborationsoverthe lattertwodecades:Whileresultshavebeenachieved and lessons have been learnt, there are also quite frequentexpressionsofdisappointment.

ItisusefultodistinguishbetweentwophasesofELSI/ELSAresearcheachwith theirphasesofdisappointment.The typicaldisappointmentoffirstELSI/ELSAinvolvementswasthelackofimpact,whichwasdiag-nosedasalackoftrueinteractionandtrueinterdisciplinarity(Nydaletal.2011),aswellasthelackofcriticalmassandproperorganisation(Kayeetal.2012).Thesociologists,ethicistsandphilosopherswerefundedtodo ELSI/ELSA research within a larger STEM (typically biotechnology)projectbuttheyhadtoomuchdistance.For instance,at theResearchCouncil of Norway, this diagnosis was explicitly endorsed, and from

the mid-2000s ELSA funding was directed towards “integrated ELSA”and “integrated projects” with real and intense interaction betweenSSHscholarsandSTEM researchers.Similardevelopments tookplaceelsewhere,drawingon longer traditionsof scholarshipof constructivetechnologyassessment (SchotandRip1997)and innovativecombina-tionsofethicsandethnographicwork(e.g.“Socio-TechnicalIntegrationResearch”,seeFisherandSchuurbiers2013).

Again, disappointments are well documented, ironically by the so-calledpost-ELSImanifestobyBalmeretal. (inaBritishcontext), latertobeelaboratedaslessonslearnt(Balmeretal.2016).Alsoinmyowncountry,self-reflectionandself-analysisbytheseintegratedELSAresear-chershashadarelativelypessimistictone(Forsberg2014,Nydaletal.2016).Takingonestepbackfromthemoreimmediateconcernsraisedinthesepapers,thedisappointmentappearedtoberelatedtotheadjus-tedroleas“integrated”ELSA/SSHscholarsontheirwayintotheSTEMlaboratories. In integratedELSA,distancewasreducedsufficiently for,as it were, CP Snow’s famous “Two Cultures” (1959) to clash, that is,betweenthenaturalsciencesononehandandthesocialsciencesandthehumanitiesontheother.SSHscholarsexperiencedthattheywerenot takenseriouslyquaresearchers,wereperhapsnotevenwelcome,wereneitheradvancingtheirowncareersnorhavinganimpactonso-ciety,orgenerallyuncomfortablewithfindingtheirroleinco-producingthe scienceand technology that theyby virtueof their ownexpertisecouldnotreallyvouchfor intermsof itsethicalandsocialdesirability.Thescientists,ontheirside,hadproblemscomingtotermswithwhatexactlytheyhadletintothelab–asortofspies?Saboteurs?Orjustanirrelevantexpense,forcedontothembythegrantconditions?Anumberof lessons were drawn, most of them quite commonsensical, such asbeing reflexiveandopen todialogueaboutourown factsandvalues;seekoutthemeaningfulcollaborativerelationshipswithscientistsratherthanforcingELSAdowntheirthroats;etc.

InEuropein2011-2012,ELSAgraduallycededtothenewEUpolicyconceptofRRI(Owenetal.2012).TheEuropeanCommission(EC)con-ceptwasinterpreteddifferentlyacrossEurope,andnotablyalsowithintheEuropeanCommission,withtheorthodoxDGRTDbureaucracyinsis-tingonthefiveorsix“keys”(ethics,genderequality,publicengagement,openaccess,scienceeducation(sic!)andsometimes“governance”)atthesametimeas theoriginalvonSchombergdefinitionwas implicitlyendorsedbymostSSHscholarswhoactedforandinteractedwiththeEuropeanCommission.IntheUK,thealternative“AREA”frameworkforResponsibleInnovationprovedinfluentialwellbeyondtheBritishIsles.RRIfunctionedasanumbrellanotonlyforELSAbutalsoanumberofothercommunitiesofpracticeandscholarship,notablythoseoftechno-logyassessmentandpublicengagement.Still,RRIactionsandprojectsrecruitedquitea fewof the sameSSHscholarswho surfed theELSAwaves. For some of us, RRI gave new promise and new enthusiasm,perhapsprimarilybecauseRRIwas seen lessasan inherentnegativeresponse to STEM (in spiteof its origin in thegovernance of the Col-lingridge dilemma) and also as an opportunity to “promote the good”by steering science and technology towards the common good and abettersociety.Again,thepresumptionwasthatsuchsteeringisnotonlypossiblebutalsothatSSHscholarsholdtheexpertisethatenablesustoengageinthissteeringandidentifyitsgoals,thistimeinactivedia-loguewithcivilsociety.Again,therewerelessonsanddisappointments,often related toRRIpracticesappearing less thanmeaningfulboth toSSHscholarsandSTEMresearchers.ToquoteabiotechPhDstudentinoneofourRRIcourses: “I am still waiting for the moment when you say


that we have to engage the citizens in our laboratory research and we tell you that it won’t work.”Indeed,inmyownsubjectiveexperience,IhavewitnessedhowdebatesamongRRIscholars/practitionersin2016-18appearedquitesimilartotheELSAdebates5-10yearsago,evenwithand without overlap in the actual people taking part. For instance, atthe 2018 international conference of S.Net (the Society for the StudyofNewandEmergingTechnologies)inMaastricht,thedifficultconver-genceworker roleofSSHscholarshiredto“doRRI” inSTEMprojectswasdiscussedinseveralofthesessions.

LESSON 1: BEYOND THE PURE IMAGINARY OF “REFLEXIVE SCIENCE”

LetusrecallforamomentthepolicynarrativethatIclaimedtounderlieELSA,RRIandotherattemptsatintegratingSSHintoSTEM:

STEMscientists,scientificpractices,thegovernanceofscienceandindeed the modern, knowledge-based society should become morereflective/reflexive. This can be achieved by involving SSH, which areinherentlyreflexive.

Above I have described some experiences of disappointment asELSI/ELSAandRRIeffortsoftenseemedtohave little impactandso-metimes were perceived as downright meaningless. Since reflexivityis what we as SSH scholars by assumption are supposed to purport,it seems timely to ask reflexive questions about the disappointment.Howmayweunderstandourownroleasparticipantsinthestriveforreflexivescience?

Thepolicynarrativeofreflexivesciencecanbeseenasasociotech-nicalimaginary,thatis,acollectivevisionofgoodandattainablefuturescience,technologyandsociety.ItwillbeusefultopursuethatanalyticallenssomewhatfurtherinthecaseofRRI.“Science”meanstwothingsin this regard. At thedistal poleof the imaginary, it is a vision of co-producedgood(i.e.,reflexive)STEMscience,good(i.e.,ethically,sociallyandenvironmentallydesirable)STEM-basedtechnologyandagoodso-cietythatcanbenefitfromthisethicallyandsociallygoodSTEMscienceandtechnology.Theprogrammeofactioncorrespondingtothisvision,issimplythesuccessfuldeploymentofSSH-informedandSSH-drivenRRIpractices.However, theseRRIpracticesarealsothemselves imagined;theyareinnowaypresentasoff-the-shelftechnologies.Soattheproxi-malpoleoftheimaginarywehavethevisionofco-produced“science”asSSH-basedknowledgeonRRI,“technology”astheRRIpractices,toolsandmethodstobeappliedontoSTEMresearch,and“society”asthere-searchandinnovationsectorthatnolongerwillgiverisetoCollingridgedilemmas or otherwise create problems in the world. Programmes ofactioncorrespondingtothisversionoftheimaginaryincludeRRIframe-worksandfundingschemes,suchastheHorizon2020SwafS(Science-with-and-for-Science)programme.

Allimaginariesarespeculative;thisiswhatmakesthemimaginari-es rather thanplansorcost-benefitanalyses.Change isgeneratedbyimaginingthenon-existentandagreeingonaprogrammeofactionthatmaybringitintoexistence.Thisimplies,however,thattherecanbenoguaranteeofsuccess.Anythingcangowrongintheattemptstorealiseasociotechnicalimaginary,andthefailuremayhaveanytypeofcause:material,social,epistemic,political.

Asfortheproximalpoleoftheimaginary,oneassumptionstandsoutinitsboldness:ThebeliefthatSTEMpracticeswillproducesubstantively“better” technologies (in thesenseof theirethical,environmentalandsocialdesirability)ifthesepracticesbecomereflexiveandsocanaccountfortheirownvalue-ladennessandtheirowncontextofimplication.Thisassumptionseemstobesharedinallstrivesforreflexivity,goingbacktoMarxistand feministcritiques, throughradicalscience,post-normalscience and the concept of socially robust technology, all the way totheRRIofthe2010s.Theexactmechanismofhowthisissupposedtohappen, varies from quite elitist beliefs in the normative expertise ofSSH,ethics,“TechnologyAssessment”expertsandthelike,tobeliefsinthepowerofdeliberationanddemocratisation.The latterwouldentailrecommendationsofbringinginarangeofstakeholders,citizensandso-cialactorsinupstreamengagementexercisestocancelthetunnelvisionofSTEMpractitionersand/or“align”researchagendaswithsociety,thatis,steerresearchfundingtowardsSTEMthataddressessocialneedsandconcerns.

Wedonotknowifthisassumptionoftheeffectivenessofreflexivityholds.Itisofcoursealwayspossibletocherry-pickexamplesthatseemtoconfirmtheassumption;hencetheindustryofprojectsthatdocument“best practices” of RRIiii. In my experience, many STEM researcherscan sympathise with the goals of RRI but they also find the workingassumptionquitenaïve.IquotedabovethePhDstudentwhosaid:“Iamstillwaitingforthemomentwhenyousaythatwehavetoengagethecitizensinour laboratoryresearchandwetellyouthat itwon’twork.”Indeed, he expressed the expectation that we were making naïve as-sumptionsabouttheimpactofupstreamengagement.

The experience of disappointment with RRI as expressed by SSHscholarsengagedinRRIprojectsandeffortshasbeenconnectedtomorethanthisproblematic(butcentral)assumption,however.Perhapsaboveall there has been frustration with practical and organisational issuesrelatedtotheprogrammesofaction.TheRRIframeworksandtheSwafSprogrammehavebeenseenastoosuperficialandnotreallyembodyingthe insights of relevant SSH scholarship (see e.g. Rip 2016); researchpolicy-makersdon’treallyunderstandRRI;evenwhenSTEMresearchersengage,theymightnotengagewiththelevelofcommitmentrequired;andwhen research fundingorganisations requireRRI fromSTEMpro-jects, they may be satisfied with mere tokens and window-dressing,notunlike“CorporateSocialResponsibility”atitsworst.ThepureideasaboutreflexivescienceoriginatingfromSTSandall theother relevantSSHfieldsbecomeco-opted,contaminatedandperverted.

Isuggestthatthistypeofdisappointmentcanbeovercomebyapply-ingourownscholarshipontoourownsituation;byanexerciseofrefle-xivity,asitwere.Indeed,ifourvisionwastoachieveimpactonalargescale,co-producinggoodnessinSTEMandthewholeworldbyfirstco-producingourownRRIknowledgeandtechnology,thenthiswasavisionofmassiveupscaling.Weknow,however,fromSTSandthehistory,philo-sophyandsociologyofscienceandtechnology,thatupscalingprocessesareopen-endedandthattheyintroducesurprise.Aboveall,otheractorswhoarenottrainedinSSHhavetobecomeenrolledintotheprogrammeofaction,andtheycannothelpbutmaketheirownsenseofthesepoliciesandpractices. Inside thebureaucracyof theEuropeanCommission, forinstance,thesuccessfuldeploymentofanypolicyconceptbothnecessi-tatesandhingesuponthedevelopmentofnumericalindicatorsandamo-nitoringsystem.Otherwiseitcannotsurvivewithintheinstitutionallogic.

The open-endedness and complexity of such processes also im-ply thatoneshouldnot trustone’sownassessmentandevaluationof

ISSUE 48 | JULY 2019 61

theprocesswhile it isunfolding.It isaparadoxthatwhileweasSSHteacherswillinstructourstudentsaboutthevirtuesofcriticaldistancetotheobjectofstudy,ELSAandRRIscholarsaretothehighestdegreebothparticipantsandobserversatthesametime;indeed,weseemtobeourownchroniclers.Thiscriticismhitsthepresentauthorasmuchasanybodyelseanditalsohitsseveralauthorsinthereferencelistofthispaper.ItisastrikingfeatureofSSHscholarswhoworkwithSTEMthatwewritequitealotaboutourselves.

NoneoftheseanalysesprovesthatRRIorotherstrivesforreflexivesciencesarefutileormeaningless.Theanalysisindicatesthetrivialcon-clusionthattherecanbenorecipeforsuccessbutalsotheslightlylesstrivial insight that success may be different from what was imaginedandmightbeidentifiedinhindsightandperhapsbyothersthantheSSHscholarswhowereinvolvedinthefirstplace.WeareremindedofHegel:“The owl of Minerva spreads its wings only with the falling of the dusk.” (Hegel1972,p.14)

At the same time,SSHscholarsare knowledgeworkers, andevenwhenconsciouslyinvolvedandengagedintheco-productionofsociety,weare involvedwithknowledgeproduction. Iwillend thissectionbyapersonalexample,takenfrommyworkfortheCentreforDigitalLifeNorway,anational,“virtual”(meaninggeographicallydelocalised)cen-treforsystemsbiologyandbiotechnology.Thecentreand itsresearchprojects are funded by the Research Council of Norway, and RRI is amandatory requirement inall researchprojectsand in theactivitiesofthecentrehubiv.IparticipateasoneoftheRRIcoordinatorsofthishub.InthiscapacityIseemyselfandmycolleaguesasknowledgeworkersin three respects. First, we teach and disseminate SSH knowledge toSTEM scientists, in particular PhD students and postdoc researchers,butalsotosomeextentthe“PrincipalInvestigators”whoareultimatelyresponsiblefortheirownimplementationofRRIintotheirrespectivere-searchprojects.Secondly,wemakesomeefforttoattendtothecoreoftheassumptionoftheRRIimaginary,namelytounderstandthepossiblerelationshipbetweenthemanymethodologicalchoicesintheSTEMre-searchanditscontextofimplication.Inthiseffortithasmadelittlesensesofarto“bringcitizensintothelab”.Rather,weworktounderstandthedownstreamimplicationsofchoosing,say,onetypeofcomputationalorbiologicalmodeloveranother.Thisisachallengenotsomuchofpartici-patorytechnologyassessmentasofWissenschaftstheorie,ofbeingabletopenetratedeepintotheepistemologicalquestionsofSTEMscience,actuallydeeperthanwhatisnormallyrequiredforSTEMdailypractice,toidentifysitesofde facto politicsinthelab.ThiskindofworkdependsoncombinedSTEMandSSHknowledgetotheextentthatithasprovendifficult todowithout“doublecompetence”, that is,personswhoaretrainedinbothSTEMandSSH.

Finally,wedoourownSSH-basedresearch,organisedintherecentlystarted Res Publica project, which is led by Dr Heidrun Åm.v The ResPublicaprojectwillamongotherissuesfocusonhowthebioeconomyisimaginedandattemptedtobecomerealisedbybiotechnology.Inotherwords,theprojectwillnotrestrictitselftothepotentialde facto politicsofminutemethodologicaldecisions in the laboratorybutalsokeepanopeneye for thede facto politicsofpolitics itself, in theconventionalsense of public decision-making and political institutions. Again, onecouldimagineafutureex-postassessmentoftheRRIendeavoursofthe2010stoconcludethattheyhadanSTSbiasandfocusedtoomuchontheimplicitmicro-politicsattheexpenseofattentiontopoliticalecono-my.PerhapsfuturehistorianswouldidentifythisbiasaspartofalargerSSHtrendatthebeginningofthe21stcenturyandrelateittoincreasing

differentiationandfragmentationofSSH.Evenworse,theymightrelatethefragmentationofSSHresearchtohowimportantissuesarelostoutofsight.TheymayevenconnectthistohowSSHstudentsandscholarsmaintainedanintersectionalistfocusonmicro-aggressionsinuniversitylifeinEuropeanandNorth-Americancountries,whilethepublicsphereinthesamecountriessawtheriseofpopulismandopenthreatstode-mocracy.Theremightbeaneedforawake-upcalltoengagewiththebigissuesandaskwhatisimportant.

LESSON 2: GOING FOR WORTHWHILE COLLABORATIONS BEYOND THE INVERTED DEFICIT MODEL

Thebigissuesdonotrespectdisciplinarybordersoreventhedistinc-tionbetweennatureandculture.ForSSHtogainimpact,itseemsthatSSHscholarshave to learnabout issuesoutsideof their usual scope,whichisanexcellentmotivationforresearch.Thishasbeenakeypointfromactor-networktheoryfordecades:Thedevelopmentofscienceandtechnology (and accordingly its governance) depends on many non-human actors: the genetically modified organisms, the nanoparticles,theCRISPR-Cassystems,theplasticinthePacificOcean–suchthingsthatSTEMresearchersknowmuchbetter thanus.Here there isa re-searchchallenge,notjustachallengetoeducateSTEMresearchersandpolicy-makerswithourperfectSSHunderstanding,anditisaresearchchallengethatisprofoundlyMode2inthesensethatitdemandscon-tributionsfromradicallydifferenttypesofdisciplinesbutalsothattheyleavetheircomfortzones.Iwillendthispaperwithanotherpersonalex-ample,notfromanRRIprojectbutfromthe“Horizon2020SocietalChal-lenge” project called MAGIC (making GRADE the irresistible choice)vi.InMAGIC,westudythescience-policyinterfaceforthegovernanceofthe water-energy-food nexus with a combination of ecological econo-mics,energetics,biosemiotics,sociologyandSTS,becausethisiscalledfortounderstandtheinteractionsbetweenthehuman,socialandnatu-ralagenciesinvolved,includingourownroleaschangeagents.Whereasitispossibletoclassifytheresearchersintheprojectas“mainlySTEM”and“mainlySSH”,thepracticeismoreusefullydescribedinGermaniclanguagesthathavelessdualisticconceptsforthe“TwoCultures”:WeareallWissenschaftler.Theremaybeoccasionswhenresearchersfromoneculture,saySSH,havetofillinknowledgegapsleftopenbySTEMandviceversa.However,theinteractiongoesbeyondseeingtheothersasemptyvesselswhoseknowledgedeficithastobecorrected.Iinterprettherichnessofinteractioninpartasaresultofthescopeandcomplexityoftheresearchtopic,namelythewater-energy-foodnexus.Inordertounderstandthebiophysicalsystemof,say,ariverandthesurroundingagriculture, one needs to understand the human, social, cultural andpoliticaldimensionsof thissystem.Conversely, inorder tounderstandandinterprettheintricaciesofpolicy-makinginthefieldofwatergover-nance,onealsoneedstounderstandwhatisatstakeinthepolicyde-bates,inbiophysicalterms.WhatwelearnintheMAGICproject,isthatSSHmethodsandtheoriesarenotvoidofimplicitassumptionsaboutthenaturethathumanstrytogovern,andthatSTEMmethodsandtheoriesalsoholdimplicitassumptionsaboutgovernanceandsociety.Partofthe


research challenge is to discover these implicit layers, explicate themandchallengethem.Thisisquitedifferentfromfillingdeficits.

Thereisnothinguniqueinthisparticularexample;therearewholejournals that identify features and approaches conducive to meaning-ful inter- and transdisciplinary work between SSH and STEM. When Iinvoke the example of MAGIC, it is to make a specific point by cont-rastingitwithfeaturesofELSAandRRIinteractionsthathavecreateddisappointment.Ihavelaunchedabovetheideaofgoingfor“big”and“worthwhile”issues;however,withfurthercommentthisideaisnothingmorethantwofinewords.AboveitwasnotedhowSTSmayhaveint-roducedabiastowardsthede factomicro-politicsoftheSTEMlabora-tory.Insomeinstancesthis“bias”maybeterriblyimportantandexactlywhatone should focuson; and itwas agreat achievement of STS todiscoverthemicro-politicsthroughthoroughempiricalresearchfromthe1970sandonwards.Still,onepotentialdisappointmentoftheELSAorRRIconvergenceworkeriscreatedfromtherealisationthattheactualmicro-politicsofaparticularSTEMresearchprojectmaybequiteunim-portantoruninteresting,orthatitmaybeimportantbutthatthereisnowillingness, neither in the practice or the governance of the science,tochangeanything.ThisispartlywhytheResPublicaprojectalsowillreturn to the “politics of politics proper”, to find other and promisingsitesfortheco-productionofthegoodfuture.IfaSTEMprojecthastheexpressandunshakeablegoalofproducingacyberneticsoldierorage-neticallymodifiedsalmon,theremaybelittleuseinspendingyears inthe lab todoRRIasakindofactivistethnographicaction researcher.ThecontrastwiththeMAGICprojectisstriking.Althoughitsmainpartisquantitativebiophysicalscience,itsgoalistorethinkandhelpchangethescience-policyinterfaceinthegovernanceofthenexus.Itaddressesa“big”issuenotbytryingtodeviceatechnicalsolutionbutbycreatingknowledgethatmayinduceinstitutionalchange.

Theopennessof theMAGICproject to theoreticaland institutionalchangefitsSSHreallywellandinparticulartheHforHumanities.SSHrarelysitswell incollaborations inwhich it is relegated toa technicalrole,definedbySTEM;thisisseenwellinthedisappointmentsdescribedabove.Indeed,beforethesplitofthe“TwoCultures”,thelaboratoryhadtobeinventedfornaturalphilosophytobecomeabletosolvetechnicalproblems. In theexampleof theMAGICproject,weaccordinglyseeamarkerofaworthwhilecollaboration:Thewillingnessofallparticipantstogobeyondthetechnicalchallengesandengagewiththeoreticalaswellaspractical-politicalchallenges.However, thismarker– indeedamarker of reflexive science already present – is sufficient but not ne-cessary. One could still strive for reflexive science, not necessarily tosolve theCollingridgedilemmabut toarrive to thepointatwhich theSSH-STEM collaboration becomes meaningful because a shared inte-restintheoreticalandpractical-politicalchallengeshasbeencultivated.Perhapswhathasbeen learnedthroughthesuccessesandfailuresofELSAandRRIendeavoursisthatSSHcannotprovideatechnicalfixtothelackofreflexivity.Rather,itbringsarepertoireandknowledgereser-voirthatmayormaynotberelevantinthecontextathand.Mechanicalandmindlessdeploymentofthatrepertoiremayendindisappointmentbecauseittriesdowhatespeciallythehumanitiesarenotatallequippedtodo,namely reduce theotherhumansubject (theSTEM researcher)to an object. For worthwhile collaborations towards reflexive sciencetodevelop, itseemsabetterstrategytocultivatecommonintellectualcuriosityandengagementtowardsthebigissuesacrosstheSTEM-SSHdivide.Partof that strategywill be to identify contexts inwhich such

commonalitiesarelikelytobepossible.Thisinsightreflectsbackonthepolicynarrativeofreflexivescience,however:ItmightmeanthatRRIorotherSSHinteractionswithSTEMwillnevercomeoff-the-shelf(DelgadoandÅm2018).

ACKNOWLEDGEMENTSTheworkleadingtothepresentarticlewasperformedaspartofthe

projectsMAGICandResPublica.MAGIChasreceivedfundingfromtheEuropeanUnion’sHorizon2020ResearchandInnovationProgrammeun-dergrantagreementNo689669.ResPublicahasreceivedfundingfromthe Research Council of Norway under grant agreement 270623/O70.Thisworkreflectstheauthor’sviewonly;thefundingagenciesarenotresponsibleforanyusethatmaybemadeoftheinformationitcontains.IammostgratefultotheeditorsofthejournalforfeedbackandtoIrmelinWNilsenforcarefulediting.

REFERENCES Balmer, A. S., Calvert, J., Marris, C., Molyneux-Hodgson, S., Frow, E., Kearnes, M., Bulpin, Pablo Schyfter, K., Mackenzie, A. and Mar-tin, P. (2016).Fiverulesofthumbforpost-ELSIinterdisciplinarycollabo-rations,JournalofResponsibleInnovation,3,73-80.

Delgado, A., and Åm, H. (2018). Experiments in interdisciplinarity:Responsible researchand innovationand thepublicgood.PLoSbiolo-gy,16(3),e2003921.doi:10.1371/journal.pbio.2003921

Fisher, E. and Schuurbiers, D. (2013).Socio-technical IntegrationRe-search:Collaborative Inquiryat theMidstreamofResearchandDeve-lopment.97-110.

Forsberg, E. M. (2014).InstitutionalisingELSAinthemomentofbreak-down?LifeScience,Society&Policy,10,1-16.

Forskningsradet (s. a.)Ethical, legalandsocialaspectsofbiotechno-logy, nanotechnology and neurotechnology (ELSA), from https://www.forskningsradet.no/en/Funding/ELSA/1253954329471

Hegel, G. F.(1972).GrundlinienderPhilosophiedesRechts.Suhrkamp,FrankfurtamMain.

Jasanoff, S., and Kim, S.(2009).ContainingtheAtom:SociotechnicalImaginaries and Nuclear Power in the United States and South Ko-rea.Minerva,47(2),119-146.

Kaye, J., Meslin, E. M., Knoppers, B. M., Juengst, E. T., Deschênes, M., Cambon-Thomsen, A., Chalmers, D., De Vries, J., Edwards, K., Hoppe, N., Kent, A., Adebamowo, C., Marshall, P. and Kato, K.(2012).Researchpriorities.ELSI2.0forgenomicsandsociety.Science,336,673-4.

Nydal, R., Myhr, A. I. and Myskja, B. (2011).NANOETHOS.ReporttotheELSAprogramme,theResearchCouncilofNorway,fromhttp://ge-nok.no/wp-content/uploads/2013/03/NANOETHOS_1_1.pdf

ISSUE 48 | JULY 2019 63

Nydal, R., Myhr, A. I. and Myskja, B.(2015).Fromethicsofrestrictiontoethicsof construction. ELSA research inNorway,NordicJournal ofScienceandTechnologyStudies,3,34-45.

Owen, R., Macnaughten, P. and Stilgoe, J. (2012). Responsible re-search and innovation: From science in society to science for society,withsociety.ScienceandPublicPolicy,39,751–760.

Rip, A. (2016).Theclothesoftheemperor.AnessayonRRIinandaroundBrussels,JournalofResponsibleInnovation,3,290-304.

Rommetveit, K., Strand, R., Fjelland, R. and Funtowicz, S. (2013).WhatcanhistoryteachusabouttheprospectsofaEuropeanResearchArea?Luxembourg:PublicationsOfficeoftheEuropeanUnion.

Sardar, Z. and van Loon, B.(2012).IntroducingPhilosophyofScience.London:IconBooks.

Schot, J. and Rip, A. (1997).Thepastandfutureofconstructivetech-nology assessment, Technological Forecasting and Social Change, 54,251-268.

Snow, C.(1959).TheTwoCultures.London:CambridgeUniversityPress

Strand, R. (2000).Naivety intheMolecularLifeSciences.Futures,32,451-470.

von Schomberg, R.(2011).Introduction.InR.vonSchomberg(Ed.),To-wardsResponsibleResearchandInnovationintheInformationandCom-municationTechnologiesandSecurityTechnologiesFields(pp.7-15),Lu-xembourg:PublicationsOfficeoftheEuropeanUnion.

AUTHOR

ROGER STRANDSenter for vitenskapsteori (Centre for the Study of the Sciences and the Humanities/Zentrum für Wissenschaftstheorie), University of BergenParkveien9,P.O.Box7805,Bergen,N-5020(Norway)E:[email protected]

KEYWORDS: SocialInnovation,SocietalImpact,SocietalTransformation,Empower-ment,ParticipatoryApproaches,CivilSociety

Endnotesi https://epsrc.ukri.org/research/framework/area/ii IwillusetheELSI/ELSAacronymsasalmostsynonymous.Theoriginalconcept,datingbacktothe“HumanGenomeProject”,wasELSI–ethical,legaland

socialimplications.AstheconceptcrossedtheAtlanticandbecameadoptedinEurope,itwasalsocriticisedforbeingtoonarrowlyconstruedintermsofidentifyingand“fixing”collateralsofthegenomicrevolutionwiththetoolsofbioethics,biolawandpatentlaw.ThechoiceofELSA–Aforaspects–thatwasmadeinsomenationalcontextsinEuropecanbeseenasanattempttoexpresstheawarenessofthiscriticismandtheintentiontohaveabroaderfocus.

iii Seehttps://www.rri-tools.eu/andhttps://www.rri-practice.eu/fortwoprominentexamples.iv https://digitallifenorway.org/gb/responsibilityv https://digitallifenorway.org/gb/projects/res-publicavi https://magic-nexus.eu/


ledgetransferandtodescribethemes,formatsandprogrammeelementstoillustratetheroleofSSHAsectorswithinuniversityoutreachactions.

KNOWLEDGE TRANSFER – WITH STRINGS ATTACHED FOR SSHA

Knowledgetransferisoftenassociatedwithutilisationofintellectualproperty,patent-orproduct-centredtechnicalinnovations.Theso-calledTTOs (“Technology Transfer Officers”) are responsible to deliver hardevidenceforbusinessesthroughlawyersandpatentoffices.ObviouslythereareseveralreasonswhytheSSHAfieldscannotbeveryinterestedinparticipatingwithinsuchkindoftransferbusiness.Itisevenhardtosay that the social sciences, the humanities and the arts are sharingthesameviewpointandgoalswithintheknowledgetransfersetup.Forpeople outside these fields, the acronym SSHA is not even graspableandthereforehastobebrought intoattentionbyawarenessactivitiesandgoodpracticeexamples.Fromacriticalpointofview,plentyofrea-sonshavebeenbroughtintodiscourseabouthowandwhythetransferofknowledgeandtheeconomictriggerbehinditwillcorrodethecorevaluesofSSHA(e.g.Castells1997,Lui2004,Boltanski/Chiapello2005).Nevertheless,thegoalandmotivationtocreatesocietalimpactthroughknowledgetransfertocommunitiesandpeers isseenasan importantissue.Ontheonehand,thejudgmentisoftendrivenbyastrongmistrust(e.g.Raunig2007)2accordingtothesystematicapproachtoincludeneo-liberalandentrepreneurialelements into the freedomofsciencesandarts.Ontheotherhand,social impactandresponsible innovationsareidentifiedascoredriversandmotivationstocontributeacademicknow-ledgeandresearchskillstoabroaderfieldofapplicationwithinsociety.Still,weseeproblemsonanindividuallevelofprecarityandstagnationarising,whichhavetodealwithfundamentalissueslike“makealiving”or feeling valuated for the work done, because academic assessmentstructuresarenotalignedtomarkandqualifytheseactions.Theclaimis:ValuationprocessesintheSSHAwithsocietalimpactaredeveloped;they“just”havetobeappliedinexistingstructuresofknowledgetrans-

INTRODUCTION

Basedonobservationswithinthestructuralframeworkprovidedby the “Wissenstransferzentrum Ost”1 (Knowledge TransferCentreEast),aninter-universitycollaborationprojectofallnine

Vienneseuniversities,thiscontributionispointingoutdifferentelementsdriven by the sectors of “Social Sciences, Humanities and the Arts”(SSHA)withintheparadigmofa“thirdmission”iofuniversities.Besidesthecorefunctionsofteachingandresearch,universityoutreachactivi-tieshavedevelopeddifferentformatstoprovideservicestopopulationswhomightnototherwisehaveaccesstothoseservices.Historicallythefieldofknowledgetransferhasbeendominatedbyvalorisationanduti-lisationmethodswithinthefieldoftechnologytransfer,whichisdealingmainlywiththesectorsof“Science,Technology,EngineeringandMa-thematics”(STEM).TheSSHAhadnotbeentakenintoaccountlargely,since the significance for societal development and contribution wereobviouslyintegraltoagreatergoodofhumankindandbythesemeansseemedobsoleteforutilisation.OnthecontrarytheSSHAfieldisforcedtogeneratearatherradicalandnewapproachofknowledgeexchangewithamultitudeofpeers–bothwithintheacademiccommunityandwithactorsandstakeholdersfromsocietyaswellaseconomy–iftheywanttobepartofthethirdmissionparadigm.Thepaperelaboratesondifferent approaches, formats and processes, which have been desig-nedandappliedwithinthecollaborationprojectsinceitsstartin2014,toexemplifysuchwillingness.Therewillbeafocusonexamplesfromtheknowledgetransferframeworkfocusingonexchangemethodsfromuniversitytosocietyatlargeandhowimpacthavebeencatalysedwithindifferentlevelsofimplementation.Basedontheseobservationsandlear-nings, thecontribution reflectson targetgroups, suchas researchers,alumni and university staff from different disciplines within the SSHAfield,withafocusoninter-andtrans-disciplinaryendeavours.Theaimistofigureoutthepotentialsandmethodologicaladaptionsforasubs-tantialintegrationoftheSSHAinexistingknowledgetransferstructures,todescribepathwaysofimpactorientednarratives,giveexamplesfromaprocessorientedandformatdrivenparticipatoryempowermentframe-work,tostrengthenthepositionofSSHAatlargeinthefieldofknow-

GEORGRUSSEGGERDOI:10.22163/fteval.2019.369

SSHA-DRIVENKNOWLEDGETRANSFERWITHINTHETHIRDMISSIONOFUNIVERSITIES

1 Seealso:http://www.wtz-ost.at(lastaccessed:14.11.2018)2 CreativeIndustriesasMassDeception,G.Raunig.Seealso:http://eipcp.net/transversal/0207/raunig/en(lastaccessed:14.11.2018)

ISSUE 48 | JULY 2019 65

fer and exchange channels.ii This cannot be done within a top downsuper-structuretreatingallSSHAfieldandstakeholderswiththesamepolicyinstruments.Itmightnotbeevenpossibletojustapplysomethinginanexistingunderstandingofknow-howtransferwithoutsharingthesamevisions,agendasandtools.

THE CASE – “KNOWLEDGE TRANSFER CENTRES” IN AUSTRIA

Thecase is referring toa funding scheme introducedby theAust-rianFederalMinistryofEducation,ScienceandResearch in2014andiscarriedoutbythefundingentity“AustriaWirtschaftsservice”tilltheendof2018.iiiOneModuleofthefundingschemeisdividingAustriainthreeregionalcentres(East,West,South).TheeasterncentreisViennawithitsnineUniversities.ivEverycentreisreceivingabouthalfamillionEuroperyeartocreate,developandcarryoutknowledgetransferacti-vitiesaddressingsocietyandeconomy.TheimportantandnewelementwithintheactionisaspecialawarenesstotheSSHAfields,designatedthroughanadditionalbudgettothementionedfundingearmarkedforSSHArelatedactivitiesonlyandthustheobligationforexistingtechno-logyandknowledgetransferstructuresto includeSSHAintheirdeve-lopmentsandactions.This rathernewandexperimentalapproach ledtothedevelopmentofseveralnewknowledgeexchangeformatstryingtoidentifyandaddressnewtargetgroups.Includingworkshops,lectu-resandseminarstoempowerresearchers,universitystuffanddoctoralstudentsforparticipatorymethodsofknowledgeexchangee.g.citizenscience,or todeveloptrainingformats foralumnibasedontopics likesocialentrepreneurshipandsocialbusiness.Theimpactonthislevelofdevelopmentcouldbedescribedas“impactbydesign”.Pre-existingpa-thwaysandapproachesmainlyfocusedonexploitationandvalorisationofproducts(e.g.patents,inventions,technologies)insteadoftheempo-wermentofcertaingroupsandstakeholderstocontributeanddistributetheirknowledgeandskillstosociety.Tochanneltheseactionsasecondlevelhastobedelineatedwithinanexternalcooperationframeworkoftheknowledgetransfercentre,toensureexternalorganisationunitscanreceive and apply knowledge on inter- and trans-disciplinary level. Byidentifyingandincludingexternalorganisationsandstakeholdersasex-perts,mentorsandpartners,awarenessraisinganddisseminationintonewareasofapplicationwasaccomplished.Furthermoreanetworkofpartnersbasedonaqualitativecooperationexperiencecanmultiplyout-comesandfomentnewactions.Theimplementationoftheknowledgetransfercentreshowed, ifaclearapplicablecooperation framework isprovidedbyuniversities,itisusedbythementionedtargetgroupsandpartnerorganisations.

THE CHALLENGE – TAILORING PROCESSES WITHOUT BLUEPRINTS

Theopportunity to createanddevelopa sustainable frameworkofSSHAknowledgetransferbetweenallVienneseuniversitiesandallre-gionalcentres inAustria isanappealingchallenge.Theestablishmentof a communication pattern for all responsible university entities andcentresisoneofthem.Anotheroneistoestablishacompliantformofinteractionto findcommongroundsand languagesofSSHAinterests.Thekeyquestionisabouttheimplementationofsupportlevelsindiffe-rentorganisationalstructuresandcultureswithintheuniversitysystem.However the project structure given by the funding entity was rathervague and had not foreseen a clear structure of contents and trans-feractivitieswithin theapplication.Theprojectpartnersdelivered theconceptualoutlineandmainemphasisoncontent.TheSSHAsectorinViennaisledbytheAcademyofFineArtsViennadealingwiththefactofminorpreliminaryworkdoneconcerningknowledge transferat thepartners, and hardly any structures established in the field. The mainaim was to create new, open, flexible and interdisciplinary formats ofexchangeandtransfertosupportabottomupprocessintheSSHAfieldbased on projects, rather than defining knowledge transfer processesastailormadeblueprints.Theorganicvisionforcapacitybuildingintheuniversitiesandtheidentificationofcooperativecommunitiesinsocietyandwitheconomystillhadtobenarroweddown,especiallyconcerningthementionedtargetgroups,contents,andchannelstodesigntoolsandformatsforseveraldifferentpatternsintheSSHAspectrum.Forthisrea-son,socalledtransferandcreativityhubshavebeendesignedtodelivertrainingprogrammeswithcertainSSHArelevanttopics.Oneofthemostintegralpartswithinthedevelopmentplanwasrelyingontheempow-erment of diverse target groups within the university system, such asuniversitystaffandresearchers.Additionally,aprioritywassetontar-getingalumni tobroadentheresponsibilityofknowledgetransferandtocreatealevelofexchangewiththeircompetencesandneeds,tooffernewconnectivityoptionsandpeerlearningformatsforallparticipants.Bythesedecisions,atrustislaidinidentifyingunknownandupcomingactorsofknowledge transferwithinabroaderangleofempowermentandknowledgeproduction,inrelationtouniversitiesbutnotnecessarilyonlyfromuniversities.KirstenLangkildedescribestheinterplaybetweenacreativeandcriticalcultureandtherealmofsocietyas“impact of cul-ture” (Langkilde,2018,p32).Theprocessofproductionisunderstoodasacontributiontosociety.Knowledgetransferchannelscansupportsuchelementsbytailoringprocessestoaddressspecialinterestcommunitiessuchassocialbusinessesorimpactinvestorstoprovideaparticipationorcooperationframework.Tointermediatebetweenacademicprojectsandexternalpartnerstheknowledgetransfercentrehelpedtosupportonbothsidesofthespectrum.


IN THE MAKING – FORMATS AND TOOLS

Firstofallamulti-levelparticipatoryapproachfortargetgroupsandcooperation partners was outlined to be adapted by different hostinguniversityandopinionleaders.AllofthedesignedtoolsandformatsofknowledgetransferwithintheSSHAsectoroftheKnowledgeTransferCentreEastareratheroffersthaninstructions.Sincenoteverypartneruniversity in thenetwork isworkingunderthesameconditionsand ishavingasimilardevelopment status,equally claimedneedshadbeenreplacedbyoffersthatcanbeused,adaptedandmodifiedbytheuniver-sitiesfortheirpurpose.Workinginanetworkmadeiteasiertoexchangeoffersandapplythemwithintheexistingstructures.Allactionsareopentoalluniversitypartnersand freeofcharge toensurean interdiscipli-naryexchange.Theofferscoverextracurriculacourses,workshopsandtrainingprogrammes,calledtransferandcreativityhubs,tocovertopicsandaspectsofdiversebackgroundsidentifiedfromtheSSHAfields.Forexampleacontinuingeducationprogrammevwithmorethanfortywork-shops,lecturesandseminarsperyearishostedbytheVienneseuniversi-ties.Differentlocationsandtopicsshouldhelptoidentifypotentials,sup-portprojectsandcatalysecommunitiesaroundspecialinterestgroups.Thelow-thresholdoftheformatsshouldraiseawarenessforupcomingandexistingelementsofknowledgetransferandattractpeerstolearnmore about contemporary developments and upcoming topics withintheSSHAtransferactivitiesanditsimpactonsociety.Experts,partnersandstakeholdersfromtheregionhavebeenincludedintheactivitiesasspeakersand trainers tocarryouta largeproportionof theeducationprogramme,tomultiplythechannelsofdisseminationandtorecognisenew actors. This rather generic approach was driven by the fact thatmostoftheactorswithintheSSHAfieldwearereachingaremotivatedtoapplytheirexpertiseinmeaningfulactions,theywanttomakealivingfromwhattheydoandarelookingforcooperationandcommunitiestoexpand their sphere of action. Supporting transferable skills and peerlearning within a consistent setup of programmes is building trust insustainablestructureswhereindividualknow-howandexpertisecanbeadjustedwithingroups, leading to connectionswithexternalpartnersandassuringthestepbystepdevelopmentofarecognisedpracticeforculturalwellbeingasintegralpartofsocietalchallenges.Fromatechni-cal levelweusedtheextracurriculacoursesto identifytheinterestofparticipantsonaquantitativeandqualitative level todecide ifamoreintensivesetupofacreativityandtransferhubisuseful.

TRANSFER FOR THE PEOPLE – IT’S ALL ABOUT CONTENT.

Withinthemultitudeofstructuresand interactionpatternsbuilt toidentifyandsupporttheneedsoftransferculturesintheSSHAandtodeliverittoalargercontextofsociety,preciseexamplesandgoodpracti-cesarethekey.Knowledgetransferandallitsimplications,especiallyintheSSHAfields,havetobechannelledintangibletopicstoobserveaso-cialorsocietalimpact.BythismeansIwanttopointoutsomeexamplestoelaborateon.Withinthefour-monthtrainingprogrammecalled“MakeyourselfanExpertHub”viwefocusedondifferenttopicssuchas,howtomakealivinginthearts,howtosetupasocialbusiness,howtocreateacrowdfundingcampaign.viiThetrainingsalwayshavebeenintroducedbyacall,assessedbyajury,carriedoutwithexternalstakeholderandcooperationpartners.Trainersandexpertsfromthefielddeliveredsta-teoftheartexamples,helpedwiththesetupofteamsandcontributetowardsprojectmanagementskillsandsupportingchannels.Bythees-tablishmentofmicro-fundingandfinancialsupport,structuresexternalfundingsourcescouldhavebeenconvincedtocooperate,socalled“im-pactinvestors”havebeenmadesensitivetothecontributionsofSSHAtosocietyatlargeandstakeholdershadthechancetounderstandbetterwhySSHAachievementsarehavingafundamentalsignificancetosoci-ety,whichisunderaheavytransformationprocess.

Usingtheexampleof the“SocialBusinessHub”weworkedclose-lywithpartners like“magdashotel”asocialbusinessoperated in theframeworkofCaritas.Theyprovideworkforrefugeesfromcrisis-riddencountries to offer them the chance to demonstrate their abilities andtalents.3Inadditionweincludedorganisationslikethe“ImpactHubVien-na”,theplatform“SocialCityVienna”andthe“SocialEntrepreneurshipCenteroftheViennaUniversityofEconomicsandBusiness”todevelopatrainingprogrammeforthe“SocialBusinessHub”.Withinthehubwecalledforprojectsfromtheuniversitycontextwithsocialbusinessideasdoco-developandco-createtheirideas.Outof25applicationstenpro-jectshavebeenselectedtobringtheirideastolife.Aftertwomonthofintensiveandproductivetrainingtheprojectspitchedtheirideasinfrontofthepartners,potentialinvestorsandstakeholderfromthefieldofso-cialbusiness.4Morethan50%ofthesupportedprojectsfoundedasocialbusiness.Othersgotofferstodevelopfurtherwithinprojectcooperationorfoundpartnerstogoonwith.

CONCLUSION

One of the greatest achievements within the logic of knowledgetransfer activities with people from a multi-disciplinary background isthebelief tocontributesomethinggoodand important forpeopleandsociety.Evenifthissoundspathetic,butthecredo“tomaketheworldabetterplace” is enablinga very contemporary cultureof knowledgebeyond the fact of economical reason. In general social, cultural and

3 Seealso:5goodreasonstostayatmagdasHotel:https://www.magdas-hotel.at/en/hotel/5-reasons(lastaccessed:14.11.2018)4 Avideoofthepresentationeventisavailablevia:https://vimeo.com/182367688(lastaccessed12.11.2018)5 ConferenceforKnowledgeExchangeandTechnologyTransferProfessionals,Vienna2018,Panel:OnArtsIncubators&CulturalAccelerators.Seealso:http://

www.wtz-ost.at/conference(lastaccessed:05.11.2018)

ISSUE 48 | JULY 2019 67

artisticapproachesareoftenmisunderstoodasphilanthropic,withami-noreconomicaldimension.Thementionedtrainingsaswellas„ArtisticIncubatorsandCulturalAccelerators“asdiscussedatthe„ConferenceforKnowledgeExchangeandTechnologyTransferProfessionals,Vienna2018“5aredealingwiththisbiasbyfocusingonsubstantialtransferandexchangeactivitiesbasedonsocio-culturalchangeandcriticalreflectiontocontributedeeplyforsocietalchallengesandthecontemporarytrans-formation of society. The empowerment of social scientist, artists andculturalworkers in collaborativeand interdisciplinarydevelopmentsofnewframeworksofactionsuchasartisticincubatorsandculturalaccele-ratorshaveledtoauniqueperspectiveoneconomicframeworkshavingthecommongoodandsocialimpactatstake.FortheSSHAsectorsandespeciallyforresearchersinthefield,thedescribedfieldsofactioncouldberatherseenasachancethanasaburden.Takingthesechallengeswithanunderstandingofdiversity,responsibility,ethicalguidelinesandsustainability for thewholesociety intoaccountwillaswell influenceandchangeencrustedeconomicaldrivenstructures.Despite thecom-mendablefact,monetaryaspectscannotbedisregarded.Furthermore,aclearvisionoffinancialneedsandcompensationfortheseactionshastobedevelopedwithintheSSHAinterests,todeliveraseriousapproachbeyondcounterculturesanddependenceonthirdpartyfundingapproa-ches.IftheSSHAfieldswanttosucceedinregardofthecontemporarysocietalchallengestheyareforcedtostepoutofthecomfortzone.Tohelpandsupportthisprocessknowledgetransferbetweenuniversitiesandsocietycanbeausefultool.

REFERENCES Buckland, M. and Gey, F. (1994).Therelationshipbetweenrecallandprecision.JournaloftheAmericanSocietyforInformationScience,45,12-19.

Boltanski, L. and Chiapello, E. (2005). The New Spirit of Captialism.London:Verso.

Castells, M. (1997).Powerof Identity:TheInformationAge:Economy,Society,andCulture.Cambridge,MA:BlackwellPublishers.

Langkilde, K.(2018).ImpactofCultures.In:WissenstransferzentrumOst(Ed.),Wissenstransfergestalten.Werkzeuge,Formate,Potenziale.Vien-na:facultasVerlag.22-33.

Lui, A. (2004).TheLawsofCool:KnowledgeWorkand theCultureofInformation.Chicago:UniversityofChicagoPress.

Raunig, G. and Wuggenig, U. (2016). Kritik der Kreativität. Vienna:transversaltexts.

AUTHORGEORG RUSSEGGERCentre for Knowledge Transfer, Project Lead: Knowledge Transfer Centre East, SSHA SectoAcademy of Fine Arts ViennaSchillerplatz3,Vienna1010(Austria)E:[email protected]

Endnotesi Survey by the Institute for Advanced Studies. (2012) (in German language): https://www.wien.gv.at/wirtschaft/standort/pdf/third-mission.pdf (last ac-

cessed:15.06.2018)ii BenneworthB.andJongbloedB.W.(2009).Whomatterstouniversities?Astakeholderperspectiveonhumanities,artsandsocialsciencesvalorisation.

(Articleispublishedwithopenaccessat:springerlink.com)iii Funding scheme and funding guidelines (in German language): https://www.aws.at/foerderungen/aws-wissenstransferzentren-und-ipr-verwertung-

modul-1a-regionale-wissenstransferzentren(lastaccessed:12.05.2018)iv ThenineVienneseuniversitieswithintheKnowledgeTransferCentreEast:AcademyofFineArtsVienna,UniversityofAppliedArtsVienna,Universityof

MusicandPerformingArtsVienna,UniversityofVienna,ViennaUniversityofEconomicsandBusiness,TechnicalUniversityVienna,UniversityofNaturalResourcesandLifeSciencesVienna,MedicalUniversityVienna,UniversityofVeterinaryMedicineVienna.

v DetailsabouttheactualeducationprogramoftheKnowledgeTransferCentreEastisavailableonline(inGermanlanguage):http://www.wtz-ost.at/veran-staltungen(lastaccessed:17.06.2018)

vi Videoexamplesofprojectsandsupportedprojectscanbeviewedviaonlinestreamingvideochannel:https://vimeo.com/akbild(lastaccessed:17.06.1018)vii Seealso:http://www.crowdfundershub.at(WebsiteinGermanlanguage,lastaccessed15.06.2018)


those interactions may help policy makers to stimulate the conditionsforimpactandthroughthisincreasinglyopenupfortheprobabilitythatresearchprojectsdogeneratenotonlyscholarlybutalso(andespecially)societalimpact(Spaapenetal.2011;Eric2010).

IntheERiCprojectiandinitssuccessortheSIAMPIprojectii,thecon-ceptofproductiveinteractionswasdeployedtostudyresearchimpactinabroadway,includingsocietalimpact.Theapproachwasappliedinseve-ralscientificandtechnologicalfieldslikeinformationandcommunicationtechnology(DeJongetal.2014),architecture(DeJongetal.2011),law(VanArensbergenetal.2010),electronicengineering(Proppetal.2010);mechanicalengineering(VanderMeulenetal.2010),inbiomedicalfields(Prins2010),andinthesocialsciencesandhumanities(Molasetal.2011).iii

Many of these productive interactions or “impact pathways” werefound when the investigations were focused on the direct or indirectlinksbetweenusersandproducersofknowledge.Morerecently,thecon-ceptofco-productionofknowledgehasbecome fashionable,pointingat collaboration between researchers and stakeholders in the processofknowledgecreation.It isexpectedthatsuchcollaboration,inwhichstakeholdersbringinlocalknowledgeaboutthetopicunderstudyandknowledge about possibilities and constraints of applying knowledge,thedisseminationanduseofscholarlyresearchoutputwillbemorefre-quent,easierandfaster(seeamongothers:Wardenaar2014;Heggeretal.2012;HeggerandDieperink2014;DjenontinandMeadows2018).Themodelofco-productionismainlydeployedinstudyingcomplexproblemssuchasclimatechangeandenvironmentalstudies,andmanyproblemsarestilltobesolved,suchasresourcingknowledgeco-productionandtheculturaldifferencesbetweenresearchersandstakeholders(Djenon-tinandMeadows2018).Furthermore,theroleofstakeholders isoftennotsomuchintheco-productionoftheknowledge,butmoreinthestartoftheprojectwhentheresearchquestionsareformulated,andintheendwhendisseminatingthenewknowledge(Wardenaar2014).

However,alsootherinteractionsmayberelevant.Asresearchisde-

ABSTRACT

Itisoftenarguedthatthepresenceofstakeholdersinreviewpanelsmayimprovetheselectionofsocietalrelevantresearchprojects.Inthispaper,weinvestigatewhetherthecompositionofpanelsinde-

ed matters. More precisely, when stakeholders are in the panel, doesthatresultinmorepositiveevaluationofproposalsofrelevancetothatstakeholder?Weinvestigatethisforthegenderissuesdomain,andshowthatthis isthecase.Whenstakeholdersarepresent,therelevantpro-jectsobtainamorepositiveevaluationandconsequentlyahigherscore.Ifthesefindingscanbegeneralised,theyareanimportantinsightforthecreationofpathwaystoandconditionsforimpact.

INTRODUCTIONThere is an increasing awareness that to generate impact, focus

should be on the relations between knowledge producers and know-ledgeusers,onrelationsthatcanbeseenasthepathwaystoimpact.AmainreasonforthisisitmaytakemanyyearsafterR&Dprojectshavetakenplacebeforeimpactbecomesvisible.Ifonewantstoevaluatethepossiblesocietalimpactofresearch,itmakessensetofocusonthecon-ditions that increase theprobabilityof impact. Therefore,more retros-pectiveresearch isneededto identify theconditionsforcontemporaryimpactofresearchdoneinthepast.

Onemayarguethatthenatureoftherelationsbetweenknowledgeproducersandknowledgeusersmayhelptoincreaseutilisationofknow-ledge,whichinturnmayleadtoimpact.Theterm“productiveinterac-tions”hasbeensuggestedfortheserelations,andoneoftheobjectsofresearchwithin this topic is to identify the varietyof interactionsandhow they are formed in different knowledge domains. Understanding

PETERVANDENBESSELAARANDULFSANDSTRÖMDOI:10.22163/fteval.2019.370

PANELCOMPOSITIONASPATHWAYTOIMPACT:DOWENEEDSTAKEHOLDEREXPERTISETOSELECTRELEVANTMISSION-ORIENTEDPROJECTS?#

# Theauthorsacknowledgethe“ExpertGroup”onthe“InterimEvaluationofGenderEqualityasacrosscuttingissueinHorizon2020“(Knolletal.2017).

ISSUE 48 | JULY 2019 69

pendentonincreasinglyscarcefunding,animportantpathwaytoimpactmay be the selection of the “correct” projects – projects that includethecreationof impactpathwaysasdiscussedabove,butalsoprojectsthatdirectlyorindirectlyfocusontherelevantsocietalissues.Itisoftenclaimed that thepresenceof stakeholders in thepanelsmay increasetherelevanceandpossiblytheimpactofresearch:extendedpeerreview(Nowotnyetal.2001)throughbroadeningpanelandpeerreviewbyin-cludingpractitioners (Cornelletal.2013). In thispaperwe investigatewhetherthisindeedworks:doesthepresenceofstakeholdersinfluencetheselectionprocess?Inthispaperweuseaninnovativemethodtoin-vestigatewhetherthisindeedworks:doesthepresenceofstakeholdersaffecttheselectionprocess?Wedosoforacasewhereselectionpanelsdifferintermsofmembership:someincludespecialistsongenderissues,whereasothersdonot.Aretheformerpanelsmorepositiveaboutpropo-salsthathaveagenderrelevantdimensionthanthelatter?

GENDER ISSUES IN RESEARCHResearchongenderandsciencehasbeenfocusedonthepositionof

womeninscience,suchasongenderbiasingrantallocation(WenneråsandWold1997;VandenBesselaaretal.2018)andinacademiccareers(Brouns2003;BenschopandBrouns2003;VandenBrink2006).

Morerecently,theeffectofunderrepresentationofwomeninscienceonthecontentofresearchhasbecomeaprominent issue.Biomedicalresearchisagoodexample,asithasbecomeclearthatinmuchclinicalresearch only male subjects were included. This has blinded the fieldfor gender differences in symptoms, diagnosis and in medication andtreatment(Chapmanetal.2013).Atthesametime,severalstudieshaveshownthatfemaleresearchersaremoreinclinedtotakegenderissuesintoaccountthanmaleresearchersdo(Nielsenetal.2017).Ifthisholds

for research and research output, it may also hold for research input:grants.Arefemalepanelmembers,and–morespecificallyforthispaper–panelmembersthatspecialiseingenderissuesmoreinclinedtoselectproposalswithagenderdimension?

In this paper, we address the question whether the availability ofgenderexpertiseingrantselectionpanelsdoesmatter.Dopanelswithgenderspecialistsandpanelswithoutgenderspecialistslookdifferentlyatproposals?And,dopanelswithgenderexpertisehaveamorepositivereportongenderrelatedproposalsthanpanelswithoutgenderexpertiseonboard?Afterhavingansweredthisquestion,theissuecomesupinwhatrespectgenderexpertiseisrelevantandisinfluencingtheselectionprocess.Toanswerthat,observationsofpanelswouldbeneeded,andthatfallsoutsidethescopeofthispaper.However,theresearchquestionisalsorelevantfromapracticalperspective.Ifwefindapositiveeffectofgenderexpertiseontheselectionprocess,thisknowledgecanbeusedforcomposingpanels,evenifwedonotknowhowitexactlyworks.

DATA AND METHOD DATA

We use a dataset covering 111 granted project proposals, and allwere considered by the applicants as gender-relevant.iv The projectsweresubmittedinvariouscallsintheHorizon2020programme,allwithatwo-stageprocedure.Table1givesanoverviewofthecallsthepropo-salswereaddressing.

Bordersecurityandexternalsecurity(BES) 2

Disaster-resilience:safeguardingandsecuringsociety,includingadaptingtoclimatechange(DRS) 5

EnergyEfficiency(EE) 5

Meetingnewsocietalneedsbyusingemergenttechnologiesinthepublicsector(EURO) 3

Overcomingthecrisis:newideas,strategiesandgovernancestructuresforEurope(EuroSociety) 15

Fightagainstcrimeandterrorism(FCT) 2

RRIuptakeincurrentresearchandinnovationsystems(Garri) 2

Informationandcommunicationstechnology(ICT) 11

Newformsofinnovation(INSO) 2

Energytransition(LCE) 1

Mobility(MG) 11

Nanotechnologies,advancedmaterialsandproduction(NMP) 1

Healthresearchandinnovation(PHC) 19

Innovativewaystomakescienceeducationandscientificcareersattractivetoyoungpeople(SEAC) 7

Smallfarmsbutglobalmarkets:theroleofsmallandfamilyfarmsinfoodandnutritionsecurity(SFS) 3

Aresourcetorecycle,reuseandrecoverrawmaterials(Waste) 9

WaterInnovation:BoostingitsvalueforEurope(Water) 6

Theyounggenerationinaninnovative,inclusiveandsustainableEurope(Young) 7

All these projects are “flagged” as gender relevant, which means that the applicants claim that their project has a relevant gender

Table 1.Numberofproposalsinthesamplebycall.


dimension.vHowever,an independentevaluationoftheproposals– interms of the “Cross-Cutting Monitoring Indicator” (CCMI) – indicateswhetheragenderdimension is reallypresent in theproposals.Of the110projects in the sample, 60get apositive CCMI indicator scorebyan“independent”projectofficer.Thismeans that thesampleconsistsof60genderrelevantapplications,and30applicationsthatlackgenderrelevance.For17projects,thevariableismissing.Obviously,manyappli-cantstrytoselltheirprojectasgenderrelevant–evenifthisisnotthecase.This isnotunexpected,astheymayhopethatthis improvestheprobabilitytogetfunded.vi

Apart from informationabout thegendercontentof theproposals,wehaveinformationabouttherelevantpanels.viiWedoknowforeachoftheprojectswhetheratleastoneexpertongenderissueswasintherelevant panel – which was the case for 71 of the projects. We alsoknow the scores theproposalshave received from thepanels. Finally,wehavetheevaluationreportsabouttheproposalsinthesecondstageoftheevaluation.Alldatawereprovidedbythefunder.Theinformationavailableenablestodistinguishfourconditionswhichwillbeusedfortheanalysis.TheseareshowninTable1.

Table 2.Sample:genderexpertiseandCCMI*.

Gender expert in panel

CCMI No Yes

No 22 11

Yes 15 45

Missing 2 15

Total 39 71

*“Cross-CuttingMonitoringIndicator”

METHODS

Inorder toassess theevaluationof theprojectproposals,and theinfluenceof(inthiscasegender)stakeholdersinthepanel,weperforma linguistic analysis of the review reports. The “Evaluation SummaryReports” (ESR)consistof (i) theprojectsummaryproducedby theap-plicantsand(ii) thereviewtext (includingthescores)producedbythepanel.Summaryandreviewwereseparatedfortheanalysis.Weusethesummariestocheckwhethergenderismentionedinthecontentoftheresearch.Thereviewtextwasusedtoinvestigatewhetherthereviewsexplicitly relate togender,andtoanalysewhether thereviewshaveamorepositiveornegativetone.

TheESRfilesareinPDFformat,andwedidsplitallfilesinareviewpartandinasummarypart.Then,thePDFfileswereconvertedintoplaintextfilesandfromthesefilesthe“standardtext”wasdeleted,suchasheadingsofsections.TheremainingpartsofthefileswereimportedintothetextanalysissoftwareCorTexT1fortermextraction.Forfindingtermsreferringtogenderandgenderissues,weusedboththesummaryandthereview.Forthelinguisticanalysisweusedonlythereviewtexts(seebelow).

Findingtermsthat refer togenderandgender issueswasdonebymanual inspection of the word lists produced by CorTexT. The review

partswerealsousedforalinguisticanalysisinordertodistinguishbet-weennegativeandpositivereviews.ThelatterwasdoneusingLIWC2,a tool for linguistic analysis of texts. The tool works with a variety ofpredefinedlinguisticcategoriesandhasbeenappliedregularlyfortheanalysisof reviews (Kaatzetal.2014a;VandenBesselaaretal.2016,2018b).Eachlinguisticcategoryconsistsofasetofwordsrepresentingthat category, which have been validated in other studies (Abele andWojciszke2014).TheLIWCprogrammecountsforeachofthecategorieshowmanytimesawordbelongingtothatcategoryispresentinareviewreport.As the reportsareofdifferent length,normalisation isneeded:thenumberistranslatedintoapercentage.Inthiscase,westartwithusingthosecategoriesthataretestedandusedinpreviousstudiesongrantdecisionandpaneldeliberation(Kaatzetal.2014a;VandenBes-selaaretal.2016,2018b):

• Ability words,suchasgift*,intell*,skill*;• Achievementwordssuchascreati*,excel*,compet*;• Agenticwordssuchasoutspoken,solid,risk;• Negativeevaluationwordssuchasnaïve,defect*,lack*;• Positive evaluation words such as intriguing, compelling, com-

mit*;• Researchwordssuchaslaboratory,result*,fund*;• Standout adjectives such as world class, outstanding, excep-

tional*.The term extraction of the review reports resulted in a list of fre-

quentlyused(stemmed)terms.This listwas inspected inordertofindadditionalreviewtermsnotincludedintheabove-mentionedlinguisticcategories.Basedonthetermextraction,thefollowingadditionallingu-isticcategoriesareadded:

• Negatingwordssuchashasn’t,don’t,can’t;• Negativeemotionswordssuchasabuse*,bitter*,bad*;• Positiveemotionswordssuchasagreeabel*,benefit,helpful;• Exclusionwordssuchasbut,either,except,just,not;• Insightwordssuchasdefine,reflect,idea*;• Certaintywordssuchasfundamental,commitment,truly.

Whyweretheseadditionalcategoriesselected?Firstly,astermex-tractionshowsthatthecategoriesmayplayarolegiventhefrequencytheyappear.Fornegationwords,anadditionalargumentisthattheex-cellentapplicantsarethenorminscience,andtheothersaremeasuredagainstthoseexcellent:“notexcellent”.Exclusionwordsmightbeusedbiasedbecauseofthesameargument.Positiveandnegativeemotionsarerelevanttoinclude,asonewouldwanttoseehowstrongsentimentsplayaroleinpaneldeliberation.

RunningLIWCgives forevery review thepercentageofwordsbe-longingtoeach linguisticcategory.Wecannowcomparetheaveragefrequenciesofthelinguisticcategoriesbetweenthoseapplicationsthathave a positive CCMI score versus a negative CCMI score, and thoseevaluatedbyapanelwithgenderexpertiseorbyapanelwithoutsuchexpertise.As therearesomemissingvalues in theCCMIvariable (17)weactuallycaninclude93projectsintheanalysis.Aswealsohavethescorestheproposalsreceived,wecanalsocomparethescoresforthefourgroupswiththeresultsofthelinguisticanalysis.

RESULTS

1 http://cortext.risis.eu/login2 http://liwc.wpengine.com

ISSUE 48 | JULY 2019 71

ARE THE SUMMARIES OF THE PROJECTS GENDER RELATED?

Manuallyinspectingthewordsusedintheprojectsummariesshowsfirstly thatwords likesex,male,and femalearehardlyused.Thewordgender isused in theprojectsummaries, inadifferentway in the fourconditions(Table2).Inthetwogroupswithproposalsthatwereflaggedasgenderrelevant,40%usesthetermgender,whereasthisisonlythecase for 27%of thenon-gender relevantproposals. In the reviews thepattern issimilar.Comparing the twosetsofproposals thathavebeenevaluatedbypanelswithgenderexpertisewiththeothertwosets,showthatpanelswith thegenderexpertsmore frequentlyevaluate in termsofgenderissues(39%),whereastheotherpanelsdothisinonly14%oftheproposals.Asatentativeconclusion,panelsincludingstakeholdersdomoreoftenevaluateproposalspartlyfromthestakeholders’pointofview.

Table 3.“Gender”inthesummaryandinthereview.

Gender expert in panel

CCMI No Yes

”gender”in No 5(23%) 4(36%)

summary Yes 3(20%) 21(47%)

”gender”in No 2(9%) 2(18%)

review* Yes 3(20%) 20(44%)

*excluding“genderbalance”intheteam

Moresophisticatedapproachestothisarepossible.Weonlyusedthetermgender,butonecouldthinkofproducingontologies(orstructuredthesauri)describinggenderrelevanttopics indetail,andusetheseforanalysing thecontentof theproposals (e.g.,VandenBesselaaretal.,2017).Thisapproach,however,isoutsidethescopeofthispaper.

ANALYSING THE REVIEW REPORTS

Weusealinguisticanalysisofthereviewreports,asdescribedinthemethodssection.Wecomparethefourgroupsofproposals,definedbythetwocorevariables: (i)availabilityofgenderexpertise inthepanel,and(ii) theCCMIscoreforgenderrelevance.Weusegroup4asrefe-rence:genderexpertisepresentandapositiveCCMIscore.

1. Group 1 (no gender expertise, negative CCMI) versus group4 (gender expertise, positive CCMI): Compared with Group 4,Group1hasasignificanthighermeanscoreonnegative emo-tions(mean=1.14vsmean=0.70,p=0.004),agentic language(mean=2.96vsmean=2.57,p=0.037)andonnegative evalu-ation(mean=2.13vsmean=1.22,p=0.000),andasignificantlowermeanscoreoninsight(mean=2.34vsmean=2.90,p=0.008)andonpositive evaluation(mean=8.12vsmean=9.10,p=0.070).Asthesescoresaregenerallynotnormallydistribut-ed,weusenexttoAnalysisofVariance(tocomparethemeans)also a non-parametric test (to compare the mean ranks). Thisshowsthatcomparedwithgroup4,group1hasasignificanthighermeanrankonnegative emotions,agentic,andonnega-tive evaluations,andasignificant lowermean rankon insightandonpositive evaluation.So,bothtestsgivethesameresults.

2. Group2(genderexpertise,negativeCCMI)versusgroup4(gen-der expertise, positive CCMI): Compared with group 4, group2hasasignificanthighermeanscoreonnegative evaluations (mean = 1.79 vs mean = 1.22, p = 0.061), and a significantlowermeanscoreonpositive emotions(mean=2.75vsmean=3.70,p=0.005)andonpositive evaluation(mean=7.57vsmean=9.10,p=0.037).Again,asthesescoresaregenerallynotnormallydistributed,weusenexttoAnovaalsoanon-par-ametrictest.Comparedwithgroup4,group2hasamarginally(non-significant)highermeanrankonnegative evaluation,andasignificantlowermeanrankonpositive emotionsandonpositive evaluation.Bothtestsgivesimilarresults.Themarginally/non-significanceisduetothesmallnumberofcasesingroup2.Theconclusionofthisanalysisisthatpanelswithgender-specialistsaremorepositiveongender-relatedprojectsthannon-gender-relatedprojects.

3. Group 3 (no gender expertise; positive CCMI) versus group 4(gender expertise, positive CCMI): Compared with Group 4,Group3hasanon-significant (smallN)highermeanscoreonnegative evaluations (mean=1.61vsmean=1.22,p=0.13),andasignificantlowermeanscoreonpositive emotions(mean=3.16vsmean=3.70,p=0.056).Asthesescoresaregener-allynotnormallydistributed,weusenexttoAnovaalsoanon-parametrictest.ComparedwithGroup4,Group3hasasignifi-canthighermeanrankonnegative evaluations,andasignificantlowermeanrankonpositive emotions.Obviously,bothtestsgiveaboutthesameresults.Thenon-significancemayalsobeduetothesmallNforgroup3.Theconclusionisthat proposals with a gender dimensionaremorepositivelyevaluatedbypanelswithgenderexpertisethanbypanelswithoutgenderexpertise.

PANEL SCORES

Comparingtheevaluationscoresforthedifferentgroupsshowsthatin panels with gender expertise, “gender relevant” proposals (ccmi =yes)getahigherscorethanthe“non-genderrelevant”proposals(ccmi=no):14.0versus13.6;andthepanelswithgenderexpertisescorethegenderproposals (ccmi=yes)higher than thepanelswithoutgenderexpertisedo:14.0versus13.1points.

CONCLUSIONS Theconclusionsarethat(i)panelswithgender-specialistsaremore

positiveaboutgender-relatedproposalsthanaboutnon-gender-relatedproposals,and(ii) thatpanelswithgenderexpertisearemorepositivethanpanelswithoutgenderexpertiseaboutproposalswithagenderdi-mension.Thisdifferenceinevaluationlanguageisalsoreflectedinthescoresasweshowedabove.Theoverallfindingwouldconfirmthestra-tegyofgettingstakeholdersintopanels:ithelpsgettingprojectsfundedthatworkonissuesrelevanttothestakeholders.Sowecantentativelyanswer thequestion whethergender expertisematter or not: our fin-dingsindicatethatgenderexpertiseinpanelsmatter.Ifthesefindingscanbegeneralised, stakeholder representation seemsagoodway forincreasingresearchprojectrelevanceandimpact.

Furtherworkisneeded,asthispaperonlyisonlyafirstmodeststep.Severalimprovementsneedtobeaddressedinthefuture.(i)Firstofall,


a better operationalisation is needed of what is “stakeholder relevantresearch”.Thisasksforontologiesthatgiveastructuredrepresentationofthetopicsrelevanttothespecificstakeholders.Withsuchontologies,itbecomeseasiertoidentifyrelevantresearchbutalsotoassesswheretherearewhitespotsintherelevantresearchportfolio.(ii)Theanalysiswasdoneusingonlythesummaryoftheproposals.Usingthefulltextmayimprovetheanalysis,althoughitisalsomoredifficulttoidentifytherelevantpartsoftheproposalandmay introducemorenoise. (iii)Onlygrantedproposalsaretakenintoaccount,buttheanalysisofthenon-grantedproposals isas important. (iv)Otheraspectsof theevaluationmaybetakenintoaccount,suchasthescientificqualityoftheconsor-tium,andearlierwork in relation to the stakeholdersneeds. (v)Sinceweonlyhaveaccesstoprojectsingenderflaggedtopics,itisnotpos-sibletoconductamorerefinedanalysisthatfocusesonthedifferencesbetweenflaggedandnotflaggedtopics:howgoodistheidentificationofstakeholder relevantprojects? (vi)Fielddifferencesshouldbe takenintoaccount.Thiscouldnotbedoneduetotherelativesmallnumberofproposals.(vii)Lastbutnotleast,expostevaluationisneededtoo.Dotheproposalsthatweredefinedasstakeholder-relevantindeedproducemore useful and more used output? And what is the quality in otherdimensions,suchasthescholarlyquality?

REFERENCESAbele, A.E. and Wojciszke B. (2014).Communalandagenticcontentinsocialcognition:ADualPerspectiveModel.Advances in Experimental Social Psychology50,195-255.

Benschop, Y. and Brouns, M.(2003).CrumblingIvoryTowers:AcademicOrganizinganditsGenderEffects.Gender, Work and Organization,10(2),194-212.

Brouns, M. (2000).Thegenderednatureofassessmentprocedures inscientificresearchfunding:theDutchcase.Higher Education inEurope, 25,193-201.

Chapman, E.N., Kaatz, A. and Carnes, M.(2013),Physiciansandimpli-citbias:howdoctorsmayunwittinglyperpetuatehealthcaredisparities.Journal of General Internal Medicine28,1504–1510.

Cornell, S., Berkhout, F., Tuinstra, W., Tàbara, J.D., Jager J. Chabay I, de Wit, B., Langlais, R., Mills, D., Moll, P., Otto, I.M., Petersen, A., Pohl, C. and van Kerkhoff L. (2013).Openingupknowledgesystemsforbetterresponsestoglobalenvironmentalchange.Environmental Science and Policy28,60–70

Djenontin, I.N.S. and Meadow, A. M.(2018).Theartofco-productionofknowledgeinenvironmentalsciencesandmanagement:lessonsfrominternationalpractice.Environmental Management61,885-903

ERiC(2010).Evaluating the social relevance of academic research: a gui-de. (In Dutch: ERiC: Evaluatie van de maatschappelijke relevantie vanwetenschappelijkonderzoek:handleiding.DenHaag,2010)

de Jong S., van Arensbergen, P., Daemen, F., van der Meulen, B. and van den Besselaar, P.(2011).Evaluatingresearchinitscontext:an

approachandtwocases.Research Evaluation20(2011)

de Jong S., Barker, S., Cox, D., Sveinsdottir, T. and van den Besse-laar, P.(2014).Understandingsocietalimpactthroughstudyingproduc-tiveinteractions.Research Evaluation23 (2014) 2,89-102

Hegger, D., Lamers, M., van Zeijl-Rozema, A. and Dieperink, C.(2012). Conceptualizing joint knwledge production in regional climatechangeadaptationprojects:Success, conditionsand levers foraction.Environmental Science and Policy18,52-65

Hegger, D. and Dieperink, C.(2014).Towardsuccessfuljointknowledgeproductionforclimatechangeadaptation:lessonsfromsixregionalpro-jectsintheNetherlands.Ecology and Society19(2),34.

Kaatz, A., Magua, W., Zimmerman, D.R. and Carnes, M. (2015). AquantitativelinguisticanalysisofNHIRO1applicationcritiques.Acade-mic Medicine90,1,69-75

Knoll, B., de Chevigne, S., Bustelo, M., Engebretsen, E. and Sand-ström, U. (2017). Interim Evaluation: Gender equality as a crosscutting issue in Horizon 2020.ReportoftheExpertGroupon“theInterimEvalu-ationofGenderEqualityasacrosscuttingissueinHorizon2020”.Euro-peanCommission.

Molas-Gallart, J., Tang, P. (2011).Tracing ‘productive interactions’ toidentifysocial impacts:anexample fromthesocialsciences.Research Evaluation20,219–226

Nielsen, M.W., Andersen, J.P., Schiebinger, L. and Schneider, J.(2017).Oneandahalfmillionmedicalpapersrevealalinkbetweenau-thor gender and attention to gender and sex analysis. Nature Human Behaviour1(11):791-796.

Nowotny, H., Scott, P. and Gibbons, M.(2001).Re-thinking Science— Knowledge and the Public in an Age of Uncertainty.PolityPress,Cam-bridge.

Prins, A. (2010).Evaluation Research in Context: the Mechanical Engi-neering case.DenHaag,ERiCproject

Propp, T. and van der Meulen, B.(2010).Evaluation Research in Con-text: the Electrical Engineering case.DenHaag,ERiCproject

Spaapen, J. and van Drooge, L. (2011). Introducing‘productive inter-actions’insocialimpactassessment.Research Evaluation20,211–218.

Staw, B.M. and Epstein, L.D.(2000).Whatbandwagonsbring:effectsofpopularmanagementtechniquesoncorporateperformance,reputati-onandCEOpay.Administrative Science Quarterly26,501-524.

Van Arensbergen, P., Pen, M., van Drooge, L., Spaapen, J.and van den Besselaar, P. (2010).Evaluation Research in Context: the law case.DenHaag,ERiCproject

Van den Besselaar, P., Stout, L., Gou, X.(2016).Alinguisticanalysisofpeerreviewreports.InProceedings of the Science Technology Indicators

ISSUE 48 | JULY 2019 73

PETER VAN DEN BESSELAARNetwork Institute & Department of Organization Sciences, Vrije Universi-teit AmsterdamDeBoelelaan1081,Amsterdam,1081HV(Netherlands)E:[email protected]

Teresa Mom Consultancy ltdMiddenweg203,Amsterdam,1098AN(Netherlands)E:[email protected]

ULF SANDSTRÖM KTH Royal Institute of Technology, Department of Industrial Economics and ManagementLindstedtsvägen30,Stockholm,11428SE(Sweden)E:[email protected]

Conference (STI 2016),Valencia,September2016.

Van den Besselaar, P., Schiffbaenker, H., Sandström, U. and Mom, C. (2018). Explaining gender bias in ERC grant selection. Proceedings Science and Technology Indicators (STI 2018).Leiden,September2018

Van den Besselaar, P., Sandström, U. and Schiffbaenker, H. (2018b).Studyinggrantdecision-making:alinguisticanalysisofpeerreviewre-ports.Scientometrics 117,313-329

Van den Besselaar, P., Khalili, A. and Sandström, U.(2017).Evaluatingresearchportfolios,amethodandacase.Proceedings Science and Tech-nology Indicators (STI 2017),Paris,September2017.

Van den Brink, M., Brouns, M. and Waslander, S. (2006).Doesex-cellencehaveagender?AnationalresearchstudyonrecruitmentandselectionproceduresforprofessorialappointmentsinTheNetherlands.Employee Relations28,523-539

Van der Meulen, B., Daemen,F., van Drooge, L., de Jong, S., Spaa-pen, J., Wamelink, F. and van den Besselaar, P.(2010).Evaluation Re-search in Context: Architecture.DenHaag,ERiCproject

Wenneras, C. and Wold, A.(1997).Nepotismandsexisminpeer-review.Nature387(6631):341–343.

Wardenaar, T. (2014). Organizing collaborative research: the dynamics and long term effects of multi-actor research programs(PhDthesisVrijeUniversiteitAmsterdam).DenHaag:RathenauInstituut.

AUTHORS

Endnotesi ERiCstandsfor“EvaluatingResearchinContext”.ItwasaprojectoftheRoyalNetherlandsAcademyofArtsandSciences,TheNetherlandsResearchCoun-

cilNWO,theAssociationofUniversities,andtheAssociationofUniversitiesofAppliedSciences.Theaimwastodevelopamethodforresearchevaluationthattakesintoconsiderationallrelevantqualityandimpactdimensions,scholarlyaswellassocietalandeconomic.

ii TheSIAMPIprojectwasfundedbytheEuropeanCommissionundergrantagreementno230330.SIAMPImeans“SocialImpactAssessmentMethodsforresearchandfundinginstrumentsthroughthestudyofProductiveInteractionsbetweenscienceandsociety”.PartnersweretheRoyalNetherlandsAcademyofArtsandSciences(KNAW),CSIC(Spain),MSH(France)andUniversityofManchester(UK).TheSIAMPIconsortiumdevelopedmethodstoassesssocialimpactofresearchprojects,researchprogrammesandresearchfundinginstruments.

iii However,thesocialimpactoftheSSHmaybestrongestininterdisciplinaryprojects,wherethesocialsciencesareanimportantpartasthesefieldsfocusoftenontheconditionsunderwhichthelargerprojectcanhaveimpact.Wecannotgointothisissuehere.

iv Whatcountsasgenderrelevantisnotfurtherdiscussedhere:wetaketheclassificationoftheproposalsintermsofgenderrelevanceasitwasdonebythefundingorganisation(theEuropeanCommission).Itisnecessarytomoredetaileddefinewhatgender(orothersocietal)relevancemeans,asdiscussedabove.

v Sinceweonlyhaveaccesstoprojectsin“genderflagged”topics,itisnotpossibletoconductamorerefinedanalysisthatfocusesonthedifferencesbe-tween‘flagged’and‘non-flagged’topics.

vi Thisisnotuncommon,alsoinotherdomains.Forexample,organisationsadvertisethemselveswithfashionablelabels(“Ourcompanyaimstoempowertheemployees”)eveniftheydonotanythingthatcouldcountasempowering.Researchshowsthatthisindeedhasapositiveeffectonthereputationofthosecompanies(StawandEpstein2000).

vii Asthisareallfundedprojects,itwouldbepossibletoincludemoreinformationaboutthepartners,astheseareintheproposals.Characteristicsoftheapplicants,suchastheirearlierworkon(inthiscase)genderrelevanttopics,mightalsoinfluencethediscussionandscoresbythepanel.Thisextensionisalsoforfurtherresearch.


Acknowledgement:ThisarticleisbasedonworkfromCOSTAction15137EuropeanNetworkforResearchEvaluationintheSSH(ENRESSH)andsupportedbyCOST(EuropeanCooperationinScienceandTechno-logy).

Inthispaper,wepursuetwomainobjectives.First,wereviewtherelevant literatureandpresent itaccordingtoatheoreticalframe-work that combines structural perspectives and consideration for

individualagency,toallowusabetterunderstandingoftheroleplayedbysenioracademicsinthesocialsciencesandhumanities(SSH)intheimplementationofthedifferentpoliciesthatconcerntheproduction,thedisseminationandtheevaluationofresearch, includingimpactrelatedpolicies.Indeedtheacademics’negotiatingpoweroftheimpactagenda–asitiscurrentlypromotedbyEuropeanpolicymakers(seee.g.Euro-peanCommission2018)andencompassestheimpactonpolicymaking,economyaswellastheenvironmentandsociety–cannotbeunderstoodinisolationoftheirperceptionandattitudestowardsthebroaderpoliti-calchangesthataffectthepracticeofacademicresearch.Secondlywediscusssomepreliminaryresultsfromtheinterviewswehaveconductedin the context of the COST ENRESSH action with 16 European seniorsociologistsactive ineightEuropeancountries, focusinghereon theirperceptionsandattitudestowardstheimpactagenda.

A. CHANGES IN THE RESEARCH POLICY MAKING

Mostcurrentresearchpoliciesandpolicyagendasinresearchcanbeconsidered,directlyorindirectly,intheperspectiveofafewtendenciesthathavebeeninitiatedorfosteredbyresearchpolicymakers,bothatnationalandEuropeanlevels,andthatconcernSTEM(Science,Techno-logy,EngineeringandMathematics)aswellas–oftenwithsomedelay–SSHdisciplines.Wewilldistinguishbetweenthetendenciestowards

internationalisation, digitalisation, managerialism, marketisation and“exoterisation”ofresearch(Vanholsbeeck2016).

Internationalisationrelatestothetendencytoencouragetheproduc-tionofresearchcontentsthatfocusonglobalphenomena,orcompari-sonsofnationalsituations,thatarepublishedin international journals–mostlyintheEnglishlanguage–communicatedatinternationalconfe-rencesandimplygeographicaland/orvirtualmobilityoftheresearcher.Insomedisciplines, internationalisationhasantedatedpolicyprescrip-tions,followingepistemologicalmotives.

Digitalisationreferstotheuseofdigitaltoolsandmediatoproduceanddisseminateresearch.

Managerialismmostlyconsistsintheadoptionof“NewPublicMa-nagement”(NPM)intheadministrationofresearch.NPMrelatestotheintroductionintothepublicsectorofadiversityofmanagingpracticesand tools fromtheprivatesector,withanemphasison thenotionsofefficiency,effectiveness,excellence,accountabilityandstandardsofper-formance(Hood1995;Deem1998;Endersetal.2009;WhitleyandGläser2014). In regards tohuman resourcesmanagement, there isa relatedtendency–whichconstitutesoneofthemostimportantchangesinthegovernanceofresearch–tofavourfundingmodesthatcombinerecur-rent with temporary project based funding, in a context of increasingscarcityofresearchbudgets(GläserandLaudel2016:121-122).

Marketisation relates to the tendency to consider universities, re-searchersandtheresearchoutputsthemselvesinthequasi-marketper-spectiveofacompetitiveknowledgeeconomy,andtoreconsiderinthisperspectivetherelationsbetweenacademiaandindustries.

By the lessusualnotionofexoterisationwedesignate thevariousprocesses of opening the production, dissemination and evaluation ofresearchoutside(exo)ofthedisciplinarycirclesoftheacademicpeers.In that respect, European and national level policies have supportedthetransferofknowledgefromresearcherstonon-academicstakehol-ders–inparticulartotheindustry–aswellas,morerecently,theco-creationby researchers, policymakers, industriesand/or citizensalikeofsolutionstosocietalchallenges,undertheinfluenceofprogrammatic

MARCVANHOLSBEECK,THEODOSIADEMETRIOU,AGNEGIRKONTAITE,ANDREJAISTENICSTARCIC,VILLEKEISKI,EMANUELKULCZYCKI,ELENAPAPANASTASIOU,JANNEPÖLÖNEN,HULDAPROPPEANDMAJAVEHOVECDOI:10.22163/fteval.2019.371

SENIORACADEMICSASKEYNEGOTIATORSINTHEIMPLEMENTATIONOFIMPACTPOLICIESINTHESOCIALSCIENCESANDHUMANITIES

1 https://ec.europa.eu/research/openscience/pdf/draft_european_open_science_agenda.pdf#view=fit&pagemode=none,consultedon20October2018.

ISSUE 48 | JULY 2019 75

1.1 INSTITUTION AS SCIENTIFIC ORGANISATION

Ofparticularinterestwhileconsideringchangesinscientificorgani-sations,theso-called“neo-institutionalist”schoolofsociologyhasrene-wedorganisationtheory,by focusingonthesupra-individualcognitiveandculturalfactorsthatexplainthesocialandorganisationalphenome-na(DiMaggioandPowell1991).Neo-institutionalistsdevelopedthecon-ceptofisomorphism,whichexplainswhyrationalactorsincreasethesi-milarityoforganisationsthathaveemergedasacertainfieldordomain,while trying tochange them (DiMaggioandPowell1983).Conceptualdistinction has been made between coercive isomorphism – involvingpressuresfromotherorganisationsonwhichtheorganisationdependsaswellassocialexpectationssurroundingthem–mimeticisomorphism–consistingintoanorganisationimitatinganotherorganisation’sstruc-turebecauseofthebeliefthatsuchimitativeprocesswillbebeneficial–andnormativeisomorphicprocess,relatingtoprofessionalnormsthatspanorganisationsbelongingtothesamefield.

Thesethreetypesofisomorphismaretosomedegreeatworkincon-temporaryacademia,fosteringsimilarmovestowardsinternationalisati-on,marketisationandmanagerialism.

First,formsofcoerciveisomorphismcanbefoundinthedrivingeffectthatperformancequantitativeindicatorshaveonthepracticestheytrytomeasure,inSSHresearchevaluationlikeinotherareasofsociallife(asexpressedforexample inCampbell’sorGoodhart’s laws,accordingtowhichameasure ceases tobeagoodmeasureonce it becomesatarget).Inparticular,bibliometricindicatorsareincreasinglyused,bothatEuropean(Vanholsbeeck2017)andnationallevel(e.g.Hammarfeltetal. 2016; Pölönen and Wahlfors 2016), to benchmark national sciencesystems and universities, but also to assess – and provide fundingto– individualsandprojects (DeRijckeetal.2016;GläserandLaudel2016).Usuallydevelopedbyprivatecompanies–suchastheinfamousImpact Factor (now provided by Clarivate Analytics) – bibliometrics ismostlybasedoninternationaldatabasesofscholarlyjournals.Assuch,theydirectlyor indirectlycoerceresearchers intheirpublishinghabits.TheycontributetotherisingproportionoftheshareofSSHpublicationsthattaketheformofarticlespublished,inEnglish,ininternationaljour-nals(HammarfeltanddeRijcke2015;Kulczyckietal.2018),evenifanystrictlycausalascriptionoftheeffectsofagivenresearchpolicyonre-searchcontentshastobeconsideredwithcaution,becauseofthemanyconfoundingvariableswhicharetoconsider(GläserandLaudel2016)2.Furthermore,someevaluationsystemsstilltakebooksandpublicationsaimed at professional and general audiences into account (Giménez-Toledoetal.2016).

Second, world university rankings which are in a significant partbasedonbibliometricindicatorshavebecomeincreasinglyimportantinthelastdecade,notleastduetotheirmediaexposure.Oftenproducedbynon-academicorganisations,theyexertsomeinfluenceonuniversi-tiesaroundtheworld,promotingaglobalmodelof“world-classuniver-sities”worthfollowing(mimeticisomorphism).

ideassuchas“mode2ofknowledgeproduction”(Gibbonsetal.1994)or– inthecontextof thepreparationofthenext“EuropeanResearchand Innovation Framework Programme “Horizon Europe” (2021-2027)–“missions”(KattelandMazzucato2018).TheEuropeanOpenScienceagenda1 – including open access to publications, open research dataandcitizenscience–andtheimpactrelatedpoliciesalsoalignwiththistendencytoexoterisation.Theconceptofexoterisationisthusbroaderthanmarketisation,sinceitincludessocialinnovation–whichcantakenon-commercialforms–andrelatestothenotionofknowledgesocietyratherthantothesoleknowledgeeconomy.

If there issomedegreeofconvergencebetweenmostof theabovementionedtrends,thetendenciestowardsexoterisationandmanageria-lismofresearcharenot(yet)congruent,sinceperformanceindicatorsthatarecurrently inusage inthemanagementof researchdonottake intoaccountinanysignificantwaytheextra-academicimpactofresearch,noropensciencepractices(O’Carrolletal.2017;Vanholsbeeck2017).

B. INSTITUTIONAL AND INDIVIDUAL PERSPECTIVES ON THE ROLE OF SENIOR ACADEMICS

The role of senior academics in the implementation of researchpolicies and science related political agendas, including the impactagenda, is better appreciated according to two theoretically differentbuteventuallycomplementaryperspectives:theinstitutionalistandthecomprehensive–intheWeberianmeaningoftheterm–approach.Thefirst focuseson the structural determinants that impact individualbe-haviours,studyinginstitutionalpressuresoncollectiveorganisations(atameso-sociological level). The second takes theoppositeperspective,focusingontheinnermotivationsandperceptionsofindividualsaswellastotheiragency,anddedicatesattentionontheimpactthatindividualstrategiesandsubjective interactionsmayhaveonorganisations (atamicro-sociologicallevel).

1. INSTITUTIONALIST APPROACH

Institutionisabroadsocialscienceconceptwhich,inthecontextofsciencesociology,cantaketwomainmeanings.Itrefersindeedtotheofficialorganisationsinwhichscienceispractised(i.e.:universities,re-searchcentres,researchunits,academies,etc.),butalsotothespecificrules,processesandstableusagesthatweighonthebeliefsandbehavi-oursofthosewhopracticescience(Gingras2017:29).

2 Forexample,ithasbeenshownthatthedecreaseinshareofpublicationspublishedinFinland,whichisalsopartiallyindicativeofpublicationlanguage,isattestedinthenationalpublicationstatisticssince1994,wellbeforetheperformancebasedfundingmodelwasestablishedinFinland(AuranenandPölönen2014).


to the disciplines or the institution of science, rather than its organisa-tions. [...] They may even treat the universities, departments and institutes they are part of as irritations, a collection of performance indicators and management demands which threaten to get in the way of real science”(DaviesandHorst2016:65).

Inthesameprofessionalperspective,therelationshipsbetweenseniorresearchersandPhDcandidates–althoughlittleresearchhasbeencar-riedoutthatfocusesonPhDdirectors’reactionstothechangingcontextofPhDeducation–constituteanotherplacewhereprofessionalvaluesmayconflictwithorganisationalprocesses(Deuchar2008;Bøgelund2015).

It should be noted though that the dominant bibliometric perfor-mance indicators that are currently used in the new public manage-mentof researcharestill linked to theprimaryprofessionalactivityofacademia.Theymostly relate indeed to theproductionandcitationofarticles inscientificpapers,andnot to theengagementof the resear-chersintheirorganisation.Henceitcanbearguedthatthoseindicators,althoughoftencriticised,arenotentirelyforeigntosomecoreacademicprofessionalvalues.

2. COMPREHENSIVE APPROACH

Institutionalist perspectives should be combined with the analysisoftheperceptionsandattitudesoftheindividualresearchers,withdueconsiderationtotheiragency.Indeedscientificorganisationsprovidein-dividualscholarswithacertainpowerto“negotiate”highereducationand research policies (Linkova 2014), not the least because Europeanuniversities do generally enjoy a high level of institutional autonomy,whiletheprofessionalnormofacademicfreedomprevailsinEuropeanhighereducation.Furthermore,someacademicsareactiveasfullorpart-timeadministratorsintheirinstitution,withoutbeingperseinanadmi-nistrativecareerpath.Senioracademicsarealsothosemostlyinchargeofleadingaresearchteamandtrainingearlystageresearchers,assuringtheirprofessionalsocialisation.

The“comprehensive” literature thatwe reviewed in thecontextofthisCOSTENRESSHproject resorts toadiversityof theoretical frame-worksandconcepts,someofthemevenreferringtoneo-institutionalism(e.g. Lam2010orTeelken2011). Theoretical framingnotwithstanding,mostresultstendtoemphasisetheambivalentattitudesofresearcherstowards the abovementioned tendencies of marketisation, manageria-lismandexoterisationof research,bringingoutat the individual levela similar attitude of “symbolic compliance” to the one that had beenobservedattheinstitutionallevel.

REACTIONS TO MANAGERIALISM

Moststudieswereviewedconcentrateonthescholars’reactiontomanagerialism,withanearlyfocusontheUKsituation.Alreadyin2001,itwascontendedthatmanagerialismwasnotentirelyembeddedinUKuniversities, and that middle and junior level academics actively keepprofessionalacademicvaluesaliveandmoderatetheharshereffectsofthechanges(Barryetal.2001).Deem(2003)hasarguedthattheattitudetowardsmanagerialismofUKacademicadministratorsvariesdependingon their intention to return later to teachingand research role. Thosewhointendtogobacktoprimaryacademictasksmitigatethenewma-nageriallanguageandkeepsomecoreprofessionalvalues.Studyingthe

Finally,effortstostandardisehighereducation–includingthirdcycleandresearchers’training–notablyviatheBolognaProcess,contributetosomenormativeisomorphismwithinacademia.

ORGANISATIONS’ REACTION TO INSTITUTIONAL PRESSURES

In a meta-analysis of neo-institutional approaches and resourcedependence theories, Oliver (1991) brought important nuances to theconceptofisomorphism,andtotheideathatorganisationsconformtothepressuresoftheirinstitutionalenvironment,benefittingfromadhe-ringtoexternalrulesandnorms.Sheproposedamorenuancedtypologyofstrategicresponsestoinstitutionalprocessandactiveorganisationalbehavioursthatvaryfrompassiveconformitytoactiveresistance,takingtheformofacquiescence,compromise,avoidance,defianceormanipu-lation.

Relying on Oliver’s typology and applying it to research organisa-tions, Leisyte (2007) studied the effects of governance models on theresearchpracticesofresearchunitsinthefieldsofmedievalhistoryandbiotechnology.Shemakesthedistinctionbetweenthreeorganisationalstrategies towards managerialism: passive compliance, symbolic com-pliance – a combination of acquiescence and avoidance consisting inpretensionofcompliance,butchangingnothingtothewayresearchisperformed–andproactivemanipulationoftherulesandnormsoftheinstitutionalenvironment.

Ithastobenoticedthatlocalspecificitieshavetobetakenintoac-count while considering the effect of managerialism on organisationsandindividualsalike(Stöckelová2012).Inparticular,informerEuropeansocialistcountries,theintroductionofmanagerialismhasaccompaniedaprocessofde-andre-politicisation(LinkováandStöckelová2012).

1.2 INSTITUTION AS A SET OF SOCIO-PROFESSIONAL VALUES

Accordingtothesecondoftheabovementionedinstitutionaldefini-tions,theinstitutionofsciencedesignatesthespecificsocialsystemofscience.Assuchscientistsarenotonlyexposedtorules,processesandstableusagescoming fromthenon-academicworld,butalsoproducetheirveryownsocio-professionalvalues,thatspantheboundariesoftheorganisationsbywhomtheyareemployed.

Inthisperspective,theprofessionalvaluesofsciencehavebeenana-lysed as a potential source of resistance to organisational changes inuniversities(Chandleretal.2002;KirkpatrickandAckroyd2003).Indeed,thevalues towhichscientistsadhere–suchasacademic freedomorthe ones identified by Merton (1973), of communalism, universalism,disinterestedness andorganised scepticism–maydiverge from thosethataresupportedby researchpolicies ingeneral,andbynewpublicmanagementinparticular.

Thediscrepanciesbetweenthevaluesofthescientificinstitutionandthemanagementofthescientificorganisationmayevencreateaclashbetween (internal) professional accountability, based on professionalvalues, and (external)managerial accountability, basedonmanagerialnormsandprocesses (LinkováandStöckelová2012).Hence somesci-entistsengageindoubleallegiance:they“rarely seem to see themselves first and foremost as organisational members. Their allegiance is primarily

ISSUE 48 | JULY 2019 77

innovation” (Anderson 2008: 256 and 267). On the basis of interviewswithAustrianhistorians,KehmandLeiðytë(2010)identifiedagenerationgap,seniorresearchersbeingmorepronetoresistancethanearlycareeracademicswhomayhavebeenprofessionallysocialisedinthenewma-nagerialcontext.Linková,studyingtheresponsesofCzechresearchersinthehumanities,socialsciences,andnaturalsciencestoresearchas-sessment,foundthatsomeacademicsengagedailyinmicro-politicsofresistanceandcritiqueswhich“are located within traditional ‘Science’ values stressing autonomy and peer judgment on the one hand or indi-vidual performance, primacy and competitiveness on the other”(Linkova2014:85-86).Bydoingso,theyrelyontraditionalscientificvaluesandstressautonomyandpeerjudgment,evenif,overall,researchersshowadaptationtothenewtypesofgovernmentality.

Thelastcategoryofarticleswereviewedconsidersonthecontrarythatacademicsmostly–andnotonly symbolically–complywith thenewmanagerialprocesses,and that resistance,whenever ithappens,isessentiallyideologicalordiscursive,onlyaminorityresistingactively.Thoseare theconclusions thatClarke,KnightsandJarvis (2012)havereachedonthebasisoftheir48interviewswithBritishbusinessschoolacademics. Leathwood and Read (2013), as well as Ylijoki and Ursin(2013),madesimilarconclusions, respectively in regards to theBritishandFinnishacademicstheyinterviewed.Arecentstudyoftheintroduc-tionofperformanceappraisalsinaregionalAustralianuniversityshow-edlittleresistanceeitherfromacademics’side,earlycareeracademicsbeing particularly compliant with the new prescriptions. (Kalfa et al.2018).

MARKETISATION

In regards to the tendency to themarketisationof research, ithasbeenargued,onthebasisoffocusedinterviewswithseniorresearchersinthreedifferenttypesofresearchsettingsinFinland(departmentsofHistoryandSurfaceScienceandSemiconductorTechnology;WorkRe-searchCentre),thattheirengagementin“academiccapitalism”dependsonhowclosetheirfieldisfromthemarket(Ylijoki2003).Thestudyshowsthatresearcherstrytoaccommodatetraditionalacademicpracticesandvaluestomoreentrepreneurialactivities,underthepressureofworkingincreasingly on short-term contracts and projects. Similarly, a studybasedon36interviewsandasurveyof734academicscientistsfromfiveUK researchuniversitiesshows theactiveagencyofacademics in theshapingoftherelationshipsbetweenscienceandbusiness(Lam2010).Most academics exploit the ambiguities of “boundary work” betweenacademiaand industry, rather thanbeingentirely“traditional”or“en-trepreneurial”.

ATTITUDES TOWARDS EXTRA-ACADEMIC IMPACT

Asfarasweknow,thereisnodedicatedresearchontheresearchers’perceptionof impactpoliciesassuch,whereversuchpoliciesdoexistinanexplicitform.Intheirabovementionedstudyoftheeffectsofthemanagerialismonresearch,KehmandLeiðytë (2010)showedthat theprescriptionstopublishforabroaderpublic,combinedtotheprescrip-tionsinfavourofmoreinterdisciplinaryresearch,haveaffectedthere-searchtopicsonwhichGermanmedievalhistoriansareworking.Further-more,thefindingsofSmith(2010)suggestthatthegrowingpressureto

negotiationofevolvingresearchpoliciesbyUKlifescientists,MorrisandRip(2006)similarlyunderlinedthatscientistsdevelopmoreorlessproac-tivestrategiestomodulatetheimpactofchangingresearchpolicies.

RegardingSSHdisciplines,KehmandLeiðytë,onthebasisofinter-viewswith researchers inEnglishmedievalhistoryunits,also showedthatthey “try to find a balance between their own research agenda and the research priorities of the funding bodies [...]. They do so by following largely symbolic compliance strategies – maintaining their own research lines and at the same time selling their research interests according to the priorities of the external research funders”(KehmandLeiðytë2010:80).Teelken (2011)analysedthe individualbehavioursof48academicandsupportstaffmembersat tenuniversities in theNetherlands,SwedenandtheUK,infacultiesofsocialsciencesandeconomics/businessstu-dies.Theresearchshowsthatacademicsdissociatethemselvesfromthemanagerial prescriptions, and appear to be only loosely coupled fromtheirorganisations,even ifbesidesymboliccomplianceandprofessio-nalpragmatism(dealingwiththemanagerialprescriptions“in a critical but serious manner”),anattitudeof“formal instrumentality”isalsoob-served (Teelken 2011: 278). Respondents do not consider assessmentassuchasundesirable,butarecriticaloftheincreasinglyquantitativeand time-consuming performance based assessment, as well as thegrowing competition for research funding. From interviews conductedwithcommunicationscholarsinFrenchspeakingBelgium,Vanholsbeeck(2012)similarlyconcludedthat thoseresearchers, rather thanfullyac-ceptingorresistingtotheprescriptionsthatsupportthepublicationof(many)papers in international journals,are ratherambivalent towardstheprescribedqualityrequirements.Someofthem“tinker”withtheseprescriptions,tryingtopublishaccordingtothe(perceived)prescriptions,whilestillallowingtimeforpublishingaccordingtotheirveryowndefi-nitionofquality.

FocusingontheuseofbibliometricsforevaluationpurposesinDutchlaw faculties, micro-politics of indicator use have also been revealed,through which scholars in advanced administrative positions try toproactivelypursue “competing normative and epistemic agendas”ratherthanpassivelyreactingtoexternally-imposedadministrativeprocedures(KaltenbrunneranddeRijcke2016:284).Comparably,FinnishandNor-wegianuniversitiesusethenationalpublicationchannel-basedqualityindicator for assessing individual academics, more particularly in thehumanities.Assuch, the indicator isusedasa replacement forpubli-cationcounts,inlackofalternativeindicatorssuchastheImpactFactorintheSSHfields(PölönenandWahlfors2016).AlsoinFinland,researchonSSHresearchers(N=92)hasshownthattheintroductionofthenewperformancebasedfundingmodelhasinvolvedwhattheauthorscallthe“publication laundering”(inFinnish“julkaisupesu”),meaningthemani-pulationofpublicationliststomeetthestandards,e.g.peer-review,ofmeasuredperformances(Sivulaetal.2015:153).

Someauthorsevencontendthatthereareattitudesofrealresistance,andnotonlyofsymboliccompliance,fromacademicstomanagerialism.Cleggmaintains thatacademicsdo resistmanagerialism,albeitpassi-velyand individually,bycreatingspacesfor theexerciseof“principled personal autonomy and agency”whichallowthemtodevelop“their own ways of practising and a personal sphere of meaning”inwhichtheycanpractisewithintegrity(Clegg2008:343).Similarly,Andersonfindsthatthe resistance of Australian academics takes many forms and followseverydayandcovertdiscursivestrategies,considering “academics’ ca-pacity — indeed, their perceived responsibility — to assess, analyse and criticize”aswellasdeemingthemasparticularly“skilled in rebellion and


viewees perceive the same isomorphic evolutions of higher educationtowards managerialism and internationalisation, and many also shareto some degree an attitude of symbolic compliance with the relatedprescriptions.

Inregardstoourinterviewees’perceptionandattitudestowardstheimpactagenda,itisquiteclearthattheimpactagendaisnotperceivedashavingcurrentlyanydirectandsignificantincidenceontheirprofes-sionallife.Therealpressureisobviouslyonproducingmorepapers,intheEnglishlanguage,ininternationaljournals,ratherthanongettingmoreinteractionswiththenon-academicworld.Insomecountriesthe-re isevena recentandvery strong focuson theuseofbibliometricsinSSHresearchassessment(e.g.Croatia,Poland),althoughdedicatedfundingtoolsforsupporting“impacting”SSHresearchhavealsobeenputinotherplaces(Belgium).Quantitativeperformancebasedevaluati-onofresearchismostlyperceivedasbeinginconsistentwithanystron-gerengagementinimpactrelatedactivities,whichsomerespondentsassociatewithlocalresearch(andpublicationsinvernacularlanguage)andperceiveashardertoproperlyquantify(Lithuania).Insomecases(likeinSlovenia),pastevaluationprocessesinvolvinggeneralpublicintheevaluationprocesstohigherextentmayhavebeenassociatedwithmoresocietally impactingresearchpoliciesthanwhatiscurrentlythecase.

However,someintervieweesmentionthatitisstillpossibletoreward–evenifslightly–impactintheassessment(likeinFinlandorinIceland)orthatitmayevenbefeasibleinsomecasestopursuea“parallelcare-er”inacademia,basedonmediaengagementandtheconductofmoreoperationalresearch.“OpenScience”(OS)and“OpenAccess”(OA)arenotconsideredaspriorities(atall)andsomeintervieweesevenperceiveOAjournalsasbeingofalowerqualityand/orreputation,orevenasfos-teringtheprevailingsciencesystem.OneBelgianrespondentunderlinesthoughthatinstitutionalOArepositoriesdoallowthedisseminationofadiversityofresearchoutputs–besidescholarlyarticles–includingthosewhomayimpactsociety.

InoneBelgianresearcher’sperspective,impactshouldnotbeconsi-deredonlyinaninstrumentalperspective,butreliesonthesociologists’dutyto“engageinthecity”inascientificallyinformedbutalsocriticalway.AnIcelandicrespondentconsidersthatinteractingwiththemediaisanintrinsicpartofhisacademicjob.

Finally,wewouldliketoemphasisethatseveralresearchers–inpar-ticularthosewhodonothaveresponsibilitiesinadministrativeareas(Cy-prus)ordonotbelongtothenewacademicgeneration(Croatia)–wishthatassessmenttakesbetterimpact-relatedendeavoursintoaccount.AsoneofourCroatianrespondentstoldus:“Theresponsibilityofscienceistowardssocietyandthecommunityastheyarefundingus,andnotjustourpersonalscientificcareerorourmotives.Thisispartofoursocialres-ponsibilityofbeingscientists.Oftenourscientificresultshavenoimpact.Nevertheless, it is our responsibility to interpret social processes evenwhenwefeelthatournotionshavenoresonance.Itisourresponsibilitytointerpretsocialprocessesandtrytobeconvincing,eventhroughnon-scientificpublicationssuchaspolicydocumentsorthelike”3.

producepolicyrelevantresearchinhealthinequalityisdiminishingtheautonomy and creativity of sociologists, and is instead promoting theconstructionofinstitutionalisedandvehicularideas.

Otherstudiesfocusontheperceptionofscholarsonsciencecommu-nication,publicengagementandvalorisationofresearch.Intheirreviewofpaststudiesandsurveysonhowscientistsviewthepublic,thegoalsofcommunication,theperformanceandimpactsofthemedia,aswellastheroleofthepublicinpolicydecision-making,BesleyandNisbet(2013)havearguedthatscientistsconsiderthepublicasgenerallyuninformedaboutsciences.Theyarecriticalofmediacoveragebutbelievethatin-teractionswith journalistsare importantforpromotingscience literacyaswellascareeradvancement,policymakersbeingconsideredasthemost importantexternalstakeholderstoengagewith.Furthermore,onthebasisofparallelsurveysofscientistsfrommultiplescientificsocie-ties,themostconsistentpredictorsofwillingnesstotakepartinpublicengagementactivitiesareabeliefthattheexperiencewillbeenjoyableandmakeadifference,aswellasthetimeavailabletoengage(Besleyetal.2018).Age,sex,scientificfieldbutalsotheresearcher’sperceptionof the public, of her peers and of her personal engagement skills areinconsistentpredictors.

Finally,asurveyconducted inBelgiumonhighereducation institu-tionsoftheBrusselsCapitalRegion(N=727)showedthatonerespon-dentontwohasexperienceinvalorisation(Dobbelsetal.2015).Thevastmajority of respondents in SSH were concerned by social valorisationrather thaneconomicvalorisation–which is themain focusofknow-ledgetransferpoliciesoftheBrusselsRegion–contrarilytotheirpeersintheexactandappliedsciences.Amajorityofresearchersagreedthatresearchersshouldcontributetovalorisation,although62%oftheres-pondentsconsiderthatacademicsshouldremainfreetovaloriseornot.Mentionedobstaclesarethelackoftime(85%),lackofskillsordedicatedfunding(64%)aswellaslackofreward(60%).Valorisationisperceivedlikeapersonalaffair,ratherthanaprofessionalopportunityornecessity.

3. DISCUSSION OF EXPLORATORY RESULTS

Weconducted16semi-structuredinterviewswithseniorresearchersinsociology,havingearnedtheirPhDforatleasteightyearsandactiveinBelgium,Croatia,Cyprus,Finland,Iceland,Lithuania,PolandandSlove-nia.Weinterviewedthemabouttheirperceivedrolesinthedefinition,thedisseminationandtheimplementationofthequalitycriteriaandrationa-lesthataretobeusedinevaluationsituations.Inparticular,wewantedtoknowtowhatextent theyconsider it important that impact is takenintoaccountintheevaluationofSSHresearch.Wehadpreviouslyagreedonabroaddefinitionofimpact,consideringitastheresultofallkindsof“productiveinteractions”(SpaapenandVanDrooge2011)throughwhichresearchersengagewithallkindsofnon-academicpublics.

Evenifwewillbringfurthernuancesanddevelopmentstotheana-lysis in a future publication, it is already possible to contend that ourresultsdonotessentiallycontradictthemostimportantconclusionsfromthestudieswereviewedabove.Inparticularwehavefoundthatinter-

3 InterviewconductedinCroatianwithafemaleseniorsociologist,1/03/2018[ourtranslation]

ISSUE 48 | JULY 2019 79

DiMaggio, P. J. and Powell, W. W. (1983).Theironcagerevisited:ins-titutionalisomorphismandcollectiverationalityinorganizationalfields.American Sociological Review48(2),147–160.

DiMaggio, P. J. and Powell, W. W. (Eds.) (1991).The New Institutio-nalism in Organizational Analysis.Chicago:UniversityofChicagoPress.

Dobbels, J., Kesbeke, W. and Ysebaert, W.(2015).Hoeonderzoekerswerkelijkdenkenovervaloriseren.THandMA,1:93-97.

Enders, J., De Boer, H. and Leišytė, L.(2009).“NewPublicManage-ment and the Academic Profession: the Rationalisation of AcademicWorkRevisited”.InJ.Enders,andE.deWeert(Eds.),The Changing Face of Academic Life: analytical and comparative perspectives. New York :Palgrave-Macmillan,36-57.

Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P. and Trow, M.(1994).The new production of knowledge: The dynamics of science and research in contemporary societies.London:Sage.

Giménez-Toledo, E., Mañana-Rodríguez, J., Engels, T. C. E., Ingwer-sen, P., Pölönen, J., Sivertsen, G., Verleysen, F. T. and Zuccala, A. A.(2016).Takingscholarlybooksintoaccount:CurrentdevelopmentsinfiveEuropeancountries.Scientometrics,107(2),685-699.

Gingras, Y. (2017). Sociologie des sciences. Presses universitaires deFrance.

Gläser, J. and Laudel, G. (2016).Governingscience:howsciencepolicyshapesresearchcontent.European Journal of sociology/Archives Europé-ennes de sociologie,57(1),117-168.

Hammarfelt, B. and de Rijcke, S. (2015). Accountability in context:effects of research evaluation systems on publication practices, disci-plinarynorms,andindividualworkingroutinesinthefacultyofArtsatUppsalaUniversity.Research Evaluation,24(1),63-77.

Hammarfelt, B., Nelhans, G., Eklund, P. and Åström, F. (2016). Theheterogeneous landscapeofbibliometric indicators.Evaluatingmodelsfor allocating resources at Swedish universities. Research Evaluation,25(3),292-305.

Hood, C. 1995.The‘newpublicmanagement’inthe1980s:Variationsonatheme.Accounting, Organizations and Society, 20,93–109.

Jiménez-Contreras, E., de Moya Anegón, F. and López-Cózar, E. D. (2003). The evolution of research activity in Spain: The impact of theNational Commission for the Evaluation of Research Activity (CNEAI).Research policy,32(1),123-142.

Kalfa, S., Wilkinson, A. and Gollan, P. J.(2018).Theacademicgame:Complianceandresistanceinuniversities.Work, Employment and Soci-ety,32(2),274-291.

Kaltenbrunner, W. and de Rijcke, S. (2017). Quantifying ‘Output’ forEvaluation: Administrative Knowledge Politics and Changing EpistemicCulturesinDutchLawFaculties.Science and Public Policy,44(2),284-293.

REFERENCES

Anderson, G. (2008).Mappingacademic resistance in themanagerialuniversity.Organization,15(2),251-270.

Auranen, O., and Pölönen, J. (2014). “Julkaisufoorumi-luokitus jakansallinenjulkaiseminen”(PublicationForumratingandnationalpubli-shing).InR.MuhonenandH.-M.Puuska(Eds.),Tutkimuksen kansallinen tehtävä.Tampere:Vastapaino,153–175.

Barry, J., Chandler, J. and Clark, H.(2001).Betweentheivorytowerandtheacademicassemblyline.Journal of Management Studies,38(1),87-101.

Besley, J. C., Dudo, A., Yuan, S. and Lawrence, F.(2018).Understan-ding scientists’ willingness to engage. Science Communication, 40(5),559-590.

Besley, J. C. and Nisbet, M. (2013).Howscientistsviewthepublic,theme-diaandthepoliticalprocess.Public Understanding of Science,22(6),644-659.

Bøgelund, P.(2015).HowsupervisorsperceivePhDsupervision–Andhowtheypractice it. International Journal of Doctoral Studies, 10(1), 39-55.

Chandler, J., Barry, J. and Clark, H.(2002).Stressingacademe:Thewearandtearofthenewpublicmanagement.Human Relations, 55(9),1051–69.

Clarke, C., Knights, D., and Jarvis, C.(2012).Alabouroflove?Academicsinbusiness schools.Scandinavian Journal of Management, 28(1), 5-15.

Clegg, S.(2008).Academicidentitiesunderthreat?.British Educational Research Journal,34(3),329-345.

European Commission(2018).Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee and the Committee of the Regions: Horizon 2020 interim evaluation: maximising the impact of EU research and inno-vation.11/01/2018.

Davies, S. R. and Horst, M. (2016). Science Communication: Culture, Identity and Citizenship.Springer.

Deem, R. (1998). ‘Newmanagerialism’andhighereducation:Thema-nagement of performances and cultures in universities in the UnitedKingdom.International Studies in Sociology ofEducation,8,47–70.

Deem, R. (2003).“NewmanagerialisminUKuniversities:manager-aca-demicaccountsofchange”.InH.Eggins(Ed.),Globalization and reform in Higher Education.Berkshire:OpenUniversityPress,55-67.

De Rijcke, S., Wouters, P. F., Rushforth, A. D., Franssen, T. P. and Hammarfelt, B. (2016). Evaluation practices and effects of indicatoruse—aliteraturereview.Research Evaluation,26(2),161-169.

Deuchar, R. (2008).Facilitator,directororcriticalfriend?:Contradictionandcongruenceindoctoralsupervisionstyles.Teaching in Higher Educa-tion,13(4),489-500.


Pölönen, J. and Wahlfors, L.(2016).“Localuseofanationalratingofpublication channels in Finnish universities”. Poster presented at the21stNordicWorkshoponBibliometricsandResearchPolicy,Copenha-gen, 3.-4.11.2016. URL: https://figshare.com/articles/Local_Use_of_a_National_Rating_of_Publication_Channels_in_Finnish_Universities_NWB_2016_poster_/4246541.

Sivula, A., Suominen, J. and Reunanen, M.(2015).A1alkuperäisartik-kelitieteellisessäaikakauslehdessä.Uusienjulkaisukäytänteidenomaks-uminenihmistieteissä2000-luvulla.Kasvatus and Aika,9(3),149–171.

Smith, K. (2010).Research,PolicyandFunding–AcademicTreadmillsandtheSqueezeonIntellectualSpaces.The British Journal of Sociology,61(1),176-195.

Spaapen, J. and Van Drooge, L.(2011).Introducing‘productiveinterac-tions’insocialimpactassessment.Research Evaluation,20(3),211-218.

Stöckelová, T.(2012).Immutablemobilesderailed:STSandtheepiste-micgeopoliticsofresearchassessment.Science, Technology and Human Values,37(2),286-311.

Teelken, C. (2011).Complianceorpragmatism:howdoacademicsdealwithmanagerialisminhighereducation?Acomparativestudyinthreecountries.Studies in Higher Education,37(3),271-290.

Vanholsbeeck, M. (2012).Entrequalitéprescriteetqualitésouhaitable.Quaderni,77,71-84.

Vanholsbeeck, M. (2016).La notion de «qualité» des publications dans l’évaluation de la recherche et des chercheurs en sciences humaines et sociales: Le potentiel de l’Open Access pour dépasser le paradoxe des prescriptions en matière de qualité et l’ambivalence de leur perception par les chercheurs en sciences de la communication.PhDthesis,UniversitéLibredeBruxelles.

Vanholsbeeck, M.(2017).LanotiondeScienceOuvertedansl’Espaceeuropéen de la recherche: Entre tendances à l’«exotérisation» et à la«gestionnarisation» de la recherche scientifique. Revue française des sciences de l’information et de la communication,11.

Weingart, P. (2005). Impactofbibliometricsuponthesciencesystem:Inadvertentconsequences?.Scientometrics,62(1),117–131.

Whitley, R. and Gläser, J.(2014).Theimpactofinstitutionalreformsonthenatureofuniversitiesasorganisations.InR.Whitley,andJ.Gläser(Eds.),Organizational Transformation and Scientific Change: The Impact of Institutional Restructuring on Universities and Intellectual Innovation.EmeraldGroupPublishingLimited,19-49.

Ylijoki, O.-H.(2003).Entangledinacademiccapitalism?Acase-studyonchangingidealsandpracticesofuniversityresearch.Higher Education, 45,307–35.

Ylijoki, O.-H. and Ursin, J.(2013).TheconstructionofacademicidentityinthechangesofFinnishhighereducation.Studies in Higher Education,38(8),1135-1149.

Kattel, R. and Mazzucato, M.(2018).Mission-orientedinnovationpoli-cyanddynamiccapabilitiesinthepublicsector.Industrial and Corporate Change,27(5),787–801.

Kehm, B. M. and Leiðytë, L.(2010).“EffectsofNewGovernanceonRe-searchintheHumanities–TheExampleofMedievalHistory”.InD.Jan-sen(Ed.),Governance and Performance in the German, Public Research Sector.Dordrecht:Springer,73-90.

Kirkpatrick, I. and Ackroyd, S. (2003). Transforming the professionalarchetype?ThenewmanagerialisminUKsocialservices.Public Manage-ment Review,5(4),511-531.

Kulczycki, E., Engels, T. C., Pölönen, J., Bruun, K., Dušková, M., Guns, R., Nowotniak, R., Petr, M., Sivertsen, G., Istenič Starčič, A. and Zuccala, A.(2018).Publicationpatternsinthesocialsciencesandhumanities: evidence from eight European countries. Scientometrics,116(1),463-486.

Lam, A. (2010). From “ivory tower traditionalists” to “entrepreneurialscientists”?Academicscientistsinfuzzyuniversity-industryboundaries.Social Studies of Science,40(2),307–340.

Leathwood, C. and Read, B.(2013).Researchpolicyandacademicper-formativity: compliance, contestationandcomplicity.Studies in Higher Education,38(8),1162-1174.

Lee, A. (2008).Howaredoctoralstudentssupervised?Conceptsofdoc-toralresearchsupervision.Studies in Higher Education,33(3),267-281.Leisyte, L. (2007). University governance and academic research: Case studies of research units in Dutch and English universities. PhD thesis,UniversityTwente.

Linková, M.(2014).Unabletoresist:Researchers’responsestoresearchassessmentintheCzechRepublic.Human Affairs,24(1),78-88.

Linková, M. and Stöckelová, T. (2012). Public accountability and thepoliticizationofscience:ThepeculiarjourneyofCzechresearchassess-ment.Science and Public Policy,39(5),618-629.

Merton, R. K. (1973) [1942].“TheNormativeStructureofScience”. InR.K.Merton,The Sociology of Science: Theoretical and Empirical Investi-gations.Chicago:UniversityofChicagoPress,267-278.

Morris, N. and Rip, A. (2006).Scientists’copingstrategiesinanevol-vingresearchsystem:ThecaseoflifescientistsintheUK.Science and Public Policy,33(4),253–263.

O’Carroll, C., Rentier, B., Cabello Valdès, C., Esposito, F., Kaunismaa, E., Maas, K., Metcalfe, J., McAllister, D. and Vandevelde, K. (Eds.)(2017).Evaluation of Research Careers fully acknowledging Open Science Practices. Rewards, incentives and/or recognition for researchers practi-cing Open Science.PublicationOfficeoftheEuropeanUnion.

Oliver, C. (1991).Strategicresponsestoinstitutionalprocesses.Acade-my of management review,16(1),145-179.

ISSUE 48 | JULY 2019 81

AUTHORSMARC VANHOLSBEECKDepartment of Information and Communication Studies, Université Libre de Bruxelles30DepageAvenue(level11),Brussels,1050(Belgium)E:[email protected]

THEODOSIA DEMETRIOUDepartment of Education, 46MakedonitissasAvenue,CY-2417,P.O.Box24005,Nicosia,1700(Cyprus)E:[email protected]

AGNE GIRKONTAITEDepartment of Sociology, Vilnius UniversityUniversitetostr.9,room309,Vilnius,01513(Lithuania)E:[email protected]

ANDREJA ISTENIC STARCICUniversity of Primorska, Faculty of EducationCankarjeva5,Koper,6000(Slovenia)University of Ljubljana, Faculty of Civil and Geodetic EngineeringJamova2,Ljubljana,1000(Slovenia)E:[email protected]

VILLE KEISKIUniversity of Jyväskylä, Department of Social Sciences and Philosophy,Opinkivi,Keskussairaalantie2,POBox35,40014(Finland)E:[email protected]

EMANUEL KULCZYCKIScholarly Communication Research Group, Adam Mickiewicz University,Szamarzewskiego89c,Poznań,60-568(Poland)E:[email protected]

ELENA PAPANASTASIOUSchool of Education, Department of Education46MakedonitissasAvenue,CY-2417,P.O.Box24005,Nicosia,1700(Cyprus)E:[email protected]

JANNE PÖLÖNENPublication Forum, Federation of Finnish Learned SocietiesSnellmaninkatu13,Helsinki,00170(Finland)E:[email protected]

HULDA PROPPEHáskóli Íslands / University of Iceland, GimliSæmundargata2,Reykjavi,101(Iceland)E:[email protected]

MAJA VEHOVECInstitute of EconomicsTrgJ.F.Kennedyja7,Zagreb,10000(Croatia)E:[email protected]


Asaconsequenceof this trend inpolicyand research,public fun-dingagenciesareexpectedtoimprovetheirsupportofSSHimpact.InSweden, four national funding agencies joined forces with a nationalNGO of SSH researchers in the design of a set of digital tools for in-clusivefundingofresearchandinnovation(R&I).Theprocessthattookplace2014-2018,isinthispaperusedasaspringboardforexpandingtheknowledgeonhowpublicR&Ifundingmaybedesignedinordertoen-hanceSSHimpact.ThemainresearchquestionconcernswhatexcludingandincludingmechanismstowardsSSHresearchersthatweredelinea-tedintheprocess,andhowthesemechanismsimpactedthedesignofdigitaltoolsforinclusivefunding.Previousstudiesonacademicimpactsupportservetotheoreticallycontextualisethesemechanismsandtools.Aparticipatoryresearchapproach,wherenewknowledgeisdevelopedjointlybyresearchersandothersocietalactors,servestoensuretheso-cialcontextualisationoftheprocessandresults.

Initially, the theoretical framework of academic impact support ispresented.Thisisfollowedbyanoutlineoftheparticipatoryresearchde-sign.Subsequently,theresultsarepresentedintermsofidentifiedme-chanismsofinclusionandexclusion,andtheirimpactonthetooldesign.Finally,conclusionsaredrawnregardinghowdigital tools for inclusivefundingmaybedesignedinawaythatenhancesSSHimpact.

IMPACT SUPPORTIn Sweden and several other European countries, public support

services for knowledge transfer, innovation and impact are offered byR&Ifundingagencies,universityinnovationoffices,technologytransferoffices,academicincubators,scienceparks,etc.Theseinstitutionsgene-rallyprovidefinancialand/ornon-financialsupportintermsofgrants,lo-ans,investments,businesscounselling,peer-to-peersupport,networks,testbeds,training, lectures,etc.Asmostofthesesupportservicestra-ditionallytargetresearchersand innovators intechnology,engineeringandnaturalsciences,needswithinsocialsciencesandhumanitiesareinsufficientlymet(cf.Bakhshietal.,2008;Brundeniusetal.,2016;Daw-sonandDaniel,2010;Howaldtetal.,2018;Lindberg,2012,2018;Lind-bergandNahnfeldt,2013;LundströmandZhou,2011;Muhonenetal.,2018;OlmosPeñuelaetal.,2014;Phippsetal.,2012;TEPSIE,2012,2014;Wutti and Hayden, 2017). Studies have identified both environmentalbarriers–e.g.lackofsupportstructuresintermsoffunding,counselling,

ABSTRACT

Inordertoexpandtheknowledgeonhowsocietalimpactofsocialsci-encesandhumanities(SSH)canbeenhancedthroughpublicfundingof researchand innovation,aprocessofdesigningdigitalised tools

forinclusivefundingisscrutinised,involvingfourSwedishfundingagen-cies and an Non-Governmental Organisation (NGO) of SSH researchers.TheagenciessharedthechallengetoattractabroaderrangeofSSHre-searcherstoapplyfortheirfunding.Excludingandincludingmechanismswere identified in the interactions, textsand imagesof the formulation,communicationandprocessingofcalls for funding.Thedevelopedtoolsdigitallyguidetheuserthroughqueriesregardingthepresentandpoten-tialdiversityofSSHrepresentationamongapplicants,reviewers,agencystaff,etc.andregardingtheformulationandcommunicationofcalltexts,assessmentcriteriaandreviewerinstructions.ThetoolsthusenhanceSSHimpactbymakingfundingmoreavailable,butfailtodemonstratehowcon-creteinteractionwithsocietalactorsmayenhancethis.

INTRODUCTIONDespitethewidelyacknowledgedimportanceofsocialsciencesand

humanities (SSH) for understanding and enhancing societal develop-ment,publicsupportstructuresforknowledgetransfer,innovationandimpact of research in society have traditionally focused natural scien-ces,engineeringandtechnology(SET) (Brundeniusetal.,2016;OlmosPeñuela et al, 2014; Wutti and Hayden, 2017). This is part of a moreencompassingpattern,wherealsopolicyandresearchoninnovationandgrowthhavefocusedindustrial,technologicalandcommercialrenewalratherthansocialtransformation(cf.DawsonandDaniel,2010;Godin,2014; Lindberg, 2012, 2018; van der Have and Rubalcaba, 2016). Theimportance of SSH research for innovation and impact is however in-creasinglyemphasisedinEUpolicystrategieson“SciencewithandforSociety” (SwafS),“ResponsibleResearchand Innovation” (RRI),“OpenScience”,etc.(EuropeanUnion,2014,2016,2017).Thisisaccompaniedbyarapidlyincreasingacademicinterestinsocialinnovation,withrefe-rencetonewfigurationsorcombinationsofsocialpracticesthatmeetsocialneeds,whereSSHknowledgeisesteemedaspivotal(Brandsenetal.,2016;Brundeniusetal.,2016;Howaldtetal.,2018;Moulaertetal.,2013;Nichollsetal.,2015).

MALINLINDBERG,SVETLANAGROSS,MILDARÖNN,LISSANORDIN,JANSANDRED,LARSWÄRNGÅRDANDCATHARINANORBERGDOI:10.22163/fteval.2019.372

INCLUSIVEFUNDINGFORENHANCEDIMPACTOFSOCIALSCIENCESANDHUMANITIES

ISSUE 48 | JULY 2019 83

networks,etc.–andactor-relatedbarriers–e.g.lackofcapacitiesandcompetencies – for realising and growing innovations with prominentsocialmotivesandcomponents (Brandsenetal.,2016;Howaldtetal.,2018;TEPSIE,2012,2014).Thisreflectsthe“reasonably settled consen-sus within the innovation community that science, engineering and tech-nology (STEM) research is more ‘useful’ to societies than other types of research, notably social sciences and humanities (SSH) research”,notedbyOlmosPeñuelaetal.(2014:384).Thefurthernotethatbyseekingto“increase and concentrate (R&I) funding on areas that bring the greatest, narrowly economic return”,governmentstendtoregardSSHresearchas“not worthy of investment”(ibid:385).

SSHdohowevermatterinsocietalprogress,“because they help us understand and address wicked problems (…) about which there is litt-le agreement on solutions”, according toPhippsetal. (2012:167). It isadvocatedthatSSHprovideinsightsinto–andinnovativesolutionsto– current societal challenges of poverty, immigration, climate change,security,health,etc.(Bakhshietal.,2008;LindbergandNahnfeldt,2013;Phipps et al., 2012; Wutti and Hayden, 2017). Traditional impact indi-catorsofpatents,licensesandspin-offcompaniesarethustoonarrowforestimatingSSHimpact(LundströmandZhou,2011;Muhonenetal.,2018;OlmosPeñuelaetal.,2014;WuttiandHayden,2017).SSHhavea well-documented tradition of engagement practices towards users,thusproducingresultsthattheseusersvalue,butthatmaybedifficulttomeasure in termsofmacro-economic impact (OlmosPeñuelaetal.,2014).Astudyof1600SpanishresearchersexposesthatSSHachieveitsimpactnotprimarilybydirectinteractionwithbusinesses,butbyindirectinteractionthroughcreatingcontentforthemedia,andbycooperatingmoredirectlywithgovernmentandcivilsocietyorganisationstoimprovethequalityoflife(ibid).AnAustrianstudysimilarlydetectsSSHimpactintermsoftransmissionofacademicknowledgeintoprofessionalpracticeand public spheres, not primarily seeking to obtain profits, but ratherraisingconsciousness(WuttiandHayden,2017).Basedonacompara-tiveanalysisof60examplesfrom16Europeancountries,Muhonenetal. (2018) proposes a framework for estimating impact that considersboth societal interaction and the societal changes it enhances. TheseexampleshighlightSSHimpactthrough“socialinnovation”,i.e.thede-velopment of new figurations or combinations of social practices thatmeetsocialneeds(Brandsenetal.,2016;Grimmetal.,2013;Howaldtetal.,2018;Moulaertetal.,2013;Nichollsetal.,2015;Phippsetal.,2012).

The public and academic interest in social innovation has rapid-lygrownduring the lastdecade,asaway tohandlecomplexsocietalchallenges (Brandsen et al., 2016; Grimm et al., 2013; Howaldt et al.,2018;Moulaertetal.,2013;Nichollsetal.,2015).Thishasservedtocon-solidatesocial innovationstudiesasamulti-disciplinaryresearchfield,providing insights intothedevelopmentofnewsolutionsfor improvedwelfare,wellbeingand relationsamongvariousgroupsandcommuni-ties, especially those perceived as economically or socially vulnerable(Cajaiba-Santana,2013;DawsonandDaniel,2010;Haxeltineetal.,2017;Ionescu,2015;PolandVille,2009;vanderHaveandRubalcaba,2016).Accordingtoseveralstudies,socialinnovationischaracterisedbyacti-veinvolvementofthosegroupsthataretobenefitfromthedevelopedsolutions,makingindividualandcollectiveempowermentacrucialcom-ponentofsuchprocesses(Brandsenetal.,2016;Howaldtetal.,2018;Moulaertetal.,2013;Nichollsetal.,2015).Thesestudiesalsocharac-terisesocialinnovationprocessesascomplexmulti-actorandmulti-levelendeavours,wherepublic,privateandcivilsectoractorsonvariousorga-nisationalandgeographicallevelareforcedtointeract,inordertopro-

perlyunderstandandaddresscomplexsocialsystems.Transformationofthesesystemsaredependentupontheinterplaybetweenstructureandagency,i.e.establishedinstitutions,regulationsandnorms,ontheonehand,andindividual’scapacitytochallengeorenforcethesestructures,ontheotherhand(Haxeltineetal.,2017;Westleyetal.,2017).Thecom-plexnatureofsocialinnovationsandthechallengestheyaddress,makesSSH expertise on human relations, social progress and organisationaldevelopmentpivotalinsuchprocesses(Brundeniusetal.,2016;Grimmetal.,2013;LundströmandZhou,2011;Phippsetal.,2012).

Aglobalmappingofover1000socialinnovationsexpose,however,thatresearchersandotheruniversityofficialsareinvolvedonlyin15per-centofthemappedcases(ButzinandTerstriep,2018).Incontrast,publicauthorities,civilsocietyorganisationsandprivatecompaniesaremorefrequently involved,amounting toabout40percentof thecases. Thiscontraststotheprominentroleofuniversitiesandresearchersintraditi-onalinnovationprocesses,focusingtechnologicalandindustrialinnova-tion.Insocialinnovation,researcherstendtobereplacedasknowledgeprovidersbyusers,beneficiariesandconsultants,inlinewiththegrass-rootcharacterofsuchprocesses(ButzinandTerstriep,2018;DomanskiandKaletka,2018;SørensenandTorfing,2015).AsnotedbyPhippsetal.(2012:167-168),“new SSH knowledge that isn’t shared cannot contribute to (…) social innovations”,callingforimproved“knowledge mobilization”tomaximisesocietalimpactofSSH,throughtheuseofknowledgebro-kersandsocialmedia.ItispredictedthatuniversitieswillbeincreasinglyinclinedtoinvestinknowledgetransferservicestosupportSSHintheco-productionofsocietallyusefulknowledge (LindbergandNahnfeldt,2013;LundströmandZhou,2011;Phippsetal.,2012). LundströmandZhou(2011)notetheestablishmentof‘socialinnovationparks’invariouspartsoftheworld,whereSSHknowledgeeitherformsthebasisfororenrichesdevelopmentofnewsolutionstosocietalchallenges.LindbergandNahnfeldt(2013)discernthatpublicsupportservicescouldenhanceSSH innovation through improved competences regarding how to de-sign,financeandscalesocialsolutions,throughrevisedproceduresandtoolstofittheneedsandprerequisitesamongSSHresearchers/innova-tors,aswellasthroughalternativewordsandimages–e.g.byreferringto “ideas” rather than “innovations” and images of people instead ofmachinery–toillustrateandinspireacademicinnovation.Bakhshietal.(2008)concludethatnationalfundingagenciesmayenhancewidercon-tributionsofSSHtoinnovationbysettingSSH-suitablestandardsforeva-luatinggoodpracticesofknowledgetransfer,bysupportingteam-basedcollaboration across disciplines, by facilitating a culture of knowledgetransfer with societal actors, and by functioning as active knowledgebrokers through face-to-face networking, personal contacts, represen-tationonexternalboards,panelsandsteeringgroups,etc.TheroleofnationalfundingagenciesasknowledgebrokersisfurtherscrutinisedbydeJongetal.(2016),exposingthatdespitetheireffortstoapplyimpactcriteriaintheirfunding,inlinewithgovernmentpolicies,itremainsun-cleartomanyresearchershowimpactshouldbeorganised,presentedandassessed.

RESEARCH DESIGNInordertoexpandtheknowledgeonhowpublicR&Ifundingmaybe

designedinordertoenhanceSSHimpact,thestudyemploysapartici-patoryresearchapproach,wherenewknowledgeisdevelopedjointlybyresearchersandother societalactors (AagaardNielsenandSvensson,


2006;ReasonandBradbury,2008).TheresearchersrepresentedaSwe-dishNGOofSSHresearchers (Humsamverkan) that joined forceswithrepresentativesfromfournationalR&Ifundingagencies(Formas–TheSwedish Research Council for Sustainable Development, Forte – TheSwedishResearchCouncil forHealth,Working lifeandWelfare,Ener-gimyndigheten – The Swedish Energy Agency, VINNOVA – Sweden’sInnovationAgency)–inaprocessofdesigningdigitaltoolsforinclusivefundingofresearchandinnovation.TworesearchersfromHumsamver-kanfacilitatedtheprocess,whiletheagencyrepresentativescontributedwith theirpracticalexperiences.During2014-2018, they incrementallydelineated excluding and including mechanisms towards SSH resear-chersintheirprocessesofformulating,communicatingandprocessingcalls for funding. This tookplaceat regularworkshops, alternatelyar-rangedindividuallywitheachagencyandcollectivelywithallagencies.Basedontheacquiredinsights,toolswerepiecewisecollectivelydesi-gnedduringcontinued, jointworkshops.The toolswerepublicly laun-ched in thespringof2018, freelyavailableatwww.humsamverkan.se(inSwedishonly).

InordertosimultaneouslyexpandtheacademicknowledgeonhowpublicR&IfundingmaybedesignedtoenhanceSSHimpact,thepartici-patingresearchersandagencyrepresentativesdecidedtoscientificallyanalysetheprocessandresults, inlinewiththeparticipatoryresearchapproach of joint knowledge development (cf. Aagaard Nielsen andSvensson,2006;ReasonandBradbury,2008).Theresultsofthatanalysisformthebasisforthispaperthatisco-authoredbythemainparticipants.Theparticipatoryprocedurehelpsattaining‘sociallyrobustknowledge’,astheresultsarevalidatedthroughcontinuousdialoguebetweenthosewhopossesspracticalexperiencesandacademicknowledgeinthestu-diedarea(Nowotnyetal.,2001).Thedatainformingthestudyconsistsofmeetingminutes,tooldraftsandthefinalisedtools,collectedattheworkshopsduring2014-2018.Thecollecteddatawasthenanalysedinthe lightof the theoretical frameworkofacademic impact support,aspartofthejointwritingprocess.Aspartofthis,athematicanalysiswasperformedinordertodistinguishexcludingandincludingmechanismstowardsSSHresearchers(cf.Guestetal.,2012).

RESULTSMotivatedbytherisinginterestinEuropeanpolicyandresearchfor

SSH impactand innovation, the fournational fundingagencies joinedforces with the national NGO of SSH researchers in 2014, in order todelineateexcludingandincludingmechanismstowardsSSHresearchersintheircallsforfunding,asabasisfordesigningdigitaltoolsforinclu-siveR&Ifunding,whichwerelaunchedin2018(availableinSwedishatwww.humsamverkan.se).Theagenciessharedtheambitiontopromotesocietalprogressthroughtheirfunding,aswellasthechallengetoat-tractabroaderrangeofSSHresearcherstoapplyfortheir funding. Inordertoimprovetheirunderstandingofobstaclesandopportunitiesforinclusivefunding,theystartedoffbyidentifyingexcludingandincludingmechanismstowardsSSHresearchersintheirprocessesofformulating,communicatingandprocessingcallsforfunding.Firstly,theydelineatedtheircallprocesses,identifyingeachphasefrominitialinitiativetillfinalfundingdecisions.Evenifeachagencyhadtheirown,specificroutesforinitiatingandmanagingcalls,commonphasesincludedidentificationofrelevantareasorchallengestoaddressinthecall,formationofastaffteam to manage the call, formulation of the call text, communicating

ofthecalltotargetgroups,reviewingapplicationsandcommunicatingdecisions.

Ineachofthedelineatedphases,crucialinteractionsbetweenstaff,targetgroups, intermediariesandother stakeholderswerepinpointed.Key textsand imageswerealso identified, including instructions fromgovernmentandtopmanagement,calltexts,websites,othermarketingmaterial,evaluationinstructions,decisionletters,etc.Thedelineatedin-teractions,textsandimageswerethenscrutinisedwithregardtotheirpotentiallyexcludingor includingeffectsonSSH researchers fromva-rious disciplines. In the interactions, such mechanisms were primarilyidentifiedinthecompositionofthestaffteam,thecontactnetworkswithintermediaries for communicating the call, the presentation forms forcommunicatingthecall,aswellasthecompositionof–andinstructionsto–reviewcommittees.Excludingorincludingeffectswereperceivedtobedependentontherepresentationandapplicationofavarietyofcom-petenceareasintheseinteractions,includingavarietyofSSH-specificones. In the texts,similarmechanismswere identifiedprimarily in thedescriptionoftheaddressedareasorchallenges,inthedemandedcom-petenceprofile, in theassessmentcriteria,aswellas in theheadingsand structureofpowerpointpresentations.Excludingor includingef-fectswereperceivedtobedependentonthechoiceandorderingofspe-cifictermsandcriteria,intentionallyorunintentionallylinkedtospecificresearchdisciplinesorideological/politicalnorms.Suchlinkscouldeit-herbeexplicitorimplicit,concreteorabstract,specificorgeneric.Bothexplicitlyandimplicitlystateddisciplines–nameddirectlyorimplicatedthroughdiscipline-specificterminology–mightgivetheimpressionthatonlyresearchersinthesedisciplinesarethetargetgroupforthecall.Theorderinwhichcertainareasorcriteriaarepresentedmightalsoaffectwhoisappealedbythecall,whereareas/criteriathatarepresentedfirstoftenareperceivedas themost important. If themost limitingareas/criteriaarepresentedfirst,anarrowerrangeofapplicantswillprobablybeappealed, than if it ispresented last. In the images,excludingandincludingmechanismswereidentifiedprimarilyintheillustrationsincalltexts,websitesandpowerpointpresentations.Theeffectswerepercei-vedtobedependentontherepresentationofavarietyofresearchers,disciplines,areas,etc.,inavarietyofformatsandsettings.

Thedelineationof theseexcludingand includingmechanismswasusedasaspringboardfordesigningasetofdigitaltoolsforinclusiveR&Ifunding.Theambitionwasthatthetoolswouldbeusefulbothfortheparticipatingagencies,aswellasotherfundingagenciesinSweden,inthestriveforimprovedsocietalimpactthroughbroadenedrepresentationofresearchersanddisciplines.BesidesimprovedSSHrepresentation,thetoolsmightservetoenhanceapplicationsfromotherunder-representedgroupsaswell,inregardtogender,origin,age,etc.Thedesignedtoolsencompassthreemainentrypoints:1)Whatdowemissout?,2)Whatarethereasons?,3)Whatcanwedo?.Whenenteringanyofthese,theuserisguidedthroughasetofawareness-raisingandpractice-orientedqueries.Threetofourmainqueriesaccompanyeachpoint,furtherspeci-fiedinanumberofsub-queries.Theuserisencouragedtouseapreviousorpotentialcallforfundingasabasis,whenrespondingtothequeries.Thefirstentrypoint–Whatdowemissout?–isfollowedbythreemainqueries:1)Whohasappliedandbeengrantedfunding–andwhohasnot?,2)Whatkindofapplicantsandgrantedapplicationswouldhavebeendesirable,3)Howcantheinsightsintowhatapplicantshavebeenmissed out be improved?. Examples of sub-queries are: How well arevariousdisciplinesrepresentedamongapplicantsandgrantedapplica-tions?, Are the approval-rates the same for various disciplines?, What

ISSUE 48 | JULY 2019 85

disciplinesoughttoberepresentedinordertoproperlyunderstandtheaddressedarea?.Toenhancethereflections,a listofallexistingSSH-disciplinesisprovided.

The second entry point – What are the reasons? – is followed byfourmainqueries:1)Howisthecalltextformulated?,2)Whatcriteriaareappliedinthecall?,3)Howisthecallcommunicated?,4)Howaretheapplicationsreviewed?.Examplesofsub-queriesare:Towhatextentisabroadvarietyofdisciplinesrepresentedamongstaff,reviewers,ap-plicants,intermediariesandotherstakeholders?,Whatexplicit,implicitorabsentdisciplinesaredistinguishableinthetextsandimagesofthecall,andtowhatextentdothesereflectabroadvarietyofdisciplines?,Towhatextentdoexistingcriteriaqualifyordisqualifyapplicantsfromabroadvarietyofdisciplines?,Dotheestablishedcommunicationchan-nelsreachawidevarietyofresearchers?.Thethirdentrypoint–Whatcanwedo?– is followedby threemainqueries:1)Doweneedmoreknowledge?,2)Doweneed tochangeour routinesand frameworks?,3)Doweneedtodevelopinternalroutinesandsupportfunctions?.Ex-amplesofsub-queriesare:Whatnewknowledgeisneededinordertounderstandandcounteracttheexcludingmechanismsidentifiedinthefirstandsecondentrypoint?,Whatnewinstructions,courses,formsorotherroutinesandsupportfunctionscanbeestablishedinordertoattainagreatervarietyofapplicantsandgrantedapplications?,Whopossessestheorganisationalpowertoaltercomprehensiveframeworksorregula-tions?.

Theentrypointsarecomplementedbythreefictionalcase-examples,ofwhichoneispresentedbelow.

A CALL THAT DOES NOT REACH ITS INTENDED TARGET GROUPS

Acall for fundingof researchaboutsustainable transport sys-temsintendstoengendernewknowledgeonhowtrafficvolu-mes may be reduced by infrastructural planning. The agencyesteemthatcomprehensivestudiesareneeded,spanningfromhow actors communicate during planning processes to hownormsanddiscoursesaffecttheirdecisions.Theagencythere-forewantstoattractavarietyofapplicantstothecall,notleastfromsocialsciencesandhumanities.Thecall ishoweverformulatedinawaythatassumesthatthereaderalreadypossessesexpertiseinthetransportarea.Itcon-tains, for example, several expressions that are specific to thearea.Thetextthussignalsthatthecallisdirectedtoaspecific,narrow group of transport experts. When communicating thecallthroughnewslettersandinformationmeetings,theagencymakesnoattempttoexplainthearea-specificexpressionstoawideraudience.Asaresult, thefinancierdoesnotattaintheaspiredvarietyofapplicants.Thefewapplicationsreceivedonlycomefromresear-cherswhoarealreadyaccustomedtoapplyingforfundingfromtransport-specificcalls.

CONCLUSIONS

ThejointdesignofdigitaltoolsforinclusiveR&Ifunding,asdepictedintheprecedingsection,servestoexpandtheknowledgeonhowsucheffortsmayenhanceSSHimpact.Similartotheargumentationinprevi-ousstudiesonacademicimpactsupport,theprocesswasmotivatedbytheacknowledgedabilityofSSHtoprovideinsightsintoandinnovativesolutionstocomplexsocietalchallenges(cf.Bakhshietal.,2008;Brun-deniusetal.,2016;Grimmetal.,2013;LindbergandNahnfeldt,2013;LundströmandZhou,2011;Phippsetal.,2012;WuttiandHayden,2017).InlinewithpreviouslyidentifiedbarrierstoSSHimpactandinnovation,theparticipating fundingagenciesandSSHresearchersperceived thenarrowrangeofSSHdisciplinesrepresentedamongapplicantsasham-peringtosuchambitions(cf.Brandsenetal.,2016;Howaldtetal.,2018;TEPSIE, 2012, 2014). The joint ambition was to address both environ-mentalbarriersintermsofgrantedfundingandactor-relatedbarriersintermsofimprovedcapacitiesandcompetenciestoprovidesuchfunding(cf.ibid).Theprocessthusacknowledgedthattransformationofcomplexsocial systems requires an interplay between established institutions,regulations and norms, on the one hand, and individuals’ capacity tochallengeandchangesuchstructures,asconcludedinpreviousstudies(cf.Haxeltineetal.,2017;Westleyetal.,2017).

The first research question, regarding what excluding and inclu-dingmechanismstowardsSSHresearchersthataredelineated intheprocess,exposesthatsuchmechanismsareatplaythroughallphasesofthecallprocesses,includingidentificationofareastoaddress,staffteamformation,calltextformulation,targetgroupcommunication,ap-plication review and decision communication. The interactions, textsandimagesineachphasearedelineatedasespeciallyrelevant.InthelightofSET-relatednormsofeconomicandcommercialimpact,pinpoin-tedinpreviousstudiesonacademicimpactsupport,thedelineatedme-chanisms mainly concern: 1) the variety of SSH-related competencesrepresentedamongtheagencystaffandreviewcommittees,2)thepre-senceofSSH-tailoredareadescriptions,termsandassessmentcriteria,in call texts, website information, power point presentations, review-er instructions, etc. (cf.Bakhshietal., 2008;Brundeniusetal., 2016;DawsonandDaniel,2010;Howaldtetal.,2018;Lindberg,2012,2018;LindbergandNahnfeldt,2013;LundströmandZhou,2011;Muhonenetal.,2018;OlmosPeñuelaetal.,2014;Phippsetal.,2012;TEPSIE,2012,2014;WuttiandHayden,2017).The identifiedexcludingmechanismsenforce–justastheincludingmechanismschallenge–theperceptionofSETbeingmoreusefulandinvestment-worthyforthesociety(cf.Ol-mosPeñuelaetal.,2014).

Thesecondresearchquestion,regardinghowtheidentifiedmecha-nismsimpactedthedesignofdigitaltoolsforinclusivefunding,exposesthatthreemainentrypoints–regardingwhatismissedout,whythisismissed,andwhatcanbechanged–wereperceivedasthemostcrucial.ByguidingtheuserthroughqueriesregardingthepresentandpotentialdiversityofSSHrepresentationamongapplicants,grantedapplications,reviewers,intermediaries,contactnetworksandagencystaff,apathwayto more inclusive funding is established. The path is further clarifiedbyqueries regarding the formulationandcommunicationofcall texts,assessmentcriteriaandreviewerinstructions,aswellasorganisationalroutinesandsupportforenhancedSSHimpact.Thetooldesignthuscon-cordswithconclusions inpreviousstudies, regarding thecrucial func-tionofpublic fundingagenciesasknowledgebrokers,byencouraging


societal knowledge transfer ingeneral, and tailored toolsandcriteria,widened communication paths, alternative words and images, etc. inparticular (cf.Bakhshietal.,2008;deJongetal.,2016;LindbergandNahnfeldt,2013).

Byunderlining theneed forsimultaneouschangesofpracticalcalldesignandstrategicorganisationalframeworks,thedevelopedtoolsre-flect the crucial interplay between established procedures/norms andindividual/collectiveempowerment(cf.Haxeltineetal.,2017;Westleyetal.,2017).Whenattemptingtomakethefundingmoreinclusivetowardsunderrepresentedgroups, conflictsmayarise in relationboth toothermissionsandtasksoftheagencies,andtoconservativeattitudesamongagencymanagers, staffandotherstakeholders.AsimilarlyhamperingfactoristhatneithertheidentifiedmechanismsnorthedesignedtoolsconsiderhowSSHimpactmaybeenhancedthroughconcrete interac-tion with users and other stakeholders from various societal sectors,highlightedaspivotalinpreviousstudies(cf.Brandsenetal.,2016;How-aldtetal.,2018;deJongetal.,2016;Moulaertetal.,2013;Muhonenetal.,2018;Nichollsetal.,2015;OlmosPeñuelaetal.,2014;WuttiandHayden, 2017). As social innovation studies underline the importanceofactiveinvolvementofusersandstakeholdersinordertoenablebothindividualandcollectiveempowerment, the intendedenhancementofSSHimpactmayhavebeenhampered(cf.Brandsenetal.,2016;Howaldtetal.,2018;Moulaertetal.,2013;Nichollsetal.,2015).Thisisespeciallyconcerning,asresearcherstendtobereplacedasknowledgeprovidersbyusers,beneficiariesandconsultants in social innovationprocesses,thusmissingoutonvaluableSSHexpertiseonhumanrelations,socialprogress and organisational development (cf. Brundenius et al., 2016;ButzinandTerstriep,2018;DomanskiandKaletka,2018;Grimmetal.,2013;LundströmandZhou,2011;Phippsetal.,2012;SørensenandTor-fing,2015).

REFERENCESAagaard Nielsen, K. and Svensson, L.(Eds.)(2006).Action research and participatory research.Maastricht:ShakerPublishing.

Bakhshi, H., Schneider, P. and Walker, C. (2008).Arts and Humanities Research and Innovation.London/Bristol:NESTAandArtsandHumani-tiesResearchCouncil.

Brandsen, T., Cattacin, S., Evers, A. and Zimmer, A.(Eds.)(2016).Soci-al innovations in the urban context.NewYork:Springer.

Brundenius, C., Göransson, B. and Carvalho de Mello, J. M. (Eds.)(2016).Universities, Inclusive Development and Social Innovation: an in-ternational perspective.Cham:Springer.

Butzin, A. and Terstriep, J.(2018).ActorsandRolesinSocialInnovati-on.InJ.Howaldt,C.Kaletka,A.SchröderandM.Zirngiebl(Eds).Atlas of Social Innovation – New Practices for a Better Future.Dortmund:Sozial-forschungsstelle,TUDortmundUniversity.

Cajaiba-Santana, G. (2013).Socialinnovation:Movingthefieldforward.A conceptual framework. Technological Forecasting & Social Change.82(1):42-51.

Dawson, P. and Daniel, L. (2010).Understandingsocial innovation:aprovisionalframework.International Journal of Technology Management.51(1):9-21.

de Jong, S. P. L., Smit, J., van Drooge, L.(2016).Scientists’responsetosocietal impactpolicies:Apolicyparadox.Science and Public Policy.43(1):102-114.

Domanski, D. and Kaletka, C. (2018). Social Innovation Ecosystems.InJ.Howaldt,C.Kaletka,A.SchröderandM.Zirngiebl (Eds).Atlas of Social Innovation – New Practices for a Better Future.Dortmund:Sozial-forschungsstelle,TUDortmundUniversity.

European Union(2017).LAB – FAB – APP — Investing in the European future we want.Reportofthe independentHighLevelGrouponmaxi-misingtheimpactofEUResearch&InnovationProgrammes.Brussels:EuropeanCommission.

European Union (2016). Open Innovation, Open Science, Open to the World.Brussels:EuropeanCommission.

European Union (2014). Science with and for Society. HORIZON 2020 Work Programme 2014-2015.Brussels:EuropeanCommission.

Godin, B. (2014).Anoldwordforanewworld,orthede-contestationofapoliticalandcontestedconcept. InK-E.Sveiby,P.GripenbergandB. Segercrantz (Eds.). Challenging the Innovation Paradigm. New York/London:TaylorandFrancis.

Grimm, R., Fox, C., Baines, S. and Albertson, K. (2013).Social inno-vation,ananswertocontemporarysocietalchallenges?Innovation: The European Journal of Social Sciences.26(4):436–455.

Guest, G., MacQueen, K. M. and E. E. Namey (2012).Applied thematic analysis.ThousandOaks:SAGE.

Haxeltine, A., Pel, B., Dumitru, A., Avelino, F., Kemp, R., Bauler, T., Kunze, I., Dorland, J., Wittmayer, J. and Jørgensen, M. S.(2017).To-wards a TSI theory: a relational framework and 12 propositions.TRANSIT.

Howaldt, J., Kaletka, C., Schröder, A. and Zirngiebl, M.(2018).Atlas of Social Innovation – New Practices for a Better Future.Dortmund:Sozi-alforschungsstelle,TUDortmundUniversity.

Ionescu, C. (2015). About the conceptualization of social innovation.Theoretical & Applied Economics,Vol.22No.3,pp.53-62.

Lindberg, M. (2012).Astrikingpattern–Co-constructionofinnovation,menandmasculinityinSweden’sinnovationpolicy.InS.Andersson,K.Berglund,J.Thorslund,E.GunnarssonandE.Sundin (Eds.).Promoting Innovation.Stockholm:VINNOVA,pp.47-67.

Lindberg, M.(2018).Relatinginclusivenessandinnovativenessininclu-siveinnovation.International Journal of Innovation and Regional Develop-ment.8(2):103-119.

ISSUE 48 | JULY 2019 87

Lindberg, M. and Nahnfeldt, C. (2013).Commercializingworklifeba-lance – outlining a model for analyzing and promoting social scienceinnovation.International Journal of Innovation Science.5(1):21-30.

Lundström, A. and Zhou, C. (2011).Promotinginnovationbasedonso-cialsciencesandtechnologies:theprospectofasocialinnovationpark.Innovation – The European Journal of Social Science Research.24(1-2):133-149.

Moulaert, F., MacCallum, D., Mehmood, A. and Hamdouch, A. (Eds.)(2013).The international handbook on social innovation.Cheltenham:Ed-wardElgar.

Muhonen, R., Benneworth, P. S. and Olmos Peñuela, J.(2018).From productive interactions to impact pathways: Understanding the key dimen-sions in developing SSH research societal impact.CHEPSworkingpaper;Vol.2018,No.02.CenterforHigherEducationPolicyStudies(CHEPS).

Nicholls, A., Simon, J. and Gabriel, M.(Eds.)(2015).New Frontiers in Social Innovation Research.NewYork:PalgraveMacmillan.

Nowotny, H., Scott, P. and Gibbons, M. (2001). Re-thinking science – knowledge and the public in an age of uncertainty.Cambridge:PolityPress.

Olmos-Penuela, J., Benneworth, P., Castro-Martinez, E. (2014).AreSTEMfromMarsandSSHfromVenus?Challengingdisciplinarystereo-typesofresearch’ssocialvalue.Science and Public Policy.41:384–400.

Phipps, D. J., Jensen, K. E. and Myers, J. G. (2012).ApplyingSocialSciencesResearchforPublicBenefitUsingKnowledgeMobilizationandSocialMedia.InA.Lopez-VarelaAzcárate(Ed.).Theoretical and Metho-dological Approaches to Social Sciences and Knowledge Management.London:IntechOpen.

Pol, E. and Ville, S. (2009). Social innovation: Buzzword or enduringterm?.The Journal of Socio-Economics.38(6):878-885.

Reason, P. and Bradbury, H.(Eds.)(2008).The Sage Handbook of Action Research.Participatory Inquiry and Practice.London:SAGEPublications.

Sørensen, E. and Torfing, J. (2015). Enhancing Public Innovationthrough Collaboration, Leadership and New Public Governance. In A.Nicholls,J.Simon,M.Gabriel(Eds.).New Frontiers in Social Innovation Research.NewYork:PalgraveMacmillan.

TEPSIE(2012).Barriers to social innovation.Brussels:EuropeanCommis-sion.

TEPSIE (2014).Social Innovation Theory and Research – A Guide for Re-searchers.Brussels:EuropeanCommission.

van der Have, R. P. and Rubalcaba, L. (2016). Social innovation re-search: An emerging area of innovation studies? Research Policy. 45:1923-1935.

Westley, F., McGowan, K. and Tjörnbo, O. (Eds.)(2017).The evolution of social innovation: building resilience through transitions.Cheltenham:EdwardElgarPublishing.

Wutti, D. and Hayden, M. (2017). Knowledge Transfer in the SocialSciencesandHumanities(SSH)–Definition,Motivators,Obstacles,andVisions.Colloquium: New Philologies.2(1):87-101.

AUTHORSMALIN LINDBERGProfessor, Industrial Design, Luleå University of TechnologyLuleå,97187(Sweden)E:[email protected]

SVETLANA GROSSPhD student, Stockholm School of Economics and HumsamverkanStockholm,11383(Sweden)

MILDA RÖNNChair, HumsamverkanStockholm(Sweden)

LISSA NORDINSenior Analyst, Formas – Swedish Research Council for Sustainable DevelopmentStockholm,11182(Sweden)

JAN SANDREDProject Officer, VINNOVA – Sweden’s Innovation AgencyStockholm,10158(Sweden)

LARS WÄRNGÅRDSenior Advisor, Forte – Swedish Research Council for Health, Working life and WelfareStockholm,10137(Sweden)

CATHARINA NORBERGResearch Officer, Energimyndigheten – The Swedish Energy AgencyEskilstuna,63104(Sweden)

KEYWORDS funding;humanities;impact;innovation;knowledgetransfer,socialsciences


BACKGROUND: SOCIAL IMPACT OF SCIENCE THROUGH PRODUCTIVE INTERACTIONS

Academic research evaluation has seen important changes in the2000s mainly related to changes in the societal role of university andthe transformation of university governance towards increased socialresponsibility and accountability. The traditional research assessmentwasbasedonpeerreviewand,subsequently,thenumberofscientificpublications.Inthe1990s,thedemandformeasuringeconomicreturnsfromresearchfundingincreasedandwascloselyassociatedwithadvan-cementofcommercialisationofuniversityresearchresults.Inthe2000s,variousEUcountriesstartedtodevelopframeworksforanalysingwidersocietal impacts of academic research, a task that was related to theintroductionof the thirdmissionofuniversities.Thus, the focus in theimpactassessmentshiftedtothedevelopmentofquantitativeindicatorsthroughwhichthesocietalimpactcouldbemeasuredandusedinallo-catingfundsforresearch(KearnesandWienroth2011).

Anotherdevelopmentinimpactassessmenthasbeentheshiftfromlinear to interactive models of science’s social impact. The fact thatsciencehasbecome“contextualised”andknowledge“socially robust”(Nowotnyetal.2001)hashadimplicationsforresearchpolicywiththeendresultof thatbeingthereplacementof linearprocessesofunder-standingthesocialimpactwithinteractiveapproaches.Thus,insteadofseeingscienceasthefountainofnewknowledge,whichwouldunprob-lematicallyflowfromuniversitiestosociety,interactivemodels,suchasproductive interactions (SpaapenandvanDrooge2011),acknowledgethat societal actors other than scientists are increasingly important increatingscience’ssocietalimpact.

INTRODUCTION

Intherecentyears,theUniversityofOuluhastakenanactiveroleinsupportingitsresearchinsocialsciencesandhumanities(SSH).Thesupporthasmanifesteditselfintheintroductionofnewstructures

suchasthe“EudaimoniaInstitute”1,establishedintheearly2010stopromoteandcoordinatemulti-andcross-disciplinaryresearchinhumansciences. “Eudaimonia” constitutesa community inwhich researchersareprovidedacollegialandsupportiveenvironmentforcarryingoutre-search. It also servesasaplatform in thenewservice concept called“Rapid Research Radicals” (3R), which aims to develop collaborativeexcellence and new openings in research. Connected to this, “TellusInnovationArena”2offersmethodsandfacilitationexpertisetosupportvarious formsofcollaboration. Inaddition to this, “OuluThinkTankofScienceandSociety”3,whichoperatesundertheauspicesofthe“Eudai-monia Institute”,wasestablished to facilitate the interactionbetweenscientistsinSSHandthebroadersociety.The“OuluThinkTank”aimstoproduce policy-relevant research of internationally high standard, andtoofferitsexpertisetodifferentparties,suchascompaniesandsciencepolicymakers.TheSSHcommunityhasalsobeensuccessfullyincludedinthedevelopmentoftheuniversity’sstrategy4andprofilingmeasures.

JUHATUUNAINEN,RAULISVENTO,PENTTIHADDINGTON,KIRSIOJUTKANGASANDSIRPAAALTODOI:10.22163/fteval.2019.373

THEOULUWAYOFSTRENGTHENINGSOCIALIMPACTOFSSHSCIENCES:FROMLINEARMODELSOFIMPACTTOPRODUCTIVEINTERACTIONSANDBEYOND

1 http://www.oulu.fi/eudaimonia/2 http://www.oulu.fi/tellusarena3 http://www.oulu.fi/thinktank4 http://www.oulu.fi/university/focusarea/understanding-humans-in-change

ISSUE 48 | JULY 2019 89

waslackingresearch-basedevidence.Tofill thisgap,SRCfundshigh-qualityresearchwithactualsocietalimpactwithanannualbudgetofaround55millioneuros. Itencourages researchers to findconcretesolutions to so-called grand challenges. Solving grand challenges re-quiresmultidisciplinaryapproachesandactiveinteractionamongaveryheterogeneoussetofsocietalactors.Thus,animportantelementofSRCprojects is active interaction between those who produce new know-ledgeandthosewhouseandfurtherelaborateit.

“BCDCEnergy”,thecaseexamplewechosetoanalyseinthispaper,isa large,multidisciplinaryandmultiorganisationalconsortiumchoseninthefirstcalloftheSRCresearchfundsin2015.TheconsortiumisledbyOuluBusinessSchool,theUniversityofOulu,andtheotherpartnersare:theCentreforWirelessCommunicationsandDepartmentofInfor-mationandCommunicationStudiesattheUniversityofOulu,theFinnishEnvironmentInstitute(SYKE),theVATTInstituteforEconomicResearch,theDepartmentofComputerScienceatUniversityofHelsinkiandtheFinnishMeteorological Institute (FMI).Theaimof theconsortiumis todevelopnewtypesofservicestointegraterenewableresourcesintothesmartgridsincostefficientways.

EPISTEMOLOGICAL DIMENSIONS OF ACHIEVING IMPACT IN “BCDC ENERGY”

Theepistemologicaldimensionofproductiveinteractionsaddresseswhatnew research resultsandunderstandingof relevantphenomenahavecontributedtothesolvingoftechnologicalandsocietalproblems.Thebasic researchquestionof “BCDCEnergy” relates to the intermit-tencyproblemcreated to theenergy system through increasingshareof renewable energy sources. Wind and solar energy production can-notbedispatchedinsimilarwaysthantraditionalpowertechnologies.Thiscreatesnewtypesofuncertaintiesintothesystem.“BCDCEnergy”searchsolutionsfromdifferenttypesofflexibilitiesboth insupplysideanddemandsideofthesystem.Insupplyside,theroleofhydropowerincompensatingintermittencygeneratedbyrenewableshasbeenstu-died.IthasbeenshownhowhydropowerhaspositivepotentialinhydrodominatedmarketsliketheNordicelectricitymarket“NordPool”(Huukietal.2017).Fromthedemandside,consumerflexibilityhasbeenstudiedbyanalysingtheroleofrealtimepricing(Huukietal.2017).Ithasbeenshownthatrealtimepricingcanplayaroleinsolvingtheintermittencyproblembuttherearealsoconstraintsrelatedtoitsefficiency(Kuhnlenzetal.2018).Consumerattitudeshavebeenanalysedthroughlargesur-veysusingthechoiceexperimentmethod. It turnsoutthatconsumersarewillingtoadaptnewtechnologiesandcontractsiftheygetaccepta-blecompensationsfromthedisutilitythatarecreatedtothem(Ruokamoetal.2018,Krishnamurtyetal.2018).

The research community of “BCDC Energy” quite early recognisedthatwithoutmultidisciplinaryknowledgetheprojectcouldeasilyhavefocusedintonarrowlydefineddimensionalfeaturesoftheongoingener-gy market disruption. Furthermore, the project’s understanding of theneedtotakeallmarketcontexts intoconsiderationbecameobvious intheworkshopsorganised.Theprojectisthusconfidentthatthebroade-ningofitsviewandresearchagendahashelpedtheresearcherstocom-municatetheirscientificfindingswithandservetheneedsoftheenergymarketstakeholders.“TheFinnishTransmissionSystemOperator”(TSO)Fingridpublishedaworkingpaperandarelatedreportentitled“Electri-city market needs fixing – What can we do?”inMay2016.Togetherwith

Acommonfeatureoftheinteractiveapproachesistheiremphasisonsituatedandnegotiatedcharacterofsciencewithinlocalsocialcontexts(HaywoodandBesley2014)occupiedbyheterogeneousgroupsofstake-holderswithspecificinterestsoftheirown(Michael2009).Thequestionisthusnomoreabouttheunilineartransferofknowledgetosociety,butaboutvariouswaysinwhichsocietalactorsareengagedinknowledgecreation and application (Spaapen and van Drooge 2011). Interactivemodelsthereforeinvolveamoreequalandcollaborativecommunicationbetweenacademicsandsocietalactors,andincreasedlevelsofnegoti-ationacrosstheblurredborderbetweenscienceandsociety(HaywoodandBesley2014).Bysodoing,theyalsohelpustoappreciatehowso-cietalstakeholdersattributevaluetoscientificfindingsandmakeuseoftheseasapartoftheirownactivities.

Among the most popular interactive concepts of science’s socialimpactisthatofproductiveinteractions.Theconceptwasdesignedtocircumventtheproblemsofattribution,temporalityandcausalityinim-pactassessmentbylookingatinteractiveprocessesbymeansofwhichimpactsarecreatedinpracticebyheterogeneoussetsofsocialactors.Theconceptseeksthustoaddresstheiterativeproductionofnewun-derstandingandmutuallearningattheinterfaceofscienceandsociety.AccordingtoSpaapenandvanDrooge(2011:212),productiveinterac-tionscanbedefinedas“exchanges between researchers and stakehol-ders in which knowledge is produced and valued that is both scientifically robust and socially relevant”.Inaddition,SpaapenandvanDrooge(2011:212)elaboratedthattheinteractionisproductiveif“it leads to efforts by stakeholders to somehow use or apply research results”.Productiveinter-actionsarethuspreconditionsofachievingsocietalimpactsofscienceor“intermediate indications of de facto social impact”(SpaapenandvanDrooge2011:216).

Inourview,productiveinteractionsareamongthemostfruitfulsug-gestionsforunderstandingthesocialimpactofsciencebuthavelimitedvalueinprovidingdifferentiatedunderstandingaboutthevariousmodesofinteractionthattakeplaceduringsuchmutualinvolvement.Whatisneededisamoretangibleframeworkwiththehelpofwhichonecoulddifferentiatebetweenvariouskindsofproductiveinteractionsandarti-culatetherolenewscientificunderstandinghasinsolvingsocietalpro-blems.Forsuchaframework,wewilluseadistinctionmadebetweenthree dimensions of societal impact of science, 1) epistemological, 2)artefactualand3)interactional-institutionalfoundationsofimpact(Mi-ettinen et al. 2015). Further, we will illustrate how such an approachcouldbeusedtoanalyseenergyresearchledbyscholarsworkinginOuluBusinessSchool,theUniversityofOulu,Finland.Wedosotostimulateself-reflectionofresearchcommunitiesandtohelpthemarticulatethesocietalimpactoftheirresearchwheneveritisneeded.

BRIGHT CLOUDS - DARK CLOUDS (BCDC) ENERGY CONSORTIUM AS A CASE EXAMPLE

The research consortium we analyse, “BCDC Energy”, was fundedby thenewlyestablishedStrategicResearchCouncilof Finland (SRC),whichissettoencourageandenablethediscoursebetweenscientificresearchandsociety.ThemajorinnovationpoliticalrationaletoprecedethecreationofSRCwastherecognitionofhowsocietaldecision-making


in the interface between science and energy markets. Therefore, thecompanies familiarwith themarket situation in theenergy sector arevitalpartnersindevelopingtheserviceplatformtheprojectaimstobuild.Consequently,theproject’sAdvisoryBoard(AB),whichinvolvesimpor-tantsocietalstakeholders,wasformedatthebeginningofresearch.Theaim was to use co-creation methods to develop the services with thekeyplayersof theenergymarkets,and to shareknowledgeof the re-cent research with them. Interactive events and workshops organisedby“BCDCEnergy”withtheABhavebeenthemeanstodevelopmodelsfornewtypesofserviceplatforms.Thesemeansallowmultidisciplinary,multi-organisational and multi-professional collaboration. Face-to-faceinteractionsintheworkshopsenablefindingalternativesolutionstochi-cken-and-eggproblemstypicaltotwo-sidedplatforms. Inaddition,themembersoftheABhavetheirownonlineforum,whichhasbeenusedtoshare,store,produceandmanageinformationrelatedtotheproject.

ExternalinteractionandcommunicationinBCDCisdesignedandim-plementedincollaborationwiththecommunications’unitsofthepartneruniversities,otherorganisationsandpartnersinvolvedintheworkoftheBCDCconsortium.Theprojecthasmanagedtoutilisetheresourceslyinginthemultiorganisationalstructureoftheconsortiumanddevelopedafruitful,reciprocalcollaborationnetworkwiththecommunicationunitsinvolved. With the communication specialists of different disciplines,astrongerandwidercompetencehasbeen in theBCDCcommunity’sreach.Thisinteractionnetworksupportsmanaginginterdisciplinaryandtransepistemicissues.Theperceptionsoftheresearchersandcommuni-cationprofessionalsonprojects’sciencecommunicationarealsobeinginvestigatedinaseparateworkpackage.

The“BCDCEnergy”communicationnetworkfunctionsasameetingpoint for highly esteemed professionals and colleagues. Face-to-facemeetingswithatimelyagendaandinformalinnature,havebeenheldsincethebeginningoftheprojectwithprojectpartnersandthe“KaskasMediaagency”.Thecommunicationunitsdesignanddevelopavarietyof methods for science communication and the communication spe-cialistsprovidesupportandempowertheresearcherstocommunicatetheirresearchinpublicandonline.Thissupportincludeseditorialhelp,repeatedlycheckedroutinesandinformaldiscussionsregardingcommu-nication. InordertoopenupscientificworkandprocessesofcreatingnewknowledgetheBCDCprojectpublishesablogandtweets.Month-lyatleastoneexpertblogispublishedonthe“BCDCEnergy”websiteandsharedviatheproject’scommunicationnetwork.Theparticipationandrepresentationofthe“BCDCEnergy”consortiuminsocialmediaissecuredviaweeklyappearancesinTwitter,wheretheresearchersalsoactivelyfollowotherpeersandstakeholders.Theprincipal investigatorofthe“BCDCEnergy”consortiumleadsbyexampleandregularlypubli-shesupdatesoftheresearchprogressasBCDCstorypostsontheBCDCwebsite.

theotherenergy-relatedSRCconsortia, i.e.,“SmartEnergyTransition”(SET)and“Transitiontoaresourceefficientandclimateneutralelectri-citysystem”(El-Tran),“BCDCEnergy”deliveredajointcommenttothisinitiative.Around-tablediscussionbasedontheargumentsputforwardinthatcommentwasthenorganisedbyFingrid.Inadditiontothis,theenergy-related SRC projects published a policy brief emphasising theneedtomoveintocoalneutralsociety5().

Finally,basedonthemultidisciplinaryresearchbytheBCDCconsor-tium,anovelterminology“CleanEnergyResearch”wascompiledtotheBankofFinnishTerminologyinArtsandScienceshostedbytheUniver-sityofHelsinki (www.tieteentermipankki.fi). In thisway, the resultsoftheprojectcanbeutilisedbywideraudiencesalsoaftertheendoftheproject.Thefactthat“TheInstitutefortheLanguagesofFinland”picked“energiasääennuste”(energyweatherforecast)asanewwordintheirFinnishwordsdatabaseindicatesthattheprojecthasachievedasignifi-cantepistemicoutcomeandanewconceptrelatedtothedisruptionoftheenergymarketsinFinland.Thenewtermandtherelatedsoftwareapplication(seeartefactualdimensionbelow)keepspreading–withoutanyeffortfromtheconsortium–toavarietyofinternetsites(e.g.,http://www.finsolar.net/aurinkoenergia/aurinkoatlas/).

ARTEFACTUAL DIMENSIONS OF ACHIEVING IMPACT IN “BCDC ENERGY”

Akeyingredientinsolvingtheintermittencyproblemisdevelopmentofenergyrelatedforecasts.Thisiswhymeteorologistswereincludedintheconsortiumfromthebeginning.Already in theconsortiumkick-offmeeting inJanuary2016 thisproved tobeagoodchoice.During themeeting dinner, business collaborators discussed with meteorologistsandeconomistsonhowtomakeenergyrelatedweatherforecastsmorepractical.Duringthediscussionsanideaofanapplicationshowingtheforecastofwindandsunshineinenergyunitspoppedup.Thissoundedlikeagoodideaandthedevelopmentstartedimmediately.InJune2016theapplication“EnergyWeatherForecast”6(http://www.bcdcenergia.fi/en/energy-weather/)wasopenedintheconsortiumwebpage.It isanopenaccessapplicationshowing24hourshourlyforecastsforwindandsolarpowerinkWh.Itshowstheseforecastsforall200measurementpointsoftheFinnishMeteorologicalInstituteinFinland.Itisscaledfor2.5kWsolarpanelsandsimilarsizewindmillssothattheyareapplicableevenindetachedsinglehouseholdhouses.Theforecastisupdatedeveryhour.Theapplicationturnedouttobeverypopular.Fromitsopeningithasreachedover12.000visits.ThedevelopmentprocessoftheEnergyWeatherForecasthasalsobeendocumented(Suorsaetal.2018).

INTERACTIONAL-INSTITUTIONAL DIMENSIONS OF ACHIEVING IMPACT IN BCDC ENERGY

The interactional-institutional dimension of productive interactionsincludesformsandforumsofcollaborationbetweenscientistsandothersocietalactors.IntheBCDCconsortium,theinvolvedresearcherswork

5 http://smartenergytransition.fi/fi/policy-brief-hyodynnetaan-energiamurros-ja-luovutaan-fossiilisesta-energiasta/6 http://www.finsolar.net/aurinkoenergia/aurinkoatlas/)

ISSUE 48 | JULY 2019 91

plinary project led by SSH-sciences. It explores how human skills andcapabilitiescanbestrengthenedintherapidlytransformingworldthatisincreasinglydrivenbynewdigitalandtechnologicalsolutions.Furtheritinvolvesambitiousaimstoinitiateco-creationactivitiesinfivefacultieswithintheuniversity,andtofacilitatesocietaloutreachandimpact(seeFigure1).Morespecifically,concreteactivitiestosupportsocietalimpactofSSHsciencesinclude:

• utilising the university’s new research organisation, profilingprojectsandservicestructurealsoforknowledgetransferandsocietalimpact(e.g.asof2017eachfacultyhastheirowndedi-cated communication specialist who helps design and imple-mentinteractionplanstoandwithstakeholders);

• reflectiveandco-creativecommunicativepracticeswithinmulti-andcross-disciplinaryresearchcommunities;

• newinteractivepracticesforresearchgroupsandcommunitiestomakesocialimpact:blogs,encouragingactiveparticipationinthesocialmedia,andresearchstoryposts;

• regular (e.g. annual) graduate school courses, seminars andworkshopsfacilitatedby“Eudaimonia”,“OuluThinkTank”andthe“GenZHub”thatprovideplatformsandforumsforSSHsci-entists and stakeholders to reflect on their practices, identifycommoninterests,shareknowledge,andestablishpossiblecol-laborationacrossdisciplinesandinterestgroups.

WHAT CAN THE UNIVERSITY LEARN? CAPITALISING SUCCESS AND INSTITUTIONALISING GOOD PRACTICES IN THE FUTURE

The interactive models of science’s social impact, together withtheneworganisationalstructures(“EudaimoniaInstitute”;“OuluThinkTank”ofScienceandSociety;“GenZHub”)provideasolidbackgroundtodevelop,spreadandinstitutionalisetheidentifiedgoodpracticesforstrengtheningthesocietaloutreachofSSHresearchattheUniversityofOulu.First,thegoodpracticesidentifiedintheBCDCproject–andotherprojects–willbespreadtotheSSHcommunityinOulu.Second,“Eudai-monia”and“OuluThinkTank”willcollaboratewithotherfocusinstitutesattheuniversitytointroduceSSHresearchonaregularbasisinvariousevents.Third,theidentifiedgoodpracticeswillbeusedtoestablishandstrengthen connections to the broader society to promote and add toimpact of SSH research. Finally, as a brand new endeavour, they willfacilitatetheactivitiesofthenewprofilingeffortoftheuniversity,“GenZ-project”,startinginAugust20187.

“GenZ”–GenerationZandbeyond:Co-evolutionofhumancapabi-litiesandintelligenttechnologiesinthe21stcentury–isacross-disci-

7 http://www.oulu.fi/university/focusarea/understanding-humans-in-change/profiling-areas

Figure 1:TheOuluway–Facilitatingcross-disciplinaryresearchinSSHsciences;respondingtoaglobalchallenge;strengtheningsocietaloutreachandimpact.


AUTHORSJUHA TUUNAINENOulu Business School, University of Oulu

PenttiKaiterankatu1,Oulu,90570(Finland)E:[email protected]

RAULI SVENTOOulu Business School, University of Oulu

PenttiKaiterankatu1,Oulu,90570(Finland)E:[email protected]

PENTTI HADDINGTONFaculty of Humanities, University of OuluPenttiKaiterankatu1,Oulu,90570(Finland)E:[email protected]

KIRSI OJUTKANGAS Research Support Services, University of OuluPenttiKaiterankatu1,Oulu,90570(Finland)E:[email protected]

SIRPA AALTO Faculty of Humanities, University of OuluPenttiKaiterankatu1,Oulu,90570(Finland)E:[email protected]

REFERENCES

Haywood, B. K. and Besley, J. C. (2014). Education, outreach, andinclusiveengagement:Towardsintegratedindicatorsofsuccessfulpro-gramoutcomesinparticipatoryscience.Public Understanding of Science 23/1,92–106.

Huuki, H., Karhinen, S., Kopsakangas-Savolainen, M. and Svento, R. (2017).FlexibleDemandandFlexibleSupplyAsEnablersofVariableEnergyIntegration.https://papers.ssrn.com/abstract=2966053

Kearnes, M. and Wienroth, M.(2011).Toolsofthetrade:UKresearchintermediariesandthepoliticsofimpacts.Minerva49:153–174.

Krishnamurthy, C. K. B., Vesterberg, M., Böök, H., Lindfors, A. V. and Svento, R.(2018).Real-TimePricingRevisited:DemandFlexibilityinthePresenceofMicro-Generation.Energy Policy 123,642-658.

Kuhnlenz, F., Nardelli, P.H.J., Karhinen, S. and Svento, R. (2018).Implementing flexible demand: Real-time price vs. market integration.Energy,149,550-565.

Michael, Mi.(2009).PublicsPerformingPublics:ofPiGs,PiPsandPoli-tics.Public Understanding of Science18/5:617–630.

Miettinen, R., Tuunainen, J. and Esko. T.(2015).Epistemological,Ar-tefactualandInteractional–InstitutionalFoundationsofSocialImpactofAcademicResearch.Minerva – A Review of Science, Learning and Policy 53/3,257–277.

Ruokamo, E., Kopsakangas-Savolainen, M., Meriläinen, T. and Svento, R. (2018). Towards flexible energy demand – preferences for dynamiccontracts,servicesandemissionreductions.Paperpresentedatthe6thWorldCongressofEnvironmentalEconomics,Gothenburg,June2018.

Spaapen, J. and van Drooge, L.(2011).Introducing‘productiveinterac-tions’insocialimpactassessment.Research Evaluation20/3:211–218.

Suorsa, A., Svento, R., Lindfors, A. and Huotari, M.-L.(2018).Know-ledgecreationandinteractioninanRandDproject:thecaseoftheEner-gyWeatherForecast,Mimeo.

ISSUE 48 | JULY 2019 93

II. THE CONTEXT IN WHICH QRIH EMERGED

The report Duurzame Geesteswetenschappen (Sustainable humani-ties) of theCommitteeon the“NationalPlan for the futureof theHu-manities”, also known as the “Committee Cohen” (Committee on the National Plan for the Future of the Humanities 2009),observedthat, intermsofresearchassessment,thehumanitiesaretoomuchatthemercyofmodelsderivedfromtheexactsciencesandmedicine.TheCommitteerecommended that thehumanitiesdevelop itsownsetofassessmentstandards. Intheyearsthat followed,theRoyalNetherlandsAcademyofArtsandSciences(KNAW)tookthistaskuponitselfandtackledthisassignmentviaseveralstudiesthatculminatedinaproposalforanade-quate evaluation system for humanities research in 2012 (Quality and relevance in the Humanities 2012).Thisproposalhasbeenquiteinfluen-tial,asitincludedasoneofthefirstnationalsystemsboththeacademicandthesocietaldimensionofscholarlyactivity.TwoothersectionsintheAcademy,thefieldsofdesignandengineeringandthesocialsciences,developedsimilarvisionsonresearchatthesametime.Thisworkwasalsoreported(KNAW2010andKNAW2013)andtogetherthesefieldsinfluenced to a great extent the model that was adopted in the new“StandardEvaluationProtocol”2015-2021(SEP),launchedin2014.

The SEP 2015-2021 enables a balanced assessment of both theacademicqualityandtherelevancetosociety.Thesetwoarethemaincriteriafortheevaluationbyaninternationalreviewcommittee,whichisbasedonaself-assessmentreport.SEPTableD1(table1)formsthebasis for providing evidence in support of the self-assessment report.Theindicatorsinthistabledeterminethecontentoftheself-assessmentreportandtheinformationgatheredforthereportformsthebasisfortheevaluationbytheassessmentcommittee.

I. INTRODUCTION

Inthispaperwereportthedesignandintroductionofanewsystemfor theassessmentof “QualityandRelevance in theHumanities”(QRiH)intheDutchevaluationcontextandreportalsothefirstex-

periencesofusingthesysteminongoingevaluations.Thedesignofthe“QRiHsystem”isanattempttomeettwochallenges:Thefirstistofindanevaluationmethodthatfitsthewaysinwhichhumanitiesresearcherscommunicatewithscienceandsociety.Inmanyofthecurrentevaluationsystems,with theusualattention tometrics fitting thecharacteristicsof research in “Science, Technology, Engineering and Mathematics”(STEM)fields,theresearchandcommunicationpracticesinthehumani-tiesarehardlyacknowledged.Thesecondchallengeistodealwiththeincreasingpressureonresearchers,inhumanitiesandallotherfields,todemonstratethesocietalrelevanceofresearch,whileatthesametimethereisa lackofconsensusonhowtoassessresearchwithregardtothe societal mandate towards greater attention for knowledge utilisa-tion.Wedescribespecificcharacteristicsofresearchcommunicationinhumanitiesandaddresshowthecommunitiesofresearchersandpolicymakershavebeeninvolvedinabottomupdevelopment.Also,wewillarguethattheformatofthenarrativeforself-evaluationaddressestheabovechallengesandgivesroomtothediversityinthecommunicationoutcomesamongtheresearchunits.Thefirstexperiencesinusingthesystemareencouraging,butdemandsustainedattentionofpanels,re-searchersandpolicymakersinmakingthesystemvalid.

ADPRINS,JACKSPAAPEN,THEDVANLEEUWENANDNELLEKEVANDENBROEK-HONINGHDOI:10.22163/fteval.2019.374

QRIH:TOWARDSAFITTINGSYSTEMFORHUMANITIESRESEARCHEVALUATION1

7 TheauthorsthankFrankvanVree,chairoftheworkinggroup,forhisenergeticandforcefuleffortsindevelopingQRiHandinrealisingtheconditionsfortheworkinggrouptodoso.

Qualitydomains

Research quality Relevance to society

Asse

ssm

ent

dim

ensi

ons Demonstrable products Researchproductsforpeers 4.Researchproductsforsocietaltargetgroups

Demonstrable use of products Useofresearchproductsbypeers 5.Useofresearchproductsbysocietaltargetgroups

Demonstrable marks of recognition Marksofrecognitionfrompeers 6.Marksofrecognitionbysocietaltargetgroups

Table 1.Qualitydomainsandassessmentdimensionsofthe“DutchStandardEvaluationProtocol”(SEP2015–2021).Theindicatorsgiveninthetablearegivenasyetemptycategories,whichcanbeusedinafield-dependentfashion.Eachdisciplinecanbringupindicatorsthatsuitbestproductionandcommunicationpracticesofthefield.


TheKNAWreports,andthesubsequentSEPprotocol,werepublishedinawidercontextofincreasingcriticism–suchasthe“SanFranciscoDeclarationOnResearchAssessment”(DORA),the“LeidenManifesto”,andthe“ScienceinTransitionmovement”–ontheperverseinfluencesofresearchmetricsinthesciencesystem,amongothertheever-incre-asing drive to “publish or perish”. As a consequence of this criticism,productivitywhichusedtobeamainqualitycriterion,nowhasbeenta-kenoutoftheSEP2015-2021,whilesocietalrelevancegainedanequalstatustoscientificqualityintheevaluationmodel.

OneofthemaincharacteristicsoftheSEPisthatitpresentsageneralframework,butwithinthisframeworkleavesroomforthevariousdiscip-linestodevelopasetofcriteriaandindicatorsthatfitbesttheproductionandcommunicationhabitsofthefield.

Inthiscontextasmallworkinggroupwasassignedbythehumani-tiesdeanstodevelopasetofassessmentstandardsspecificallyforthehumanities.iTheworkinggroupworkedalongthreelinesofactivity.First,we researched production and communication practices in the Dutchacademichumanitiesresearch.Second,welookedatwhatwashappe-ning in a few neighbouring countries, Norway and Flanders (“CurrentResearchInformationSysteminNorway”–CRISTiNandVABBdatabasesystems), and the United Kingdom (focusing on the impact pathwaysdevelopedinthe“BritishResearchExcellenceFramework”–REF2014).Third,wedevelopedQRiHinawaythatthiswouldfit intothecurrentnationalevaluationsystemforacademicresearch,SEP2015-2021.Andwedidthisbottom-upthatisweengagedtheresearchandpolicycom-munityinthehumanities.Thethreelinesofactivitywillbeexplainedinmoredetailinthefollowingparagraphs.

III. RESEARCH PRODUCTION AND COMMUNICATION IN THE HUMANITIES

A brief analysis of the production and communication practices oftwolargefacultiesofhumanities(LeidenandAmsterdam)showsawidediversityintypesofoutputanduseoflanguages,andalsoindicatesdif-ficultieswhenrelyingonresourcesthatareoftenusedintheevaluationofSTEMfields,suchasWebofScience.Theresearchinformationsys-temsofthesefacultiesshow–inlinewithotherresearch(VanLeeuwen2013)–thatthelargestportionofresearchoutputisnotinjournalsbutinbookchapters,andlistsalsoawidevarietyofothertypesofresearchoutcomes,suchasbooks,professionalpublications,bookreviews,orpu-blicationsaimedatthegeneralpublic(figure1).

Also, the output in peer reviewed journals, accounting for 16% ofthetotalofresearchoutputofthetwohumanitiesfaculties,appearstoberepresentedonly tosomedegree in journalsmentionedor listed in“WebofScience”(WoS).Theshareof“WebofScience”sourcejournalsandof journalsmentioned inWebofSciencealsovariesaccording tothedomainsofscholarlyresearch.In“EconomicHistory”,abouthalfofthe output in reviewed journals is in WoS journals, whereas in manyotherfields,suchas“CultureStudies”,“ReligionandTheologyStudies”oramongresearchersfrom“Archaeology”theshareofWoSjournalsisbelow20%.(Seefigure2.)

Figure 1.Relativesizesofresearchoutputpertype,LeidenandAmsterdamUniversities.

ISSUE 48 | JULY 2019 95

Figure 2.ShareofoutputinjournalswithWoSsourcestatusormenti-onedinWoSinLeidenandAmsterdamoutput.Totalofpapersinpeerreviewedjournals(100%).

Anotherwell-knowncharacteristicofhumanitiesresearchisthefactthatcommunicationoccursinmanymorelanguagesthanEnglish.Thischaracteristicvariesacross itsvariousdomains.Considerabledifferen-cesoccurinthelanguageofjournalsselectedbypanelsinthefieldofhumanitiesresearchersasprominentpublicationchannels(seealsopa-ragraph IV). For example, the selection for digital humanities consistsalmost completely of journals in the English language oriented, while

“Medievalstudies”showsthatmorethanhalfoftheselectedjournalsisinotherlanguagesorinDutch(seefigure3).

Thesefindingsclearlyindicatethatclassicalbibliometricapproachessuch as citation analysis or WoS status of journals are insufficient tosupportresearchassessmentinthehumanities,whichisconsistentwithotherliterature:asimilarstudyofdifferentpublicationculturesamong“SocialSciencesandHumanities”(SSH)andSTEMresearchers,baseduponbibliometricanalysisof referencebehaviorbyauthorspublishingin journals covered in theWoS, shows the lesser relevanceof journalbasedassessmentinSSHcomparedtoSTEM:inSSH,between10%and40%ofallreferencesareaddressedtojournalliteratureinWoS,whileforSTEMthisamountsupto95%(vanLeeuwen2013).Additionally,takingitfromtheperspectiveofoutputproducedbyawholeuniversity,classicalbibliometricanalysisbaseduponWoSmakestheresearchconductedinmostSSHdepartmentsnearlyinvisible,whiletheinternaloutputregis-trationsystemclearlyshowsthepresenceofawidevarietyofscholar-lycommunication typesbeingpresent (vanLeeuwenetal.2016).Thissituationdisqualifiestheexistingbibliometric toolboxforSSHand lawresearch assessments, as quantitative analysis only deals with a verysmall portionofwhatactuallyhasbeenproduced, acrossa varietyofcommunicationchannels.

OneoftheassumptionsatthestartoftheQRiHprojectwasthatinthehumanities,publicationsforwideraudiencesandforstudentscanberegardedasaveryimportantexpressionofsocietalrelevance,eventothepointthatthelinebetweenacademicandnon-academicpublica-tionsoftenisdifficulttodraw(Sivertsen2016).Theworkinggrouptookthisideafurtherbyproposingthecategoryof“hybrid”publicationasarelevantcategoryforhumanities,definedaspublicationswithscholarlystatusalsoaddressingwideraudiencesofacademicsandnon-academicreaders.

Figure 3.Languageorientationofselectedjournalsin17humanitiesfields.


IV. LEARNING FROM OTHER RESEARCH EVALUATION SYSTEMS

DevelopingtheQRiHsystemalsoincludedreviewingcomparablesys-temsinsomeotherEuropeancountries,suchastheimpactcasestudiesintheBritishREFexercise,thepublicationdatabases“CurrentResearchInformation System in Norway” – CRISTiN (used in Norway) and theBelgian“VlaamsAcademischBibliografischBestandvoordeSocialeenHumaneWetenschappen”–VABB-SHWusedinFlanders.Acomparisonbetweenthecontentofthedatabasesandthewaysinwhichpublicationandotheroutputdataareusedintheallocationoffundsisuseful,eventhough,unliketheSEP,theindicatorsderivedfromtheVABB-SHWandCRISTiN are applied in a performance-based research funding system(PRFS)distributinginstitutionalgrantstotheuniversities(Ossenbloketal.2012).

CRIStiN is the national research information system of Norway. ItdocumentsallscholarlyarticlesbyNorwegianresearchers,andcomple-mentsthelibrarysystemdatabaseBiBSYS,whichfocusesonbooks.TheFlemishVABB-SHWacademicdatabaseisdevelopedspecificallyforthesocialsciencesandthehumanitiesbecauseitisfeltthatthesefieldswerenotadequately represented inWoSdatabase,whichservesas theba-sisforallocatingfundsamongSTEMdomainsinFlanders.Bothsystemscontainseveralthousandsofjournaltitlesanddistinguishbetweenthemin different ways. The Norwegian system makes a difference betweenlevel 2 publications (in international journals) and level 1 publications(other journals,manyof themNorwegian, thatmeetscientificcriteria),andlevel0fornon-scientificpublications.Level1and2areindirectlytiedto financial distribution in the universities. The Flemish system uses asimilardistinctionbetweenWoSjournalsandnonWoSjournals.VABB-SHWisdirectlycoupledtoallocationof“BijzondereOnderzoeksfondsen”(“SpecialResearchFund”–BOF)usedtoreallocatefundingbetweentheuniversities via points given to 5 different types of publications. Booksget4points,articles1.ThepolicycontextofthesesystemsdiffersfromtheDutchpolicycontext theQRiHsystemhas tooperate in.But thesesystemsareworthwhileinvestigatingbecausetheyfacepartlythesameproblemsQRiHfaces.ThemainissueishowtovaluepublicationmediathatarenotpartofWoSorotherinternationaldatabases.

IntheDutchSEPevaluationsystem,acentraliseddatabaseforjour-nalsandpublishersorotherbibliometricindicatorsisabsent.Also,otherthantheBritishREFsystem(Sivertsen2016), theevaluationoutcomesdonot includedirect fundingconsequencesbetween institutions. Theimplicationoffinancialconsequencesofsystemsisthattheinformationis very much focused on competitive elements and comparisons bet-ween groups of researchers which may be sensitive of the indicatorsused(Hammarsfeltetal.2015).AsOssenbloketal.argued,researchersworkingintheFlemishVABB-SHWhavepublishedincreasinglyinWoSjournalstothedetrimentofpublicationsinthelocallanguage,followingits rating system based on the WoS status. By contrast, the share ofpublicationsinNorwegianremainedstable,occurringintheNorwegianCRISTiNsystemsthatincludesastimulustopublishinthelocallanguagenexttopublishinginWoSjournals(Ossenbloketal.2012).

TheFlemishandNorwegiansystemsalsodifferfromthefieldorien-tationofQRiHtoaddressthespecificneedsofthehumanitiesonly,as

Theclaimofthehybridcharacteristicsofhumanitiespublicationscanbesubstantiated. Inaquestionnairesendout topanels in the fieldofhumanitiesresearchers(seealsoparagraphV),thevariousparticipatingpanelsreactedpositivelytotherequesttoidentifyexamplesofsuch“hy-brids”.Severalofthesuggestedworksweresubsequentlyanalysedforreferencesinscholarlyliterature(usingGoogleScholar)andreferencesto be found in non-academic environments using the search enginesGoogle and BING (Prins et al. 2016), demonstrating the actual use inboththescientificandsocietalsphere.

Google Scholar cites

# net societal stakeholders*

AnnemarieMol(2003)TheBodyMultipleDukeUniversityPress

3359 132

JosévanDijck.TheCultureofConnectivity:ACriticalHistoryofSocialMedia.Oxford:OxfordUP,2013.

729 132

JamesC.Kennedy,NieuwBabyloninaanbouw.Nederlandindejarenzestig(Amsterdam1995:Boom)

280 153

ErnstvandeWetering.Rembrandt.ThePainteratWork,AUP,1996.

150 170

TrudyDehue(2008)Dedepressieepidemie,Amsterdam:Augustus

103 215

Table 2.Fivefrequentlyusedhumanitiespublicationsusedbothoninter-netandcitedbyGoogleScholar.*Netsocietalstakeholders:Libraries,repositories,webshopsandotherinternetfindsnot-relevantformeaningfulcommunicationareexcludedfromtheseresults.Alsoexcludedarereferencesfromscholarlyjournals.ii

Use of humanities publications by non-academic stakeholders according to their sector. 100% = total of use by relevant stakeholders.

Theseexamplesillustratetheneedtodevelopspecificindicatorsforhumanities research. The division between the academic and societalsphereisseeminglylessclearorstrictinthehumanities,meaningthatsocietalproductioninthehumanitiesisnotaspinoffderivedfromacade-micproduction,butcanbeanintricateoutcomeofscholarlyproduction.Theexamplesalsoshowthemutualentwinementofacademicandso-cietalproductivityinthehumanities.

ISSUE 48 | JULY 2019 97

theVABB-SHWsystemintendstoinformdecisionmakingforfundinginboth thehumanitiesand the social sciences, and theCRISTiNsystemaddressestheentirenationalfieldofscientificandscholarlyresearch(Si-vertsen2016).However,therearealsosimilaritiesbetweenthetwo:theorganisationofdomainpanelstoincludespecificexpertise,acentralisedsupervisingbodyorauthority,andacentralcollectionofinformationonpublications. QRiH adopts the first two of these similarities betweenCRISTiNandVABB-SHW.

Another important evaluation system is the 2014 British ResearchExcellence Framework (REF UK). In REF, experience has been gainedof what are called impact case studies. These studies focus on theimpact on society and describe, among other things, the project, theparticipants and their share in the project, the nature and scope ofthe impact, and what the project actually yields. The case study re-ports followed a specific structure and were no longer than 5 pages

LookingattheUKREFexercise,wefocusedontheseimpactcasestu-diesthatwereintroducedspecificallytoevaluatethesocietalimpactofresearch.Impactwasdefinedratherbroadlyasresearchhaving“an ef-fect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia”

.TheREFwebsitehasadatabasewithabout7000impactcasestudies,allwritten in themandatory format.These functionedasasupport tospecificallyunderlinethesocietalrelevanceoftheresearchconducted.

Comparison with these other systems led us to believe that someelementscouldbeveryuseful intheDutchsituationwhileotherwereless suitable. For example, the fact that there is no direct connectionbetweenoutputandfundingintheDutchsystemwewereabletorefrainfrom levels for journalsandotherkindsofpublications.Consequently,wewereable todevelop listsof journalsandpublisherwithout levelsandwithoutametricssystem.Atthesametime,wetriedtobeselectivehereandlimitthenumberofentriesintheQRiHdatabase,nowinclu-ding2210uniqueentriesforthemostimportantjournalsandpublishers.Theselectionwasgiventopanels,anelementwetookfromtheFlemishandNorwegiansystems.Furthermore,wedecidedtodevelopindicatorsfor quality and relevance of research bottom up, that is, the researchcommunity – via the research schools – was asked to come up withproposals.

V. BOTTOM UP: THE INVOLVEMENT OF EXPERTISE FROM THE VARIOUS DOMAINS.

Todevelopadetailedviewonthepublicationculturesinthedifferentresearchdomains in thehumanities,we senta request to theboardsof the17 researchschools inwhichDutchhumanities isorganised toassemble panels of researchers – junior as well as senior and promi-nentones -.Over theperiodof twoyears,more than200 researchershaveparticipatedinthesepanels.Wehaveaskedthepanelstoansweraquestionnaireaboutvariousaspectsoftheirpublicationculture,inclu-dingtheimportanceofpeerreview,typicalformsofoutput,andtolistjournalsandbookpublishersrelevantforthevariousaudiencesintheirdomain,suchaspublicationchannelsaimingatspecialties,disciplinaryandmultidisciplinaryaudiencesandalsodewidercontextofhybridau-diences(combiningbothacademicaudiencesandgeneralreadership).

Overall,panelshavereactedpositivelytotherequestbutexpressalsoconcernsthatlistsofjournalsandbookpublishersmightleadtothede-velopmentofametrics-basedsystem.Thequestionaboutoutcomesofresearchandcommunicationtypicalforthescholarlydomainhasledtolistingsofvariousformsofcommunicationusuallyoverlookedinoutputcounting.Thelistsincludecataloguesformuseumexhibitions,filmsanddocumentaries,designsandsoftwareprogrammesandother formsofoutput.

InthevariousstagesofdevelopingtheQRiHsystem,wehaveheldmeetingswithboardmembersandpolicymakersofHumanitiesfacultiesandwiththeboardsoftheresearchschools.Althoughthesemeetingshaveresultedinpositivereactionsabouttheinvolvementofthepanels,concerns remained not only over the possibility that QRiH in the endwouldleadtoametricssystembutalsothattheoutcomesoftheworkofthepanelscouldbetoorestrictiveincasesofmultidisciplinaryschol-arship,orwithrespecttodomainsnotcoveredbythepanels.Thecom-mentsraisedduringthemeetings,andinthenumeroustalksandphoneconversationwithpolicymakersand researcherseventually led to theproposalthatthesystemshouldbebasedonthenarrativeastheleadingformatforself-evaluations.

QRiH in a nutshellThebasicstructureofQRiHistheformatofthenarrativefor

theself-evaluationoftheresearchunit.Thenarrativeshouldaddressboththescientificandthesocietalmissionofthe

researchandbesupportedbyconcreteevidence.Indicatorsinthesixcellsoftable1shouldbeelaboratedinwaysthatfitthehumanities.Theworkinggroupdecidedtopublicisethediffe-

rentelementsonawebsite()andusethewebsiteasaworkinprogress.ResearchersinthehumanitiescanusethewebsiteinSEPevaluations,andatthesametimeshareexperiencesand

dosuggestionsforimprovement.Thisiswhatishappeningrightnowbecausemanyofthehumanitiesfacultiescurrentlyare

goingthroughaSEPevaluation.

Thenarrative,muchincommonwiththeformatdevelopedintheBritishREFsystem,shouldallowtheinstitute,thegrouportheprogrammetoindicatewhatthecoreoftheresearchis,howitshouldpositionitselfandwhichstrategyisbeingpursuedinordertoachievetheobjectivesandsharetheresearchresultswiththeacademicworldandsociety,aswellasthesuccessofthoseresults.Theintentionisthatclaimsofproductivi-ty,useandrecognitionputforwardinthenarrativeshouldbesubstanti-atedwithevidencethatcanbederivedfromindicatorsproposedbythevariousdomainpanelsandauthorisedbyanationalpanel (authorisedindicators)orbyput forwardself-formulated (reasoned)evidencewiththehelpofabroadlistofindicatordefinitions.

AsQRiHistobeusedinthecontextofSEPevaluation,itsformatofthenarrativeimpliesaslightbutimportantalterationoftheSEPformat.TheoriginalSEPformatconsistsof6cells,inwhichthevarioustopicsoftheself-evaluationaretobeelaborated(seetable1).AlthoughtheSEPformatleavesopenwhichkindsofevidenceistobeputforward,thusleavingroomforvariationforthediverseacademicdisciplines,thefor-matcanbereadasmakingacategoricaldistinctionbetweenthespheresofacademicresearchandsociety,distinguishingResearchQualityfromSocietalRelevance.Formanyscholarlyactivitiesinthehumanities,how-ever, this is too restrictive. The narrative of the QRiH aims to address


thisissuebyallowingviathenarrativeformtodemonstratequalityandrelevance both in the academic and societal sense either as separatespheresorcombined.

Thesystemissustainedbyawebsite8()providinginformationaboutauthorisedandreasonedindicatorsforeachcellintheSEPprotocol:the-reareindicatorsforproducts–forpeersandforsocietalparties,indica-torsofusebypeersandinsociety,andindicatorsforrecognitioninbothspheres.Atemplateforthenarrativecanbedownloadedtostructuretheinformationinthenarrativeinsuchwaythatitwillfitthesizeofmaxi-mum15pagesrequiredaccordingtotheSEPprotocol.Andthewebsiteprovidesvarioustools,includingexamplesofhybridpublications,domainprofilesandlistsofjournalsandpublishersselectedbythedomainpa-nelsasexemplaryforthecommunicationamongspecialties,thedomainorinmultidisciplinarycommunication.

VI. INTERMEDIARY REVIEW OF QRIH

In2018mostof theDutchhumanities researchunitsare tobeas-sessedaccordingtotheSEP2015-2021.ThedraftversionofQRiHwasintroduced inDecember2017and theworkinggroupagreedwith thedeansofthehumanitiesfacultiesthatresearchunitstobeassessedweregoingtousetheQRiHasaguidingprinciple.Thisofferedtheopportunitytoinquireafterthefirstuserexperiences.Forthispurpose,wedraftedaquestionnairewithquestionsabouttheusefulnessofQRiHinpreparingtheself-evaluation.Ofthetwentyunitstobeevaluatedin2018-2019,sofar,seventeenhaveactuallypreparedself-evaluations,andpossiblyhaveworkedwiththeQRiHsystem.Fourteenhaveresponded.

Thepreliminaryimpressionsbasedonthe14receivedandcompletedquestionnairesarethatQRiHisoverallappreciatedasatoolthatgiveshumanitiesresearcherstheopportunitytoreportinawaythatisrepre-sentativefortheiractivities,especiallyviathenarrative.QRiHappearstobewidelyknownbydirectorsandpolicymakers;onlyonepolicyofficer(newattheposition)wasnotfamiliarwithQRiH.Mostrespondents(11)indicatedtohaveusedQRiH(moreorlessextensively)whilewritingtheself-assessment report. Two respondents indicated that they had notused QRiH because they had started their self-assessment procedurebeforeQRiHwaspublicised.Mostrespondents indicatedtohaveusedtheformatofanarrative,whichwasreceivedwithenthusiasm. Inge-neral,thesetof(qualitativeandquantitative)indicatorsisexperiencedashelpful.

Somerespondentsindicatethattherearetoomanydifferentgroupsofindicatorsandthatindicatorsforsocietalproductivity,useandreco-gnitionshouldbemorespecified,preferablyalsoinanauthorisedform.Also,asQRiHisdevelopedwiththehelpofdomainpanelsfromtheva-riousresearchdomains, thedistinctionsamongresearchdomainsthatarevisibleonthewebsiteseemtobeconfusingforusers,especiallyforresearchunitswithamoremultidisciplinaryfocus.Italsoappearedthatthedomainprofilesdevelopedasasoftbenchmarking toolby thedo-mainpanels,appearedhardlytobeused,becausetheywerenotseenasrelevant.Arguably,thisrelatestothefactthatmanyresearchunitscoverseveralresearchdomains.

Remarkably, the lists of journals and publishers, organised per re-search domain, in the other systems a guiding element, seem to behardlyusedbyDutchresearchers.Thereasonforthisisnotyetclear.Itcouldbeduetothegroupingofjournalsandpublishersindomains(andresearchunitscoversometimesseveraldomains),butalsobecausenotall people find the listsuser friendly.Another reasoncouldbe that insomedomainsthereisongoingdiscussionaboutthecontentofthelists.

Additionally,inthecontactswithsomedirectorsitappearsthatthedistinctionofQRiHand theSEPprotocol isnot yet clearenough, lea-ding some to revert to the SEP protocol. In part it is argued that theSEPprotocolisanestablishedformat,whileothersalsoindicatethattheinformationsystemsforresearchoutputarealignedtotheSEPprotocol.

Althoughthefirstreactionsingeneralarequitepositive–inparticu-larwithregardtothepossibilitiesofferedbythenarrative-somespecificaspectsofQRiHarehardlyusedorinneedoffurtherelaboration.Togetabettersenseofwhysomeaspectsarehardlyused,in-depthinterviewswillbeheldinthefallof2018.

VII. CONCLUSIONS AND DISCUSSION

TheintentionoftheQRiHsystemistoofferanassessmentsystemspecificallydesignedforthehumanities,adaptedtothespecificcharac-teristicsofthescholarlyworkinthehumanitiesanddevelopedwiththehelpofitsresearchersandpolicymakers.ThesystemworksintheDutchcontextofevaluation,meanttofunctionwithinthebroaderscopeoftheSEPevaluationprotocol.ThisSEPprotocolischaracterisedbyanequalattentionforboththeacademicandsocietalaspectsofproduction,useandrecognitionofresearchandlacksthelinkageoffinancialconsequen-cesthatistypicaltoothersystems.AlthoughtheDutchSEPprovidesaformat flexibleenough forawidevarietyofdisciplines, itsapplicationfor assessing research units in the humanities has lacunae that QRiHintendstomend.AbriefanalysisofthecharacteristicsofproductionandcommunicationofHumanitiesresearchrevealsnotonlythatthetypesofcommunicationarefarmorediversethanjournalarticles,booksandbookchapters,or that thecommunication includesvarious languages:thedistinctionbetweenacademicandsocietalcommunicationisoftennotveryrelevant,leadingtoformsofcommunicationdistinctlydifferentthaninSTEMfieldssuchashybridpublications.Thefactthatweaimatdeveloping a special indicator for hybrid publications met with consi-derableenthusiasmindiscussionswithresearchersattheUniversityofAmsterdam.Therefore,QRiHoffersthepossibilitytoaddressthevariousaspectsofqualityandrelevancebothinthescholarlyandthesocietalspheresinaflexiblenarrativeform.Theformatofthenarrativeissuppor-tedbysetsofauthorisedandreasonedindicatorsincludingalsolistsofprominentchannelsofcommunicationamongspecialists,indisciplinaryandinmultidisciplinarysettings.

Developingand implementing theQRiH systemnot simply the int-roductionofasetof indicatorstobedulyappliedbypolicymakers,re-searchersandcommitteemembers.ThedevelopmentandintroductionofQRiHtook–andstilltakes–placeincomplexsetsofcontextseachposingconstraintsandpossibilities.Thefirstistheexistingstructureand

7 https://www.qrih.nl/en

ISSUE 48 | JULY 2019 99

demandsof theprotocol for theevaluationof research in theNether-lands,theStandardEvaluationProtocol–SEP.Anotherhighlyrelevantcontext are the available evaluation systems and approaches in othercountries,suchastheBritishResearchExcellenceFramework(REF),theFlemish VABB-SHW system, or the Norwegian CRISTiN approach. Yetanother is the organisation of the field in universities, faculties, insti-tutesandresearchschools,andtheinformationavailableateachlevelor organisation to sustain and support a specific research evaluation.A most crucial context consisted of the expectations and anxieties ofresearchersinthefieldandoftheboardmembersoffaculties,institutesandschools.TheintroductionofQRiHandtheideaofanarrativetode-monstratequalityandrelevanceofhumanitiesresearchcouldtakeplacebyaccommodatingtoeachofthese,andbychallengingthesecontexts.

ThedevelopmentofQRiHisalong-termprocess,fortwomainrea-sons.First,bothresearchersandpolicymakersshouldfeelastheownersofthesystem,forwhichweaimedtohaveabottomupprocess.Second,developingthevariouspartsofthesystem,inparticulartheindicators,isademandingendeavor.Fromthereactionsviathequestionnaireandinconversations,itissafetoconcludethatwearehalfwaynow.QRiHhasshownthepossibilitiestodesignasystemfortheevaluationofresearchin the humanities that does justice to the disciplinary diversity of thefield,andtothediversityofitsoutcomesandwaysofcommunication.Itsmaincharacteristic,thepossibilitytodemonstratetheacademicandso-cietalqualityandrelevanceofresearchprogrammesinacomprehensivewayviathenarrative,guidedbyaformatandabroadsetofwelldescri-bedindicators,authorisedorother,iswellreceivedamongresearchers,boardsandpolicymakers.

Wehavereasontoassumethatthecharacteristicofthenarrativehascontributedtoachangeinexpectationsamongresearchers.Atthestartoftheproject,theattempttoformulateindicatorsforqualityandrelevan-cewasmetwithdistrustandanxietyamongsomeresearchers.Inviewoftheabsenceofsharedviewsabouthowresearchinthehumanitiesistobepubliclyaccountable,combinedwiththedisputeoverresearchin-dicatorsinmanycountries,thiswasunderstandable.Thebottomuppro-cessprovedveryvaluable:byexchangingexperiencesandinformationwithresearchersandpolicymakers,andduringthevariousdiscussionsdistrustgavewaytocriticalapprehension,butalsoaraisingsensethatthenewsystemprovidespossibilitiesforthebetter.Needlesstosay,theprocessofdevelopmentandintroductionofQRiHisstillgoingon.Thenextstepswillbetheanalysisofthequestionnairesthatweresentouttoallparticipating researchschoolsand lookat theconsequences forQRiH.Also,thesetofindicatorswillbeelaboratedfurthertostrengthenthesupportingevidenceforthenarrative.

VIII. REFERENCESCommissie Nationaal Plan Toekomst Geesteswetenschappen (Com-missie Cohen). (2009). Duurzame Geesteswetenschappen, AmsterdamUniversityPress,Amsterdam

Committee on the National Plan for the Future of the Humanities(CommitteeCohen).(2009).SustainableHumanities,AmsterdamUniver-sityPress,Amsterdam

Declaration, D. O. R. A. (2012).TheSanFranciscoDeclarationonRe-searchAssessment.Puttingscienceintotheassessmentofresearch.

Dijstelbloem, H., Huisman, F., Miedema, F. and Mijnhardt, W.(2014).Science inTransitionStatusReport:Debate,ProgressandRecommen-dations.

Hicks, D., Wouters, P., Waltman, L., Rijcke, S. D. and Rafols, I.(2015).Bibliometrics:theLeidenManifestoforresearchmetrics.Nature.doi:10.1038/520429a

Koninklijke Nederlandse Akademie van Wetenschappen. (2010).Kwaliteitsbeoordelingindeontwerpendeenconstruerendedisciplines,Amsterdam.[QualityEvaluationindesignandengineeringdisciplines]

Koninklijke Nederlandse Akademie van Wetenschappen(2012).Qua-lityandRelevanceintheHumanities.Towardsanadequatesystemfortheevaluationofresearch.Amsterdam:RoyalNetherlandsAcademyofArtsandSciences.

Koninklijke Nederlandse Akademie van Wetenschappen(2013).Naareen raamwerk voor kwaliteitsbeoordeling van sociaalwetenschappelijkonderzoek,KNAWAmsterdam

Ochsner M., Hug, S. and Galleron, I., (2017).The futureof researchassessment inthehumanities:bottom-upassessmentprocedures.Pal-graveCommunications.3:17020doi:10.1057/palcomms.2017.20.

Ossenblok, T. L., Engels, T. C. and Sivertsen, G.(2012).Therepresen-tationofthesocialsciencesandhumanitiesintheWebofScience–acomparisonofpublicationpatternsandincentivestructuresinFlandersandNorway(2005–9).ResearchEvaluation,21(4),280-290.

Prins, A.A.M. and Spaapen, J. B. (2017).ServingVariegatedAudien-ces:FromRankingOrientedEvaluationtoMisssionOrientedEvaluation,ftevalJournal forResearchandTechnologyPolicyEvaluation,44,Sep-tember2017,pp42-49

Quality and Relevance in the Humanities. (2017).QRiH,aninstrumentfor describing, systematically, quality and relevance in humanities re-searchintheNetherlands.https://www.qrih.nl/en

Dijstelbloem, H., Huisman, F., Miedema, F. and Mijnhardt, W.(2013).ScienceinTransition.Whysciencedoesnotworkasitshould,andwhattodoaboutit,positionpaper.https://scienceintransition.nl/en/

Spaapen, J.B. and Prins, A.(2016).Contextualevaluationofmulti-,in-ter-,andtransdisciplinaryresearch.In:Hubert,B.andMathieuN.etal.(Eds)InterdisciplinaritésentreNaturesetSociétés,PeterLang,2016,p.273-290

Van Leeuwen, T.N. (2013). Bibliometric research evaluations, Web ofScienceandtheSocialSciencesandHumanities:aproblematic relati-onship? Bibliometrie - Praxis und Forschung, 2013. 1-18 (http://www.bibliometrie-pf.de/article/viewFile/173/215)

Van Leeuwen, T.N., van Wijk, E. and Wouters, P.F.(2016).BibliometricanalysisofoutputandimpactbasedonCRISdata:AcasestudyontheregisteredoutputofaDutchuniversity,Scientometrics,106(1),1-16

ISSUE 48 | JULY 2019100

VSNU, KNAW and NWO. (2014).StandardEvaluationProtocol2015–2021,protocolforresearchassessmentsinTheNetherlands.Voorburg:VSNU,KNAW&NWO.

AUTHORSAD PRINSSupport in ResearchManagement

Oosterkade8b,Groningen,9711RS(TheNetherlands)E:[email protected]

JACK SPAAPENRoyal Netherlands Academy of Arts and Sciences,Kloveniersburgwal29,Amsterdam,1011JV(TheNetherlands)E:[email protected]

THED VAN LEEUWENCentre for Science and Technology Studies Leiden UniversityP.O.Box905,Leiden,2300AX(TheNetherlands)E:[email protected]

NELLEKE VAN DEN BROEK-HONINGHRathenau InstituteAnnavanSaksenlaan51,TheHague,2593HW(TheNetherlands)E:[email protected]

Endnotesi Theworkinggroupwaschairedbyprof.dr.FrankvanVree(theUvA,nowdirectorofNIOD-KNAW)andconsistedofThedvanLeeuwen,AdPrins,Jack

Spaapen,MartinBoeckhoutandDavidDuindam.TheKIGAcademicAdvisoryCommitteeconsistedofProf.RemiegAerts(RU,UvA),EmeritusProf.GeertBooij(Leiden),Prof.MarthaFrederiks(UU,2014-2015),Prof.MaryKemperink(RUG),Prof.AnthonieMeijersTU/e,andProf.FrankvanVree(chair;UvA,NIOD).NellekevandenBroek-Honinghismemberofthepresentworkinggroup.

ii ThenumbersdifferfromthoseinPrins2016,afterclassificationsofmorewebsites

ISSUE 48 | JULY 2019 101

synergies,butthecoreofeveryprojectmustlieinSSH,whicharelistedundersections5and6inthe“FieldsofResearchandDevelopmentclas-sification”(FORDclassification)(OECD,2015).TheprinciplesofappliedresearchanddevelopmentarepromotedinthesensedescribedintheFrascatimanual(OECD,2015),whichsaystheresearchsolutionmustbepracticeoriented,novel,creative,uncertain,systematicandreproducib-le/transferable.TheETAprogrammeisimplementedundertheAct(Act,2002), with regard to the Regulation (GBER) (Commission Regulation,2014)andthe“StateAidFramework”(Framework,2014).Thefundingisnotprovidedinthedeminimismode.

Item Value

Programmeduration 2018-2023,5callsforproposals

Totalexpenditure 111millionEUR

Publicaid(statebudget) 92millionEUR

Expectedaverage/max.amountofaidperproject 190thousandEUR/3millionEUR

Fundingintensityratesoftheprogramme/perproject

upto80%/upto80%

Originofco-financing privateandotherpublicresources

Overheadswith/withoutHRExcellenceinResearchAward

upto30%/upto20%

Min./expectedaverage/max.durationoftheproject 12months/36months/48months

Table 1.GeneraltermsandconditionsReference:ETAProgramme,2017.TechnologyAgencyoftheCzechRe-public

EligibleapplicantsforfundingmusthavearegisteredofficeintheEU,theEuropeanEconomicAreaortheSwissConfederationandfittothedefinitionofthefollowingentities:

• Research and knowledge dissemination organisations.Theresearchorganisationcanbesupportedupto100%oftheireli-

ABSTRACT

Thisarticlereferstotheimplementationofthe“ETAProgrammefor Applied Research, Experimental Development and Innova-tion in Art, Social Sciences and Humanities”. The programme

addresses dynamic social, economic, globalisation-related, cultural ortechnologicalchangesof the21st centurywithallocationof92millionEURofstateaidfor6years,until2023.TheETAprogrammeintroducestheso-calledapplicationguarantor,whichshouldbothincreaseapplica-bilityoftheresearchresultsofSSHandbroadenthespectrumofR&Dsolution users. It is also aimed at supporting the so-called innovationecosystemofSSHconsistingofinterdisciplinarycollaboration,combina-tionoftechnicalandnon-technicalresearchcontentandusageofbasicresearchdiscoveriesofSSHforapplication.SeveralsupportedresearchprojectswillbementionedaswellaspointsfortheongoingdiscussiononhowtoexploittheinnovationpotentialofSSH.

INTRODUCTION THE ETA PROGRAMME

R&D Programme “ETA – Programme for Applied Research, Experi-mentalDevelopmentandInnovationinArt,SocialSciencesandHumani-ties”(ETAprogramme,2017)wasdevelopedandisimplementedbythe“TechnologyAgencyof theCzechRepublic” (hereinafterTACR).Basedonfindingsoftheevaluationactivitiescarriedoutin2014-2017,thefinalversionoftheprogrammewasadoptedbytheGovernmentoftheCzechRepublicinJanuary2017.Atpresent(October2018),theimplementationoftheETAprogrammeisinthemidofits2ndcallforproposals(seealsoTable1.“Generaltermsandconditions”).Theprogrammesupportssocialsciences,humanitiesandart(hereinafterSSH)toaddressthedynamicsocial, economic, globalisation-related, cultural or technological chan-ges,whichthehumanandsocietyarecurrentlyfacing.Othernon-SSHfieldsarealsowelcomedintheprogrammemainlyfor interdisciplinary

MARCELKRAUSDOI:10.22163/fteval.2019.375

ETAPROGRAMMEARTS,SOCIALSCIENCESANDHUMANITIESASANINTEGRALPARTOFTHEINNOVATIONECOSYSTEMOFTHE21STCENTURY

ISSUE 48 | JULY 2019102

gibleexpenditureonR&Dactivitywithintherespectiveproject.The co-financing can be ensured from private or other publicsources.

• Enterprises.Themaximumoftheallowablefundingintensityissetupwiththerespectoftheirsizeandfinancialperformanceaccording to the Regulation (Commission Regulation, 2014).Companies carrying out the project alone or in collaborationwith other participants must demonstrate the ability to co-financetheprojectonlyfromprivatesources.

• Other natural and legal persons.Forthe1stand2ndcallforproposals,theothernaturalandlegalpersonsarelocalauthori-tiesorlegalentitiesinwhichlocalauthoritiestakepartintheroleoffoundersormembers.Thesearemainlymunicipalities, city quarters, regions, microregions or local action groups(hereinafterlocalauthorities).Theseentitiesareconsideredaseligible for funding only if at least one research organisationorenterprise isamongtheprojectapplicants.Theintensityofsupportisbasedontheschemeapplicabletoenterprises,themaximumreachesupto80%oftheireligibleexpenditures.Therestcanbeaddedfrompublicorprivatesources.

MISSION, VISION, OBJECTIVES AND CHALLENGES OF THE 21st CENTURY

Mission: Themissionof theprogram is to support theapplicationcultureofacademicstaffandotherprofessionalsfromSSHfields(R&Dsolutionproviders)andtostimulate interest inexploitationof theirso-lutionsbySSHapplicationsphere (R&Dsolutionusers, suchasminis-tries, municipalities, health, social or cultural organisations, schools,universities, churches, researchorganisations,enterprises,NGOsetc.).Vision:ThevisionoftheprogrammeistoencourageresearchcreativityofSSHcommunity,whereSSHandnon-SSHscientificfieldsarelinkedwitheachotherandconnectedwithR&Dsolutionsusersand/ortargetgroupstosuchanextent,thatSSHbecomesafullyintegratedpartoftheinnovationecosystem.Objectives:Theobjectiveoftheprogrammeistosupporttheinvolvementofart,socialscienceandhumanitiesinappliedresearch, experimental development and innovation projects and useof their researchoutputs in theformofneworsubstantially improvedexistingproducts,procedures,processesorservicesinpractice.Challen-gesofthe21stcentury:Allprojectsshallbeaimedatmitigatingthreatsandexploitingopportunitiesinthecontextofthecurrentandthefuturechallengesofthe21stcentury.Suchchallengesaffectthedynamictrans-formationsofcontemporarysociety,intheareasof:

a.Human and society in the context of dynamic social andtechnological transformations and challenges of the 21st

century:(1)theprinciplesoftheFourthIndustrialRevolution;(2)digitisation, virtual realityandartificial intelligence; (3)mediaand social networks; (4) social services, social work, socialhousingandsocialinclusion;(5)familypolicy;(6)demographicchange–agingandfragmentationofsociety;(7)socialinsuranceschemes;(8)migrationandintegration;(9)equalopportunitiesformenandwomenandprinciplesofnon-discrimination;(10)health,psychosocialdevelopmentandspirituality;

b.Human and the environment for his / her lifeinthecontextofsustainabledevelopmentofthelandscape,regions,townsandmunicipalitiesand thebuildingculture: (11)globalisationand

regionalisation; (12) architecture, urbanism and living space;(13)sustainabilityandtheenvironment;(14)physicalandvirtuallinking;

c. Human and the economy in the context of discovering newcompetitiveadvantagesandcompetencedevelopmentfor the21st century: (15) educational challenges; (16) employment;(17) health and safety at work; (18) sustainable growth andnewcompetitiveadvantages;(19)innovativeculture,acreativeecosystem; (20) design, design thinking and innovation; (21)newstrategicnon-materialresources;(22)digitalandcreativeeconomics;(23)mediaandtechnology;(24)businesscreation,business culture and business ethics; (25) clustering andstrategicnetworking;

d.Human and the social system in the context of interactionbetweenthecitizenandthestate,publicpolicies,governanceandcitizen-orientedpublicservices.(26)citizenparticipationingovernmentandcommunity life; (27)protectionof intellectualpropertyrights,openinnovation,bigdata;(28)strategicsupportfor research, development and innovation; (29) responsibleresearch, development and innovation and corporate socialresponsibility;(30)creationandevaluationofpublicpoliciesandinterventions;(31)citizen-orientedpublicservices.

Eachprojectmustbefocusedonatleast1ofthe31so-calledchallen-gesandopportunitiesofthe21stcentury.

INNOVATION ECOSYSTEM OF SSH

Inordertofostersustainabilityoftheintervention,theETAprogram-me also aims at supporting the system in which applied R&D in SSHtakesplace.Itbelongstocrosscuttingcurrentandfuturechallengesofthe21stcenturynotonlyforthehumanandsociety,butalsofortheSSHas such. Three aspects of the so-called innovation ecosystem of SSHhavebeenidentified:

1. Interdisciplinarity–breakingdownthebarriersbetweendis-ciplines.Supportofthisaspectshouldresultinahigherperme-abilityofdifferentknowledgeofSSHandnon-SSH fieldsandin an increased synergy effect of their innovation potential.Challengesandopportunitiesofthe21stcenturyaresocomplexthattheirsolutionsoftenliebeyondtheboundariesofvariousscientific disciplines. In addition, the innovation potential ofsomefieldsofSSHcanbebetterexploitedinconjunctionwithotherdisciplines.Thus,thisaspectoftheinnovationecosystemofSSHpromotestheconvergenceofknowledgeinbetweenofSSHorbetweenSSHandtechnical,lifeornaturalsciencestoacquirenewknowledgeandwaysforapplications.

2. Responsibility – producing more responsible research out-comesandinnovation.Supportofthisaspectshouldminimisethe negative undesirable effects of innovation on humans orcertain social groups and strengthen the fair distribution ofbenefitsarisingfromuseof theR&Dsolutions inpractice. In-novations–whethertheyareproducts,procedures,processesorservices–shouldbedevelopedwithregardtopossiblesideeffectstheymayhaveonothergroupsofthepopulation.With-outsufficientreflectionoftheirnon-technicalaspects,researchoutputs for somesocialgroupsmaybepotentiallydangerousor exclude them from use. In addition, linking technical and

ISSUE 48 | JULY 2019 103

non-technicalresearchcontentwillenableSSHtousetechnol-ogysolutionstodelivertheirinnovativepotentialtosociety.Inevaluating this aspect, it is necessary to assess whether theproject proposal respects the value of social justice and ben-efitsfordifferenttargetgroups.Projectsthatpracticallyaddresstheoverlookeddimensionofsocialresponsibility(e.g.integrateknowledgeofage,ethnicity,sex,orgender)intheresearchcon-tentarealsowelcomed.

3. Connectivity – utilisation of innovative potential of the SSHknowledgeanddiscoveries.Supportofthisaspectshallhelptobuildabridgebetweenbasicandappliedresearch.Theinnova-tivepotentialofdiscoveriesandknowledgeofSSHforsocietyoftenremainslatent.Certainoutcomesofbasicresearcharenotusuallyusedinpracticeinrespectivesocialareas.Thisaspectwillsupportprojectsthatbuilduptheirpracticalresearchsolu-tiononanexistingknowledgefrombasicresearch.Duringtheevaluationprocess, it isnecessary toassesswhether theuseofspecificknowledgeordiscoveriesforapplicationsisjustifiedandfeasible.

Eachprojectmustbebasedonatleastoneaspectoftheso-calledinnovationecosystemofSSH.

PROGRAMME LOGIC MODEL

Theprogrammelogicmodel isbasedonthe“EvaluationReferenceModel”for“TAFTIE’s1Taskforce”(TechnolopisGroup,2014),whichcon-sistsoffourcomponents:Inputs:Toreachthemission,achievethevisonandfulfiltheobjectives,theGovernmentoftheCzechRepublichasallo-cated92millionEURwhichrepresentupto80%oftotalexpendituresoftheprogramme.Theother20%shallcomefromotherpublicorprivatere-sources.Outputs:Researchwork–resultofthefundingwillbemeasuredbye.g.numberofsupportedprojects,formofcollaboration,involvementoforganisationsinappliedR&Dactivitiesornumberandtypeofresearchresultssuchascomprehensiveresearchreports;certifiedmethodologies,proceduresandspecialisedmaps;audiovisualworks;organisationofaconference, workshop or exhibition; scientific publications; dictionari-es,textbooks,teachingmethodsandtools,psychodiagnosticmethods,mappingandplanningstudies,evaluationandimpactstudies,software;data structures and files, hardware prototypes, game simulations andsimulators,ICTapplications,patent;prototype;functionalsample;busi-nesscreation(start-ups,spin-offs)etc.Outcomes:Theimmediatebene-fitsforbeneficiariesorpartnersofthesupportedprojectsareexpectednotonlyinformofinnovationcomingfromusageofresearchresultsinpractice(innovationofproducts,procedures,processesorservices),butalsoinformofstrongerinnovationecosystemofR&Dsolutionproviders:interdisciplinary collaboration; combining technical and non-technicalresearch content in one R&D project and more intensive exploitationofoutputsfrombasicresearchforapplications.Impact:IftheproducedoutcomesaremadewithinthesustainableinnovationecosystemoftheSSHandusedinday-to-daypracticeoftheR&Dsolutionusers,thenthepositiveimpactoftheinnovationpotentialoftheSSHwillbeachievedaswellasnewqualityoflifeofhumanandsociety:

a) Impact on the human and society: qualityofhuman life is im-proved; sustainable environment for human life is supported;competitivenessoftheCzechRepublicisimproved;efficiencyandqualityofpublicpolicies,publicadministrationandpublicservicesisincreased.

b) Impact on the SSH innovation ecosystem:boundariesbetweenscientificareasarepermeable,researchoutcomesandinnovationaremadeontheresponsibleway;innovativepotentialoftheSSHdiscoveriesispracticallyused.

SPECIFIC TERMS AND CONDITIONS APPLICATION GUARANTOR

OneofthemainchallengesoftheETAprogrammeistochangetheway of thinking, that the applied R&D only takes place between aca-demia and businesses. The ETA programme stresses the relationshipbetween“R&Dsolutionproviders”and“R&Dsolutionusers”inordertoadvancetheexistingsupportstructures.InthecaseofSSHapplicationspherethenaturalusersoftheR&Dsolutionsmaynotonlybeenterpri-sesbutanyentityinpublicspace.Therefore,theETAprogrammeintro-ducestheso-calledapplicationguarantor,whichrepresentstheusersofR&Dsolutionsintheproject.Themaintaskoftheapplicationguarantoristocontributetomakingtheoutcomesoftheprojectfitforuseinpracti-cethroughverifyingtheirreliabilityandusability.Inaddition,applicationguarantorcanplayanimportantroleinthedevelopmentofparticipativeresearchmethodsthroughitsproximitytotheprojecttargetgroup.Therelevantapplicationguarantor isanentity thatcanuse themainR&Doutputsforitspracticeandthusfulfiltheprojectaim.Nevertheless,notalloftheseentitiesmaybeeligibleapplicantsforstateaidforresearch(accordingtoActAct,2017),Regulation(CommissionRegulation,2014)and“StateAidFramework”(Framework,2014).Hence,theETAprogram-medistinguishesbetweentwokindsofapplicationguarantors(herein-afterAG):

a. Internal AG – the entities performing the role of AG can befinancially supported if they belong to the eligible applicants(researchorganisations,enterprisesorlocalauthorities).Inordernottobreakthelegislationrelatedtotherulesofstateaidforresearch,whentheAGisanenterprise,itmustbecomeanapplicantoftheprojectandthereforealwaysactasaninternalAG.

b.External AG – if AG does not belong among the eligibleapplicants,itactsasanexternalAGintheproject(e.g.ministries,public authorities, health or social organisations, schools,cultural organisations, Non-Governmental Organisations(NGOs)etc.).EligiblecostsoftheexternalAGcannotbecoveredfromtheprogrammeresources.Yet,therepresentativesoftheexternalAGmightbeemployedbytheapplicantsfortheR&Dprojectpurposes.EntitiesintheroleofAGintheprojectmusthavetheirregisteredofficeintheCzechRepublic.

EachprojectmusthaveatleastonerelevantAGforthemainresearchoutputs,regardlessifit´sinternalorexternal.

1 TheEuropeanNetworkofInnovationAgencies(TAFTIE)

ISSUE 48 | JULY 2019104

TARGETED INVESTIGATORS AND EQUAL OPPORTUNITY FOR CAREER IN RESEARCH

TheETAprogrammeseekstopromotetheprinciplesofequality,di-versity,responsibilityorsocialjusticenotonlyviathescaleofactivitiesandresearchresultsofselectedresearchprojects,butalsointhewaythe projects are organised and conducted. The research teams of theETA programme may consist of scientists and researchers, universityteachers, doctoral students, post-docs, artists, designers, architects,employeesofmunicipalitiesor staffofexternalapplicationguarantorsandotherpractitionersorexperts(seealsothefigurenumber1).Thus,wewould like toencourageapplicants topaygreaterattention to thebenefitsofdiverseresearchteams,andalsotoequalopportunitiesformenandwomenforthedevelopmentoftheirresearchcareers.Forthosereasons,severalrulesandrecommendationhavebeendeveloped,pub-lishedandusedforpromotion,evaluationandrealisationoftheprojects:e.g.genderdiverseteamisconsideredpositive inevaluation;researchreferencesoftheteammembershavetofittotheprojectaims,butwedonotputanylimitationintermsoftimewhentheresultwasachieved– in order not to disadvantage those, who experienced some careerbreak(maternityleave,parentalleave,illnessetc.);orhigherflatrateforindirectcosts(from20%upto30%)–whichwerecommendtospendonactivitiesaimingatthework-lifebalanceoftheteammembers–butforthoseonlywhoare“HRExcellenceinresearchAward”holders.

MARKET-ORIENTED AND PUBLIC-ORIENTED RESEARCH PROJECTS

SSHcanbeusefulforthesocietybycreatingaddedvalueofmarket-orientedproductsorservicesbygivingtheme.g.anelementofrespon-sibilityorsocialjustice.HowevercertainSSHresearchsolutionscannot,or even should not be delivered to the benefit of human and societythrough market mechanisms. Thus, the ETA programme supports re-searchsolutionsboth,market-oriented(sellonthemarkettocostumers)orpublic-oriented(providedfreeofchargetothetargetgroups).Whileitisoftendifficulttoseparatethesetwotypesofprojects,asmanyprojectscontainbothcomponentsatonce,theprojectproposalsmustoptforthepredominantcomponent.Nopriorityisgiventooneofthesetwotypesofprojects,themarket-orientedandpublic-orientedprojectsaretreatedduringtheevaluationprocedureinthesameway.

Figure 1:Projectlogicmodel.

ISSUE 48 | JULY 2019 105

Public-relevantprojectsdominate.Themostfrequentmainscientificfo-cusofthesupportedprojectsis“ManagementandAdministration”(23projects),“SociologyandDemography”(12projects)and“Urban,Regio-nalandTransportPlanning”(11projects)(TACR,2018).Theavailableallo-cationforthe1stcallforproposalsallowedtosupportallprojects,whichhavebeenevaluatedpositively.Eventhoughtheirdistributionamongthesphereofart, social sciences,andhumanities isnon-proportional (themajorityoffundedprojectbelongrathertosocialsciences,minoritytohumanitiesandonlyafractionseemstobelongtothefieldsofart),threeexamplesofeachspherewerecollected(table2).

RESPONSE AND DISCUSSIONEXAMPLES OF FUNDED PROJECTS

Inthe1stcallforproposals,306projectweresubmittedand94pro-jectswerefunded(successrate32,4%).Abudgetof18millionEURwasspent,anaverageof193.000EURperproject. Thesupportedprojectsinvolved180applicationguaranties(withduplicities)suchasministries,charities,elementary schools, small andmedium-sizedcities,enterpri-ses,museums,philharmonics,regions,umbrellaorganisationsetc.,mostofwhichareexternalAG.Themajorityofapplicantsareresearchorgani-sationsincl.universities(150participations)andsmallenterprises(22).

Scope Name of the project Application guarantor(s) Total costs

ART

Audiencevalue Customerlifetimevalueintheenvironmentofculturalinstitutionsofliveart

PhilharmonicorchestraHradecKrálové;Collegium1704;Novofest

184.000

Bigdataandartisticresearch

Decentralisedcollection,analysis,visualisationandinterpretationoflargedatainanartisticpractice

FacultyofFineArts,BrnoUniversityofTechnology

84.000

Designofsmartfurniture DevelopmentofasmartfurnitureprototypeforthenewpermanentdesigncollectionoftheMuseumofDecorativeArtsinPrague

mmcité1a.s. 144.000

SOCI

ALSC

IEN

CES

Liveablecitiesandcommunities

Guidelinesforplanningofpublicspaceindigitalera CentralBohemianInnovationCenter 153.000

Earthprotectionfromasteroids

Amultidisciplinaryanalysisofplanetarydefencefromasteroidsasthekeynationalpolicy

MinistryofTransport 392.000

Environmentaleducation Solarenergy,waterinthecountryside,vegetation:anewmethodologyoftrainingmunicipalitiesandschools

Cities:DačiceandTčeboč;GymnasiumJírovcovaandJ.V.Jirsíka;NerudovaElementarySchool

139.000

HU

MAN

ITIE

S

Ethicsandautonomousmobility

Ethicsofautonomousvehicles Prototypums.r.o.;KeenSoftwareHousea.s.;MinistryofTransport

173.000

Industry4.0andsocialchange

Developmentoftheframeworksforasocialchangeintherealityoftheindustrytransformation

ConfederationofIndustryoftheCzechRepublic 253.000

Historicalliteracy Historylab:usingtechnologytofosterhistoricalliteracy–softwareforhistoryeducation

Antikomplex(NGO) 293.000

Table 2.ExamplesofsupportedR&Dprojectsfromthe1stcallforproposals(totalcostinEUR).Reference:Resultsexamplesoffundedprojectsin2018,TechnologyAgencyoftheCzechRepublic

DISCUSSIONBasedonexperiencewiththedesigningoftheETAprogramme,with

theimplementationofthe1stand2ndcallsforproposalsandfeedbacks,takingintoaccounttypesofsubmittedprojectproposalsandtheirmostfrequentweakpointsandqualities– the followingareasremainchal-lenging:

• Structural level: Institutionalisation of drawing on innovationpotentialofSSH

• Political level: Blindness of legislation and R&D policy to theneedsofSSHforinnovation

• Academiclevel:Innovativemind-setofSSHcommunityThemostfrequentreasonsforprojectproposalsrejection,isthelack

onSSHinthecoreoftheresearchproject.Theremaybeseveralreasons

forthis:ThecommonapplicantsoftheTACR’sprogrammeportfoliomain-lycomefromthe“Science,Technology,EngineeringandMathematics”(STEM) fields or natural sciences. The SSH academic community hasbeenhistoricallysupportedprimarilyintheareaofbasicresearch.Mo-reover,thesystemofknowledgetransferbetweenSSHandapplicationguarantors(municipalities,schools,culturalorganisations,NGOs,publicadministration,enterprisesetc.)isstillemerging.Eventhoughthethirdroleofuniversitiesisrecognisedasanintegralpartoftheirmission,theassessmentand institutional financingofSSHarebased substantiallyontheirpublicationperformance,notapplicationoftheirresults.Thus,thedemand-orientedresearchattentionoftheSSHcommunitymightbemorestimulated.Viceversa,thedemandontheR&Dsolutionsideoftheapplicationguarantorsisratherlow,astheymightnothaveenoughca-

ISSUE 48 | JULY 2019106

waythatbetterreflectsthespecificnatureofSSH,andstructuralexploi-tationoftheinnovationpotentialofSSHforsocietyofthe21stcentury.

REFERENCESAct (2002).ActNo.130/2002ontheSupportofResearch,ExperimentalDevelopmentandInnovationfromPublicFundsandontheAmendmentofCertainRelatedActs.

Commission Regulation(2014).No651/2014of17thJune2014decla-ringcertaincategoriesofaidcompatiblewiththeinternalmarketinac-cordancewithArticles107and108oftheTreaty-OfficialJournaloftheEuropeanUnionL187,26thJune2014,inparticularArticles25,28and29.ETA programme(2017).FundingprogrammeforAppliedResearch,Ex-perimentalDevelopmentandInnovationinSocialSciencesandHumani-ties,TechnologyAgencyoftheCzechRepublic,January2017.

Framework(2014).FrameworkforStateAidforResearch,DevelopmentandInnovation-OfficialJournaloftheEuropeanUnionC198,2014.

Government Office of the Czech Republic (2017).Definitionofthere-sults,SeparateAnnexNo.4Methodologyofevaluationofresearchor-ganizationsandprogramsoftargetedsupportofresearch,developmentand innovation, Ref. No. 26822/2017-OMP. Approved by Resolution oftheGovernmentoftheCzechRepublicon29thNovember2017No.837.

OECD (2015):FrascatiManual2015.Guidelines forCollectingandRe-portingDataonResearchandExperimentalDevelopment,TheMeasu-rementofScientific,TechnologicalandInnovationActivities,OECDPu-blishing,Paris.

Technopolis Group (2014). Evaluation Reference Model For TAFTIE’sTaskforceBenchmarkingImpact,EffectivenessandEfficiencyofInnova-tionInstruments.Amsterdam.

TACR(2018).TechnologyAgencyoftheCzechRepublic,#TAčRvDatech,ProjectProEval, EUOperationalProgrammeEmployment.Retrievedon7thofOctober,2018from:https://visual.tacr.cz/eta_en.php

AUTHORMARCEL KRAUSDepartment of Strategy and Analysis, Technology Agency of the Czech Republic

Evropská36,Praha616000(CzechRepublic)E:[email protected]

KEYWORDSart,socialsciences,humanities,appliedresearch,innovation,R&Dpro-gramme

pacity–whetherfinancial,timeorhumanresourcesrelated–toinnova-te.ItseemsthatthehigherimpactofSSHonhumanandsocietyhindersthelowinstitutionalisationoftherelationshipbetweenR&Dsolutionpro-vidersandR&Dsolutionusers,andweshouldaskwhatdoesitprevent?

Notallpartnershipscanbefullydeveloped,sinceonlyresearchorga-nisationsorbusinessescanreceivethestateaidforresearch.Manyim-portantpartnershavetoplaytheroleofexternalAGs(withoutfinancialsupport),althoughitwouldbemoreappropriateforthemtoactivelypar-ticipateonresearchactivities.StateaidrulesdonothavetobeappliedtomanyresearchprojectsofSSH,howeveraclearmethodologytorecog-nisewhichprojecthastoandwhichdoesnothavetoispoorlyavailableforSSH.Compliancewiththeintensityofsupportapplicabletobusines-sesandtheneedtoco-financefromprivateresourcesisoftenunrealisticformanyR&DsolutionusersofSSH.Deminimismodeofsupportmightbetoodangerousforthem.NeithernationalnorEuropeanlegislationofthestateaidforresearchisfriendlytothesenewtypesofpartnerships.Furthermore,consideringthefieldsofarts,theETAprogrammecreatesanincentiveforanexpandedoutlookattheartisticresearch:focusoninnovation.ButtheartisticresearchhasnotbeenfullyrecognisedyetasanintegralpartoftheR&Dpolicy,muchlessasapartofinnovationpolicy–neitheronnational,noronEuropeanlevel.Whatisthereasonforthisomission?

Thesecondmostfrequentreasonfortherejectionoftheprojectpro-posal,isthelackofnoveltyandinnovativenessoftheR&Dsolutions.Itisnotclearwhatnoveltyororiginalitymeansintermsofapplied-orientedSSHresearch:e.g.whethertheprojectaimstomovethecurrentpracticeforward,offersanovelandoriginalR&Dsolutionthathasnotbeenusedinpracticeyetorifitintroducesexistingconceptsintoanotherenviron-ment or context? It seems to be difficult to understand, how to buildupabackground,onwhichtheoriginalityofthenew(orsubstantiallyimprovedexisting)creativeR&Dsolutionswillbevisibleandthusbetterassessable.Thewillingnesstotakerisk,whichconsiste.g.ofapreviouslyuntestedinterdisciplinarypartnershipornewresearchmethodsortheircombining,isstilllow.Manyrejectedprojectproposalshaveremainedinthecurrentbordersofthetraditionalresearchparadigms,whichnega-tivelyaffectedtheirinnovativeness.Isitpossiblethatthereasonforthisdeficiencyliesespeciallyinthetwopreviousquestions?

CONCLUSIONS TheETAprogrammeisanewtoolforsupportofinnovationecosystem

throughscaling-upSSHpathwaysinordertoboosttheirpositiveimpactonthehumanandsociety.Itsnoveltylies–atleastwithinthenationalcontext–inthefactthat,throughtheroleofanapplicationguarantor,itencouragestheSSHcommunitytofindpartnerswhocanusetheirR&Dsolutions.Andviceversa,theinterestonthesideoftheR&DsolutionsusersforcooperationwithactorsofSSHisstimulated.Eventhoughtheprogrammeisatthestartofitsimplementation,firstexperiencesshowthatitcreatesanappropriatetoolinharnessingtheinnovationpotentialofSSH.However, there isaneed todeepen the relationshipbetweentheSSHcommunityandthefundingorganisationandtoconstantlyre-flect the way the ETA programme is implemented. It should be takenintoaccountthatnotonlyorganisations,butalsomembersofresearchteamsmightbefirsttimeapplicantstotheTACRprogrammeportfolio.AnimportantpartofthesuccessoftheprogrammeisboththeparalleladjustmentoflegislationandpoliciesonnationalandEUlevelsinsucha

ISSUE 48 | JULY 2019 107

disciplinary, (3) intersectoral, (4) innovative, (5) impactful and(6)inclusive.

2. Secondly, toanalysekeyaspectsof thepractical implementa-tionofthemodelatahighereducationinstitution:inthiscasetheUniversityofDeusto2.Byanalysingprocessindicatorsandoutcomes,thispaperfocusesona.the evolution of the implementation of the “6i Research

Model”overthelastdecadeandhowithasbeensustainedinpractice;

b.theresultsproduced;andc. thechangeswhichtheinstitutionhasundergonetoaccom-

modateandsupporttheevolvingmodel.Focusingontheimplementationofthe“6iResearchModel”model

attheUniversityofDeusto,thesecondpartwillrespondtothefollowingresearchquestions:

1. Howdidthe“6iResearchModel”evolveovertimeandhowhasitbeensustained?

2. Whatkindofimpactoninstitutionalchangedidthemodelin-volveintermsofstructuresandresources,mechanisms,initia-tivesandoutputs?and

3. Is Deusto steadily evolving into a research ecosystem for im-pactful research excellence, while adopting the “6i ResearchModel”?

Basedonlessonslearned,wewilldrawsomeconclusionsforfutureapplicationsandscalingupthemodel tootherhighereducation insti-tutions.

A MULTIFACETED MODEL

Building collaborative inter- and trans-disciplinary communities re-quiresdeepreflectionandaclear,well-plannedstrategy.

INTRODUCTION

“Our current infrastructures dissuade interdisciplinary research”(Moedas,2017), immersedastheyareinthesocalled“interdis-ciplinarity paradox” (Woelert and Millar, 2013). Interdisciplinary

researchisincreasinglyfosteredatapolicyleveltotacklecomplexlocaland/orglobalproblems,butitis,atthesametime,poorlyrewardedbyfunding instrumentsandacademicstructures (Bromham,DinnageandHua,2016).

Navigating through thisparadox,universitiesarecreativelydevelo-ping ways to integrate the growing demands posed to academic life.These are, at times, conflicting in terms of aims and interests (basicresearchvs.closertothemarket innovations,collaborationvs.compe-tition).Inthisway,severalEuropeanhighereducationinstitutionshavemadeattemptsatenhancinginterdisciplinaryresearchthroughvirtual,physicalorcombinedapproachesonissuesofrelevanceatamoregloballevel.ThisisthecaseatTrinityCollegeThemes;UniversitádeBolognaIntegratedResearchTeams;UniversityofSussexStrategicResearchPro-grammesandLundUniversityStrategicResearchAreas,tonamebutafew.Inmostcases,thesenewendeavourscoexistwithmoretraditionalwaysofmanagingresearch(disciplinedriven,“SocialSciencesandHu-manities”(SSH)vs.“Science,Technology,Engineering,ArtsandMathe-matics”(STEAM),etc.).

Theaimofthispaperistwofold:1. Firstly,tointroducethemainfeaturesandelementsofaninno-

vativeresearchmanagementsystem,the“6iResearchModel”.Emergingfromabottom-upinitiative,themodelistheresultofourquestforaclearholisticvisiontodeviseacomprehensiveresearchmanagementmodel,withdiversemechanisms,struc-turesandmeasurement tools.The“6iResearchModel” takesitsnamefromthe integrationofsixelementsthatareusuallymanagedinadisconnectedmanner:(1)international,(2)inter-

ANTONIACARO-GONZALEZDOI:10.22163/fteval.2019.376

THE“6IRESEARCHMODEL”:EVOLUTIONOFANINNOVATIVEINSTITUTIONALSTI1POLICYFRAMEWORKATTHEUNIVERSITYOFDEUSTO

1 Science,TechnologyandInnovation2 IacknowledgetheinvaluablesupportoftheInternationalResearchProjectOfficestafffortheircommitmenttoimplementingthemodelandfortheenor-

mouseffortincollectingthebackgrounddataneededforthisresearchproject.

ISSUE 48 | JULY 2019108

With the focus on social impact valuation-driven research, the “6iResearchFramework”adoptsasystemthinkingapproachandisbasedonthreeinnovative,interrelatedandmutuallyreinforcingpillars:

• An evolving the “6i Research Model”: this ismadeupofacombinationof(i)nternational,(i)nterdisciplinary,(i)ntersectoral,(i)mpact,(i)nnovationand(i)nclusionfeaturesanddimensions.

• A self-feeding flexible governance systemwhich integratestop-down and bottom-up uptakes with well-rounded flexiblegovernancesupportstructuresandmechanisms.

• A dynamic processwhichcombinescompetitiveandcollabora-tive researchendeavourswitha focusonexcellenceand realimpact.

Researchhasshownthatacollaborativecultureisastrongpredic-torofcreativity(DeCusatis,2008,Barczak,LasskandMulki,2010)and,accordingtoWaddelandBrown(1997),inter-sectoralpartnershipscan“help reduce duplication of effort and activity that works at cross-purpo-ses; they can also stimulate innovation and unusually creative solutions if the diverse goals of participants can be addressed”(p.1).Takingthisintoaccount,the“6iResearchModel”departsfromthefirmconvictionthatinterdisciplinarity isabsolutelyuseful forunderstandingcomplexprob-lems,suchashumanmobilityorclimatechange(Repko,2012). Italsoassumesthatengagingininternationalinterdisciplinaryandintersecto-ralcollaborationshelpsto:a)identifyglobalpriorities;b)developmoreresponsibleandaccountableresearch;andc)strengthenthecapacitiesrequiredtobeabletotackleglobalandlocalchallenges.

Sinceresearcherssufferfromanumberoflimitationsintermsoftheirindividualagency,careerdevelopmentandstability(i.e.securefundingforresearch),newformsofresearchercollaborationsandpartnershipswithnon-academicstakeholdershaveenormouspotentialforgeneratinginnovative ideasandstronger social impact.Studiesalsodemonstratethatpeopleareinclinedtocollaborate,providedthatthereisreciproci-ty,whichisthebasisoftrust(Thomson,PerryandMiller,2007).Never-theless, inorder to take interdisciplinarityseriously,eachpersonmustbe“secureinhisorhercompetence”,asbeinginterdisciplinarymeansbeingintentionalingroupformationanddecisions,whileincorporatingdifferentapproaches,methodologiesandprocedures(HallandWeaver,2001).Alongtheselines,creatingacollaborativeculturerequirestheco-operationofpeopleatdifferentlevelsandareasoftheorganisationandrequirestrustandleadership,reciprocity,commitment,dialogueandthesharingofideasandprojectsthatgiveasenseofbelonging,teamworkandresult-orientedprocesses.

Inordertoprovidesuchbasis,the“6iResearchModel”proposesput-tingforwardan orchestrated multi-layered and flexible interventionwhichincludes:

• awell-defined visionat a strategic level, integrating targetedinitiativesaroundthe6iaxe;

• clear,underlying,governingprincipleswhichinclude(a)apeo-ple-centredapproach;(b)buildingtrustand(c)havingconfluent“win-win”goals;

• anumberofsupportstructuresandmechanisms,putinplacetocreativelyandsteadilymakeprogressintheimplementationphasewithahighlyprofessionalisedbodyofresearchmanagersandadministrators;and

• adefinitionandimplementationofspecificmeasurestovalueimpact at a project level, with established specific rewardingmechanismsforassessingsocialimpact.

Themodelalsomakesuseofadialogicalblendofcollaborationvs

competition to achieve excellence in research. Although perceived asopposites,the2017“LeagueofEuropeanResearchUniversities”report(LERUreport)arguesthatbothcollaborationandcompetitionareneces-sarytoachieveexcellenceinresearchanditsimpact,wheneverresearchexcellence and social impact are complementary to, or compete with,eachother(AkkerandSpaapen,2017).

Alastkeyelementofthe“6iResearchModel’s”engineisthedefini-tionofindicatorsofprogressandachievementsregardingcollaborativeendeavours and inter- and trans-disciplinary integration. As with anysharedeffortandteamworkingeneral,theobjectivesofthemodelanditsrespectiveinterventionmustbeclearlydefinedandmutuallyagreedbyallmembers,includingthequantitativeandqualitativeindicatorsthatprovideanevaluationofachievement.

METHODSThis research is framed within a broader investigation focused on

understandingthemultilevelprocessdynamics,resultsandimpactsofthe6iinnovativeresearchmanagementmodelathighereducationins-titutions.Basedonthesystemthinkingapproachwehaveenvisionedamodelcapableofdevisingholisticandadaptableimplementationstothecharacteristicsofeachinstitution;andabletorespondtomorehumani-sticandsocialpurposes.

Using a methodological approach that combines a myriad of datacollection instrumentswithquantitativeandqualitativemethodologies(dataandpolicyanalysis,surveys,in-depthinterviews,discourseanaly-sis),the“6iResearchModel”isbeingassessedasimplementedattheUniversityofDeustoduring theperiod2010-2018.Thecombinationofdatacollectioninstruments,methodologiesandtriangulationofresearchresultshasenabledus to identifyanddescribe thechangeprocesses,whileunderstandingthem,capturingandreconstructingtheirmeaning.

Inordertoanswerthequestionsrelatedtothesecondobjectiveofthispaper(whichistoanalysethecasestudyoftheimplementationofthemodelat theUniversityofDeusto),wehave, fromtheuniverseofdatacollectionmentionedabove,specifically focusedonthecombina-tionoftwovariables:

a.The timeline, to analyse the evolution of the “6i ResearchModel”overtimefrom2010toOctober2018,and

b.Thekeyenablingelements,suchas(b1)theuniversity’sstrategyand its backing on policies developed for and introduced todrive the different actions, (b2) the supporting structures,(b3) the driving mechanisms, initiatives and instruments,whichhavebeensequentially introduced togeneratechangeand (b4) capacity building, which prepares researchers andresearch managers to engage in the process. Table 1 showsthesecondvariablecontainingthemainelementsinterveningintheprocess,aswellasthesourcesusedinordertocollectevidencerelatedtoeachindicator.Thispaperisfocusedonthedescriptiveanalysisof theprocess forwhich the typeofdatausedismainlyquantitative.

Variable Indicators Sources

ISSUE 48 | JULY 2019 109

b1)Policyandstrategy Institutionalpoliciesaddressingmanagementofthe6i.

-“DeustoStrategicPlan2015-2018”includingspecific“MasterPlans”for:a)Internationalisation;b)“Interdisciplinaryandintersectoralcollaborations;andc)SocialImpact”;

b2)Supportingresearchstructuresandstaff

DeustoResearchsupportstructuresandstaff -Recordskeptbythe“InternationalResearchProjectOffice”indicating:a)Thenumberofsupportstructurescreatedorre-organisedbyyear;-Annualrecordskeptbythe“HumanResourcesDepartment”showingthenumberofemployeeshiredbythemainsupportstructureresponsibleforchannellingthestrategy(IRPO);

b3)Mechanismsandinitiatives

-Internationalproposalsandprojects-Interdisciplinaryplatforms-Coregroups-Concertedactions-DIRSi-COFUNDproject-Self-createdandexternalinitiativestodriveinnovationandsocialimpact.-Disseminationinitiatives-“DeustoSocialImpactLabel”,“Deusto-SantanderAward”

-Recordskeptbythe“InternationalResearchProjectOffice”indicating:a)Numberofproposalssubmittedtointernationalprojectsandthenumberofconcertedactions(yearlyprogressreportstothe“BasqueGovernmentFrameworkProgrammeandMasterProgrammes”);b)Theanalysisofintra-platformdynamicsreliesonthedatacollectedfromtwoplatforms(“AgeingandWellbeing”,“Gender”)sincetheseweretheplatformswithspecificdataavailable.Foreachplatform,thedataincluded:theyearofcreation,thenumberofproposalssubmittedinrelatedtopics,numberofmeetingsheld,numberofcoregroups.c)NumberoftopicspublishedfortheDIRS-COFUNDselectionprocess.d)NumberofCOFUNDERsenrolled.e)Internalinitiativesandparticipationinexternalinitiativestodriveinnovationandsocialimpactaswellasdisseminationinitiatives.f)Numberofactionsregardingsocialimpactevaluationandrecognitiongrantedperyear.

b4)Capacitybuilding Specific6i-relatedtrainingprovidedtoresearchersandresearchmanagers.

-Recordskeptbythe“InternationalResearchProjectOffice”andthe“HumanResourcesDepartment”indicatingthenumber,natureandbasicfactsaboutin-houseandexternaltrainingsessionsattendedbyDeustoresearchersandmanagers.

Some indicators, such as international proposals and projects, act

bothasprocesscatalysersandresults,havinganimpactonandplayingaroleininstitutionalchangeinaself-feedingmechanism.

THE “6I RESEARCH MODEL”: AN IMPLEMENTATION IN MOTION AT UNIVERSITY OF DEUSTO

Theprocess,asimplementedattheUniversityofDeusto,hasbeenstudiedbycombiningtwoanalyticalvariables:a)time;andb)elementsinterveningintheprocess.Forthisreason,datacollectedunderthefourelementsincludedinthesecondvariable–b1)policyandstrategy;b2)support research structuresand staff; b3)mechanismsand initiatives;

Table 1.Datacollectionandanalysis.i)"DeustoInternationalResearchSchool”

Figure 1.Processevolutionofthe“6iResearchModel”attheUniversityofDeustoSource:preparedbytheauthorbasedondatagathered.

andb4)capacitybuilding–havebeenexaminedlongitudinallyfortheperiod2010-2018todescribetheprocessandthechronologicalevolu-tion of the “6i Research Model”. Figure 1 graphically summarises theaggregatedindicatorsundereachvariableandelement,andresultsarereportedinsequence.

ISSUE 48 | JULY 2019110

2010-2011 – BOTTOM-UP INCEPTION

Policy and strategy.Atthestartofthedecade,researchatDeustowascarriedoutinadisconnectedmannerandprojectsgravitatedmorearoundtheworkofindividualresearchinterests.Weweredoingmanythingsrelatedto6idimensions,andhadbeendoingsoformanyyears,onlywecalledthemdifferentnamesastheyweredissociatedfromeachotherandtookplaceindifferentplaces.

However,theUniversityofDeustohadasolidbaseonwhichtobuild:• Over130yearsofhistory thatbackedsolid relationshipswith

companies,SMEs,regionalclusters,entities,policymakers,oth-eracademic institutionsandsocialorganisations.Thishasal-lowed“DeustoResearch”toblendcompetitiveness,innovation,andtechnologyinordertotacklechallengesforcommunities,companiesandpublicbodiesintheregion.

• A robust number of externally evaluated and accredited re-searchers, research teams and units at the University with aprovenrecordofresearchexcellenceandengagementwithso-ciety(37researchteams,9researchinstitutes,13chairs)3;

• Acommittedsenior leadershipwithadeepknowledgeof theinstitution,theindividuals,thesystemandtheinternaldynam-ics.Therearethreeelementsprovidingthedrivingforceforthisleadership:firstly,flexibility,withroomformanoeuvreintermsoffindingsolutions,proposingideas,introducingchangesandcreatively introducing innovations in research management;secondly, alignment with the defined strategy; and finally, afirmconvictionthatcollaborationisthedrivingforcerequiredtoachievehigherscientificcompetitivelevelsandcloserlinkswiththeneedsofsociety.

Therefore,basedonintuitionandanemergingvisionofamoreinte-gratedwayofmanagingresearch,webasicallystartedtojointhedots.Thefirststepswereinformalmeetingswithresearchersandtransferofknowledgeofficersworkinginthefieldofageing.Wegatheredtodis-cuss,meet,takestock(ofexistingexpertise,ongoingprojectsandpubli-cations)andplanthestepsforward.

Supporting research structures and staff. In 2011, the Internati-onalResearchProjectOffice(IRPO)wascreated.Madeupof3experi-encedadvisors,theIRPOteamwasassignedwiththetaskofdrivingtheuniversity’sresearchforwardby identifyingopportunitiesto internatio-nalisetheuniversity’sresearchandbuildbridgesbetweentheuniversityandstakeholders.

Mechanisms and initiatives. In 2010, despite submitting six pro-posals, launched by international calls, only one research unit at theuniversityhad included internationalprojects in itsportfolio.However,bytheendof2011,Deustohadmorethantripleditssubmissionstoin-ternationalprojects(21submissions)andthenumberoffundedprojects(3 fundedprojects).Though thesedatashowthe initial results, itwasclear from the early phases of the process that both learning how towriteproposalsandtheparticipationininternationalprojectswerekeymechanismsformovingthestrategyforward.

Furthermore, in 2011, the first interdisciplinary research platform,“AgeingandWellbeing”,emergedasabottom-upinitiativealignedwiththe “European Innovation Partnership on Active and Healthy Ageing”.The “Deusto Interdisciplinary Research Platforms” are flexible mecha-

nismsorganisedaroundsocietal challenges forestablishingcollabora-tive inter- and trans-disciplinary research partnerships between diffe-rentresearchteamsandexternalactors.Bygatheringresearchersfromdifferentdisciplinestopromoteactive,healthyandmeaningfulageing,the“AgeingPlatform”pavedthewaytootherinterdisciplinaryplatformswhichweretoemergeinthefollowingyears.Thepathtoconstructingthe“6iResearchModel”wasunderway.

2012-2013 – GROWING STRUCTURES AND BUILDING CAPACITY

Supporting research structures and staff development. In 2012,withthesupportoftheVice-Rectorfor“ResearchandTransferofKnow-ledge”, Deusto organised its research structure around the “DeustoAdvancedResearchCentre”(DARC).Thiswasmadeupoftwosupportresearch units: the “DEIKER-Deusto Research Results Transfer Office”andthe“IRPO-InternationalResearchProjectOffice”.Inthesameyear,IRPOalsoincreaseditsstaffbyhiringtwomoreexperiencedadvisorsandonejuniormanager.Thiswasanimportantincreaseinresourcesdirectedtowardstheimpulseofmechanisms and results.

Mechanisms and initiatives. With less proposals submitted in2012thaninthepreviousyear(15),thenumberofinternationalprojectsfundedwashigher (5) thanprevious results,which, in fact,meantanincreaseinthesuccessrateandhavingfourresearchunitsinvolvedininternationalprojects.In2013,thereweremoreresearchersinvolvedinthe internationalisationof research(8researchunitscomparedto4 inthepreviousyear).Thesesubmittedelevenmoreproposalsthanin2012,threeofwhichwerefunded.Thelowsuccessratewasjustifiedduetosomeunitsthatwerejuststartingtobuilduptheircapacityinthisfield,havinghadlittleexperienceinwritingproposals.

2012-2013 was also the period in which the first proposals withinthe“AgeingandWellbeingInterdisciplinaryPlatform”wereprepared(2proposalsin2012and5in2013).Theplatformalsostartedtoholdtwoperiodicmeetings(oneeverysixmonths).Envisagedascohesiontools,thesemeetingsfacilitatedspacesforexchangingideas,networkingandplanningbetweenplatformmembers.Oncepilotedandbasedonlessonslearned,regulargeneralplatformmeetingswereintroducedsuccessivelyovertheotherinterdisciplinaryplatforms,adjustingthecontentanddy-namicsforeachspecificcontextandfield.

Capacitybuilding.Withmore staff, the IRPOmanaged toorganiseonein-housetrainingsessionin2012andfourtrainingsessionsin2013.Thefocusofthesesessionswastoinstructresearchersonhowtoapplyforinternationalcompetitiveproposalsandfunding.

2014-2015 – GAINING CLARITY: ORGANISING STRUCTURES AND TOP-DOWN SUPPORT

Policy and strategy. Since2010,“DeustoResearch”hadbeenstea-dily developing a clearer vision for challenge-driven research alignedwith the Europe 2020 and the “Basque Country Smart SpecialisationStrategies”, with advanced research units and experts contributing toknowledgegenerationandinnovativesolutions.Nevertheless,itwasin

3 2017data.

ISSUE 48 | JULY 2019 111

2015thatthefirstfour“i’s”intheresearchmodel(internationalisation,interdisciplinarity,intersectoralandimpact)wereincludedinthe“Deus-to2018StrategicPlan”(2015-2018).

Withtheestablishmentoftheseinternalpoliciesandrecognitionme-chanisms,themodelreceivedbackingatthehighestinstitutionallevelfromtherector’steam,with:

1. Theintroductionofthefoundingprinciplesandgoverningele-mentsintotheagendaandstrategy;

2. Thedevelopmentofavaluationsystemataresearchandinno-vationpolicylevelwithintheuniversity,includingthreespecific“Master Plans” in the “Deusto 2018 Strategic Plan”, creatingsynergieswithotherstrategicareasoftheuniversity,suchasa“CommitmenttoSocialJustice”;

3. Securingaportionoftheresearchsupportbudgettopromotejointparticipationininternationalresearchprojects;

4. Settingaflexiblestructureandsupportmechanismstocreate,developandestablishinterdisciplinaryplatforms;and

5. Thedefinitionofprogressindicators,againstwhichthismulti-layeredprocesshasbeenregularlymonitoredandevaluated.

Supporting research structures and staff. Attheendof2014/be-ginningof2015,the“DIRS-DeustoInternationalResearchSchool”wascreatedundertheDARCstructuretocoordinatedoctoraltrainingattheuniversity.Inthesameperiod,theIRPOhiredthreemorejunioradvisors.

Mechanismsandinitiatives.Internationalproposalscontinuedtobethekeymechanismforengaging researchersandunits in the“i strat-egy”. The number of proposals submitted to international calls nearlydoubledin2014(goingfrom26proposalssubmittedin2013to40in2014–7ofthemreceivedfunding).ThiswastheresultofagoodpositioningstrategyfortheinitialcallsundertheHorizon2020programme.In2015,thenumberofsubmissionstointernationalprojectsreacheditshighestlevel(53proposalssubmittedand9projectsfunded)4.Consequently,thenumberofresearchunitsworkingoninternationalprojectsliterallydou-bledfrom8in2014to16in2015.

Asignificanteventin2014wastheemergenceofanewinterdiscipli-naryplatformfocusedon“Genderissues”.Meanwhile,the“AgeingandWellbeing”platformkeptincreasingthenumberofproposalssubmitted(risingfrom5submissionsin2013to14proposalsin2015).Inaddition,asaresultofthedevelopmentandapprovalofthespecific“MasterPlan”toboostinterdisciplinarycollaborations,threemoreplatformsemergedin2015(“B-Creative-CreativeCulturalIndustriesandCities”;“SocialJus-ticeandInclusion”;and“StrengtheningParticipation”).

In2015,theplatformsalsoofficiallystartedtounfoldintocoregroupsas performing mechanisms for collaborative endeavours. These coregroupsweresmallergroupsofexpertsworkingtogetherwiththeirlocalandinternationalpeersandstakeholdersaroundspecificsocietalchal-lengesonspecificproposalsorprojects.Thesehadundergone testingduringtheprevioustwoyearsandwerefoundtobeviablemechanismsforfocusingcollaborationon:

1. buildingwin-winsituationsbetweenresearchers;2. tangible work aligned with the agenda, the results expected

andtheinterestsofdifferentresearchunits;and3. creatingmeetingspacestobuildtrustandpersonalrelationships.

Thedatashowanincreaseinthenumberofactivecoregroups,froma number of timid informal exchanges in 2010 to the current regular,

content-specific,adhoccoregroupmeetingsheldon the twostudiedinterdisciplinaryplatforms.

Capacity building. Inordertomanagetheincreasingdemandandtoprovidetrainingandsupporttoresearchers,theIRPOorganised9trai-ningsessionsin2015,includingin-houseandexternaltraining.

2016-2017 – HARVESTING RESULTS AND BOOSTING MECHANISMS

Policyandstrategy. Internationalisation, interdisciplinaryand inter-sectoralcollaboration (the first3“i’s”)were thedriving forces thatar-ticulatedDeusto’sresearchresponsetosocietalchallenges,andsocialimpact(the4thI)wasincorporatedsteadilyintotheresearchandinno-vationpolicyandinternalrewardmechanisms.In2016-2017,anevolvingmulti-layeredprocessof“SocialImpactValuation”wasfinallyinplace.Theprocessencompassedprogressatfourdifferentlevels:

1. Reflection and state-of-the-art knowledge production that re-sulted in the establishment of an evaluation criteria set con-trastedwithinternational,nationalandregionalexperts;

2. Thegenerationofsupportunits,dependentontheseniorman-agerappointedtothespecific“StrategicMasterPlan”andtwoperformingbodies:thesteeringandtheevaluationcommitteesin charge of planning, implementing and evaluating progressandresults;

3. Trainingof social impactmanagers in chargeof theeverydayimplementationoftheproposedactionplan;and

4. The launchofconcretevaluationmeasuresand initiatives:aninternalcallwasdevelopedand launched: the“DeustoSocialImpact Briefings”. “Deusto Social Impact Briefings” are briefpublicationstodisseminatetheresearchresultsofprojectstospecificstakeholdersandawideraudience.

Mechanisms and initiatives. In2016,Deustoachieved itshighestnumber of international funded projects (12) while the “Ageing andWellbeing”platformmanagedtosubmit12proposalsbetween2016and2017.In2017,the“GenderPlatform”alsostartedtoincreaseresultsandpresented4proposalsforinternationalcalls.

Inthesameyear,4interdisciplinaryresearchareaswereidentifiedinalignmentwiththe“BasqueSmartSpecialisationStrategy”andthein-tersectoralcollaborativeframework:“Energy,Territory,HealthandIndus-try4.0”.Inaddition,specificcommitteeswereassignedtheresponsibilityofmonitoringtheimplementationoftheactionplanenvisagedunderthe“MasterPlan”onsocialimpact.

Interdisciplinaryco-operationsteadilyincreasedwithinthe5interna-tionalinterdisciplinaryplatforms.Thisprocess,coordinatedbyresearchmanagersatthe“InternationalResearchProjectOffice”,crystallisedinthe creationof a collaborative culture (i.e. exchangeof ideas, sharingknowledge,buildingtrust)basedonregularformalandinformalgathe-rings,meetingsandexchanges(six-monthlygeneralplatformmeetingsandmorefrequentadhocmeetings,whichweretopic-specificorpro-ject-based,wereheldregularlyonademandbasis.

Framedwithinthethen4istrategyandbackedby42partnerorgani-sations,theUniversityofDeustoreceivedaprestigiousMarie-Sklodows-kaCurieCOFUNDproject.ThiswasledbytheVice-Rectorof“Research

4 ThiswasalsoaresultoftheDeustoTech’sstrategytoinvestinhiringarenownedconsultancytoboosttheirinternationalisationstrategy,helpingitsunitsprepareahighnumberofEuropeanproposals.

ISSUE 48 | JULY 2019112

andTransfer”asaninstitutionalprojecttochannelacollaborativecul-tureamongPhDprogrammes, research teams, support structuresandinterdisciplinaryresearchplatformsandareas.Itwasachallengingpro-posaltoprepare,withmultiplenegotiationsandmultilevelcoordination.However,itwassuccessfullyevaluatedandfundedandhasallowedtheuniversityto:

a.Leveragethecoordinationlevelbetweendifferentdepartments,researchsupportunits,andPhDprogrammesattheuniversity;

b.Introduce innovations in the selection process and theidentification of topics offered in the two call for candidatesopenundertheproject;and

c.Offer 8 doctoral positions to attract talented youngresearchersofexcellencetoDeustoPhDprogrammes,teamsandplatforms.

In terms of social impact, in 2016, we organised the first “DeustoConference”,which,togetherwithnon-academicstakeholdersparticipa-tinginresearchprojects,addressedissuesrelatedtothesocialimpactofuniversityresearch.Inaddition,theuniversitysetuptwonewrelatedin-itiativesintheperiod:the“DeustoResearchSocialImpactLabel”,whichrecognisesimpactfulresearchprojects,and,forthefirsttime,thesocialimpactdimensionwasintroducedintothe“Deusto-SantanderResearchAwards”.Asecond“DeustoConference”washeldinMarch2017.Fur-thermore, in 2017, DIRS-COFUND topics evolved and were evaluatedusingtheexisting4imodel.Another8positionswerepublishedinthesecondcall,resultingintheenrolmentofanewcohortof8“EarlyStageResearchers”(ESRs),whojoinedthe8previousones.

Capacity building. In 2017, the amount of in-house and externaltrainingprovided to researchers reached itspeak,with twiceasmanysessionsheldthaninthepreviousyear(14trainingsessionsin2017com-paredto7sessionsin2016).

2018 – BROADENING THE MODEL

Policyandstrategy.The“6iResearchModel”gaineditslasttwo“i’s”inthisyear:innovationandinclusion.Weare,atpresent,incorporatinginnovationandentrepreneurshipinamoresystematicway.Apartfromtheexistinginnovationinitiatives5,wearedevisingmechanismsandac-tionstoaligninnovationwithintheresearchstrategy.

AfundamentalunderlyingprincipleofthemissionandvisionoftheUniversityofDeustoisinclusion(the6thI).Wearecurrentlytakingstockof the way this dimension is being tackled within the model. A clearexampleofthisisthatanyonewhowishestoiswelcometocontributeto the Deusto interdisciplinary platforms in a variety of different roles(aspartofacoregrouptoprepareaproposal,asarepresentativeoftheplatformatrelevantinternationalorlocalevents,etc.).Specificmetho-dologiesandindicatorsarebeingdevelopedtocapturetheinclusionofdisciplines,roles,researcherswithintheinterdisciplinaryplatforms,thepreparationof internationalproposals, the implementationofprojects,etc.

Figure 2illustratesthe“6iResearchModel”andshowsthealignmentoftheuniversity’s“StrategicPlan”(inthecentre)withtheverticaland

5 Therearealreadyanumberofoutstandingbutdissociated innovation initiatives,unitsandresearchersatDeusto.The innovationdimensionof the“6iResearchModel”willbuildonthisrichbodyofalreadyexistinginitiatives,researchersandunits.Itwillfigureoutsuitablecollaborativemechanismstointegratetheresearch-innovation-transfercontinuumofknowledge-socialimpact.

Figure 2.Deustoimplementationofthe“6iResearchModel”.

ISSUE 48 | JULY 2019 113

horizontalinterconnectionsbetweenthelevelsandelementsinaconti-nuouslyevolvingself-feedingprocess.

Supporting research structures and staff. Asoftoday’sdateinOc-tober2018,IRPOstaffnumbershavebeenbolsteredbythehiringofaprojectmanagerandonejunioradvisor.Theincreasedsupportstructurewillmakeitpossibletokeepupwiththecontinuousworkload:

• providingsupporttostaffinpreparingproposalsforcompetitivecalls;

• takingonthepreparationofambitious initiatives,suchasthecoordinationofaHackathonwithintheAALForum2018andtheBiscaySilverWeekheldinSeptember;

• capacitybuilding(7trainingsessionshavebeenheldonlythisyear);

• boostingtheactionplanforinnovation(innovationradarpilots,socialimpactlicensing);and

• improving the communication and dissemination of researchresults (generationofnewsforthe interdisciplinaryplatforms,“DeustoResearch”website).

Mechanisms and initiatives. The monitoring of the performanceindicatorsforthe3MasterPlansshowsthedrivingforceofthestrategyintermsofthedynamicsgenerated,blendingcollaborationandcompeti-tiveness.ThisblendresultedintheparticipationbyDEUSTOinatotalof167researchproposalsbetween2015and2018,with35ofthembeingsuccessfullyfundedunderHorizon2020andotherrelatedprogrammes.Thisrepresentsasuccessrateofover20%,meaningthattheUniversityofDeustoisshowingacompetitiveperformanceabovethenationalandEuropeanaverage.

Figure 3.Internationalproposalssubmittedandprojectsfunded(FP7,H2020andrelatedprogrammes)

For the last two “i’s”: innovation and inclusion, driven byinitiativesfromtheEuropeanUnionsuchas“InnovationRadar”,Deusto started to run “Innovation Radar” pilots with selectedresearchprojects carriedoutby theuniversity. Ithasalsocol-laborated with local industry partners to launch an initiativecalled the “Social Impact Licensing Strategy”, which is aimedatscreeningtechnologiesand/orservicesprovidedby“DeustoResearch”toevaluatesocietalmarkets.

Finally,inrelationtoinclusionandalignedwiththeinternationalisati-onofresearch,thewiderongoingresearchprojectwillincludeinitiativesinwhichDeustohas takenpartwhicharedirectly relatedto inclusion(i.e. the“EuropeanScience forRefugees initiative”,which isaimedatopeningdoorsforrefugeescientiststoEuropeaninstitutions)6.

6 InclusionhasalsoreceivedspecificobjectivesandactionswithintheMasterPlanentitled‘CommitmenttoSocialJustice’,whichispartofthe“Deusto2018StrategicPlan”.

ISSUE 48 | JULY 2019114

CONCLUSION

Thispaperhad twoobjectives: the firstwas to introduce themainfeaturesofthe“6iResearchModel”andgiveabriefaccountofitsmul-tifacetedcomposition.Thesecondwas toanalysehowthemodelhasevolvedinpracticeduringitsimplementationattheUniversityofDeustointheperiod2010-2018.

Firstly, tosummarisethemodel,6istandsforsix researchdimensi-onsthatareusuallymanagedinadisconnectedmanner:international,interdisciplinary,intersectoral,innovative,impactfulandinclusive.Alongtheselines,the“6iResearchModel”isamultidimensionalsystemthatcombineskeyelementsinordertosustainamulti-layeredinterventionthat:(1)includes6iinawell-definedvisionandstrategy,(2)definescleargoverningprinciples,(3)providesmechanismsandstructurestosupportinternational, interdisciplinary, intersectoral, impactful, innovative andinclusivecollaborationand(4)definesspecificmeasuresforevaluatingtheon-goingprocess.

The combination of a system thinking approach with a hands-onpractical implementation,whichisembeddedintherequisitesandas-sessmentmechanismsofuniversitylifehashelpedusenvisionamodelcapableof

a.devising more holistic implementations open to futuredevelopmentsandcollaborations;

b.beingabletoadapttothefeatures,characteristicsandeverydaybusinessofeachinstitution;and

c. proposingresearchquestionsandinnovationsthatrespondtomorehumanisticandsocialpurposesincollaborationwithotherresearchersandstakeholders.

Secondly,bycombiningtwomainvariables (timeandkeyenablingelements),wehaveexplainedthemainfeaturesandevolutionofthepro-cessovertheperiodinquestion.Changesintroducedundereachofthesub-variables(policyandstrategy,supportstructures,mechanismsandinitiatives,andcapacitybuilding)havelongitudinallygenerateddifferentinstitutional responses that accommodate the ever-evolving researchmanagementprocess.

Theresultsobtainedfromtheanalysisoftheimplementationofthemodel at Deusto show how a process that integrates these 6 usuallydisconnectedelementsintoanorchestratedstrategycanpavethewaytogrowingarobustmodel.Thefirminstitutionalcommitmentto6iatDe-usto,togetherwiththeinnovativecombinationofinstitutionalstrengthsandelements,demonstratesacomplexself-feedingdynamic.Inthisdy-namic,bottom-up initiativesand top-downsupportcombineanddriveeachother,integratingaroundtheordinarydeliveryofresearchresultsatacademicinstitutions(i.e.researchprojectfunding).

Thisself-feedingprocesscanbeclearly illustratedby theevolutionof the “Deusto Interdisciplinary Research Platforms”. Emerging as abottom-upinitiativein2011toaddressboththeagencyofresearchersandtheuniversity’sresearchmanagementstructure,researchplatformswere backed at the highest level over time and incorporated into theuniversity’sstrategicplan.Inaddition,theyaresteadilybecomingakeypartoftheuniversity’sresearchstructurethroughwhichtochannelin-ternational, interdisciplinary and intersectoral collaborations. Togetherwithresearchexcellenceunitsandgroups, theplatformsarefosteringthe inclusion and engagement of researchers and stakeholders in im-pactful research. Specifically, the five “Interdisciplinary Research Plat-forms”andthefour“InterdisciplinaryAreas”havehelpedtoaggregate

expertiseandcriticalmasstostriveforresearchexcellencealignedwiththe“Europe2020Strategy”andthe“BasqueSmartSpecialisationStra-tegy”(RIS3).

In termsofpeoplemanagement,engaginguniversitystaff toworkintointerdisciplinarycommunitiessuccessfullyisalong-termandcom-plexprocess. Interdisciplinarycommunities,suchasthe“Deusto Inter-disciplinaryResearchPlatforms”,areliving,dynamicpeople-centredsys-tems,withfearsandemotions,knowledgeandexpertise,attitudesandpersonalities,interestsandpersonalhistoryandrelationshipswithintheinstitution.There isnothingmore intricate inanorganisationthanthepeoplethatcompriseit,andingeneral,notenoughimportance,effortsandresourcesarededicatedtotheirdevelopmentanddemands.

Byinnovativelylinkingtheindividual,collectiveandinstitutionalle-vels, the evolving “Deusto Research Collaborative Framework” is ena-blingconditionstoovercomebarriersanddevelopsuccessfulandsus-tainableinterandtrans-disciplinary,intersectoralcollaborations.Thisiseasingthepathfordeliveringindicatorsofsustainable,real,collaborati-veefforts,whileatthesametimemovingtowardsdefiningmeaningfulresearchquestionsandrealimpacts.

Finally,onelimitationofthisworkisthat,bytakingconcreteevidenceasareference,thisresearchoptstoanalyseinstitutionalchangefromamore “tangible” perspective. To complement this, further studies thatarecurrentlyinprocess,aspreviouslymentioned,willbroadenthescopeanddeepentheunderstandingofthe“6iResearchModel”fromamoresociologicalapproach.Usingacombinationofquantitativeandqualita-tivedatasetsandmethodologies,theevolutionandprocesswillanalysethedrivers, thebarriersand the roleof theagencyof individualsandhumaninteractionontheprocess.

REFERENCES Akker, W., and Spaapen, J. (2017). Productive Interactions: SocietalImpactofAcademicResearchintheKnowledgeSociety.Brussels:LERU.

bmwfw (2016). Hochschulmobilitätsstrategie des BMWFW zur Förde-rungtransnationalerMobilitätanösterreichischenUniversitäten,Fach-hochschulenundPrivatuniversitäten.bmwfw.

Barczak, G., Lassk, F., and Mulki, J. (2010). Antecedents of TeamCreativity:AnExaminationof TeamEmotional Intelligence, TeamTrustandCollaborativeCulture.Creativity&InnovationManagement,19(4),332–345.

Bromham, L., Dinnage, R., and Hua, X. (2016). Interdisciplinary re-searchhasconsistentlylowerfundingsuccess.Nature.Nature.Vol,534:684–687.

Hall, P., and Weaver, L. (2001) Interdisciplinary education and team-work:alongandwindingroad.SendtoMedEduc.Vol,35(9):867-75.

Moedas, C.(2017).TheEuropeanOpenScienceCloud:Thenewrepublicofletters.SpeechgivenattheEOSCSummit.

Waddell, S, and Brown, L. D.(1997).Fosteringintersectoralpartnering:Aguidetopromotingcooperationamonggovernment,business,andci-vilsocietyactors(Vol.13).Institutefordevelopmentresearch(IDR).

ISSUE 48 | JULY 2019 115

Thomson, A., Perry, J., and Miller, T. (2009). Conceptualizing andMeasuringCollaboration.JournalofPublicAdministrationResearch&Theory,19(1),23–56.

Woelert, P., and Millar, V.(2013).The‘paradoxofinterdisciplinarity’inAustralianresearchgovernance.HigherEduaction.Vol,66(6).

AUTHORANTONIA CARO-GONZALEZDeusto Advanced Research Centre. International Research Project Office, University of DeustoAvdaUniversidades24,Bilbao,48007(Spain)E:[email protected]

ISSUE 48 | JULY 2019116

Conflict of interestAllauthorsdeclarenoconflictof interest insenseof the“Uniform

RequirementsofManuscriptsStatementofMedicalJournalEditors”1.

INTRODUCTIONTheLudwigBoltzmannGesellschaft(LBG),anon-profitresearchor-

ganisation,addressescomplexsocialchallengestogetherwithpartners,bydevelopingandtestingnovelformsofcooperationbetweenscienceandnon-scientificactors inadynamicsocialenvironment. Inthisway,theLBGaims todevelopeconomicandsocial solutions thatpositivelysupportsocialchangeandcanbeuseddirectlybycivilsociety,politicsandtheprivatesector.LBG’s“ResearchandInnovationPolicy”empha-sises the targetedandcoordinated transgressionof theboundariesoforganisations,disciplinesandsystems(OpenInnovationinScience)ai-ming to improve the societal impactof research. Thus,novel formsofengagement increase theopportunity togenerate innovativeproblem-solvingapproaches.

Inthiscasestudy,the“Villageproject”,weinvestigatedifferentmea-suresaimingtodriveevidence-basedchange,towardsmakingasustai-nableimpactforchildrenthathaveaparentwithamentalillness.First,we introducean innovativeapproach toengage thepublic ingenera-tingsocietalrelevantresearchquestionsandestablishinginternationaland interdisciplinary “Research Groups” on mental health of childrenand adolescents. Second, we introduce educational programmes forresearchersandadolescentstoenrichresearchwithmeaningfulyouthengagementandtransferknowledgeamongdifferentstakeholdersandpeoplewithlivedexperience.Last,wefocusoncommunityengagement,awareness raising formentalhealthandworking togetherwithpeop-lewith livedexperienceasgamechangers inadvocating for informeddecision-makingonacommunityandpolicylevel.

AcknowledgementWethankthemembersofthe“ResearchGroupVillage”fortheircon-

tributionsinco-creatingtheresearchconceptandimplementationofthiswork.Thisgroupcomprisessevenacademicinvestigatorsbasedwithintheirhostinstitutions:

“Principal Investigator” (PI) Dr. Jean Paul, PhD, BASc, BSc (Hons)(“MentalHealthResearchProgramme”LudwigBoltzmannGesellschaft;MedicalUniversityofInnsbruck/Innsbruck;Austria),and“Co-Investiga-tors”(CoI))Dr.IngridZechmeister-Koss,MA(LudwigBoltzmannInstituteforHealthTechnologyAssessment/Vienna/Austria),Dr.MelindaGoodye-ar, PhD, MBSc (Monash University/Melbourne/Australia) Diplom-Kffr.Annette Bauer, MSc (London School of Economics/London/UK), Univ.Prof.Dr.HannaChristiansen,PhD(PhilippsUniversity/Marburg/Germa-ny), Dr. Batool Fatima, Dr.PH, PMDCP, MA (Aga Khan University/Kara-chi/Pakistan),andDr.IngunnOleaLund,PhD(TheNorwegianInstituteofPublicHealth/Oslo/Norway).TheteamissupportedbyDr.RaphaelaKaisler, the research group and relationship manager for the “MentalHealthProgramme”attheOpenInnovationinScience(OIS)centre,Lud-wigBoltzmannGesellschaft(LBG).

“The Village” is a research project funded by the Austrian FederalMinistryofEducation, Science,andResearchthroughtheOpenInno-vationinScienceCentreattheLudwigBoltzmannGesellschaftGmbH,hostedattheMedicalUniversityofInnsbruck,withcollaborationofCo-investigatorinstitutions.ThestudyissupportedbyexpertisefromProf.Lesley Stirling (linguistics, The University of Melbourne/Australia) andProf.GeoffWong(realistapproach,OxfordUniversity/UnitedKingdom).

AuthorswouldalsoliketoacknowledgetheworkandcontributionsoftheInnsbruck“Villageteam”:ProjectcoordinatorRosannaBuchauer;PhDstudentsDanielPurtscheller,FranziskaHuck,andLisaDobener;andstudentassistantsJuliaKapferer,ElenaSeis,Anna-ElenaPinggera,andJohannaKrönner.

Thestudyisperformedaccordingtothe“DeclarationofHelsinki”anditslateramendmentsandwasapprovedbythe“HumanResearchEthicsCommittee” of Monash University Melbourne, Australia in addition tointernalapprovalfromtheLondonSchoolofEconomics,London,UK.

RAPHAELAE.KAISLERANDJEANL.PAULDOI:10.22163/fteval.2019.377

EVIDENCE-BASEDPRACTICEANDPOLICESFORIMPACTONMENTALHEALTHOFCHILDRENANDADOLESCENTS

1 www.icmje.org

ISSUE 48 | JULY 2019 117

CREATING EVIDENCE-BASED PRACTICE USING A COLLECTIVE IMPACT FRAMEWORK

Aiming to generate societal impact in the field of “Mental HealthofChildrenandAdolescents”,LBGadoptedanovelapproach towardsforming highly interdisciplinary “Research Groups”. This aligns to theEuropean Union’s Horizon 2020 scheme (European Commission, 2013)toaddresssociety’s“GrandChallenges”andrecognisesthecentralrolesocialsciencesandhumanitiescanplaythroughtrulycollaborativeandadditiveresearchfrommultipleparadigmsforresearchtocreatesocialimpact(MaxwellandBenneworth,2018).LBG’sgoalwastoengagewithdifferentstakeholdersandthepublicthroughouttheentireresearchpro-cesstodevelopnovelsolutionstochallengesinthefieldofmentalhealth,whichdirectlyimpactsociety.Therefore,LBGstartedthe“OpenInnova-tion in Science” initiative, with the aim of systematically opening upprocessesofscientificdiscoveryinanefforttoenrichresearch,throughnewknowledgedrawnfrombeyondtraditionaldisciplinaryboundaries.

ADDRESSING SOCIETAL RELEVANT CHALLENGES THROUGH PUBLIC ENGAGEMENT IN RESEARCH

Sauermann and Franzoni (2015) showed that user contribution incrowdsourcingissignificantinmagnitudeandspeedofcrowd-sourcingknowledge.LBG’s“TellUs!”2wasEurope’s firstcrowdsourcingproject,generatingresearchquestionsonmentalillnessinvolvingpatientsandfamilymembersandhealthcareprofessionals.Fourhundredhigh-qualitycontributionswereanalysedandclusteredbyanexpert juryregardingtheirimportance.Outofseveralimportanttopics,securingmentalhealthforchildrenandadolescentsemergedasakey issue.Additional inter-viewswithexpertsinthefieldemphasisedtofocuson“childrenofmen-tallyillparents”emergingasthemaintopic.

Basedonthisresult,LBGannouncedaresearchcallrepresentinganinteractiveworkshop,“IdeasLab”3,tobringtogether30researchersforamulti-dayevent,duringwhichresearcherswerespecificallyencouragedtothinkout-of-theboxanddissolvedisciplinaryboundaries.Applicantswereaskedtocompleteanapplicationviaanonlineplatformcomprisingsixquestionswithregardtotheirprofessionalbackground,expertiseandinterestscontributingtorealisingthegoalof the“IdeasLab”,andap-proachtoteamwork.Intotal,136researchersappliedtoparticipateinthe“IdeasLab”,andfurtherassessmentbytheevaluatorsconsistingofthementors,anorganisationalpsychologistandtheprogrammemana-

ger.Thereof30applicantsfromadiverserangeofdisciplineshadbeeninvited toparticipate in the“IdeasLab”.During the5-dayevent inVi-enna,theresearchersweresupportedbymentors,internationalexpertsrepresentingavarietyofpediatricandadolescenthealthfields,provid-ingongoingfeedbackonthedevelopmentofprojectideasinthe“IdeasLab”. The mentors changed their role to become live peer-reviewersforthefinalpresentationsandprojectproposalsonthelastdayofthe“IdeasLab”givingfundingrecommendationstoLBG.Additionally,“pro-vocateurs”orguestspeakers,includinginternationalmentalhealthre-searchersandexpertsbyexperience(youngadultswhoseparentshaveamentalillness),wereinvitedtoinspireresearchersandidentifygapsinthementalhealthservicesystem.

TheLBGOIScentredevelopednovelevaluationcriteriafortheprojectproposalthatwerebasedonopeningupdisciplinaryboundaries,foste-ringpublicengagementintheresearchprocess,andestablishingnewformsofstakeholderinteractionandcollaborationthatleadtointerdisci-plinaryandtransdisciplinaryresearch.Thesefollowingcriteriawereap-pliedtofindinnovativesolutionstoexistingchallengesinmentalhealthbyinvolvingthepublicintheresearchprocess:

1. novelty, revolutionary and high-quality approach to complexchallenges,

2. interdisciplinaryresearch,3. engagement, stakeholder/user engagement throughout the

entire researchprocess includingdisseminationactivitiesandinvolvementofpatientsandfamilymembersinresearchactivi-ties,

4. feasibility,thecapabilitytodelivertheirprojectasahigh-qualityinterdisciplinary activity, provided both through the presenta-tionoftheir jointproposalandtheiractivityduringthe“IdeasLab”,and

5. impact,clearrelevancetoandthepotentialtomakeadistinc-tive and novel contribution towards addressing the researchchallengesinthisareacreatingaddedvalueforsociety.

Asaresultofthe“IdeasLab”,two“ResearchGroups”,“DOT–DieoffeneTür[Theopendoor]”4and“Village–HowtoraisetheVillagetoraisethechild”5wererecommendedforfundingwithacombinedbudgetofEUR6millionduringfouryears(2018-2021).Toensurepublicengage-mentandinterdisciplinaryresearchthroughouttheresearchprocess,the“ResearchGroups”areembeddedinadynamicnetworkworkingcloselywithexistingnetworksandpatientorganisationsandaresupportedbya“ResearchGroupandRelationshipManager”tofostercommunityenga-gementandcollectiveimpact.

“RESEARCH GROUPS’” GOVERNANCE STRUCTURE

To empower people with lived experience in decision-making, weincludedtheirvoicesinthe“AdvisoryBoard”ofthe“ResearchGroups”,which advises and evaluates the research activities twice a year. The

2 www.redensiemit.org3 www.ideaslab.lbg.ac.at4 www.dot.lbg.ac.at5 www.village.lbg.ac.at

ISSUE 48 | JULY 2019118

“AdvisoryBoard”consistsoftwoacademicexpertsinthefieldofmentalhealthor specificmethodologieswithin theproject,anopen innovati-onexpert,apeerPIresearcher,andtwopeoplewith lived-experience.Therecommendationsofthe“AdvisoryBoard”arediscussedandagreeduponbythe“SteeringCommittee”includingarepresentativeoftheLBGandtheMedicalUniversityofInnsbruck(the“Villageproject’s”universityhostorganisation).

Besidestraditionalscientificmeasures,suchaspeer-reviewpublica-tions,disseminationandoutreachactivities,weintroducednewassess-mentcriteriaregardingthemeaningfulengagementofpublicinresearchactivities:

1. inclusionofpeoplewith‘livedexperience’inresearchactivitiesandcommunityengagement,

2. co-developmentofinterventionswithstakeholders,3. implementationandevaluationoftheirpractice,4. policy recommendation and engagement of policy makers in

researchactivities,5. up-scaling strategies for sustainable impacts for children and

adolescents.Additionally,tofostercontinuousengagementofpeoplewithlived-

experience in the research process, we established the “CompetenceGroup”asanewadvisorybodyconsistingofsixyoungadultswithlivedexperience(“Childrenofparentswithamentalillness”–COPMI).Thisgroupconsultsboth“ResearchGroups”ontheirresearchactivitiesonaregularbasis.Inthisway,weensuretheresearchsupportsinclusionofexpertisebasedonownexperiences.Asanextstep,governmentalfun-dingshouldbeappliedtoincreaseawarenessofvaluablecontributionofpeoplewithlivedexperienceinresearchandsustainabilityoftheirwork.

THE “VILLAGE PROJECT”: CO-DEVELOPMENT WITH STAKEHOLDERS TO CHANGE PRACTICE

“Childrenofparentswithamentalillness”(COPMI)oftenneedaddi-tionalsupportstoleadthehappyandhealthylivestheydesire.However,insomecases,thosesupportsareeithernotavailableornotfoundbyfamilies,resultinginnegativelong-termoutcomesforthesechildren.The“Villageproject”aims to increase identificationandstrengthen formalandinformalsupportsaroundchildrenwhentheirparentshaveamentalillness(Christiansenet.al.,submitted).Thisprojectwillbeco-developedwithstakeholdersandwillimplementandevaluatetwopracticeapproa-ches,focusedonthechildandonprinciplesofcollaborativecare.Akeychallengeisthatmuchofthe‘hard’evidenceofwhatworksforwhom,andwhatisgoodvalueforCOPMIislargelylacking.Inthelightofthislackofevidence,ithasbeenargued(Nicholson,2009)thatthefollowingshould be emphasised: involving practitioners and people with lived-experience as equal partners in research; the appropriate applicationof mixed-methods to explore the issues; and the development andapplication of appropriate child-specific outcome measures to betterunderstand the needs and impacts on COPMI (focusing on child’sself-esteem and resilience). After a scoping phase, synthesising theinternational evidence on barriers and opportunities for support for

COPMI, we will provide information on the mental health and socialserviceswithinTyrolinAustria,theprojectsite.

Continuingpublicengagement in research tomakean impact, the“Village”projectaimstoimprovethesituationofchildrenwhohavemen-tally ill parents (COPMI) in Tyrol, Western Austria. In order todeveloppracticeapproaches tobetter identify and support these childrenandtheir parents, we needed an in-depth understanding of the regionalTyroleancharacteristicsintermsofexistingsupportstructuresandthesocietalcontextinwhichtheyareembedded.ThisworkwasledbyCoIIngrid Zechmeister-Koss, and the following welfare-state sectors weresystematicallyanalysedintermsofpotentiallyrelevantbenefits:‘Healthcare’,‘children/families’,‘socialaffairs’and‘education’.Theinformationonavailablebenefitswasfirstlycategorisedaccordingtowelfarestatesectors,andthensynthesisedintoanoverviewofservicesthatcouldbepotentiallyrelevantintheprocessofidentifyingandsupportingCOPMIsandtheirfamilies(Zechmeister-KossandGoodyear,2018).

Tyrol is a region in the Western part of Austria, constituting ninepolitical districts. From roughly 750.000 inhabitants, around 140.000persons(19%)aredependentchildren(0-18years).Thevastmajoritylivesindual-parentfamilies.CatholicreligionplaysanimportantroleinTyrol.85%ofTyroleansareAustriancitizens.50%ofthepopulationisactivelyworking in paid employment, the remainder is either retired (20%), ineducationorinotherformsofactivity(parentalleave,householdleadingonly,militaryservice).Regardingtheidentifiedbenefits,bothin-kindaswellascash-benefitsare relevant.Whilebenefits forchildren/familiesare mostly cash benefits with only limited publicly funded child-carefacilities,intheothersectors,in-kindbenefits(e.g.publiclypaidhealthor social care services) are dominant. We identifiedabroadvarietyofbenefitsthatmaybeutilisedto identifyandsupportCOPMIsandtheirfamilies. However, only one of the existing services (available in twodistricts)directlytargetsCOPMIs.Intermsofsetting,avastmajorityofservicesareoffice-basedandamuchsmallerproportionofprovidersof-feroutreachservices(e.g.infamilies’homes).Theavailableservicesarecharacterisedbyahighproportionofpublicfunding,however,accesstopubliclyfundedservicesmayberestrictedviagate-keeping(e.g.referralsfromchildandyouthservice)orshortageofcapacities(e.g.psychothe-rapy,childcare).Theexistingservicesshowageographicalvariationwithmore(typesof)servicesavailableintheurbanthanintheruralregions.Servicesarecharacterisedbyhighfragmentationintermsofgovernance(federal,regional,municipality),financing(taxes:federal,regional;soci-alinsurance)andserviceprovision(publicandprivateproviders).

Theseresultsandascopingofinternationalbestpracticeexampleswillinformtheco-developmentphasewithstakeholdersinTyrol,whichwillbemadeupofsixco-designworkshopswhichbeganinNovember2018.Duringtheco-developmentphase,wewilldeveloppracticeapproachesand tools to identify COPMI and to support them in everyday life bystrengtheningnetworksamongformalandinformalsupportsystemsinTyrol.Thiswillbesupplementedwithtrainingmaterialforimplementingthe practice approaches and thirdly, key-indicators for evaluating thepractice approaches will be defined. The development of the practiceapproaches and evaluation indicators will be done in a participatorymanner (co-design) involving representatives of stakeholders andparticularlyincludingpeoplewithlivedexperience.Community-capacitybuildingapproaches,concernedwithdevelopingasupportivenetworkof allies around a person, utilising principles of collaboration, person-centeredness, andprevention, can increase resilience at an individualandcommunitylevel,aswellasbecost-effective(Knapp,Bauer,Perkins

ISSUE 48 | JULY 2019 119

andSnell,2013;Wistow,Perkins,Knapp,BauerandBonin,2016);howthis relates toCOPMI isnotyetknown,and thisprojectwillgenerateevidencetoaddressthisgap.Aparticipatoryandco-developedapproachto the development of screening approaches and collaborative care,that is evidence-informed and evidence-generating, has not yet beenimplementedforCOPMI–neitherworldwide,norinAustria.Tothisend,wewillfacilitateaseriesofdesignworkshopswithstakeholdersatthestudysitetodevelopthecomponentsofthepracticeapproachesbasedontheresultsfromthescopingphase.

Practicaleffortstoinitiatethepracticeapproachesarecentraltotheinstallationphaseofimplementationandincludeactivitiessuchas:de-velopingthecompetenceandconfidenceofstaffthroughtrainingandcoachinginthenewapproach,aswellasmonitoringprogressthroughregularcheck-insandsupervisionofstaffatstudysites.Thetrainingpro-tocoldevelopedintheworkshopswillincludethetheoreticalbasisandunderlying values of the programme, use adult learning theory, intro-ducecomponentsandrationalesofkeypractices,provideopportunitiestopracticenewskills tomeet fidelitycriteria,and receive feedback inasafeandsupportivetrainingenvironment.Thelengthoftrainingwillbedeterminedbytheextentofchangetotheexistingprogrammeandpracticemodel,but typically the face-to-facecomponentwill runovertwodays.Asignificantactivityistosupporteachsiteinusingthenewpractice approaches, as well as the research protocols. Champions ofchange will be identified during the workshops. These professionals,“Villagefacilitators”,willbetrainedandsupportedtofacilitatetheformalandinformalchild-focusedsupport.Oncethenewpracticeapproachesandassociatedsupportivesystemsarebeingused,strategiestopromotecontinuousimprovementandrapid-cycleproblemsolvingwillbeapplied.Theresearchteamwillworkwiththestudysitestousedatatoassessimplementationprogress,identifybarriers,potentialsolutions,anddrivedecision-making.

An additional feature of this project is the central focus ofunderstanding and listening to the ‘child voice’. COPMI support inadult focusedserviceshasso farbeenmostlyparent-centred,andnotlikely to identify or develop an evidence-informed support plan thatmeets the needs and listens to the ‘voice’ of the child. Incorporatingthechild’svoiceinpracticeapproachesislikelytocontributepositivelyto better outcomes, but this knowledge has not yet been developed.The importance of ‘assent’ and supporting children to develop theirown ‘voice’ in healthcare is becoming increasingly recognised withinthe broader field of child health research. This follows the “UnitedNationsConventionon theRightsof theChild” (UnitedNations.Gen-eral,1989),acknowledgingtheethicalimperativeandrightsforchildrento be provided with their own health information. Although researchin healthcare communication is increasingly recognised as importantin improving health outcomes, in both the areas of mental healthand paediatrics, rigorous research investigating naturally occurringhealthcare interactions involving the child is extremely limited. Inparticular,goodhealthcareservicedeliveryisdependentuponclearandopencommunicationbetweenpatientsandtheirtreatingteam.Improvingcommunicationwithinhealthcareencounterscanreducemedicalerrors,andactasatherapeuticlevertosupportpatientempowerment(RoterandHall,2006).Limitedresearchhasshownthatchildrenretainsomeinformation better than their parents, and an increased proportion of

doctor-childcommunicationcomparedtodoctor-parentcommunicationcan increase parental satisfaction (Pantell, Stewart, Dias, Wells andRoss,1982).ForCOPMI,thesechildrenmayalsonothavethesupportoftheirparentsinhealthcareinteractions.Consequently,supportinghealthprofessional-child communication could be argued to be even moreimportantwithCOPMItoensurechildren’sconcerns,needs,andwishesarediscussed.Thisprojectprovidesauniqueandvaluableopportunityto investigate children’s perspectives and interactive capacity withintheCOPMIsetting,andtoobservechangesover time, inparallelwiththe broader interventions of this project. This project will importantlyaddressknowledgegapsinthisareaanddrivepracticechange.Evidencecollectedduringtheinvestigationofthe‘childvoice’willcontributetotrainingapproachesand informthedesignofpracticechangeswithinthebroaderproject.

EMPOWERMENT OF COMMUNITY AND LEADERSHIP

Toempowerpatients,familymembersandthewiderpublictoengageinresearch,LBGoffersapublictrainingprogramme“SCIENCE4YOUTH”6

that was launched in September 2018 addressing adolescents andyoungpeoplewithlivedexperience.Thisprogrammeaimstotrainado-lescentsscientificprinciplesandmethods inorder toworkasaco-re-searcherinresearchgroupsandteams.Inaflipped-classroomapproach(Moffett,2015)withinteractivevideotutorialsandquizzes,participantslearnabouttheresearchprocess,howtoapplyopeninnovationinsci-ence(OIS)methods,developtheirownresearchprojectsandapplytheirnewlygainedknowledgeworkingtogetherwiththe“ResearchGroups”(internships).NineteenadolescentsformhighschoolsacrossAustriaap-pliedfortheprogramme,thereof16females,thatarementoredbypreand post doc researchers. Each mentor supports two mentees duringthewholeprogrammeanddevelopmentoftheirownresearchprojects.Additionally,menteesaresupportedbyabuddysystem,eachadolescentworkintandemwithapeer.Withthismutuallearningapproach,poten-tiallynewinsightsonhowtoactivelyinvolvethecommunityinresearchwillbeestablishedandtherelationshipbetweenyoungpeopleandre-searcherswillbestrengthened.Theseactivitiesaimtoempoweryouth,inordertoestablishyouthleadershipinmentalhealthanddevelopyouthpartnerships with the government to make informed health decisionsandberepresentedinnationaldecision-makingboardsdrawingontheirexperienceandexpertise.

COLLECTIVE IMPACT AND COMMUNITY ENGAGEMENT

Avarietyofroutescanbeappliedtocreatechangeandimpactformentalhealthpracticeandpoliciesforchildrenandadolescents.Besi-desprovidingrigorousscientificevidenceandsystematicallyincreasingcompetencesofindividuals,itiscriticaltostrengthenadvocacyinordertoraiseawareness,identifyandconnectadvocatesandfosterdecision-

6 www.ois.lbg.ac.at/en/methods-projects/science4youth

ISSUE 48 | JULY 2019120

makingprocessestosteerpoliticalchangeonmanylevels.

ADVOCACYRaisingawarenessforthetopicthroughmulti-channelbroadcasting

activitiesisoneofthemainpillarstoincreasethepotentialforsuccess-fuladvocacy.Thiscanbedonethroughavarietyofmethods;however,crowdfunding isoneof themainmethodsapplied in thiscontext thatallowssimultaneouslyraisingmoneyandawareness.Inordertocreatesuccessfulcrowdsourcingandcrowdfundingcampaigns,itisabsolutelynecessarytotransformscientificmessagesintocommonlyunderstanda-blelanguagewithaclearscopeandprecisecalltoaction.Thisapproachwillidentifyindividualswhohavenotbeenawareofthetopicbeforeandreachindividualswhoarewillingtosupporttheimplementation.Addi-tionally,crowdsourcinghelpsresearchersandpractitionerstoreflectontheirownworkandallowsfornewstructuresandapproachestoemerge.Raisingmoneyandawareness isacomplementaryeffort thatwillun-derpinthebasistostrengthenandencourageadvocatesasafirstinitialstep. Furthermore, creating awareness will lead to the representationofpatientsandpeoplewith(lived)expertiseindecision-makingboardsthatinfluenceprioritysetting,makingthetopicmorepressingandthus,relevantforpoliticalagendasettinganddecision-making.

Wewillfosternewwaysofcollaborationandstructuresamongsta-keholdersthatallowacross-disciplinaryexchangeofpracticeandexperi-ence.Additionally,possibleawarenesscampaignsinschoolswillinformandactivatestudentsandtheirfamiliestofindpeersupportprovidingself-helpgroupsforCOPMI,professionalsupportandreferraltospecificnetworks. Further, we will engage with policy makers in our researchactivities early in the process to present evidence-based practice andstrategiestoupscaletheprojectincludingpeoplewithlived-experienceintheexchange.

VALUING COMMUNITY CONTRIBUTIONS

Engaging thegeneralpublic in research iscrucial todrivepracticechangetotacklesociallyrelevantchallenges.However,itisequallyim-portanttovaluethecommunity’scontributionsandactonalevelplayingfieldtofostersustainedengagementandcollectiveimpact.Weenvisioncapacitybuildingactivities thatwill be rewarded tomaintainpeople’sowndevelopment.Forexample,wewillprovidepublicspacetoinformandfosterdiscussionaboutmentalhealthbetweenthepublic, resear-chers and people with lived-experience, create a peer network wherepeoplewithlived-experiencesharetheirexpertise,traininterestedpeo-pleinresearchprinciplesandpublicengagement,andfostercommunityownershipbyconductingyouth/community-ledresearchinitiativesandprojects.Closecollaborationwithstakeholderswillbecrucialtosuccess-fullydrivetheseactivities.Aninitialstrategytothisendhasalreadybeeninitiatedthroughthedevelopmentofanonlinediscussionforumhostedonthe“Villageproject’s”website7.These initiativesmaybesupported

byadditionalgovernmentalfunding,cooperationwithindustryanddo-nations.

In conclusion, creating evidence-based practice, using a collectiveimpactframeworkandcommunityengagement,willfosterasustainableimpactonchildrenandadolescentstotrulydrivesystemchange.Theseactivitieswillbuildcapacitywithinacommunity,nationalandEuropeanlevelraisingawarenessofpolicy-makersoncurrentchallengesinmentalhealth.Nevertheless,advocatingforchangeonacommunityandpoli-cyleveliskeyforsuccessfulimplementationofsystemchangetherebyvaluingcommunities’contributionanddevelopmentinmentalhealth.

REFERENCES Christiansen, H., Bauer, A., Fatima, B., Goodyear, M., Lund, I. O., Zechmeister-Koss, I. and Paul, J. L.(submitted).Howtoraisethevillagetoraisethechild.Protocolofapracticeapproachtoimproveoutcomesforchildrenofparentswithamentalillness.Frontiers in Psychiatry.

European Commission (2013).EUR 25271 — The Grand Challenge — The design and societal impact of Horizon 2020.RetrievedfromLuxem-bourg:PublicationsOfficeoftheEuropeanUnion

Knapp, M., Bauer, A., Perkins, M. and Snell, T.(2013).Buildingcom-munitycapitalinsocialcare:isthereaneconomiccase?Community De-velopment Journal,, 48(2,1),313-331.

Maxwell, K. and Benneworth, P.(2018).Theconstructionofnewscien-tificnormsforsolvingGrandChallenges.Palgrave Communications, 4(1),52.

Moffett, J.(2015).Twelvetipsfor“flipping”theclassroom.Med Teach, 37(4),331-336.

Nicholson, J.(2009).Buildingtheevidencebaseforfamilieslivingwithparental mental illness. Australian e-Journal for the Advancement of Mental Health (AeJAMH), 8(3),222-226.

Pantell, R. H., Stewart, T. J., Dias, J. K., Wells, P. and Ross, A. W.(1982).Physiciancommunicationwithchildrenandparents.Pediatrics, 70(3),396-402.

Roter, D., and Hall, J.(2006).Doctors Talking with Patients/Patients Talk-ing with Doctors: Improving Communication in Medical Visits, SecondEdi-tion:SantaBarbara:ABC-CLIO.

Sauermann, H. and Franzoni, C.(2015).Crowdscienceusercontribu-tionpatternsand their implications.Proc Natl Acad Sci U S A, 112(3),679-684.

United Nations. General, A. (1989). Convention on the Rights of theChild,20November1989.Annu Rev Popul Law, 16,95,485-501.Wistow, G., Perkins, M., Knapp, M., Bauer, A. and Bonin, E.-M.

7 www.village.lbg.ac.at/getinvolved

ISSUE 48 | JULY 2019 121

(2016).CirclesofSupportandpersonalization:Exploring theeconomiccase.J Intellect Disabil., 20,194-207.

Zechmeister-Koss, I. and Goodyear, M. (2018). Supporting childrenwho have mentally ill parents in Tyrol: A mapping of existing supportstructures and epidemiological dimensions. LBI-HTA Projektbericht Nr.113a,Wien: Ludwig Boltzmann Institut für Health Technology Assessment.

AUTHORSRAPHAELA E. KAISLEROpen Innovation in Science Center, Ludwig Boltzmann Gesellschaft, Nussdorferstrasse64,Vienna,1090(Austria)E:[email protected]

JEAN L. PAULResearch Group Village, Ludwig Boltzmann GesellschaftAnichstrasse35/2,Innsbruck,6020(Austria)

Department of Psychiatry, Psychotherapy and Psychosomatics, Division of Psychiatry I, Medical University of InnsbruckVinzenzgebäude–Haus11/2.Stock,Anichstraße35,Innsbruck,6020(Austria)

E:[email protected]

ISSUE 48 | JULY 2019122

ges facing Europe.”2Alsointhe“Fifth”,“Sixth”and“SeventhFrameworkProgrammes”collaborationwithcivilsocietywasvalued.

With Horizon 2020 (FP8), the involvement of stakeholders and thediscussionsonsocietal impactof researchprojects increased.Yet, theinterimevaluationofHorizon2020(H2020)showedthatonemainareaforimprovementisbringingresultstocitizensandinvolvingthemmore.“There is a need for greater outreach to civil society to better explain re-sults and impacts and the contribution that research and innovation can make to tackling societal challenges, and to involve them better in the pro-gramme co-design (agenda-setting) and its implementation (co-creation).”(EC,DGRTD2017a,p.21).Also,thereportfromthe“High Level Group on maximising the impact of EU Research & Innovation Programmes”callsformobilisingand involvingcitizens throughco-designandco-creationofprogrammesandprojectsatEuropean,nationalandregionallevels(EC,DGRTD2017b).

Consequently,HorizonEurope(FP9)willdemandevenfurthercitizeninvolvement3.However,itisoftenforgottenthatcitizensspeakdifferentlanguagesandsufficientfundingneedstobeavailableforinterpretationandtranslations.Policyreviews,publishedbytheDirectorate-GeneralforResearch and Innovation, provide tools and analysis to policy makers,buttheyareoftennotwideenoughdistributedanddiscussedandonlyavailableinEnglish.

Closecollaborationwithstakeholderscouldbeonewayofinvolvingcitizens.Inthisregard,itisimportanttorememberthatpeopleworkingwithresearchpolicies,programmesandprojectsarecitizens,aswell.

Toincreasecitizens’involvementinHorizonEurope,acriticalreflec-tiononstakeholderinvolvementinH2020projectsandadiscussionontoolsforachievingsocietalimpactisnecessary.Here,societalimpactis

ABSTRACT

Projects funded by “Framework Programmes for Research andInnovation”shouldincreasinglyinvolvecitizensandcreatesocie-tal impact. Inparticular,SocialSciencesandHumanities (SSH)

researchersfocusonsocietalchallengesandcollaboratewithdifferentstakeholders.Theinvolvementofstakeholdersinproposaldevelopmentand project implementation is needed to secure citizens involvement.However,successfulstakeholderinvolvementinproposalsandprojectsis time consuming and needs a strong management structure, whichshould support and monitor activities, which can lead to societal im-pacts.Furthermore,Horizon2020andHorizonEuropeindicatorsshouldplacegreaterimportanceontheinvolvementofstakeholders.

1. BACKGROUND AND PURPOSE Closecollaborationwith stakeholdershasbeenademand for “Eu-

ropean Framework Programme” projects for many years. The “FourthFrameworkProgramme”(FP4,1994–1998)alreadycontainedaspecificprogrammecalled“TargetedSocio-EconomicResearch”(TSER).TheTSERprogrammeencouragedtheinvolvementofstakeholdersinresearchpro-jectstoachieveabetteruptakeofprojectresultsbypolicymakersandcivilsociety:“In line with the Commission’s White Paper on Growth, Com-petitiveness and Employment, the research activities aim at rationalising future decision-making at decentralised, national or Community levels in order to develop a shared knowledge base on the socio-economic challen-

BETTINAUHRIGDOI:10.22163/fteval.2019.378

IMPACTOFSOCIALSCIENCESANDHUMANITIESFORAEUROPEANRESEARCHAGENDA–VALUATIONOFSSHINMISSION-ORIENTEDRESEARCH1

RETHINKINGSOCIETALIMPACT–COLLABORATIONWITHSTAKEHOLDERS

1 TheauthoracknowledgestheSTI2018Leidenconference,fromwhichthistemplatewasadapted.2 FP4-TSER–Specificprogrammeoftargetedsocio-economicresearch,1994-1998.RetrievedOctober25,2018from:https://cordis.europa.eu/programme/

rcn/465_en.html.3 Information on Horizon Europe: https://ec.europa.eu/info/designing-next-research-and-innovation-framework-programme/what-shapes-next-framework-

programme_en.

ISSUE 48 | JULY 2019 123

searchmethods,aswellascommunicationanddisseminationactivities.Developing the proposal and implementing the project becomes evenmore time-consuming,butbrings theproposaland theprojectscloserto theexpected impactsdescribed in thecall topicandenhances thepossibilitiesfortheuptakeofresearchoutcomesbystakeholders.

2.2 INVOLVEMENT OF STAKEHOLDERS AT DIFFERENT LEVELS

Inadditionto involvingCSOsasfullconsortiummembers,wecon-tacted possible members for an international or European advisorygroupsupportingtheimplementationoftheprojectalreadyduringtheproposalphase.Thememberscamefromacademia,public,privateandsocialpartnerorganisations,industryorCSOs.Someofthemreviewedtheproposalsbeforesubmissionand in thiswaycontributed toexcel-lentproposals.Ifaproposalwasapproved,membersfromtheadvisorygroupwereinvolvedintheimplementationoftheproject,forexampleindiscussionsofmethodsandresearchquestionsandinsupportingdisse-minationactivities. Inmostcases, theprojectcovered their travelandhotelcoststoattendprojectmeetings(max.twiceayear),butdidnotfinanceanyworkingtime.TheselimitedfundingoptionsmakeitdifficulttoconvincepeopleworkingatCSOs to joinadvisorygroupsatprojectlevel.TheirinvolvementneedstobeapprovedbytheirboardsandmanyboardmembersanddirectorsofCSOswouldliketoseesomefinancialcompensationfortheirinvolvement,whichmakesitlesslikelyforthemtoapprovesuchinvolvement.

Anevenmoreimportanttoolforstakeholder involvementhasbeentheset-upofstakeholdergroupsorcommitteesatnational levels.Themembersofthesegroupscanagaincomefromacademia,public,privateorsocialpartnerorganisations,industryandCSOs.Theirinvolvementinprojectshascontributedtomorepublicityoftheresearchprojects.Groupmembershavenotonlysupporteddisseminationactions;theyhavealsohelped in finding interviewees and drafting “Policy Briefs” describingresearch findings relevant for stakeholders5. “PolicyBriefs”, translatedintonationallanguages,havebeenveryusefulfortheworkofCSOs.Inallprojects,somemembersfromthenationalstakeholdergroupswerealso invited to project conferences. In projects, coordinated by NOVA,nationalstakeholdergroupshavenotreceivedanyfunding,onlytravelcostsand,ifnecessary,translationswerecoveredbytheprojectbudget.

Furthermore,stakeholderscanbe involved in the researchprojectsthough different activities, like advocacy meetings, focus groups andthematicworkshops6.

2.3 DEDICATED IMPACT MANAGEMENT

definedas“social improvements e.g. via the use of project results by poli-cy makers or other societal actors”(Net4Society2017,nopagenumber).Such‘use’oftenhappensaftertheendoftheprojectandisveryoftennotpartoftheprojectevaluation.Forexample,“ASIRPA(AsianSocietyforInternationalRelationsandPublicAffairs) found that the average time lag for impact that comes from applied research was 19.9 years. For fun-damental research, much longer time lags are needed.”(ScienceEurope2017,p.17).

2. METHODS AND ACTIONS TO INCREASE STAKEHOLDER INVOLVEMENT

Thefollowingmethodsandactionshavebeendevelopedandusedbytheauthorandcolleaguesfordraftingproposalsand implementingprojects since FP4. The focus is on proposal development and projectimplementation.

2.1 STAKEHOLDER INVOLVEMENT FROM THE BEGIN-NING AND “CIVIL SOCIETY ORGANISATIONS” AS CON-SORTIUM MEMBERS

Successful proposals on call topics in H2020 Societal Challenge 6“Europe ina changingworld– Inclusive, innovativeand reflective so-cieties”containacleardescriptionofimpact.Theyoutlinetheproject’scontribution to the scientific/academic impact, societal (incl. political)impact and economic impact. In order to develop a project proposalthatconvincesevaluatorsand,atthesametime,isfeasible,itisvitaltoinvolvestakeholders from thebeginningof theproposaldevelopment.Discussionswithrepresentativesfromorganisations,whichshouldworkwiththeresearchresults,areneededtodeveloptheresearchquestions,the concept and the work packages to produce the promised outputsandtocontributetotheexpectedimpacts,whicharedescribedinthecalltopictext.

Insuccessfulresearchproposals,submittedbytheNorwegianSocialSciencesresearchinstitute(NOVA)4,“CivilSocietyOrganisations”(CSOs)wereinvolvedintheprojectdesignandhavebeenmembersinthepro-ject consortium from the beginning of the project. Since H2020, theycanbeaprojectpartnerunderthesamefinancialconditionsashighereducation institutionsandresearchorganisations. Inmostcases, theirinvolvementdemandsmoreopennessandleadstomorediscussionsdu-ringtheproposalprocessandtheprojectimplementation.Differentwaysofworkingneedtobediscussedandagreedupon.This influencesre-

4 Since2007,theauthorhasbeenemployedatNOVA,whichmergedwiththeOsloandAkershusUniversityCollegeofAppliedSciences(HiOA)in2014.In2018,HiOAwasgrantedthestatusofauniversityandchangeditsnametoOsloMetropolitanUniversity(OsloMet).

5 ExamplesforPolicyBriefsfromH2020SSHprojectscanbeviewedat http://ec.europa.eu/research/social-sciences/index.cfm?pg=library&lib=policy_briefs.6 The H2020 project DANDELION – “Promoting EU funded projects of inclusive, innovative and reflective societies” described several different tools for

disseminationandstakeholderinvolvement,http://www.dandelion-europe.eu/en/infobase/guides-to-maximise-impact-of-ssh-projects/guides-to-maximise-impact-of-ssh-projects1.html,viewedOctober27,2018.

ISSUE 48 | JULY 2019124

Since2016,weinclude,animpactmanagerintheimplementationoftheH2020researchprojectsduetothemanyprojectmanagementtasks.We found that it works best if it is already clear during the proposalphasewhowillhavethispositionincasetheproposalisapproved.Ourexperiencesshowthattheinvolvementofanimpactmanagercaneasethe communication between the consortium members and leads to astrongerfocusonachievingimpact.

2.4 CASE STUDY DARE - TOOLS FOR ACHIEVING SO-CIETAL IMPACT

IntheongoingH2020project“DARE”(DialogueAboutRadicalisationandEquality)7,theimpactmanagerhasbeeninvolvedfromthebeginningoftheproposalprocess,whichstartedinthesummerof2016.Togetherwith the coordinator, the impact manager invited CSOs to the projectduringtheproposaldevelopment.Thisaffectedtheprojectdescriptionandimplementationin,amongothers,thefollowingthreeways:

1. The“PlanforExploitationandDisseminationofResults”(PEDR)is very detailed and specific. In the proposal, we already in-cludedadetailedplandescribingdisseminationandexploita-tionactivitiesineachworkpackage,thetargetaudiencesandusers,aswellasrelatedoutputandimpactmeasures.ThePEDRisregularlyupdatedthroughouttheprojectduration(May2017–April2021).

2. The management structure contains an “Impact Sub-Commit-tee” (ISC), which supports and monitors the dissemination,exploitationandimpactactivitiesandischairedbytheimpactmanager. The ISC meets regularly online and approximatelythreetimesface-to-faceeachyear.TheISCalsowritesinternalimpactreportseveryninemonths.

3. By October 2018, nearly all consortium members had estab-lished“NationalStakeholderGroups”(NSGs),withwhomtheydiscussthedevelopmentoftheprojectandwhichtheyinvolveindisseminationactivities. For theDAREconsortium, it is im-portantthatallDAREpartnerscreatetheNSGtheyrequireandmeetwiththeirNSGtoreflectontheirtasksinDAREandtheirnational context when it fits (approximately two times eachyear).ThetypesofstakeholdersandsizeoftheNSGthereforediffer,withmostNSGshavingbetweensixand12members.AllpartnerswriteminutesoftheirNSGmeetings,whichareavail-ableforallconsortiummembersandwhichareaveryimportantresourcefortheimpactmanagementandmonitoring.

InApril2018,theprojectpublisheditsfirst“PolicyBrief”writtenbymembersoftheISCandthecoordinator.Duringthethirdprojectmee-ting inMay2018, the ISCorganisedan impactworkshopforallDAREcolleaguesdiscussingtheirexperiences,questionsandideasrelatedtoworkingwithsocietalimpact.Alreadynow,itisevidentthattheinvolve-mentofanimpactmanagerandan“ImpactSub-Committee”hascreatedastrongerfocusonimpactforallconsortiummembers.

3. FINDINGS AND

RECOMMENDATIONS3.1 IMPACT MANAGEMENT AT ORGANISATION LEVEL

ThegoodpracticeexampleresultingfromDAREleadstothequesti-onofwhetherimpactmanagementshouldalsohaveamoreprominentrole at organisation level: Could an impact manager, employed at themanagementlevelofanorganisation,easeandenhancethecollabora-tionwithcitizens,stakeholderinvolvementinprojectsandtheuptakeofresearchresultsbyindividuals,organisationsandinstitutions?

Severaluniversities,especiallyintheUK,alreadyemployimpactma-nagers.Amongothertasks,theysupportandcollectthedescriptionsofimpactcasestudies.Excellentimpactcasestudiescanleadtoadditionalfundingbynationalauthorities8.Impactcasestudiesareusedforcolla-borationwiththemediaandenhancethecommunicationwithcitizens.Ofcourse,thecreationofanimpactmanager’spositionrequiresfurtherpersonnel resources.Establishing impactmanagementatorganisationlevelwouldhelptoadvancetheprojectoutcomesaftertheendoftheprojectandwould furthermoregive timeand resources for impactas-sessments.

3.2 REVISED INDICATORS FOR SOCIETAL IMPACT

Involvement of stakeholders in research projects should count notonlyforevaluatorsdealingwithproposalsbutalsofortheoverallevalua-tionofresearchprojectsandtheprogrammeevaluation.

ApublicdebateonrevisedindicatorsforHorizonEurope(EC,DGRTD2015)isthereforeneeded.Theorientationonthe“TechnologyReadinessLevels”(TRLs)ofaprojectneedstobequestionedandbroadened.Formeasuringsocietalimpact,alongertimeframeaftertheendofaprojectisneeded,and,insteadofTRLs,programmeevaluatorsanddeveloperscouldconsiderthe“SocietalReadinessLevels”(SRLs)ofaproposalandproject.CooperationwithstakeholderscouldbeoneindicatorforsocietalimpactandbeincludedinthedescriptionoftheSRLs.Thisisreflectedbythe“InnovationFundDenmark”,whichhaspublishedadescriptionofSRLs.SRLsarealreadyconsideredforthedevelopmentofindicatorsforHorizonEurope(EC,DGRTD,2018a).Table1belowexaminesthediffe-rencesbetweenTRLsandSRLs.ItdemonstratestherelevanceofSRLswhenmeasuringsocietalimpact.

Levels TRLi SRLii

7 DAREhasreceivedfundingfromtheEuropeanUnion’sHorizon2020researchandinnovationprogrammeundergrantagreementNo725349.http://www.dare-h2020.org/.

8 ThisisforexamplethecaseinUK,where“HigherEducationInstitutions”canreceiveadditionalstatefundingbasedontheirimpactcases.Furtherinforma-tioncanbefoundonthewebsiteofthe“ResearchExcellenceFramework”:http://www.ref.ac.uk/.

ISSUE 48 | JULY 2019 125

1 Basicprinciplesobserved. Identifyingproblemandidentifyingsocietalreadiness.

2 Technologyconceptformulated. Formulationofproblem,proposedsolution(s)andpotentialimpact,expectedsocietalreadiness;identifyingrelevantstakeholdersfortheproject.

3 Experimentalproofofconceptprovided. Initialtestingofproposedsolution(s)togetherwithrelevantstakeholders

4 Technologyvalidatedinlab. Problemvalidatedthroughpilottestinginrelevantenvironmenttosubstantiateproposedimpactandsocietalreadiness.

5 Technologyvalidatedinrelevantenvironment. Proposedsolution(s)validated,nowbyrelevantstakeholdersinthearea.

6 Technologydemonstratedinrelevantenvironment.

Solution(s)demonstratedinrelevantenvironmentandincočoperationwithrelevantstakeholderstogaininitialfeedbackonpotentialimpact.

7 Systemprototypedemonstratedinoperationalenvironment. Refinementofprojectand/orsolutionand,ifneeded,retestingintherelevantenvironmentwithrelevantstakeholders.

8 Systemcompleteandqualified. Proposedsolution(s)aswellasaplanforsocietaladaptation.

9 Actualsystemproveninoperationalenvironment. Actualprojectsolution(s)proveninrelevantenvironment.

SRLsandstakeholder involvementshouldbe linked to the“United

NationsSustainableDevelopmentGoals” (SDGs). Inparticular,SDG17‘Strengthenthemeansofimplementationandrevitalisetheglobalpart-nershipforsustainabledevelopmentthroughcapacitybuilding’couldbestudiedtoimprovestakeholderinvolvement.9

3.3 FUNDING FOR COMMUNICATION, DISSEMI-NATION AND IMPACT MANAGEMENT AFTER THE END OF A PROJECT

Tosecurethefocusonprojectresults,future“FrameworkProgram-me”projectsshouldreceiveadditionalfundingaftertheendofthepro-ject tocontinuewithcommunicationanddisseminationactivities (seealso3.1),whichcanleadtosocietalimpacts.TheinterimevaluationofH2020madeclearthat“the projected social and economic impacts, for example on the creation of spin offs, on employment or the development of new innovation, are difficult to measure (in terms of causality with the pro-jects financed), in particular because they might happen at a point beyond the lifetime of the project. This needs to be taken into account in future impact evaluations. It is also difficult to predict if stakeholder collaboration across different types of organisations will last beyond the duration of the projects.”(EC2017,page969).

i SeeEC,DGRTD,2018a,page10.ii “Innovation Fund Denmark” (n.d.). Societal Readiness Levels (SRL) defined according to Innovation Fund Denmark, Copenhagen.

RetrievedOctober27,2018from:https://innovationsfonden.dk/sites/default/files/2018-08/societal_readiness_levels_-_srl.pdf.

Table 1.ComparisonofTRLsandSRLs.

3.4 COLLABORATION WITH STAKEHOLDERS AND CITIZENS’ INVOLVEMENT

Citizens’involvement,asdemandedbythemembersofthe“High Le-vel Group on maximising the impact of EU Research & Innovation Program-mes”(EC,DGRTD2017b),shouldbediscussedindetail.Studiespublis-hedbytheEuropeanCommission(EC)andacademicnetworksdescribemanydifferentpossibilitiesinvolvingcitizensinEUpoliciesandresearchprogrammes (Van den Brande 2017; Science Europe 2018; CIMULACT2018).Collaborationwithstakeholders,asdescribedabove,createsse-veralpossibilities forcitizens’ involvement.HowthiscanbeorganisedcouldbediscussedwithCSOs,whichhaveexperienceswithFPprojects.

Furthermore,andbearinginmindtheriseofpopulism,itisimportanttoreflectonthechallengescreatedbycitizens’involvement.Itcouldbeimportanttoagreeonjointvaluesbeforestartinganyformofcollabora-tion.Here, itcouldbeusefultorefertothefundamentalvaluesoftheEuropean Union and the Council of Europe: human rights, democracyandtheruleoflaw.

Eventhough,theECorganisedworkshopswithstakeholdersandim-plemented stakeholder consultations (EC 2018), Horizon Europe is notwellknownbyregionalandnationalCSOs.Tochangethis,Net4Society10could,inclosecollaborationwithforexampleSDGWatchEurope11andtheDirectorate-General forResearchand Innovation,organise “FutureSearchConferences”(Weisbord,M.andJanoff,S.1999)involvingCSOs

9 ForcloserinformationonSGDssee:https://sustainabledevelopment.un.org/topics/capacity-building.Furthermore,theH2020projectDANDELIONcontrib-utedtothediscussionofSRLs(Dandelionn.d.)andinJune2018DGRTDpublishedadetaileddescriptionofkeysocietalimpactpathwaysandprogressindicators(EC,DGRTD,2018a).

10 Net4SocietyistheInternationalnetworkofNationalContactPoints(NCPs)forSocietalChallenge6inHorizon2020,http://www.net4society.eu/.11 SDGWatchEuropeisaEuropeancross-sectoralcivilsocietyallianceadvocatingfortheimplementationoftheSDGs,https://www.sdgwatcheurope.org/

about-us/.

ISSUE 48 | JULY 2019126

European Commission, Directorate-General for Research and Inno-vation (EC,DGRTD).(2018b).Mission-oriented research and innovation - Inventory and characterisation of initiatives.Luxembourg:PublicationsOfficeoftheEuropeanUnion.

Mazzucato, M. (2018).Mission Oriented Research & Innovation in the European Union.Luxembourg:PublicationsOfficeoftheEuropeanUnion.

Net4Society(2017).Increasing Impact! RetrievedOctober27,2018from:http://www.net4society.eu/_media/Net4Society4_D3_1_1_Factsheet_Impact_final.pdf.

Science Europe (2017).Symposium Report – Building a Scientific Narra-tive on Impact and Societal Value of Science. Brussels.

Science Europe(2018).Briefing paper on Citizens Science.Brussels.

Van den Brande, L. (2017).Reaching out to the citizens: A new oppor-tunity ‘About us, with us, for us’. Luxembourg:PublicationsOfficeoftheEuropeanUnion.

Weisbord, M. and Janoff, S.(1999).Future Search – Collaborating for Change.SanFrancisco.

AUTHORBETTINA UHRIGNorwegian Social Research (NOVA), Oslo Metropolitan UniversityStensberggata26,Oslo,0170(Norway)E:[email protected]

andotherstakeholders todiscussthemainsocietalchallenges,whichwillbeimportantforthedesignandimplementationofHorizonEurope.PublicengagementhasbecomeoneofthemaindemandsfordevelopingmissionsinHorizonEuropeandmissionsshouldhavesocietalrelevance(Mazzucato2018;EC,DGRTD2018b).

4. CONCLUSIONSRethinking collaboration with stakeholders in H2020 research pro-

jectsandlinkingittocitizens’engagementinHorizonEurope,inparticu-larinmissionsandprojectsfundedundertheGlobalChallenges,couldbeusefulforwideningthediscussionsonthedesignandimplementa-tionofHorizonEuropeand the revisionof indicators.Professionalandclearlydefined impactmanagementcouldease thecollaborationwithstakeholdersandtheworkwithproposals,projectsandprojectoutcomestoachievesocietalimpacts.

5. REFERENCES CIMULACT(2018).Inspiration Catalogue for consulting different groups.RetrievedOctober29,2018from:http://www.cimulact.eu/wp-content/uploads/2018/04/D5.1_Inspiration-Catalogue-for-consulting-different-groups-compressed.pdf.

DANDELION (n.d.). Inclusive, innovative and reflective societies-sensiti-ve valorisation concept. Retrieved October 27, 2018 from: http://www.dandelion-europe.eu/imagem/IIRS_sensitive_Valorisation_Concept.pdf.

European Commission (EC). (2017). Commission staff working docu-mentInterimevaluationofHORIZON2020.PartO.Europeinachangingworld–InclusiveInnovativeandReflectiveSocieties.Brussels.RetrievedOctober 27, 2018 from: https://ec.europa.eu/research/evaluations/pdf/archive/h2020_evaluations/swd(2017)221-annex-2-interim_evaluation-h2020.pdf#view=fit&pagemode=none.

European Commission(EC).(2018).Horizon Europe Stakeholder Consul-tation. Synopsis Report.Brussels.

European Commission, Directorate-General for Research and In-novation (EC,DGRTD). (2015).Horizon 2020 indicators - Assessing the results and impact of Horizon. Luxembourg: Publications Office of theEuropeanUnion.

European Commission, Directorate-General for Research and Inno-vation (EC,DGRTD).(2017a).Key findings from the HORIZON 2020 inte-rim evaluation.Luxembourg:PublicationsOfficeoftheEuropeanUnion.

European Commission, Directorate-General for Research and Innova-tion (EC,DGRTD).(2017b).LAB – FAB – APP - Investing in the European Future we want.Luxembourg:PublicationsOfficeoftheEuropeanUnion.

European Commission, Directorate-General for Research and Inno-vation (EC,DGRTD). (2018a).A NEW HORIZON FOR EUROPE - Impact Assessment of the 9th EU Framework Programme for Research and Inno-vation. Luxembourg:PublicationsOfficeoftheEuropeanUnion.

ISSUE 48 | JULY 2019 127

Bynow,theFPisundoubtedlyoneofthelargestfundinginstrumentsfortheEuropeanSSHscholarshipsthroughitsvariousinstruments.Re-searchonimpactandperformanceofSSHinFPshavebeenmainlythetaskofexpertgroupssetupbytheEuropeanCommission(Watson,J.,et al., 2010, Hetel, L., et al., 2015, Birnbaum, B. I., et al., 2017, BadeStrom,T.,etal.,2018,Challis,L.,etal.,2003,Cerletti,C.,etal.,2001.).Inresearchjournals,theapproacheshaveappearedrelativelyscarcely(Ge-orghiou,L.,etal.,2002,Must,Ü.,2010a,2010b,Schindler-Daniels,A.,2014.).Theaimofthispaperistomonitorandanalysetheevolution(oroverlapping)oftheSSHthematicpatternthroughthethreeframeworkprogrammessince2002.

INTRODUCTION

TheEuropeanUnionEU“ResearchandDevelopmentFrameworkProgrammes” (FPs) had been in operation for ten years befo-re socio-economic research was included under the “Fourth

Framework Programme” (1994-1998). It was directly related to the re-sultsoftheMaastrichtTreaty(Reillon,V.,2017). Itwasaperiodwhenthe need for “soft power” arose. Joseph Nye’s (Nye, J., 1990) “softpower”approachadoptedduringthefifthenlargementof theEUwasconsideredtheEU’smostsuccessfulforeignpolicyinstrument(Rehn,O.,2007,Tulmets,E.,2008).Certainly, thisgavean impetustothefurtherdeepeningofthesocialdimensionoftheFrameworkProgramme.Inthesuccessiveframeworksmoreandmoreelementsofthesocialsciencesandhumanities(SSH)researchwereadded(Table1).

ÜLLEMUSTDOI:10.22163/fteval.2019.379

THECHANGINGPATTERNOFSOCIALSCIENCESANDHUMANITIESINTHEEUFRAMEWORKPROGRAMMES1

RETHINKINGSOCIETALIMPACT–COLLABORATIONWITHSTAKEHOLDERS

1 TheauthorsacknowledgetheSTI2018Leidenconference,fromwhichthistemplatewasadapted.

FP Period SSH Work Programme

FP4 1994-1998 Targetedsocio-economicresearch.

FP5 1998-2002 Improvingthesocio-economicknowledgebase.

FP6 2002-2006 Citizensintheknowledge-basedsociety.

FP7 2007-2013 Socio-economicSciencesandHumanities.

H2020 2014-2020 Europeinachangingworld–inclusive,innovativeandreflectivesocieties.

Table 1. EUFrameworkProgrammeswithelementsofSSHresearch.

ISSUE 48 | JULY 2019128

FP Programme Projects Words (abstracts, titles)

Lexical density

FP6SpecificProgramme“IntegratingandStrengtheningtheEuropeanResearchArea”Priority7:CitizensandGovernanceinaknowledge-basedsociety

166 8,415 25,1693

FP7Cooperation.Theme8:Socio-economicSciencesandHumanities.

255 13,532 20,2335

HORIZON2020SocietalChallenge6.Europeinachangingworld–inclusive,innovativeandreflectivesocieties(asofMarch2018)

277 14,520 22,3898

Table 3.FPSSHfundedprojectsinFP6,FP7andH2020.

Sincethegoalwastomonitorsubstantivechangesacrossframeworkprogrammes,wecleanedthedataofpunctuationmarks,numericvalues,articles(a,the),prepositions(on,at,in),conjunctions(and,or,but)andauxiliaryverbs,suchas“to be” (am, are, is, was, were, being), “do” (did, does, doing), “have” (had, has, having).

Thefinalanalysisandcomparisonsbetweendifferentdatasetsweremadeonthebasisofthe200mostusedwords2.

RESULTSLEXICAL DENSITY

Lexicaldensity is the termmostoftenused fordescribing thepro-portionof contentwords (nouns, verbs,adjectives,andoftenalsoad-verbs)tothetotalnumberofwords.Byinvestigatingthis,wereceiveanotionof Information packaging;atextwithahighproportionofcontentwordscontainsmoreinformationthanatextwithahighproportionoffunctionwords(prepositions,interjections,pronouns,conjunctionsandcountwords).Largemajorityofthespokentextshavealexicaldensityoffewerthan40%,whilealargemajorityofthewrittentextshavealexicaldensityof40%orhigher(Johannson,V.,2008).

AsweseefromFigure1,thelexicaldensityofworkprogrammesofdifferentFPshasdeclinedovertheyearswhileintheabstractsandtitlesofprojectsithasremainedroughlythesameandissignificantlyhigherthaninworkprogrammes.

Figure 1. LexicaldensityofdifferentFPworkprogrammesandprojects.

METHODS

Weused textualanalysis forconducting thesurvey.Thesetofdo-cuments toperform theanalysis isbasedon twosources:a)FPWorkProgrammes2002-2020(Table2).

Work Programme Words Lexical density

FP6SpecificProgramme“IntegratingandStrengtheningtheEuropeanResearchArea”.Priority7:CitizensandGovernanceinaknowledge-basedsociety.WorkProgramme2002-2003.

15,103 12,99

FP6SpecificProgramme“IntegratingandStrengtheningtheEuropeanResearchArea”Priority7:CitizensandGovernanceinaknowledge-basedsocietyWorkProgramme2004–2006.

12,606 13,5174

FP7CooperationWorkProgramme:SSHWorkProgramme2007Cooperation.Theme8:Socio-economicSciencesandHumanities.

20,943 10,3328


20,726 10,3445


15,014 12,9679


21,558 11,2302


22,894 11,8808


26,934 10,5332


26,821 10.6446

HORIZON2020WorkProgramme2014–201513.Europeinachangingworld–inclusive,innovativeandreflectivesocieties

52,043 7,5956

HORIZON2020WorkProgramme2016–201713.Europeinachangingworld–inclusive,innovativeandreflectivesocieties

59,111 8,1711

HORIZON2020WorkProgramme2018–202013.Europeinachangingworld–inclusive,innovativeandreflectivesocietiesi

31,565 9,9034

i The text of the work programmes is still changing. As of March2018,materialshavebeenusedforanalysis.

Table 2. FPSSHWorkProgrammes2002-2020byWordCountandLe-xicalDensity.

Andb)StatisticsonfundedprojectsfromtheCORDISprojectdata-base(=Projects)(https://cordis.europa.eu/).Weusedtitleandabstractwordsfortextualanalysis(Table3).

2 Infact,thenumbervaried.Thereasonisthesameamountofoccurrenceofdifferentwords.

ISSUE 48 | JULY 2019 129

Figure 3.TheoverlappingofwordsinFP6,FP7,andHorizon2020pro-jects.

Whencomparingtwodatasets,wecanseethattheaverageproporti-onbetweenoverlappedanduniquewordsinworkprogrammesisratherbalanced,but in caseofprojectwords the situation isdifferent– themajorityofwordsareunique.Atthesametime,theanalysisofFPpro-jectandworkprogrammestextswithtwooverlapsindicatesthatthereiscontinuitybetweensuccessiveframeworkprogrammes.Forexample,FP6 project words are overlapping with FP7 to an extent of 58.8% (inworkprogrammes57,1%),thewordsofH2020projectsoverlapwithFP7toanextentof55.3%(inworkprogrammeseven65,6%).

Thesubjectwewereexaminingwashowmuchthewordsofworkprogrammesandprojectsoverlap(Figure4).

Figure 4.Theoverlappingofwordsbetweenworkprogrammesandpro-jects.

AsweseefromFigure4,thetextsofworkprogrammesandprojectswerethemostinlineduringthe7thFP(48,1%overlapping),thepictureisdifferentin6thFP(uniquewordsconstitute68,3%),andinH2020(uniquewordsconstitute70,9%).Onthebasisoftheexistingmaterial,itseemsthatinmajoritycasesthereisnooverlapbetweenworkprogrammesandprojecttexts(titlesandabstracts).Wecanonlyassumethattheresultscouldbedifferentiftousethefulltextsoftheproposals.

PATTERN OF WORDS

Weanalysedtowhatextentwordsoverlapintheworkprogrammesofthethreesuccessiveframeworkprogrammesandwhichuniquewordscharacterisespecificprogrammes(Figure2).

Figure 2.TheoverlappingofwordsinFP6,FP7,andHorizon2020workprogrammes.

In20%of thecases thewordsoverlap inall three frameworkpro-grammes.Theseincludewordslikeprogramme, participant, democracy, public, research, Europe.However, thenumberofuniquewords is thesameasoverlappingwords:inFP6andinH202020%,inFP717%.Someexampleofuniquewords:inH2020–business, ICT, inFP7–foresight, emerging, family,inFP6–associated, target, embryonic. Theintroductionofnewwordscanalsobefollowedinworkprogrammes.Forexample,st-artingfromthe“7thFrameworkProgramme”,thecorewordsintroducedcrisis, identity, digital, heritage, reflective, urban.

In caseofwords fromprojects, thegeneraloverlappingoccurs in14% of cases (Figure 3). Partially words overlap with the same themostoverlappingwordsintheworkprogrammes(programme, research, Europe) butinmajoritycasesthewordsaredifferent(human, education, approach, engage). Incaseofprojects,theproportionofuniquewordsismuchbiggerthantheproportionofoverlappingwords:inFP7andinH202021%,inFP626%.

Somecommonlyusedwordschangeovertime.Forexample,whileradio andtelevisionwereamongthemostcommonlyusedwordsinthe“6thFrameworkProgramme”, in theH2020projects these termshavenotoccurredandthemostwidelyusedwordsincludesoftware, digital, online.

Wecanalsomonitorthefrequencyofusageofwordsovertime.Forexample,theterm“innovation”:inthe“6thFrameworkProgramme”,itrankedthe87thbyitsuse,seventhinthe“7thFrameworkProgramme”andsecondintheH2020.

ISSUE 48 | JULY 2019130

Societal Challenges and Industrial Leadership.Luxembourg:PublicationsOfficeoftheEuropeanUnion,2015.46pp.

Johansson, V. (2008).Lexicaldiversityandlexicaldensityinspeechandwriting:adevelopmentalperspective.Lund University, Dept. of Linguistics and PhoneticsWorking Papers 53,61-79.

Must, Ü.(2010a).CollaborationofthesocialscienceresearchersofCen-tralandEasternEuropeancountriesinEuropeanresearchprogrammesduring theperiod1994-2006. In: Internationalisation of Social Sciences in central and Eastern Europe. The “catching up” – a myth or a strategy?London,NewYork,Routledge,99-114.

Must, Ü.(2010b).CollaborationinEUFrameworkProgramme–thecaseofthesocialsciencesandhumanities.Innovation–The European Jour-nal of Social Science Research,23(1),79-83.

Nye, J.(1990).BoundtoLead:TheChangingNatureofAmericanPower. NewYork:BasicBooks.336pp.

Rehn, O. (2007).TheEU’ssoftpowerand thechanging faceofworldpolitics.EuroklubiHelsinki,20April2007.SPEECH/07/245.

Reillon, V. (2017).EUframeworkprogrammesforresearchandinnova-tion.EvolutionandkeydatafromFP1toHorizon2020.EuropeanParlia-mentaryResearchService.38pp.

Schindler-Daniels, A.(2014).ShapingtheHorizon:socialsciencesandhumanitiesintheEUframeworkprogramme“Horizon2020”.ZeitschriftfurErziehungswissenschaft,17(6),179-194.

Smith, J.(2002).ImplementationoftheEuropeanResearchAreaintheSocialandHumanSciences,especiallyasregardsthecoordinationandopening-upofnationalprogrammesDiscussionPaper.Luxembourg:Of-ficeforOfficialPublicationsoftheEuropeanCommunities,78pp.

Tulmets, E. (2008). TheEuropeanUnion:A “SoftPower”withCivilianMeans?Kultura i Polityka,2-3,60-74.

Watson, J., Kitchener, M., Gutheil, M., Ward, B., Zadrozny, T., Ackers, L. and Harrap, K.(2010).EvaluationoftheimpactoftheFrame-workProgrammeontheformationoftheERAinSocialSciencesandtheHumanities(SSH).Brussels:EuropeanUnion,2010,152pp.

AUTHORÜLLE MUSTArchimedes FoundationVäike-Turu8,Tartu,51013(Estonia)E:[email protected]

CONCLUSIONS

Textualanalysisisonewaytotrackthechangesinframeworkpro-grammesovertime.Ontheonehand, itshowsthatthe languageisalivingentitythatchangesovertime.Ontheotherhand,theterminologyshowstheprioritiesexistinginthegivenperiod.

Someresults:a.Lexical density of work programmes of different “Framework

Programmes”hasdeclinedovertheyears.Ithastobestudiedinmoredetailwhetherthisisduetotheadditionofagreaternumberofnon-lexicalwordstothetextorduetothechangeinthelanguageuseofthetextwriters;

b.Overlapping words reflect the core vocabulary which doesnotchangeovertime,andwecanmonitorthefrequencyofits use. Also, the introduction of new words/terms into workprogrammescanbemonitored;

c. The words used more often in work programmes andprojects generally do not coincide. At the same time, it canbe observed that there is continuity between successiveframeworkprogrammes.

REFERENCESBade Strom, T., Lemaire, C., Zacna, J., Arango Montanez, J. and Birnbaum, B. I.(2018).IntegrationofSocialSciencesandHumanitiesinHorizon2020:Participants,BudgetandDisciplines3rdMonitoringreportonSSHflaggedprojectsfundedin2016undertheSocietalChallengesandIndustrialLeadershippriorities.Luxembourg:PublicationsOfficeoftheEuropeanUnion,58pp.

Birnbaum, B. I., Keraudren, P., Strom, T. and Vavikis, T. (2017).Integ-rationofSocialSciencesandHumanitiesinHorizon2020:Participants,BudgetandDisciplines.2ndMonitoringreportonSSH-flaggedprojectsfundedin2015undertheSocietalChallengesandIndustrialLeadershippriorities.Luxembourg:PublicationsOfficeoftheEuropeanUnion,64pp.

Cerletti, C., Gaune-Edscard, M., Gordillo, V., Krull, W., Taylor-Gooby, P. and Vanek, T.,(2001).2000ExternalMonitoringReportontheSpecificProgrammeforResearchandTechnologicalDevelopmentintheFieldofImprovingtheHumanResearchPotentialandtheSocioeconomicKnow-ledgeBase.30pp.

Challis, L., Montes Ponce de Leon, J., Runeberg, K. and Walsh, J.(2003).2002SpecificMonitoringReportontheSpecificProgrammeforResearchandTechnologicalDevelopmentinthefieldofImprovingHu-manPotentialandtheSocioEconomicKnowledgeBase.55pp.

Georghiou, L., Rigby, J. and Cameron, H.(2002).AssessingtheSocio-economic Impacts of the Framework Programme. Policy Research inEngineeringScienceandTechnologyPREST,UniversityofManchester,England.PREST,369pp.

Hetel, L., Møller, T-E. and Stamm, J.(2015).IntegrationofSocialSci-encesandHumanitiesinHorizon2020:Participants,BudgetandDiscipli-nes.Monitoring report on SSH-flagged projects funded in 2014 under the

ISSUE 48 | JULY 2019 131

comeontheconditionofclearcommitmentsandevidenceforameasu-rablesocietalimpact.AcloserlookatthevarietyofSSHresearchpolicyandfundinginstrumentsmightrevealtheirimpactdrivenorientationandfunders’attemptstolureresearchersoutoftheirivorytowers.Itiswor-thwhiletoobservehowresearchersrespondtoresearchpolicypushonbehalfofresearchfunders.InthatrespectLithuaniacanprovideagoodcase study for various typesof research instrumentsand theiruptakebySSH researchersas thegovernmentaimsat financing research forthesakeofabettersocietalimpact.However,itisobviousthatwithoutaclearerunderstandingofwhatresearchimpactisexpectedtobeandwithoutaspecifiednotionof the impactof theSSHresearch, theaimcannotbeachieved.

SSH RESEARCH POLICY IN LITHUANIA: BACKGROUND AND LANDSCAPE

A large-scale funding of competitive research (alongside with thebasic fundingofacademic institutions)was introduced in2008by theResearchCouncilofLithuaniathatwasmadeupoftwocommittees,theCommitteeofNaturalandTechnicalSciences,andtheCommitteeoftheSocialScienceandHumanities.Bothcommitteesparticipatedonequalbasesininitiatingcallsforproposalsandtheirevaluationproceduresforthemaininstrumentofblue-skyresearch,theso-called“ProjectsofCol-laborativeResearch”.Thisactivitywasbasedonabottomupapproachallowingresearcherstochooseforanytopictheyprefer.Inthecaseofinitiatingpolicydrivenresearchinstruments,suchasnationalresearchprogrammes,thecommitteesactedwithintheremitsoftheirrespectiveresearchareas.BynowtheSSHcommitteehasparticipatedinallstagesofthelifecycleoftwofinalisedandtwoon-goingnationalprogrammes,mostly meant for either social sciences or humanities with a possiblemixedapproach.Thematicareasofthenationalprogrammeswerequite

INTRODUCTION

Scientific research is confronted with a number of opposition-basedchoices:interdisciplinaryordisciplinary,fundamental(ba-sic)orapplied,academicresearchorinnovation,blue-skyormis-

sion(policy,agenda)drivenresearchaimingmoreateitheradvancementofknowledgeorpractical societal impact.ThechoicesaremadeevenmorecomplicatedbythetraditionaldualityofSocialSciencesandHu-manities(SSH)andScience,Technology,EngineeringandMathematics(STEM)researchdeeplyingrainedintotheirmethodologiesandabilitiestoservethemosturgentneedsofsociety.However,theessenceofanyresearch,cutelysummedupbytheinitiatorsoftheconference Impact of Social Sciences and Humanities for a European Research Agenda – Valuation of SSH in mission-oriented research, is as follows:“Scientific research is about transformation – how to enable it, or how to avoid it.”(Königetal.2018:4).Thetransformativepowerofresearchanditsmissi-ontoinfluencesocietyandtobeinfluencedbyitsneedshasbeenwidelydiscussedbypoliticiansandresearchers,especiallyinthecaseofSSHresearch. Growing push for transparency of public funds and accoun-tabilitycoming fromcitizenscombinedwithcriticismsagainstSSH forbeingsociallyinefficient,ideologicallybiasedandlivinginanivorytowercausedactivitiesdirectedtowardstheimprovementofsocietalimpact.

Societal impactof theSSH research isa frequentlyusedbut insuf-ficientlyconceptualisednotion.Henceabadneedtodefineitfromtwodifferentperspectives:usageorbottom-upapproachthathelpstoidentifyitspresentmostwidelyspreadsensesandconnotationsandtop-downapproachaimingatre-thinkingthetransformativerelationshipbetweenscienceandsocietyandre-definingthenotionofimpact.Thesameholdsfortherelatednotionsofsocial andpoliticalimpact,socialbenefits,im-pactevaluationorvalorisation, etc.Anyattemptspriortore-definingSSHimpacttomeasureandaccountforsocialorsocietalimpact(letalonetoprovideindexingsystems)aredeemedforvagueandimpreciseoutcomes.

Whateverthenotionofimpactnowadaysmaybe,researchfundingorganisationsonbothnationalandtransnationallevelusuallyprioritisepolicy driven and mission-oriented research. Blue-sky research is wel-

RŪTAPETRAUSKAITĖDOI:10.22163/fteval.2019.380

SPECIFICSSHRESEARCHANDGENERALRESEARCHPOLICYINSTRUMENTS.THECASEOFTHENATIONALRESEARCHPROGRAMMESAND“NEEDDRIVENSSHRESEARCH”INLITHUANIA

ISSUE 48 | JULY 2019132

ofadifferent,i.e.morebroad,reflectiveanddescriptiveapproachtotheimportantissuesofthesocietythanapurelyinstrumentaluser-orientedproblem-solutionapproach.TheacademicandsocietaloutcomesoftheSSHprogrammesneedanex-postevaluationbutevenafastglanceattheoutcomesrevealsthetraditionalacademicresult, i.e.,anadvance-mentofknowledgeanditstransferintheformofacademicpublicationsandconferences.Rareattemptstoproviderecommendationsandtoin-fluencepoliticalactivitiesofthestateinstitutions,mostlymadebysocialscientists,cannotbeseenasveryeffectiveorimpactful.

Thusimpact-orientedrequirementsofthenationalSSHprogrammesarehardlymetbyresearchersastheimpactisnotyetanessentialpartoftheirresearchdesign.Awarenessofthecoreevaluationcriteriathatin-clude“potential impact through the development, dissemination and use of project results”(Proceduraldescription2012:12)doesnotinspirere-searcherstodevotetheirresearchtosocialchallenges.ThegapbetweenSSH research communities and state institutions still exists, deprivingthelatterofthepossibilitytomakeresearchbasedpoliticaldecisions.Di-rectsocietalimpactofpolicydrivenresearchduetoitspurelyacademicnatureisalsohardlyvisibleandmeasurableexceptfortheresultdisse-minationactivitiesandpopularisationofthemostprominentresearch.Itcanbestatedthatpolicydrivenresearchinstrumentsthatprevailinthecountrydonotprovidesatisfactoryfeedbacktothepolicyitself.

COMPETITIVENESS OF THE SSH RESEARCH FUNDING INSTRUMENTS

FourproblemorientedSSHnationalresearchprogrammesandtwostatepriorityprogrammeswithobject-limitedresearchascomparedtotheonlyoneinstrumentofblue-skyresearchprovideascaleofpossibi-litiesfortheLithuanianSSHresearcherstochoosefrom.Theirwilling-nesstocompeteisreflectedinthesuccessratesofsevenprogrammescalculated forall thecallsofproposalsofeachfinalisedandon-goingprogramme.

specific,chosenbyaspecialcommissionoutofnumeroussuggestionsby research communities due to their importance to the state and itssociety.ThenamesofthenationalresearchprogrammesinSSHillustratetheirspecificityandnationalimportanceasexpressedbytwoprogram-mes in the humanities: “State and Nation: Heritage and Identity, andModernityinLithuania”.Asforthesocialsciences,theprogrammesdif-ferintheirthematicscope.“SocialChallengesfortheNationalSecurity”isnarrowerthan“WelfareSociety”1.

Oneprogrammeofadifferent type (nomatter that ishas thewordnationalinitstitle) followingthestatepriorityoftheLithuanianstudiesisthe“NationalProgrammeoftheLithuanianStudies”.Itcouldbeplacedinbetweenfreelychosenandprescribedthematicareas.Itwaslimitedfromthepointofviewoftheobjectratherthanthetopicofresearchandcon-finedtotheinvestigationofspecifictopics.Thetopicshadtoberelatedtothe“past and present of the state of Lithuania, its society and culture as well as the development and present state of the Lithuanian language and nation”(2006:2),asprescribedby“TheProgrammeoftheResearchPrio-rityoftheLithuanianStudies2007-2008”(2006:),allowingresearcherstoinvestigatetheirspecificallychosentopicswithinthisarea.i

The most relevant research funding instrument impact-wise at theResearchCouncilofLithuaniaisanationalresearchprogramme.Despiteresearchareaspecificprogrammestheoveralldescriptionofthenationalprogrammeasan instrumentmeant tobeuniversalandequallysuita-ble forall fieldsof research. Itsmostdistinctive feature related to theimpactof theprogrammeasawhole ispresented in itsaim.Nationalresearchprogrammesaremeant„to solve problems, crucial for the state and its society, with the help of research“(Proceduraldescription2012:2).Moreover, in theproceduraldescriptionof the initiationofanatio-nalprogrammeitisstatedthat„the problem meant to be solved by the national programme should be such that it could be solved by Lithuanian researchers within a period of 3-7 years.“ (Proceduraldescription2012:3).Thelatterrequirementimpliesthattheproblemhastobewell-definedandconcrete,ademandfeasibleexclusivelyforthenaturalandtechnicalsciences.Socialsciencesandhumanities,nomatterthattheycompriseawiderangeoffieldsandsubfieldsfromthepointofviewoftheirresearchobjects,methodsandapproaches,cannotformulateanyproblemofsoci-alrelevancethatcouldbesolvedinsuchashortperiodoftime.ThetitlesoftheSSHnationalprogrammesandtheiraimsareclearmanifestations

1 Formoreaboutthenationalprogrammesseehttps://www.lmt.lt/.

Type Main area Duration Name Success rate

Nationalresearchprogramme Humanities 20102013

“StateandNation:HeritageandIdentity” 40,13%

Nationalresearchprogramme SocialSciences 20102013

“SocialChallengesfortheNationalSecurity” 39,28%

Nationalresearchprogramme Humanities 20172022

“ModernityinLithuania” 33,87%

Nationalresearchprogramme SocialSciences 20152020

“WelfareSociety” 12,83%

Stateprogramme Humanities 20092015

“NationalProgrammeoftheLithuanianStudies” 46,28%

Stateprogramme Humanities 20162024

“TheStateResearchandDisseminationProgrammeoftheLithuanianStudies”

30,68%

Blue-skyresearch SocialSciencesandHumanities

N/A ProjectsofCollaborativeResearchonResearchers’Initiative 26,62%

Table 1.Successratesofthepolicy-orientedprogrammesandblue-skyresearch.

ISSUE 48 | JULY 2019 133

Success rates should be interpreted on the bases of the type andtimingoftheprogrammes.Themainsplitbetweenpolicydrivenorstatecontractedresearchandblue-skyresearch(sixprogrammesversusone)gives the ratioof33,85%to26,62% in favourofblue-sky researchasmorepreferredbyresearchers.However,onenationalprogramme„Wel-farestate“providesanexceptionasithasacomparativelylowsuccessrate and high competitiveness due to its broad thematic scope andopenness for interdisciplinary approaches. Previous national researchprogrammeswerelesspopularincomparisonwiththepresentonesduetothefactthatatthetimeoftheirinitiationcompetitiveandcollabora-tiveresearchwasstillinitsinfancy.Onthebasesofsuccessratesoftheaboveprogrammessupplementedbyinformationprovidedbyindividualresearchersitcanbestatedthatblue-skyresearchorbroadscopepolicydrivenresearchismorepreferredbytheLithuanianSSHcommunitythanspecificagendadrivenresearch.Needlesstosaythatmorecompetitiveresearchismorepromisingquality-wise.

“NEED DRIVEN SSH RESEARCH”AlltheabovepresentedstatecontractedandpolicydrivenSSHre-

searchinstrumentshasadoublepurposetopromoteacademicoutputsandimpactondecisionmakersatthestateinstitutionsandbroaderso-ciety.Usually,researchoutputanditspossibleimpactareneededfasterthanprojectdurationandthelifecycleofaprogrammewouldallow.Astatedemandforpromptsolutionsincaseofburningissuesofnationalcybersecurity,informationalwars,wasteoffood,refugeeintegration,tomentionafew,causedappearanceofanewinstrument,theso-called“NeedDrivenResearch”.Thenewinstrumentwasinitiatedonrequestsforspecificappliedresearchfromthegovernment,ministriesandotherstateinstitutions.Itwasmeanttobemoreflexibletime-wiseasthedu-rationofaprojectwasshortenedupto1-2yearsandcallsforproposalsorganisedeveryyearstartingwith2015.

Themajordistinctionofthe“NeedDrivenResearch”isthelistofto-picssuggestedbyministriesoranyotherstateinstitutionsandapprovedbythecommitteesoftheResearchCouncilofLithuaniaassuitableforresearchandexperimentaldevelopment.Thecommitteespayattentionto all the evaluation criteria for the approval of the topics suggested,however,themostimportantcriterionisthenecessityfortheresearchor experimental development to meet social challenges and to solvepracticalproblems.Apossibilitytoinvestigateaproblemnamedbystateinstitutionsapplyingmethodsof research is oneof themost frequentbottlenecksfortheapprovalofthetopicbytheSSHcommittee.Itishardforthegovernmentalinstitutionstoformulatethetopicofresearchandresearchquestionsinascientificway.Moreover,sometimestheyneedamoremodestoutcome,suchasasetofrecommendationsorafeasibilitystudy,insteadofafull-fledgedresearchproject.

Everytopicsuggestedbythegovernment,itsministriesoranyotherstateinstitutionshastobejudgedwhether:

a. ithasastrategicvalueandimportanceforthestateandsociety,b.theproblemhastobesolvedurgently,c. its solution requiresmethodologyof researchorexperimental

development,d.the results of research will substantially contribute to the

solutionoftheproblem,

e.theplannedresearchdoesnotoverlapwithanyotherpreviouslyfinancedresearch.(Proceduraldescription2016:2)

Themostvaluedoutputsofthe“NeedDrivenResearch”aredifferentif compared to the national or any other research programmes. Apartfrompublications,theyincludespecialappliedoutputssuchaspracticalrecommendations,newmethods,evaluativemethodologies,newtech-nologies,networks,forecastsandscenariosoftheactivitiesplanned,oranythingelsethatcanhave,accordingtothecontractor’sview,apromptimpact for the state institutions and society at large. No matter that“NeedDrivenResearch”isageneralresearchpolicyinstrument,SSHre-latedtopicsprevail(71%)astheyturnouttobeofparamountimportanceforthestateaffairs,especiallyforitspolicies.

“NeedDrivenResearch”asaresearchpolicyinstrumentcannotbeeasily comparedwith theother instruments from thepointof viewofitssuccessratesasit isbasedonatwo-stepprocedure.Pre-proposalsareevaluatedbyajointcommissionofsocialpartnersandexpertsfromtheResearchCouncilofLithuania,themostsuitableproposalsfromthepointofviewofevaluationcriteria(suchasfeasibilityoftheproject,com-petencesoftheresearchers,andsocio-economical,politicaloranyotherimpact)aresuggestedforafullproposalphasewheretheyarere-evalu-atedbyexpertsandthejointcommission.Thereforetherearetwosuc-cess rates: thoseofpre-proposalsandfullproposals.Thepre-proposalsuccessrate(17,28%)isfairlylowincomparisonwiththesuccessratesofotherresearchpolicyinstruments,however,itincreasesupto33,85%for the full proposal submissions. Ingeneral, on the scaleof researchinstrumentsrangingfromfreetopicblue-skyresearchtoalimitedtopicchoiceresearchinstruments,„NeedDrivenResearch“issituatedatthefarendoftheresearchpolicy.InordertoprovethevalueoftheresearchstakeholdershavetoreporttotheResearchCouncilimplementationofitsoutputs.

Blue-skyresearchPolicydrivenresearchprogrammes

Policydrivenresearchprogrammes

Policydrivenresearch

Projectsofcollaborativeresearchonresearchers‘initiative

StateprogrammesonLithuanianstudies

Nationalresearchprogrammes

“NeedDrivenResearch“

Table 2.Thescaleof research instruments fromfree to limitedchoiceoftopics.

TheinstrumentofNeedDrivenResearchisfairlynewthereforehardtoevaluate,nevertheless,itlooksquitepromisingfromthepointofviewof itssocietal impactofSSHresearch. Itsmaindrawback is thesameas incaseofnationalprogrammes, i.e. topdownapproachtospecificproblemsandtheirsolutionleavingSSHresearcherswithevenlesstimeandmorestringentrequirementforpracticaloutputs.

ISSUE 48 | JULY 2019134

CONCLUDING REMARKSImpact-orientedresearchpolicyexecutedbytheResearchCouncilof

Lithuaniahasitsplusesandminuses.Itispraiseworthytopromotetheideaofmission-orientedandtransformativeresearch,toraiseawarenessamongresearchersandtosupport thecultureofsocietal research im-pact.However,itisobviousthatpolicydrivenresearchinstruments,es-peciallyofageneraltype,arenotalwayssuitablefortheSSHresearch.Traditional impact (both academic and societal) assessment methodsdo not reveal multilateral impact of the SSH research that remains tobere-definedtakingintoaccountpossiblesideeffectsandunintendedconsequences.Bottomupapproaches,suchasblue-skyresearch,couldbeabetteralternativeforsociety-orientedresearchprovideditsimpactis conceived,definedandassessed inmultipleways. Inanycase, im-pact,especiallysocietalimpact,oftheSSHresearchhasnotonlytobecarefullyplannedbeforemadeduringtheprocessofresearchbutalsoidentified, reflected and evaluated from a long-term perspective. NoonecoulddoitbetterthanSSHresearchersthemselves.Inspiteofallhighlyappreciatedattemptstoservethestateandsociety,policydrivenresearch instruments deprive SSH community of its blue-sky researchaswellasofachanceforuncertainbuthigh-gainopportunitiesandalong-termimpactonsocietythatishardtomeasureandtoevaluateim-mediately(Nowotny2016).

REFERENCES König, T., Nowotny, H., Schuch, K. (2018).ImpactRe-Loaded.In:Pa-thwaystoimpactfromSSHresearch.ImpactofSocialSciencesandHu-manitiesforaEuropeanResearchAgenda.ValuationofSSHinmission-orientedresearch.Programmebooklet,28-29November,2018Vienna.P.4,https://www.ssh-impact.eu.

Nowotny, H.(2016).TheCunningofUncertainty,Cambridge:Polity.

Procedural Description of the Initiation and Execution of the Pro-jects of the National Research Programmes. (2012). Retrieved Oc-tober 31, 2018 from https://e-seimas.lrs.lt/portal/legalAct/lt/TAD/TAIS.416490?jfwid=-9dzqnu3vw.

Procedural Description of the Initiation and Execution of the Projects of the Need Driven Research.(2016).RetrievedOctober31,2018from(https://www.e-tar.lt/portal/lt/legalAct/9bf9aba0296711e6acf89da936cb7409.

The Programme of the Research Priority of the Lithuanian Studies 2007-2008. (2006).RetrievedOctober31,2018 fromhttps://e-seimas.lrs.lt/rs/legalact/TAD/TAIS.284561.

AUTHORRŪTA PETRAUSKAITĖResearch Unit (Lab or Department), Vytautas Magnus University

K.Donelaičiog.58,Kaunas,LT-44248(Lithuania)E:[email protected]

Endnotesi TheprogrammespresentedheredonotcoveralltheresearchpolicyinstrumentsusedattheResearchCouncilofLithuania.Sevenprogrammeswerechosen

duetotheirrepetativenature(allofthemhadmultiplecallsforproposals)andcomparablestatebasedcallbudgets.AfewuniquecallsforproposalsaswellasprogrammesfinancedfromtheEuropeanstructuralfundswerenottakenintoconsideration.

ISSUE 48 | JULY 2019 135

PILOT PHASE OF THEMATIC SSH-CONSORTIA

Withintheframeworkofanewresearchpolicyin2012(DOZA,2012a),GhentUniversitylaunchedacompetitivecallforconsortiatostrategicallysupportseveralSSHdisciplines.Apilotfundingoffiveyearswasgrantedtofiveconsortia includingastrengtheningof themiddlemanagementthroughacoordinatoronapostdoctorallevel.Allfiveconsortiareceiveda “carte blanche” to develop the consortium and to define and focusonspecificprioritieswithinbroadermissionsofinterdisciplinarity,inter-nationalisation,academicexcellenceandsocietalvaluecreation.Itwasimportantinthisinitiativethattheconsortiacouldstarttooperatefromscratchanddeveloptheirownaims,mission,andeventuallyoutput.Thefiveconsortiawerefundedthroughtheuniversityinternal“SpecialRe-searchFund”(BOF)andhostedatdifferentfaculties.

“CRIME, CRIMINOLOGY AND CRIMINAL POLICY” (CCCP)

AttheFacultyofLawandCriminologytheconsortium“Crime,Crimi-nologyandCriminalPolicy”2hasbeen installed. Theconsortiumdealswiththedomainofdevianceandits(policy)response,aswellasintheareasofcrimeandsecurity.Theconsortiumbringstogether16scholarsfromdifferentdisciplinesinsixdepartmentsworkinginter-disciplinaryonsecurity,crimeanddeviancerelatedtopicsinlocal,national,Europeanandinternationalcontexts.Therangeoftopicscoverse.g.researchintovulnerablegroupsindetention,policingandpolicemobility,desistancefromcrimeanddruguse,(youth)crimeprevention,butalsoothercom-plexcross-borderphenomenasuchascybercrime,terrorismandorgani-sedcrime,orprivacy,informationexchange,bigdata,lawenforcementresponses,policiesandlaws.Theconsortiumfostersknowledgetrans-lation and exchange, strengthening societal value creation leading tosocietalimpactandstimulatessynergiesandcooperationwithexternalacademic,policyandpracticepartnersfromdifferentdisciplines.

INTRODUCTION

GhentUniversity isoneof thebiggestUniversities inBelgiumwith11faculties,117departmentsand650researchinstitu-tes containing around 9000 employees and 41.000 students

(Ghent University, 2016). Ranked best Belgian University on 61 in theShanghai ranking in2018GhentUniversity ishome to17highlycitedauthors and more than 55 grantees since the start of the EuropeanResearchCouncil (ERC) fundingscheme. In sucha largeorganisation,Ghent University considers the potential for top-down steering of re-searchstrategy limited.Therefore,GhentUniversityappliesa rangeofdecentralisedresearchstrategyinitiativesincludingSSH-focusedones.

Duetoanationalandregionalfocuson“objective”distributionofre-searchfundsandawillingnesstobecomeaworld-renownedknowledgeeconomy,performance indicatorsareoften limited toquantitativeandindividualoutputandtraditionalfiguressuchasnumberofpublications,PhD’sandcitations.WhencomparedtotheSTEMM(Science,Techno-logy,Engineering,MathematicsandMedicines)disciplines,thissystemis widely known to disadvantage the Social Sciences and Humanities(SSH) in part because of a different research and publication culture.Inmeasuringperformance, traditionally lessattention isgiven to indi-catorssuchas(interdisciplinary)cooperationorservicetosocietywhilemanySSHdisciplinesshowespeciallyhereastrongpotentialandsomealreadywell-developedgoodpractices.Inaddition,characteristicssuchasahighlevelofindividualityofresearchers,less“big”fundingduetosmallerresearchgroupsaswellasahighteachingloadandlackof(pl-anned)societalvaluecreationinSSH,urgedGhentUniversity’sResearchDepartmenttosupportjointinitiativesinSSHtostrengthenresearchex-cellencyandimpactthroughthestimulationofcooperation.

Oneof these initiativesentailedtheset-upof interdisciplinarySSHresearchconsortia.TheSSHresearchconsortiamaybeconsideredthecounterpartofGhentUniversity’sbusinessdevelopmentcentersasfun-ded by the “Industrial Research Fund” (IOF)1 which were installed tobridgethegapbetweenstrategic fundamental research, industrialco-operationandtechnologicalinnovation.TheSSH-consortiaarecomple-mentarytotheseSTEMMinitiativesandhavethepurposetobridgethegapbetweenSSH(fundamental)research,interdisciplinarycooperationandsocietalimpact.

NOËLKLIMA,STEFANMEYSMAN,JULIECARLIER,ALEXISDEWAELEANDESTHERDESMETDOI:10.22163/fteval.2019.381

GHENTUNIVERSITY’SINTERDISCIPLINARYSSH-CONSORTIA–ASTRATEGYTOENHANCETHESOCIETALIMPACTOFRESEARCH

ISSUE 48 | JULY 2019136

PIRENNE CONSORTIUM FOR MEDIEVAL STUDIES

ThePirenneConsortiumforMedievalStudies3,primarilybasedattheFaculty of Arts and Philosophy fosters cross-disciplinary research intothemedievalperiodandadvancesknowledgeexchangebetweenthesedifferentfieldsofstudy,aswellaswithsocietalpartnersandthegeneralpublic.TheconsortiumbringstogetherallmedievalistsatGhentUniver-sityunderthepromotor-boardof29seniorscholars.Itdrawsmorethan100 members from four faculties and ten departments, covering bothScience, Technology, Engineering and Mathematics (STEM) and SSH.Research on original medieval heritage such as texts, images, maps,artefactsandsitesisthecorebusiness,includingthedevelopmentandintegrationof “DigitalHumanities”methods.However, theconsortiumalsohousesexpertiseincollaborationwitharchives,libraries,museums,andotherculturalheritageinstitutions,aswellaswithseveralsocietalsectorssuchaseducation,policyandtourism.

GHENT CENTRE FOR GLOBAL STUDIES

TheGhentCentreforGlobalStudies4ishostedattheFacultyofPo-litical andSocialSciences.Asan interdisciplinary researchplatform itunitesscholarsfromSocialandEconomicGeography;International,EUand Conflict and Development Studies; Economics, Sociology, GlobalHistoryandEthics;HumanRightsLawandInterculturalPedagogy.Withatotalof11researchgroupsfromsixdifferentfacultiestheconsortiumfocuseson thecriticalstudyofglobalisation,withspecialattention tothe interactionofglobaland localprocesses.With its interdisciplinaryresearch and education – on urbanisation, rural transformations, eco-nomicgovernanceandmigration–theCentreaimstocontributetothesocietaldebateon,andevidence-basedpolicy-makinganddevelopmentcooperationfor,sustainabledevelopment.

“WORKING TOGETHER FOR MENTAL HEALTH” – PSYNC

“WorkingTogetherforMentalHealth”–PSYNC5refersto“psycholo-gy’and‘synchronise”.ThisresearchconsortiumishousedwithintheFa-cultyofPsychologyandEducationalSciences.Itsobjectiveistodevelopacommonstrategytotranslateclinicallyrelevantresearchtotheclinicalfieldandtothebroadersociety.Theconsortiumisdedicatedtoimpro-vingthementalhealthofallcitizens,runningresearchprojectsinclosecollaborationwithdiversestakeholdergroups,withaclearfocusonge-nerating realworld impactandprovidingsocietal innovation.PSYNC’smaingoalsarereachingvulnerablegroups,stressingtheimportanceoflifecycleperspectiveonmentalhealth,increasingmentalhealthliteracyand health promotion, safeguarding ethical perspectives, and develo-pinginnovationsinthetreatmentofmentalhealthdisorders.

“INNOVATION AND ALL INCLUSIVE GROWTH” – CIG

Theconsortium“InnovationandAllInclusiveGrowth”–CIG6foundits base at the Faculty Economics and Business Administration. Theconsortium’sgoal istoactasaneconomicandscientificbaseforeve-rything which concerns innovation, entrepreneurship and all-inclusivegrowthatGhentUniversity.Research focusesondifferent topics fromdifferentanglessuchastechnologicalinnovationandentrepreneurship,“CorporateSocialResponsibility” (CSR), corporateandentrepreneurialfinance,businessarchitectureandmodelling,innovationandgrowthatmacrolevel.Thisconsortiumdecidednottocontinueitsworkafterthepilotphaseandthereforewasnotpartofsomeofthelatermentionedevaluationmechanismsafterthefiveyearpilotphase.

Allconsortiahavecreatedtheirownstrategicplanandmodusope-randi,anddevelopedtheirownsupportstructureinlinewiththeirthe-maticfocusandtheirmemberswhoareresearchersfromdifferentfacul-tiesanddepartments.Theconsortiaandhowtheyoperatedhavebeenevaluatedondifferentoccasionsandfromdifferentangles.Beforeitwasdecided to provide continued funding, four out of five consortia wereevaluated on three criteria that were discipline-specific, consortium-specificandcoordinator-specific.

EXTERNAL PEER EVALUATION (DISCIPLINE-SPECIFIC)

In2016,theconsortiareceivedafirstevaluationthroughanexternaldiscipline-specificpeerreviewevaluationcoordinatedbytheGhentUni-versityResearchDepartmentincollaborationwiththefacultiesofArtsandPhilosophy,FacultyofLawandCriminology,FacultyofEconomicsandBusinessAdministration,FacultyofPsychologyandEducationalSci-encesand theFacultyofPoliticalandSocialSciences.Thisevaluationwasnotconsortiumspecificbutratherdisciplinespecific.However,theconsortia have been considered being part of the respective facultieswhichalsoreceivedtheattentionof theevaluators.Theconsortiaandtheirrolehavebeenevaluatedasvaluablepartsoftherelevantfacultiesanddisciplines.Thefocusoninterdisciplinarycooperationwasconside-redanassetinthefacultystructures.

STAKEHOLDER PEER EVALUATION (CONSORTIUM-SPECIFIC)

In2017,anewtargetedcallwaslaunchedforthecontinuationofthefivepilotconsortia.Forthisevaluationthepanelconsistedofmembersnot only from the University Research Council, but also of individualsfrom non-academic stakeholder groups. External members came fromthepublicsector,civilsocietyand internationalorganisations.Allcon-sortiahavebeenevaluatedinatwo-foldmanner.First,eachconsortiumwasconsideredretrospectivelybyevaluatingtheoutputsandoutcomeswithregardtotheformer“GhentUniversityResearchPolicyPlan”(2012

ISSUE 48 | JULY 2019 137

–2016)underwhichtheconsortiawereinstalled(DOZA,2012).Second,theconsortiawereassessedfromafuture-orientedperspective.Thepa-nellookedintotheambition,theorganisation,thestrategyofthecon-sortiumanditsmatchwiththecurrent“GhentUniversityResearchPolicyPlan”(DOZA,2017).

Asaresult,thedecisionwasmadetocontinuefundingforfourofthefiveconsortiaandstructurallyembedtheroleoftheresearchcoordinatorasasharedpositionwithalong-termcontract.Themajorstrengthsofthefourwerethestimulationof interdisciplinaryresearchineachthe-maticareabutalsothestimulationonacross-consortiumlevelsuchase.g. jointevents,projects,knowledgeexchangewhichwasconsideredamajoraddedvaluefortheresearchandimpactagenda.Also,thein-volvementofthecoordinatorsinseveralcentralUniversityresearchpo-licyworkinggroupswasapositiveoutcomeof theorganicbottom-updevelopmentoftheconsortiaduringthepilotperiod.

As a consequence, the objective was to build on the developedstrengthsandparticularitiesofeachconsortiumandthenaturallygrowncooperationbetweenthemthroughknowledgesharingandresearchpo-licyinvolvement.Withaconsolidationoftheexistingconsortianewop-portunitieswithregardtointerdisciplinarity7andsocietalvaluecreation8leadingtosocietalimpactwouldbecreated.Theconsortiawillbeevalua-tedeveryfiveyearsontheirorganisationandmanagement,theirinterdis-ciplinaritythroughcooperationandjointinitiatives,theirsocietalimpactthroughimpactcasestudiesandtheplanningofthefuturefiveyearperiod.

INTERNAL EVALUATION (COORDINATOR-SPECIFIC)

During the pilot phase, the coordinators in some of the consortiachanged due to staff turnover. After the decision to fund the existingconsortia permanently, the acting coordinators have been evaluatedseparately by a Ghent University panel including members from theconsortia,theResearchDepartmentandtheGhentUniversityResearchCouncil. This evaluation examined the coordinator’s profile, approachandstrategytomanagetheconsortiumforthenextfiveyears.Thepanelgavepositiveadvicetoextendtheircontractstowardsindefiniteappoint-ments.Allfourcoordinatorscouldshowtherelevantthematicexpertiseandmanagementskillstocoordinatetheconsortiaonapermanentba-sis.Allhavealsobuiltupaclosecollaborativerelationshipwithvariouspolicy officers within the Research Department and strengthened theinformationflowandthecooperationbetweenthecentraluniversityle-velandtheconsortiumresearchersfromthedifferentdepartments.Theprofile, skills and approaches of the coordinators will also be used todefine the requirements for the recruitment of future coordinators foradditionalconsortia.

CONSORTIA ARE EMBEDDED IN THE BIGGER RESEARCH POLICY AGENDA

TheSSH-consortiaareembeddedinthegeneralresearchpolicyandwerealsopartofGhentUniversity’spolicyinitiativefocusingontheex-

cellenceinthehumanities,socialandbehaviouralsciences.Thisspeci-fic initiativebroughttogetherdifferentincentiveswhichweretargetedspecificallyat the facultiesLawandCriminology,ArtsandPhilosophy,PsychologyandEducationalSciences,andPoliticalandSocialSciencesandwasintendedtostrengthenresearchqualityandresearchstrategy,takingintoaccounttheidiosyncrasiesofresearchinthesefields(DOZA,2012b). Ghent University’s intention to enhance research excellencethroughhigherresearchquality,visibilityandrecognitionaccompaniesthe initiatedSSH initiatives.Next to theSSHconsortia,budgetswerereservedforadditionalprofessorandtenuretrackpositionsandareformofthesabbaticalrules(DOZA,2012b).Fromotherresearchpolicy initi-atives,suchasthe“researchspearheads”,alsoknownastheMRPin-itiative(“MultidisciplinaryResearchPartnerships”)(DOZA,2010),GhentUniversitylearnedaboutthedemandtodevelopmethodsandincentivesrelevant for SSH with regard to the bibliometric bias, the high indivi-dualityofresearchersandless“big”fundingduetosmallergroupsaswellasthehighteachingloadandthelackof(focused)societalvaluecreation.Atthesametime,anewpolicyplanonsocietalvaluecreationcalled“IM-pact”wasdevelopedby theResearchDepartmentwhereintheSSH-consortiaandtheirstructuralembedmentplaysoneofthekeyrolestostimulateinterdisciplinarycooperationandenhancesocietalva-luecreationofresearch(DOZA,2015).

TheexperiencesfromthepilotledGhentUniversitytoconsidertheSSH-consortiaasagoodpracticeandledtoawishtoexpandtheinitia-tiveacrosstheuniversity.Thefocus,theworkingandthedevelopmentofthefourSSH-consortiaareconsideredtobeaninspirationforothernewinterdisciplinaryconsortiaworkinginotherresearchareas.GhentUniver-sityintendstoextendtheinitiativewithsixmoreconsortiatostrengthenitsgeneralprofilewithregardtointerdisciplinarityandsocietalimpact.

CONSORTIUM COORDINATOR WITH A PERMANENT ASSIGNMENT

Thestructuralembedmentandlongtermvisionrequiresthesusta-inablepositionofthecoordinator.Againstcommonuniversitycustomsfixed termassignmentswould in thiscaseweakenthepositionof thecoordinatorandhampertheworkinganddevelopmentoftheconsorti-um.Interdisciplinarycooperationandsocietalimpactcreationtaketimeand require consolidated and sustainable relationships both with andbetweenresearchersandnon-academicstakeholders.Tolimittheriskofahighfluctuationofstaffandrelatedlossofexpertise,GhentUniversitydecidedtoprovidefixedcontractsforthecoordinators.

Thecoordinatorsareknowledgebrokerswhopromotecollaborationandnetworkingwithintheconsortium,betweentheconsortiaandwiththeResearchDepartment.Theydevelopexpertiseinfacilitating,promo-ting and appreciating interdisciplinarity and take initiatives to supportinternal interdisciplinary cooperation. Within and across the consortiaandincollaborationwiththeResearchDepartmentthecoordinatorsela-borategenericandthematicinitiativesandshareknowledgewithregardtoresearchpolicy,interdisciplinarityandsocialimpact.

Thecoordinatorsarealsomonitoringthesustainabilityandlong-termstrategyof theconsortiumandoptimise the involvementandcommit-mentoftheresearchers intheconsortium.Thecoordinatorsassistthe

ISSUE 48 | JULY 2019138

researchersduringtheideaphaseforacquiringexternalfinancing.TheyfollowandinfluencetheEuropeanresearchpolicy,bothgenericallyandthematicallyfortheconsortiumandforGhentUniversity.

The coordinators also manage societal value creation and societalimpactactivities.Therefore,theydevelopexpertiseinanumberofvaluecreationandimpactrelatedtopicsrelevanttotheconsortium.Asanten-nae,thecoordinatorsareincontactwithseveralstakeholdersplayinganactiveroleintheexpansionofthestakeholdernetworkoftheconsorti-um,e.g.bysettingupastructuraladvisoryboardofsocietalstakehol-ders.Finally,thecoordinatorsdevelopandusemodelsforthedesignofvaluecreationandimpactprocessesandfortheevaluationofimpact,inlinewiththeirresearchexpertise.

Eachof thecoordinatorshasgenericexpertisewithin the researchareasoftheconsortiumatPhDlevelandisabletoassessstrategicallythepotentialofprojectsandother initiativeswith respect toscientificandsocietalimpact.Theyareknowledgebrokers,providetechnicalas-sistanceandsafeguardthepathwaystoimpact,definedbyeachoftheconsortia.Thecoordinatorsreceiveanannuallump-sumbenchfeethatcanbeusedtosupporttheirworkagendaandwherecoordinatorsthem-selvesactasbudgetholders.

JOINT CONSORTIA PATHWAYS TO IMPACT – AN EXTRA DIMENSION

Theworkofacoordinatorinthethematicconsortiummakesaround70-80%ofthetotalworkload.Theother20-30%arededicatedincross-consortium activities and generic Ghent University work. Besides theinterdisciplinaritywithineachoftheconsortiathecollaborationbetweenthecoordinatorsandwiththecentralResearchDepartmentbringsanex-tradimension.Withinthisextradimensionofinterdisciplinarity,impact,relevantoutputsandoutcomesaswellasknowledgeand informationexchangecouldbegenerated.Thisconcernsuniversity-wideinitiativesinfunctionofknowledgesharing,expertisebuildingandtrainingcoopera-tionwiththeResearchDepartmentandotherGhentUniversitypartnersintheareaofinterdisciplinarity,societalvaluecreation,impactandre-searchpolicy,includingprospectingfundingopportunitiesandpromoti-onofbestpractices.

All coordinators are members of the “Impact Task Force” and theAlpha-EUworkinggroupatcentralUniversitylevel,participateinwritingof position papers9 (e.g. Ghent University, 2017a; 2017b; 2018), orga-nise jointworkshops10, informationsessions11, lectures12,participate injointprojects13andplantoorganisean interdisciplinary impactaward.All jointactivitiesgenerateoutputs feeding thecommonobjectives toincreasethesocietal impactofGhentUniversitySSHresearch.Also, itadds to theoptimisationof the impact researchpolicyandevaluationatGhentUniversityandtothedefenceofGhentUniversity’sinterestsatEuropeanlevele.g.throughinputonthedevelopmentofEUFrameworkProgrammes.ThisexchangeonGhentUniversity’s(EU)researchpolicyishighlyvaluedbyallinvolvedactorsandshowsalreadyarangeoftangi-bleresults.

LESSONS LEARNED

ThepilotexperimentoftheSSH-consortiabroughtarangeofpositiveexperiencestothesurfacebutalsopointsofpotential improvement inthefuture.TheSSH-consortiawereestablishedinanenvironmentwhereinterdisciplinarity and openness for other disciplines is necessary, butnotself-evident.Thestructuralsupportforresearchersinviewofinter-disciplinarycollaborationshowedpositiveeffects.Thecoordinatorssti-mulateresearcherstowardsmorecooperationandcollaborationwithintheirconsortiabutalsowithexternalparties.Theorganisationofinter-disciplinarityrequiresgoodleadershipbythecoordinatorsbutalsofromtheprofessorsandresearchers involved.Thedifferentconsortiumpilottracksshowed that justacoordinator inagroupof researchers isnotsufficienttogaineffectsfromaconsortium.Dedicationandcommitmentisrequiredfromcoordinatorsandtheprofessorsandresearcherstobeabletoreachanotherlevelofcooperation.

Animportantstepintheprocesswasthedecisiontomaketheco-ordinatorpositionfixedtermandextractthemfromthe“usual”acade-miccareertrackofaresearchorientedpostdoc.Someoftheconsortialosttheircoordinatorsduringthepilotphaseandevenoneconsortiumstepped out during the pilot phase. Some researchers left for a fixedtermpositionelsewhereorfollowedtheirregularresearchtrackontopoftheirconsortiummanagementduties.Asafepositionwithanautono-mousbudgetfromthestartpreventsahighleveloffluctuationamongthecoordinators,whichgoesalongwithalossoftheacquiredexpertise.Thiskindofinitiativeshouldnotjustbea“steppingstone”forpostdocsontheirjumptothenextprojectcontractorthenextstepontheirwaytowardsaprofessorship.Nevertheless,teachingandresearchactivitiescanbeofaddedvaluetostayintouchwithacademicexpertise,andtodisseminatethecoordinator’sexpertiseoninterdisciplinarityandimpactrelatedtopics.Theprofileoftheconsortiumcoordinatorsrequiresexper-tiseinresearchandtopicknowledge,butinaddition(research)manage-mentandpolicyexpertiseandexpertiseonsocietalvaluecreationandresearchfunding.Thispositionisdifferentfromapureresearchpositionandshouldbefilledwithpeoplemotivatedtobuildtherelevantexpertiseas a pivot point between research, research policy, funding, outreachandmanagement.

Inaddition,awellorganisedresearchinformationsystemisrequiredineachconsortiumbutalsooncentralleveltoavoidthelossofinforma-tionandknowledge. Informationmanagementanddataexchangestilldepended very much on individual researchers. Therefore, incentivesareneededforparticipatingresearcherstovaluetheirengagementandcommitment within a consortium might help to convince researchersmuch quicker to dedicate more effort and energy in interdisciplinarycooperationandsocietalvaluecreationoftheirresearch.Thiscouldbedone e.g. through including open science incentives in their personalcareergoals.

Itwilltaketimeuntiltheresultsfrominterdisciplinarycollaborationdevelopeffect.Thefive-yearperiodhasshownthatthisisaprocessofbuildingtrust,dedicationandcommitment.Thisneedsalsotobecreatedandmaintainedbetweencoordinators,researchersandcentraluniversi-tyresearchpolicydepartments.Aninteractiononregularbasisinstruc-turalworkinggroupsraisesthetonusofjointactionsbetweenthethreeparties.Mutualrecognitionandunderstandingisimportanttofruitfullybringtogetherthedifferentworkinglevels.

Startingwithapilotonasmallscalehasproventobetherightway.ThelearningeffectfromthepilotevaluationsputstheResearchDepart-

ISSUE 48 | JULY 2019 139

mentinapositiontoimmediatelycallforpermanentconsortiabuildingonthestructuresandcooperationgrownduringthepilotphase.

Astrongasset intheprocesshasbeenthebottom-upapproach inthedevelopmentofthestructureandworkingoftheconsortia.There-searchersdonotconsidertheconsortiaanextrainstitutionwithheavyadministrativeburdenwhichoperatesindependentfromtheotherexis-tingstructuressuchasresearchinstitutes,departmentsorfaculties.Thebottom-upapproachmadeitpossibletofullyadapttheconsortiatotheneedsof the researchersand tobuildacomplementarystructure thatis intertwinedwithallotherstructures.A fully functioningconsortiumbringsassetstothecentralresearchpolicyleveloftheUniversity.

Theboonorbaneofthebottom-upapproachwasthediversityoftheconsortiaandtheirworkingwhichisdifficultytomeasureandcompareaccordingtostrictandmeasurableindicators.GhentUniversityhascho-senforpanelstoevaluatetheworkindividually.Clearguidelinesonhowtheconsortiawillbeevaluatedperiodicallyarenecessary.GhentUniver-sitydecidedtofocusonfourdomains:theorganisationoftheconsortiumand internal procedures, the interdisciplinarity of the working, impactcasestudiesandthefutureplanning.

TheSSH-consortiawereinspiredandconsideredcomplementarytothe“IndustrialResearchFunds”(IOF)businessdevelopmentcentresthatwereestablishedoveradecadeago.Howeverthebondsandcooperati-onbetweenbothinitiativesaredevelopingveryslowly.ThesameappliesfortheexchangeandcooperationwiththeUniversitytechnologytrans-fer officewhichwasnot fully exploitedduring thepilotphase. In thecaseofGhentUniversity’spilotbottom-upapproach,anexchangewithSTEMMdisciplineswasinsomecasesexistentbasedonsingleprojectsor individualcollaborationmoments.Thismightbetakenawayforthenext cohort of interdisciplinary consortia at Ghent University but alsoforUniversities thatwant tostartwithsuchan initiative. It iscertain-ly recommended toengage immediatelyandstructurallywithSTEMMresearchers that do have relevant connection with the topic. In somecasesthisisnotpossibleorrelevant.However,itwillhelptoopensiloresearchingandopensbordersfornewcooperationventures.

FUTURE PERSPECTIVES OF THE SSH-CONSORTIA

ThefourSSH-consortiawillcontinueonpermanentbasisembeddedin the researchpolicystructureofGhentUniversity. Insomeconsortia(where relevant) exchange and cooperation with STEMM researcherswill be further stimulated and extended especially with regard to thenext“EuropeanFrameworkProgrammeHorizonEurope”.Theconsortiawill work through a range of specific pathways to impact and also arangeofjointones.Theinitiativeswillinter aliacovertheenhancementof impact literacyamongresearchersand informing researchpolicyatGhentUniversitylevelandEUlevel.Anewcallwillmakethenumberofconsortiagrowfromfourtotenwhichalsowillleadtonewchallenges.Thecooperationandexchangeinfrastructurebuiltduringthepilotphaseprovidesa situationwherenewconsortiawith theirnewcoordinatorsare able to be immediately integrated. Finally, the consortia and theResearchDepartmentwillcontinuetoexchangeknowledgeon impactmeasurement.

REFERENCES

DOZA (2010).StrategischSpeerpuntenbeleidOnderzoek.Ghent:GhentUniversity.[eng.StrategicSpearheadsPolicyResearch]

DOZA (2012a). Ghent University – Research Policy Plan 2012 – 2016.Ghent:GhentUniversity.

DOZA(2012b).Eenhefboomnaarexcellentieindehumane,socialeengedrags-wetenschappen.Ghent:GhentUniversity.[eng.Aleverforex-cellenceinthehuman,socialandbehavioralsciences]

DOZA (2015).Strategischprojectmaatschappelijke valorisatie vanon-derzoek– IM-pact.Ghent:GhentUniversity. [eng.Strategicprojectso-cietalvaluecreationofresearch–IM-pact]

DOZA (2017). Ghent University – Research Policy Plan 2017 – 2021.Ghent:GhentUniversity.

Ghent University(2016).FocusonGhentUniversity.RetrievedSeptem-ber 10, 2018 from: http://unigentdemo.online-magazine.nl/en/magazi-ne/11623/815119/focus_on_ghent_university-_cover.html.

Ghent University (2017a).Societal impactofSSHresearch:perspecti-vesonco-creationandworkingwithsocietalreadinesslevels.RetrievedSeptember15,2018from:https://www.ugent.be/en/research/position-papers

Ghent University(2017b).PositionPaperIEH2020The‘SSHembeddingChallenge’.RetrievedSeptember15,2018from:https://www.ugent.be/en/research/position-papers.

Ghent University (2018). Call on the European Union to create high-qualitydedicatedandembeddedSSHresearchopportunitiesinFP9.Re-trieved September 15, 2018 from: https://www.ugent.be/en/research/position-papers.

AUTHORS

ISSUE 48 | JULY 2019140

NOËL KLIMAIDC Crime, Criminology and Criminal Policy, Ghent UniversityUniversiteitstraat4,Ghent,9000(Belgium)E:[email protected]

STEFAN MEYSMANIDC Pirenne Consortium for Medieval Studies, Ghent UniversitySint-Pietersnieuwstraat35,Ghent,9000(Belgium)E:[email protected]

JULIE CARLIERIDC Ghent Centre for Global Studies, Ghent UniversityUniversiteitstraat8,Ghent,9000(Belgium)E:[email protected]

ALEXIS DEWAELEIDC PSYNC – Working Together for Mental Health, Ghent University HenriDunantlaan2,Ghent,9000(Belgium)E:[email protected]

AND ESTHER DE SMETResearch Department, Ghent UniversitySint-Pietersnieuwstraat25,Ghent,9000(Belgium)E:[email protected]

Endnotes1 https://www.ugent.be/techtransfer/en/support-for-academics/iof.2 https://www.ugent.be/crime/en.3 https://www.ugent.be/pirenne/en.4 http://www.globalstudies.ugent.be/.5 https://www.ugent.be/psync/en.6 http://www.innovationtoday.ugent.be/.7 Under‘interdisciplinarity’weunderstand:thevariousgradationsandmodalitiesofcooperationoutsideyourowndiscipline.Thisincludesalsocross-,multi-,

andtransdisciplinarity.Thedegreeofintegrationbetweenvariousdisciplinescanvary.8 (Societal)valuecreation(inBelgiumandtheNetherlandsoftenreferredtoas‘valorization’)istheprocessofcreatinganaddedvaluetoscientificknowledge

andexpertiseoutsidetherealmofscience.Ifthecreatedaddedvalueisaimedatorisofspecificimportancetoacommunityofexternalstakeholders(rang-ingfromthegeneralpublictoveryspecificgroupsofstakeholders)thevaluecreationisdeemed‘societal’.

9 E.g.seealsohttps://www.ugent.be/en/research/position-papers.10 E.g.InterdisciplinaryPhDworkshoponsocietalimpact.11 E.g.MatchmakingeventbetweenSSHandSTEMMresearchersandresearchmanagerstofosterinterdisciplinaritytowardsSTEMMdisciplineswithregard

toimpactofresearch.12 E.g.JulieBayley‘Impact:buzzwordorbaseline?DevelopingstrategiesforimpactandsupportingimpactliteracyforSSHresearch(ers)’.13 E.g.ACCOMPLISSH–ACcelerateCO-creationbysettingupaMulti-actorPLatformforImpactfromSocialSciencesandHumanitieshttps://www.accom-

plissh.eu/;ENRESSH–EuropeanNetworkforResearchEvaluationintheSocialSciencesandtheHumanitieshttps://enressh.eu/.

ISSUE 48 | JULY 2019 141

“an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia”3

Incontrast tomore traditionalmethods formeasuringsocietalandeconomicbenefit,likecountingpatentsorspin-offcompanies,wesawtheREFdefinitionasmoreopentodisciplinarydifferencesandcompa-tiblewiththemultitudeofpathwaystoimpactdocumentedinempiricalstudies.4

INTRODUCING SOCIETAL IMPACT TO THE SSH INSTITUTIONS

Choosinganexistingmethodtoassesssocietalimpactmadethetaskof introducing a new evaluation dimension to the national evaluationsysteminNorwaymoremanageable.ThemaineffortoftheRCNthenwentintoconvincingtheNorwegianhighereducationinstitutionsthattheREFimpactcasetemplatecouldactuallybeusedtodocumentthesocietalbenefitsresultingfromSSHresearchinameaningfulway.

Whenplanning theevaluationofhumanities research in2013, theinternationaldebateonthepublicvalueofthehumanitieswasmakingitswavesfeltalso inNorway.Therewasastrongresistance inacade-miaagainst thinkingofhumanities research in terms ifusefulness.Atthe same time, proclamations on the essential role of the humanitiesforthedevelopmentofsocietyweremanifold.Inotherwords,therewasadiscrepancybetweenthefeelingof importanceinacademiaandtheabilitytodocumenthowresearchresultshadbeenputintouseandtopointouttheactualbeneficiaries.

Theimpactcasemethodalsoreceivedvarioustypesofcriticismfromtheresearchercommunity.Themostcommonobjectionswerethatthecasesonlycoveredasmallpartofthesocietalrelevanceofaninstituti-on,theyimpliedalinearrelationshipbetweenresearchandimpact,theywerenotreflectingthecomplexitiesofresearcher–userrelationsandnotcoveringtheimportantimpactstakingplacewithinacademia.

Withthisinmind,theRCNinvitedrepresentativesfrominstitutionsthattookpartoftheevaluationtoanimpact-workshop.TheaimoftheworkshopwastoexplorehowtheinstitutionscouldusetheREFimpactcasetemplatetodescribethepathwaysfromresearchtosocietalimpactaccordingtotheREFdefinition.TheparticipantswereintroducedtotheREFcase-modelbyProfessorHelenSmall–aliteraryscholarandfromCambridgeUniversity–whohadhadaleadingroleinherfaculty’sim-pactcasesubmissionstotheREF.

INTRODUCTION

In this paper we present how evaluation of societal impact of re-searchwasintroducedinnationalresearchevaluationsinNorwaywithinsocialsciencesandthehumanitiesthroughanadaptationof

the“ResearchExcellenceFramework”(REF)2014impactcasemethod.Wefocusonthepracticalaspectsofthisintroduction,theprocessesofevaluationandtheimpactoftheimpactevaluationonthediscourseonsocietalbenefitsof“SocialSciencesandHumanities”(SSH)research.Fi-nally,wediscussthelimitationsoftheimpactcasemethodandindicatesomepossiblewaysforward.

THE INCLUSION OF SOCIETAL IMPACT

TheResearchCouncilofNorway(RCN)hasbeenperformingnation-wideresearchevaluationsforover20years.Theintervaloftheseevalu-ationsisapproximately10yearswhichmeansthatnearlyallacademicsubjectshavenowbeenevaluatedtwice.Theaimofthesubject-specificevaluationsistoprovideacriticalreviewoftheNorwegianresearchsys-teminaninternationalperspective,andtoproviderecommendationsonmeasurestoencourageincreasedqualityandefficiencyofresearch.TheevaluationshelptoensurethattheRCNhasthenecessaryinformationonwhichtobaseitsstrategicresearchfundingandeffortsvis-à-vispub-licbodies.Theevaluationsalsoserveasatoolfortheinstitutionsthem-selvesintheirongoingeffortstorefinetheirownstrategicandscientificframework.1Thereisnodirectlinktofunding.

Traditionally,thenationalresearchevaluationshavefocusedonthequalityandefficiencyofresearchactivitiesatthenational,institutionalandgrouplevel.Asaresponsetothepoliticalexpectationsofharvestingsocietalbenefitfromincreasedinvestmentsinresearch,theRCNdeci-dedtoincludesocietalimpactasadimensionofthelatestevaluationsofthehumanities(2017)andsocialsciences(2018).ThelargemajorityofresearchersinNorwaywithintherelevantdisciplineswereincludedinthetwoevaluations.

Themainmethodused toassesssocietal impact in the twoevalu-ationswasborrowedfromthe“2014ResearchExcellenceFramework”intheUK.Themethodwaschosenfortwomainreasons:1)Itwaswelldocumented,testedandevaluated2,and2)thedefinitionofimpactusedin theREFwas judged tobesufficientlybroad to includemostof theexpectedsocietalbenefitsfromSSHresearch:

JONHOLMANDANETTEASKEDALDOI:10.22163/fteval.2019.382

EVALUATIONOFSOCIETALIMPACTINNORWEGIANSSHEVALUATIONS

ISSUE 48 | JULY 2019142

Duringtheworkshop,manyparticipantstooktheopportunitytodis-cusshowtheycouldusetheREF impacttemplatetodescribespecificsocietalbenefitsfromresearchattheirinstitutions.Inthisway,thework-shopproducedachangeinthediscoursefromanessentialistquestionofwhat impact istoapragmaticquestiononhow to documentthesocietalbenefitsofresearch.Thischangeinattitudewascrucialforthesuccessof theevaluationexercise.There isa fundamentaldifferencebetweentheeffortofunderstandingandconceptualisingacertainphenomenonlikethesocietalbenefitfromresearchtothetaskofactuallyestablishinganewpracticeofdocumentingsocietal impact.Thedebateonhowtodocumentandassessthesocietalbenefitsfromresearchshouldthusnotbelimitedtoadiscussionofthemeaningofacertainconceptortheoryon the roleof science in society. Inorder to informpolicy, thedebateshouldalsotakeintoaccounthowpoliticalexpectationsforsocietalben-efitsfrominvestmentsinresearchareimplementedthroughevaluationexercisesorregulatoryregimes.

AninterestingexampleofthisimplementationperspectiveisgivenbyMartaNataliaWróblewska,whoinarecentPhD-thesesinvestigatestheprocessofconstructingthenotionofimpactintheBritishREF.InspiredbyMichelFoucauld’stheoryofgovernmentality,Wróblewska(2018)ar-guesthattheresultingunderstandingandpracticesofsocietal impact“is a response to a set of struggles over issues related to selecting a new direction for the economic development (knowledge-based economy), re-shaping the role of universities in society (third mission, entrepreneurial university), as well as class issues and tensions between particular aca-demic disciplines.”5 According to Wróblewska, the rules which wereintroducedwith theREFguidelinescouldbeconsideredasempty and unfinalised before they were taken into use and translated into con-cretepracticesattheresearchinstitutionsandthusformingan impact infrastructureconsistingofprofessionalroles,teachingframeworksandspecifiedproceduresandtimeframes.

THE EVALUATION PROCESSTheRCNcollecteda totalof404 impactcases fromtheparticipat-

ing institutionsand researchgroups for theevaluationsofhumanitiesandsocialsciences(170casesweresubmittedtohumanitiesevaluationand234casestosocialsciencesevaluation).Thesubmissionofimpactcaseswasoptionalandforthatreasonthenumberofimpactcaseperresearchervariedalotamonginstitutionswithanaverageofonecaseper13researchers.

Forbothevaluations theRCNcarriedoutabriefdescriptiveanaly-sisof thecategoriesof impact thatwasreported in the impactcases.Thepurposesoftheseanalyseswerenottoevaluatethecases,buttodescribe trends in the submitted material. The analysis showed thatresearch leadingup to the reported impactwascommonly conductedingroups,thatthegeographicalreachwasnational,andthatthemostcommonchannelfromresearchtoimpactwasuser-orienteddissemina-tion.Forthesocialsciencecases,themostcommonbeneficiaryoftheimpactwerepoliticalinstitutions,andtheprincipalregisteredeffectwaspolitical.Thegeneralpublicwasthemostcommonbeneficiariesforthecaseswithinhumanities,andtheprincipaleffectregisteredwascultural.

TheRCNalsodidamappingoftheimpactcasesontothethematicprioritieswithinHorizon2020(H2020)societalchallengesandthosein-dicatedbytheNorwegiangovernment’slong-termplanforresearchandhighereducation.Thesomewhatsurprisingresultwasthattherewasa

greatermatchwiththeEuropeanprioritiesthanwiththeNorwegianpri-orities.ThiswastoagreatextentduetothepresencetotheSSH-relatedtheme“Europeinachangingworld”inH2020.

Theevaluationoftheimpactcaseswascarriedoutbythesamein-ternationalpeerswhoevaluatedthequalityofNorwegianresearch. Theevaluationpanelsfoundseveralgoodandvariedexamplesofsocietalim-pactamongthesubmittedcases.Inthehumanitiesevaluationthecom-mitteewas“favourably impressed with the range and depth of societal im-pacts from the Humanities”6,andintheevaluationofsocialsciencestheevaluatorsfoundthattheresearchhad“considerable relevance to a large range of public and private societal actors and activities”7.Theevaluatorshighlighted64casesasexamplesofgoodpractice. Thesewerecasesthatdocumentedconcreteandsignificantproofofimpactonsociety.

CHALLENGESDespite this, the evaluators experienced a number of difficulties

whentryingtoassesssocietalimpactinthetwoevaluations,andtheevaluation task was described as “particularly challenging”8 in theevaluationofsocialsciences.Theevaluators foundthat therewasanunevenunderstandingofthemeaningofimpactamongtheparticipat-inginstitutionsandresearchgroups.Amajorityofthesubmittedimpactcasesmerelydescribedcommunicationactivities,ratherthanprovidingdocumentationofsocietalimpact.Forthisreason,thepanelsfounditdifficulttoassessseveralofthesubmittedimpactcases,andtheyrec-ommendedthattheinstitutionsdevelopedamorestrategicapproachtoimpact,andalsothatthedifferencebetweenimpactandengagementwasbetterdefinedfortheinstitutions.Inaddition,theevaluatorsrec-ognisedthatthereweremanymethodologicaldifficultieslinkedtotheassessmentofsocietalimpact,andtheysawaneedforfurtherdevelop-mentofthemethodsforassessingandevaluatingsocietalimpact,andalsoformoresophisticatedtoolsforgatheringandarticulatingevidenceofimpact.

TheRCNhasusedimpactcasedescriptionsasthemainsourceforevaluating societal impact also in other recent evaluations (includingevaluations of research institutes and thematic evaluations). The re-porteddifficultieshavebeenthesameinmostoftheseevaluations.Inmanycasesthedistinctionbetweensocietalimpactanddisseminationisnotclear.WetakethisasanindicationthatresearchersandinstitutionshavenotfullyunderstoodtheexpectationsembeddedintheREFimpactcasegenre.Thedifferent interpretationof impact,andalso the failuretodocumentactualchange,madeitdifficultfortheexpertstoassessanumberofcases.

The RCN recognises, in order to make robust assessments of thesocietal impact of research, that there is a need to combine differentmethods.Forthatreasonuser-surveysandinterviewswereincludedinsomeofRCN’srecentevaluationsinordertoaddausers’perspectivetotheassessmentofsocietal impact. Itwashoweverproblematic tousetheresultofthesurveysinmostoftheevaluations.Theresponseratewassometimesverylow,andtheinternalresponseratevariedbetweenthedifferentsetsofquestions.Asaresult,theevaluatorsplacedmoreemphasison impactcases thanonsurvey resultswhenassessing thesocietalimpactofaninstitution.

ISSUE 48 | JULY 2019 143

THE IMPACT OF THE IMPACT EXERCISE

Despitethemanymethodicalchallengesinimpactevaluations,theRCNhasreceivedpositivefeedbackfromtheinstitutionsandresearch-ersontheusefulnessoftheimpact-exercise.Severaloftheimpactcasesproduced for the evaluations have been used by the institutions andresearchers themselves e.g. published on the institutions websites orincludedintheresearcher’sCV.Wealsoseesignsofamoresystematicapproachintheinstitutionsinidentifyinganddocumentingthe(poten-tial)societalimpactofresearch.

The impact case method has also given valuable new knowledgeinthevarietyofwaysinwhichSSHresearchcreatessocietalbenefits.WehavethusmovedfromasituationwitharathervaguediscourseonSSH-researchasageneralsocietalgoodtoacollectionofconcreteevi-dencethatcouldbeusedinadebateonhowresearchfundingshouldbeattributedinordertoobtainspecificsocietal(orcommercial)aims.Asanexample,theimpactcasesfromthehumanitieswereusedinpolicy-advicetothegovernmentrelatedtothewhitepaperonthehumanitiesthatwaslaunchedduringtheevaluation.9Itishoweverimportanttonotethatacollectionof404impactcasescannotgivearepresentativepictureofthesocietalimpactofSSHresearchinNorway.

THE WAY FORWARDSo,wheredowegofromhere?Thereisarisingdemandfrompolicy-

makersandfundersthatpotentialsocietalbenefitshouldbeconsideredthroughthewholelifecycleoftheresearchprocessontotheapplicationofresults.Inthisperspective,thedifficultiesreportedbytheevaluationcommitteesinassessingtheactualimpactofNorwegianSSHresearchisacauseofconcern.BasedonourexperiencewiththerecentevaluationsintheRCNwewouldarguethattherearetwoaspectsthatneedstobeaddressedinthetimetocome:

• furtherdevelopmentoftheimpactinfrastructureattheinstitu-tions,

• furtherdevelopmentofthemethodsforassessingandevaluat-ingsocietalimpact;

FURTHER DEVELOPMENT OF THE IMPACT INFRASTRUCTURE:

The evaluation committees’ recommendation to the institutions totakeamorestrategicapproachtoimpactdocumentationisinourviewaresultofanunderdevelopedimpactinfrastructureattheinstitutions.Thisisnotonlyaproblemforpolicy-makersandfunderssearchingforareturnontheirinvestmentsinresearch.Itisalsoaproblemfortheaca-demicinstitutionsthemselvesthatareconfrontedwithanexpectationtodocumentsocietalbenefitsfromtheirresearch,butlackingtheimpactinfrastructurethatwillhelpthemtoidentify,documentandlearnfromhowresearchproducedattheirinstitutionsinthepasthaveledtoposi-tive(ornegative)effectsinsociety.

FURTHER DEVELOPMENT OF THE IMPACT METHODOLOGY:

Theevaluationcommitteesalsocallsforfurtherdevelopmentofthemethodsforassessingandevaluatingsocietalimpact,andformoreso-phisticated tools forgatheringandarticulatingevidenceof impact.Asearliernoted,theRCNrecognises,inordertomakerobustassessmentsofthesocietalimpactofresearch,thatthereisaneedtocombinedif-ferentmethods.Inadditiontothis,itmightalsobeusefultochangethefocus. Ina recent reportby twoNorwegianevaluationexpertson theconceptandpracticeofsocietalimpact10,itisarguedthattheobjectofevaluationshouldshiftfromtheresearchresultsandtheirdisseminationtowardstheprocessofinteractionbetweenresearchersandusers.Theyalsoarguethattheevaluationofimpactneedstoberelatedtotheactualgoalsoftheresearchperforminginstitutions.

The RCN is currently investigating the possibility of creating a na-tionalevaluationprotocolinNorwaythatwillallowthehighereducationinstitutionstotakealargerresponsibilityfortheevaluationoftheirownactivitiesasitisdoneundertheDutch“StandardEvaluationProtocol”.11Ourhypothesis is thatevaluation resultswill bemore relevant for thestrategic development of each institution if the evaluation criteria arealignedwiththeirstrategicgoal.Givingthehighereducationinstitutionsa greater responsibility for the evaluation of their own activities, willprobablyalsotietheevaluationprocessesmorecloselytotheresearchprocesses,creatingloopsoffeedbackfromevaluationresultstotheman-agersofresearchprojects,groupsanddepartments.

CONCLUSIONOneofthemainlessonsoftherecentevaluationsofSSHinNorwayis

howapragmaticapproachtoassessingsocietalimpactcontributedtoachangeinthewaythatacademicsandinstitutionalleaderstalkaboutthesocietalbenefitsfromresearchinNorway.Althoughbetterdefinitionsandconceptualisationsofevaluationcriteria–suchassocietalimpact–arealwayswelcome,ourexperience is that theevaluationprocess in itselfcreatedanewunderstandingofthephenomenontobeevaluated.

Recommendations provided by evaluation experts based on the re-centevaluationsinNorwayandcasesofinternationalbestpractice,couldindicatethatfutureevaluationexercisesinNorway–includingsocietalimpact – should be more closely linked to the purposes and strategicgoalsoftheresearchorganisationsinordertoallowtheseorganisationstoexperimentwithdifferentkindsofevaluationsmethodsandprocessesthataremoreintunewiththeactualresearchprocessesandthemulti-tudeofwaysthatresearchersinteractwithpartnersoutsideofacademia.

Sofar,thenationalresearchevaluationsinNorwayhaveservedanimportant function in the implementation of national policies for re-searchandhighereducation.TheimpactofthelatestevaluationsinSSH–changingthewaythatsocietal impactofSSHresearchisconceivedanddiscussed–isanexampleofthistransformativerole.InthechoiceoffuturemodelforresearchassessmentinNorway,thereisthusaba-lancetobestruckbetweentheneedforabetteradaptationofevaluationcriteriatothestrategicgoalsofeachinstitutionandtheuseofresearchevaluations as a policy instrument at the national level. It remains tobeseenifitwillbepossibletomovetheevaluationprocessesandste-wardshipclosertotheresearchinstitutions,whileassuringatthesametimethatsuchinstitutionalevaluationsrespondtonationalpolicyneeds.

ISSUE 48 | JULY 2019144

REFERENCES

Manville, C., Guthrie, S., Henham, M-L., Garrod, B., Sousa, S., Kirtley, A., Castle-Clarke, S. and Ling, T.(2015).Assessingimpactsub-missionsforREF2014:Anevaluation.RANDCorporation,SantaMonica,Calif.AndCambridge,UK.

Gulbrandsen, M. and Sivertsen, G.(2018).Impactianvendtforskning:begrepsavklaringogpraksis.NIFUReport[Impactinappliedresearch–conceptsandpractice.Forthcomming]MinistryofEducationandResearch(2017).HumanioraiNorge[TheHu-manitiesinNorway](Meld.St.202016-2017)).Oslo.

REF2014 Reserach Excellence Framework(2011).Assessmentframe-workandguidanceonsubmissions,Bristol

The Research Council of Norway(2017).EvaluationoftheHumani-tiesinNorway,Oslo.

The Research Council of Norway (2018).EvaluationoftheSocialSci-encesinNorway,Oslo.

Thune, T. et al.(2014).Noderikunnskapsnettverket:Forskning,kunns-kapsoverføringogeksterntsamarbeidblantvitenskapeligansatteiUH-sektoren. [Internalandexternalcooperationpatternsamongacademicstaff in the Higher Education sector in Norway] NIFU Report 2014:23.Oslo.

VSNU, KNAW and NWO (2014).Standard Evaluation Protocol 2015 – 2021,Voorburg.

Wróblewska, M. N.(2018).ThemakingoftheImpactAgenda:Astudyindiscourseandgovernmentalitygovernmentality(PhDThesisSynopsis,p.7).WarwickUniversity,Coventry,UnitedKingdom,Retrievedfrom:htt-ps://tinyurl.com/y2nkhlwe25February2019

AUTHORS

JON HOLMThe Research Council of NorwayDrammensveien288,0283Oslo(Norway)E:[email protected]

ANETTE ASKEDALThe Research Council of Norway

Drammensveien288,0283Oslo(Norway)E:[email protected]

Endnotes1 https://www.forskningsradet.no/en/Subjectspecific_evaluations/12335579717342 CatrionaManvilleetal.(2015)3 REF2014ReserachExcellenceFramework(2011);484 TaranThuneetal.(2014)5 https://tinyurl.com/y2nkhlwe6 TheResearchCouncilofNorway(2017);87 TheResearchCouncilofNorway(2018);368 TheResearchCouncilofNorway(2018);369 MinistryofEducationandResearch(2017).10 MagnusGuldbrandsenandGunnarSivertsen(2018)11 VSNU,KNAWandNOW(2014)

ISSUE 48 | JULY 2019 145

INTRODUCTION

Theaimofthepaperistwofold:a)toanalysethewaysinwhichstakeholdersareinvolvedinsocialscienceresearch(SS)fundedunderEuropeanresearchprojects,inordertoidentifyelements

–organisationalcharacteristicsoftheproject,typesofstakeholders,typeofinvolvement,thatcanincreasethelikelihoodofproducinganimpactonpolicyandsociety;b)todiscussconsequencesoftheempiricalevi-dencesforresearchevaluationbothattheex-antelevel(elementscha-racterising the design of the project) and ex-post level (achievementsandpracticesindicatingthatanimpactisproducedorsignallingthatanimpactmightoccur).

Thepaperdealswith social impact,which ismostlydefinedasaneffectthatresearchcouldproducebeyondtheacademiccontextintermsofbenefitsonsocietalandinstitutionalchallenges,includingalsoimpactonthepoliticalside(Penfieldetal.,2014;Reale,Primeri,Fabrizio,2017).TheinteresttodeepenissuesofsocialimpactinSSderivesfromthelimi-tationsofusingthetraditionalapproachbasedoninput-output-outcomemeasurements; SS are characterised by effects that are more difficultto be singled out than those produced in other areas of science, andmeasurementsprovideverypoorandoftenbiasedunderstandingofthephenomenon(Realeetal.,2017).Thepaperfollowsthetheoreticalap-proachesfocusingonresearchprocess(SpaapenandvanDrooge,2011)andcontributiontotheimpactgeneration(Mayne,2012),insteadofat-tributionofimpacttoresearchactivities;inthisrespectitisofcrucialim-portancetoshedlightaboutthegeneratingmechanismsthattransformknowledge into actionable goods, and the network of actors involved(Jolyetal.,2015).

Socialimpactcouldbestrengthenedbyparticipatoryinvolvementofdifferent social actors through productive interactions (Molas-Gallart,2012; Weik et al. 2014); the positive effects of these interactions arecloselyrelatedtothewaysinwhichresearchersandstakeholderscom-municateaboutresearch,itsgoalsandsocietaldemand(Molas-Gallart,2012).Thus,socialimpactispointedoutasaconsequenceofaprocessinwhichknowledgeandexpertisecirculatestoachievespecificobjectivesthatarerelevantfortheprogressofsociety(SpaapenandVanDrooge,2011). A participatory approach could deeply affect the sustainability

ofresearchsoitmustbeimplementedsincethebeginningofprojects(Talwar,2011).Underaslightlydifferentconceptualisation,socialimpactisgeneratedthroughtranslationofactorsinvolvedintheprocess(Jolyetal.,2015),whichco-definetheirinterestsalongtheso-calledimpactpathway(Walkeretal.,2008;Jolyetal.2015).Inbothcases,theroleofstakeholdersisatthecoreofimpactproduction,andunderstandingfea-turesaffectingtheirinvolvementisstillalowexploredissue.Thispapercontributestodemonstratekeydeterminantsofimpactinthedifferenttypesofinteractionswithstakeholders,discussingwhatimplicationsthiscanhaveonevaluationcriteriaandmethodsofresearchprojects.

THEORETICAL BACKGROUND

AND CONCEPTUAL FRAMEWORK

Participationofstakeholdersinresearchactivitiesgainedamomen-tumfromthelaunchoftheactionsonscienceandsocietyandsciencewith society within the “European Framework Programmes” (EUFP);studiesdevelopedonthistopicfiguredouttheimportanceofinvolvingnon-academicpartnersinresearchprojectstoimprovethelikelihoodtoproduceanimpactfromresearchactivities(Langetal.2012;Realeetal.,2017).Participationofstakeholderscouldallowtheextensionofresearchresults towardsapracticalpathbut inabroaderperspective theycanprovidetotheprojectageneralinsightfocusedtotheproblemfield.Thismeansthatstakeholdersinvitedtocollaboratewithresearchersshouldbethosei)moreaffectedbythechallengefacedbytheresearchproject,andii)morestimulatedtooffertheirknowledgetodefinearangeofop-tionsforresultsimplementations(Wiek,2014).Thecooperationwiththestakeholders includesthepossibilitytokeepintouchwitheachofthecategories,placingasuniquepointofreferencethecompetencesneces-sarytoreachtheproject’saiminthebestpossibleway.Thismeansthatresearchcooperationisopentoactorsfrompublicinstitutions,corporatesector,andnot-for-profitorganisations(Langetal.,2012).

EMANUELAREALE,SERENAFABRIZIOANDLUCIOMORETTINIDOI:10.22163/fteval.2019.383

STAKEHOLDERS’ROLETOPRODUCEIMPACTFROMSOCIALSCIENCERESEARCH:WHATLESSONSFOREVALUATION?1

1 TheauthorsacknowledgetheSTI2018Leidenconference,fromwhichthistemplatewasadapted.

ISSUE 48 | JULY 2019146

Interactions and collaborations between researchers and stakehol-ders take different forms. The literature outlined several types of sta-keholders’ involvement, which can be typified around three main ca-tegoriesof contribution,namely: i) co-creation of knowledge betweenacademicsandnon-academics(Weiketal.,2014;Edelnbonsetal.,2011;SpaapenandvanDrooge,2011;deJongetal.,2013);ii)unpackingtheresearchobjectivesintosub-taskthataremoremanageableforprodu-cingusableresults(Belletal.,2012);iii)discussionanddisseminationofresearchresultsaftertheirproductioninordertofacilitategeneratinganimpact(SpaapenandVanDrooge,2011;Weiketal.,2014). It isworthtorecallalsotheworkofMuhonenetal., (2018),whichdeveloped60casestudiesonsocialsciencesandhumanitiespathwaystosocietalim-pactbypayingattentionnotonlytoproductiveinteractionsbutalsotothechangestheymediate.Basedontheempiricalresults,theauthorsdevelopedatypologyoffourpathwaystosocietalimpact,articulatedintwelvemodels,whichstartsfromtheclassicalpipelinemodel,ofresultstransferringfromresearchtosociety.Themodelsarepresentedinhier-archicalprogression,accordingtothedeviationfromthebasemodel:ashigharethelevelofcomplexityintermsofinteractionbetweenresearch,societyandintermediatinginstitutionsashighwillbethedeviationfromthepipelinemodel.Thepathwaymodelsbelongtofourgeneraltypolo-gies,namelydissemination,co-creation, reactingtosocialchangeanddrivingsocialchange.

Projects can have one or more types of stakeholders’ involvementbutdirectparticipationindicatesthegoalofasocialeffectofresearch,anelement toassesswith instruments other than standard academicindexes(Penfieldetal.,2014;Weiketal.,2014).Inthesameline,Talwaretal.(2011)distinguishbetweentwomaincategories:a)unidirectionalapproach,whensocialactorsareinvolvedinthefinalphasesofthepro-ject, foraweaksupport in resultsconsolidationand/oraconsultationwithresearcherstoimplementresultsinanapplicativeway;b)interacti-veapproach,whenstakeholdersareinvolvedalsointheearlyphasesoftheprojectandcontributetodefinetheresearchgoaland/ortodesigntheresearchstrategy.Whileinthefirstapproach,contributionofstake-holders isbasically limited toelaboratea tool touse research results,inthesecondone,stakeholdersareinvitedtoprovidetheirexpertisetobroadentheknowledgebaseusefultodefineallaspectsoftheproblem,beyondthescientificanalysis,andimplementingtheusabilityofresultsthroughoutallthephasesoftheproject.

However,theadvantagetohavearelevantapplicativecoreinapro-jectcouldexposetheresearchtotheriskthatpursuingapplicativere-sultsbecomeprevalentwithrespecttotheachievementofhigh-qualityscientificoutputs.Inamoregeneralway,severalcontributionsunderlinethatalargeinvolvementofstakeholdersinaresearchprojectcouldfo-cus the analysis on solving a single problem (or a restricted range ofproblems), channelling research efforts to a punctual objective at theexpenseofresultsofgeneralapplication,alsorelevantforothercases(Talwaretal.,2011;Langetal.,2012).

Weassumealinkbetweenthedegreeandthewayofstakeholders’involvement in the project and the emergence of social impact of re-search.Wethusconsiderthatoneofthekeyfeaturesforgeneratingim-pactisthecapabilityoftheprojecttobuildacommonlanguagebetweenthedifferentactors,scholarsandnon-scholars.Underthiscondition,ex-changesareabletocreatenewknowledgeandmutualunderstanding,whichislikelytoproducetransformativechanges.Also,weconsiderthatthementionedresultcanbeachievedwhencontinuousinvolvementofstakeholdersisatstake,inthedifferentphasesoftheproject,andsta-

keholdersshowconcretewillingnessandinteresttocontributeinasub-stantialwaytotheresearchachievements.Weanalysetherelevanceofroleheldbystakeholders,respecttotheresearchers,andhowtheyjoint-lycontributetotheresearchactivity,withtheexpectationthatamoreex-tensiveandeffectiveco-participationinresearchcreatestheconditionsbothfordisseminationofresultsinbroadandarticulatedterms,overtheoriginalboundaries,andforgeneratingimpactpathway.

METHODOLOGY AND DATAThepaperisbasedontwoindepthcasestudiesoftheprojects“Stra-

tegiesforinclusionandsocialcohesionfromeducationinEurope”–IN-CLUD-EDand“MakingPersonswithDisabilitiesFullCitizens”–DISCIT,fundedrespectivelyunderthe“EuropeanFrameworkProgrammes”FP6andFP7 in social sciences,wherea social impactbecamevisible justaftertheprojectcompletion.Thecasestudiesselectedaretwooutof22topsuccessstoriesdevelopedunderthe“Evaluatingtheimpactandout-comesofEUSSHresearch”project (IMPACT-EV),whichare illustrativeexamplesofsuccessfulmodesforstakeholders’involvementinresearchactions.Casesfollowastandardisedstructure,developedthoughtrian-gulationofinformationfromdifferentsources,namelyinformationfromdocumentaryanalysis(characteristicsofthecallunderwhichtheprojecthasbeenfunded,reportsanddeliverablesproduced,otheradministra-tivedocuments),dataand indicatoronresearchoutputs (bibliometricsandotherweb-basedresources),interviewswithresearchers,coordina-tors,andstakeholdersinvolvedintheactivities.Fouraspectsofactors’involvementhavebeenconsidered:

• Modalitiesandcommunications–projects’organisationalfea-tures;

• Timing–timelyinteractionsduringtheprojectandafterthepro-jectcompletiondeterminingtheimpactpathway;

• Language – capability to develop mutual understanding be-tweenresearchersandstakeholders;

• Outcomes–co-creationof resultswith transformativeeffectsonscienceandsociety.

Theprojectsanalysedbothpresentabroadinvolvementofstakehol-dersinordertomaximisetheimpactinpoliticalandsocialterms.

INCLUD-ED emphasises the role of the dialogic and participativecollaborationamongresearchersandstakeholders(end-users, local in-stitutions)inthedevelopmentofeducationalstrategiesforthesocialin-clusionofvulnerablegroups(IMPACT-EV,2017b).Theprojectfocusedonstrategiesthatcouldcontributetosocialinclusionofvulnerablepeople,decidingaboutkeyelementsandactionstoimprovesocialandeduca-tionalpolicies.“SuccessfulEducationalActions”(SEAs)–thusevidence-basedsolutionsabletoachievegoodresultsinmanydiversecontexts,were identified as examples of positive achievement in the inclusionofvulnerablegroups;SEAsweretransferredtoothercommunitiesandcontexts to improve social cohesion. INCLUD-ED produced significantachievementsoneducationalpractices,decreasingtheratesofschoolfailureandimprovingthefamilies’involvement.Theprojectputforwardthehypothesis thatsocialexclusion ismoreaconsequenceofactionsimplemented thananeffectderiving from thesocial characteristicsofthecontext(IMPACT-EV,2017b).Theconsortiumwascomposedbyanin-terdisciplinaryresearchteamcoveringanthropology,economics,history,research methods, political sciences, linguistics, sociology and educa-

ISSUE 48 | JULY 2019 147

tion,comprisingfifteenacademicorganisationsfromfourteendifferentEuropeancountries.Theorganisationalstructureincludesten“WorkingGroups”(WGs),different“FreeTaskOrientedGroups”(FTOGs),anda“Pa-nelofExperts”(POE),whichsupportedtheconsortium,andan“AdvisoryCommittee”(AC)composedofrepresentativesofvulnerablegroups.Theproject also includedhorizontal typeof actionsand structures, whoseaimwastomonitoringandharmonisingtheactivities,solvingproblemsthatmightemerge,andcombiningtheresultsandprogressesmadebythedifferentgroups.

DISCITmaingoalswerea)tohelpdefinitionofanew“EuropeanSo-cialModel”of inclusionandcohesionthroughtheanalysisofpoliticalandinstitutionalinstrumentsexistinginthecountriesinvolved,andb)toindicateawaytoremoveandpreventphysical,attitudinal,socialandor-ganisationalbarrierstoafullandeffectiveparticipationtothesocietyofpersonswithdisabilities(IMPACT-EV,2017a).Toachieveitsgoals,DISCITconsidereddifferentformsofstakeholders’engagement.Eightresearchinstitutions,fromeightdifferentcountries,andtwointernationalorga-nisationsofdisabledpeople’s rights–“TheEuropeanDisabilityForumand The Mental Disability Right Initiative”, composed the consortium.Organisationscontributedtothedraftingoftheprojectandhelpedtheconsortium to set up the analysis in general terms without make thedifferencesbetweentypesofdisabilitiesirrelevant.Furthermore,twoas-sociationsofdisabledpeoplehelpedtoidentifythespaceofinterventionoftheprojectamongthedifferentsocialareas,contributingdecisivelytodefinethechangeofperspectivethatcharacterisesDISCIT:theideathatdisabilityisnotaparticularcaseofeachareaofinterventionbutitisauniquetopicwithseveralarticulations.

Theconsortiumwassupportedbyone“EuropeanStakeholderCom-mittee” and nine “National Stakeholder Committees”, one for eachCountryinvolvedintheproject.Thesecommitteesincludedmembersof“Disabledpeople’sorganisations”(DPOs)andrepresentativesofgeneraldirectorates(limitedtothe“EuropeanCommittee”),administrativeandpolitical institutions at national and local level. Committees contribut-edindifferentways:providinginformationaboutsocialandregulatorypeculiarities within countries and commonalities between countries,refiningthedocumentaryandempiricalsurveytoolsoftheproject,hel-ping insampleselection for the interviews,andproposing themselvesasintermediariesbetweentheresearchersandthedisabledpeoplein-terviewed,inordertohelpthelattertoovercomeanyembarrassment.

Periodicalforumatinternationalandnationallevelwereorganisedtofacilitatemutualexchangesbetweenresearchersandstakeholders,dis-cussingresearchdevelopmentandincentivisingdisseminationofpolicybriefsbasedonresearchresults.Allinall,theseforumsproducedmoreresultsthanexpected,favouringaharmonisationoflanguagebetweendifferentgroupsofstakeholders(representativesofassociationsandin-stitutions)andfacilitatingthecreationofnetworksfortheexchangeofinformationandbestpracticesatinternationallevelbetweenDPOs.

FINDINGS

The case studies highlighted that both projects show significantevidencesrelatedtothethreedimensionsofstakeholders’involvementinvestigatedinthepaper;however,differencesemergedfromtheana-lysis,whicharerelatedtotheorganisationalandstructuralfeaturesoftheprojects.

INCLUD-ED

Stakeholderinvolvementwasakeyobjectivefromthebeginningoftheproject,affectingthemethodologicapproach,shapingtheresearchquestionsandthearchitectureofthewholeresearchactivities.Thecol-laborationbetweenresearchersandstakeholdersconcernedboth theknowledge-exchange dimension and the concrete implementation inspecific social contexts through specific sub-task. However, the mostsignificantevidencewasthelong-termimpactofthemodelimplemen-ted,throughaconstantdisseminationofmainresultsdeeplyinvolvingalso a large network of stakeholders (IMPACT-EV, 2017; Reale et al.,2017b).

MODALITIES AND COMMUNICATIONS

Diversevoices-associationsofvulnerablegroups,families,teachers,localdecisionmakersenforcedthevalidityandrigorousnessofthesci-entificprocessthuscontributingtotheco-productionresearchresults.

“I remember that it was very egalitarian collaboration because they were first of all introducing each of us, at each meeting we were the first who were talking in the centered explaining each community we were re-presenting and I remember being very diverse, so people, researchers, but also women, immigrants or people with disabilities, so the meeting was very diverse and there were the researchers were presenting the results or part of the results corresponding to the part we were supposed to discuss, and they were asking maybe questions or maybe very open debate on what do we think or what do we believe that concrete strategies they were presenting may affect our community or not.”

(EndUser)“From my point of view is exactly the same methodological structure

of the entire project that eases the portability, because it is based on the communicative theory of Habermas, this means that every time we sim-ply did the interviews, as you are doing with me, stakeholders, etc., we are focused on the one hand to receive the information and on the other hand to give ourselves a contribution, support for change precisely”2

(Researcher)

2 EnglishtranslationfromtheItalianoriginal:“[…]dalmiopuntodivistaèpropriol’impiantometodologicostessodell’interoprogettochefacilitalatrasferi-bilità,perchésiccomesibasasullateoriacomunicativadiHabermas,questosignificacheognivoltacheanchesemplicementefacevamoleinterviste,comeleistafacendoconme,aglistakeholderseccetera,noiciimpegnavamodaunaparteariceveredelleinformazionimaallostessotempoafornirenoistessiunapporto,unsupportoinvistadelcambiamento.”

ISSUE 48 | JULY 2019148

TIMING

Maineventsoccurredovertheprojectlifeandbeyond,documentedontheofficialwebsite3:

• 10technicalmeetingswith“AdvisoryCommittee”(eachmeet-ingconsistedofapresentationof theresultsofdifferentpro-jects/subtasksandadiscussionandreflectionbetweenrepre-sentatives of different vulnerable groups), meetings with thepanelofexpertsandmembersofresearchteam;

• 15disseminationeventsaroundEuropelikeConferences,Con-gresses,publicmeetingsandlaunchofprojectwebsite;

• 7institutionaleventsmainlyattendedbyrepresentativesfromtheEuropeanCommission–DirectorateGeneralforResearch,representatives of Member State governments, social actors,researchersandscholars;

• 13trainingseminarsattendedbymembersoftheresearchcom-munity, government representatives and Non-GovernmentalOrganisations(NGOs).

Theseeventshavebeenscheduledduringeachyearbecause theyhaddifferentpurposesandinvolveddiversetypesofstakeholdersinor-dertodiscusssteadilyshortandmedium-termachievementsofthepro-ject,tosharedifferentpointsofviewonthemethodologicalapproach,and to implement themodel throughspecific trainingseminars. Thus,theworkwasbasicallydevotedtofollowapathtogainimpactoninte-restedcommunities.

“We were meetings twice a year, at the meetings we were discussing the results of the project, they were making right, so I remember that re-searchers from INCLUD-ED project were presented us the results or the development of the project and then we were discussing about that.” (End User)

“I remember that we had, a year if I’m not long we meet with the ex-pert group and the advisory committee every year and we were presenting […] all day presenting the results, they had them in a bag but of course some people might not read report, so we synthesised the main points, and we were discussing with them, the AC, the Advisory Committee, and expert group. The contributions from the expert group were not that diffe-rent from the one’s that we could come up as research consortium, even they were a lot of policy makers at high level impact.” (Researcher)

LanguageINCLUD-EDputintoactionthecriticalcommunicativeresearchme-

thodology (Flecha and Soler, 2014) which was crucial for the projectsuccess, because it allows integrating and including knowledge fromdifferentdisciplinesandorientations,usingbothquantitativeandqua-litativemethodsandtechniquestoanalysedata;furthermore,thecom-municative methodology allowed researchers to apply mixed-methods

approachtopursueimpact.“While the voices of vulnerable groups have traditionally been excluded from research, the communicative methodolo-gy depends on the direct and active participation of the individuals obser-ved throughout the research process.”(INCLUD-EDwebsite6).

“I remember that main researcher of INCLUD-ED it was talking and he was very interested on our opinion, we felt that, we felt that we are not attending the meeting because this is part of the project but because they wanted to know what we think what we believe and what we disagree with them. I remember they were asking all the time to criticize them, to disagree with them because this is good for improving and in the way that we felt that they were taking our opinion into account.4(EndUser)

“[...] I have often also found critical points of view that are not even critical in dialectical sense; in other cases I have found resistance also to the type of interview because being a dialogic interview when the other dialogues must give you his time not only to answer but also to listen to you, and it is not said that everyone wants of this thing because you al-ready put yourself in a very strong relationship, it is more challenging, not just intellectually as time, it is really challenging from a relational point of view..5 (Researcher)

OutcomesThemembersoftheAdvisoryCommitteehadaccesstotheINCLUD-

ED resultsandmetperiodicallywith thecoordination team todiscusstheresearch.Moreimportant,theysuggestedrecommendationsonhowthefindingscouldbeusedtohaveagreatersocialandpoliticalimpact;thoserecommendationswereproblematisedwiththeresearchers.

“They were very motivated because they really give importance to our words, and then in further meetings we could see during the years of the project, during the different meetings we have we could see also the improvement they were achieving they were explaining that. […] I remem-ber a concrete neighbourhood in Spain they were telling us about and that people who never have a job before they are now getting jobs or starting to organise themselves and I remember that for me was important.” (EndUser)

Stakeholdersplayedafurtherimportantrolewithrespecttothepoli-ticaldimensionofimpact,sincepolicymakerswerewellattentivetotheinstanceofchangingcomingfromsocietytestifyingthegoodnessofthetransformationssuggestedthroughevidencesofSEAs:

“If we make lobbing with policy makers, we don’t get results. If we get social impact and social actors who are beneficiaries of social impact go to policy makers with us, this has political impact. Even with friends, even with policy makers that are friends of mine…” you are very nice and…” but nothing. We will remain friends. … Do not ask to policy makers what are thinking, because they think “Well, they are researchers, they are co-ming here for resources, for applying”. (Researcher)

3 http://creaub.info/included/Lastaccess:20/06/20184 http://creaub.info/included/5 EnglishtranslationfromtheItalianoriginal:“[...]spessohotrovatoanchepuntidivistacritici,anchecriticiinsensodialettico;inaltricasihotrovatoineffetti

delleresistenzeancheallatipologiadiintervistaperchéessendoun’intervistadialogicanelmomentoincuidialoghidevidareall’altroilsuotempononsoloperrispondertimaancheperascoltarti,enonèdettochetuttiabbianovogliadiquestacosaperchégiàtiponiinunarelazionemoltoforte,èpiùimpegna-tiva,nonsolointellettualmentecometempo,èproprioimpegnativadalpuntodivistarelazionale”.

6 EnglishtranslationfromtheItalianoriginal:“Mentreilprogettoeraancorainitinere,abbiamoorganizzatodellepresentazionialivellolocale,pressopresidisanitarieamministrazionilocali.Nonsodireseabbiamoavutounimpattopoliticooseabbiamoavutouneffettosulleloropraticheconquestiincontrimapossodirecheabbiamoavutolapossibilitàdipresentareilnostroapproccioadunitàdibasedelserviziosanitarioedellapubblicaamministrazione,avendoconlorounproficuoscambiodiopinionisullametodologiaesullinguaggiodautilizzare.Abbiamoavutolapossibilitàdiesportareunpo'delprogettoneipostidovevorremmochefosseapplicatotuttiigiorni”.

ISSUE 48 | JULY 2019 149

The effects in terms of political impact were in fact remarkable.INCLUD-EDfindingshavebeenappliedonEuropean resolutions,com-municationsandrecommendations;theSEAsweretransferredthroughacrossEurope,producinginmostofthecasespositiveeffects.However,insomecases institutionalbarriersemergedthatconstrainedthepos-sibilityof research toproducean impact in specificnational contexts.TheeffectsproducedattheEuropeanpoliticallevelwereveryimportant:threeresolutionsbyEuropeaninstitutionsonearlyschoolleavingwereapproved,mentioningresultsobtainedthroughINCLUD-ED.Furthermo-re,tworesolutionsbyEuropeanUniononthethemesofsocialandedu-cationalinclusionofchildrenofmigrantsandRomapeoplewerealsoim-plemented,usingevidencesfromINCLUD-ED.Otherevidencesrelatedtonationalcontextconcern:recommendationsofthe“EducationMinistryonEducationandFormationStrategy2020”inwhichthedevelopmentofSEAsintheBasqueCountry‘appearsasapracticetofollow’;evidencesfromstateandregionallegislation,andfinally8agreementswithpublicadministrationsforimplementingofSEAsindifferentcountries(IMPACT-EV,2017b).

DISCIT

DISCITischaracterisedbycollaborativeeffortsinvolvingresearchersandstakeholdersassociations,representativegroupsofdisabled,deci-sionmakersthathelpedtodefinetheproblemsrelatedtodisabilityasacommonareaofintervention,withseveralarticulations.Thischangeofperspectiveallowedtocalibrateasbestaspossibletheinstrumentsofdirectinvestigationandtodefinethestructureofresultsinordertomakeiteasiertoproposetheirintegrationininstitutionalsettings.Atthesametime, the project created a stable forum for the interactions betweenresearchers,institutionsandorganisationsofpeoplewithdisabilities,todiscusstheproblemsofthedisabledpeople,toencouragetheexchangeanddisseminationofgoodpracticesandtocreatecommonunderstan-dingsbetweeninstitutionsandassociations(IMPACT-EV,2017a).

MODALITIES AND COMMUNICATIONS

Stakeholders’ involvement was directly related to the theoreticalframework used for analysing “Active Citizenship” (EC-EESC, 2012),whichwasarticulatedinthreesteps:a)toreviewtheinitialconditionsof the disability policy system and their configuration with respect toindividualswithdisability,theirfamiliesandtheirinclusioninlocalcom-munities,injobmarket,andsocialandcivilactivities;b)toanalysetheeffective implementationof themeasures indaily lifeofpersonswithdisabilities; c) to figureouthowthe resultsof thementionedanalysisinteractswithrespecttothethreepillarsoftheActiveCitizenshipaction,namelySecurity,AutonomyandInfluence(EC-EESC,2012).

Astothefirstpoint,stakeholdersgaverelevantfeedbacksontheef-fectiveapplicationof laws,highlightingthelevelsofprotectionforthe

variousgroupsofdisabledpeople.Thishelpedtheresearcherstohaveamorecompletevisionofthestateoftheart.Theeffectsofthisapproachare reported in the interview to the representative of one of the twoDPOsincludedintheconsortium:

“I think that one specific thing that my organisation bring to the con-sortium was this specific knowledge of the positions of rights of persons with mental disease that we discussed with other partners of the consor-tium. I have also a background as researcher at the university and I was a legal advisory of the organisation during the project but it was obvious that the project itself, all the other partners, at the very beginning needed this input from this specific area, it is not easily deducible from the official documents, because the attribution of rights for some categories of people is different from the prescriptions of the law.” (Stakeholder)

ThesecondpartoftheDISCITresearchconcernedtheinvestigationof the conditions of people with disabilities through a data collectionbasedon interviewswitha largeaudienceofdisabledpeople. In thisphase,thestakeholdersinvolvedintheprojectprovidedtheircontributi-ontothequestionnaireonwhichtheinterviewswerebased:

“I had the opportunity to talk with the stakeholders about the questi-onnaire. Feedback used to correct the methodological part were greater in the qualitative part, but in general it was a useful debate because it allowed me to focus on the types of indicators used subjects other than re-searchers, giving me a more balanced view of the problem.”(Researcher).

Also,stakeholdersactivelyparticipatedintheinterviews,proposingthemselvesasintermediatesbetweenresearchersandintervieweesandhelpingthelattertoovercometheembarrassmentoftalkingtostrangersabouttheirconditionofadisabledperson.

Membersof theDISCITconsortiumpaidparticularattentionto theorganisationofmeetingswithsocial,politicalandresearchinstitutionstodiscussthenewpointofviewfromwhichtheprojectaimedtoaddressthe issue of disability. The effects of these meetings were double: ontheonehand,theprinciplesunderlyingtheapproachweredisseminatedindependentlyoftheresults,layingthefoundationsforadiscussionondisabilityindiscontinuitywithrespecttothepast;ontheotherhand,themembersoftheconsortiumcouldgathertipstocorrectsomeelementsoftheirmethodologyofanalysis.Accordingtothemembersoftheproject,DISCITorganisedorhasbeeninvolvedintheorganisationofmorethan60internationalinitiativesoverthethreeyearsoftheproject.Inaddition,the national groups have taken charge of organising meetings of thesametypewithlocalinstitutionstoallowwidespreadcommunication:

“While the project was still in progress, we organised local presenta-tions in hospitals and local administrative offices. I cannot say whether we have had a political impact or if we have had an effect on their practices with these meetings but I can say that we have had the opportunity to present our approach to basic units of health service and public administ-ration, having with them a fruitful exchange of opinions on the framework and the language to be used. We had the opportunity to export some of the project to places where we would like it to be applied every day.”7 (Researcher)

7 EnglishtranslationfromtheItalianoriginal:“Mentreilprogettoeraancorainitinere,abbiamoorganizzatodellepresentazionialivellolocale,pressopresidisanitarieamministrazionilocali.Nonsodireseabbiamoavutounimpattopoliticooseabbiamoavutouneffettosulleloropraticheconquestiincontrimapossodirecheabbiamoavutolapossibilitàdipresentareilnostroapproccioadunitàdibasedelserviziosanitarioedellapubblicaamministrazione,avendoconlorounproficuoscambiodiopinionisullametodologiaesullinguaggiodautilizzare.Abbiamoavutolapossibilitàdiesportareunpo’delprogettoneipostidovevorremmochefosseapplicatotuttiigiorni”.

ISSUE 48 | JULY 2019150

TIMING

Collaborations between researchers and stakeholders were imple-mented throughseveralmeetingsorganisedover theprojectduration,open to thenetworkofactors involved. Theproject calendar includedthreeplenarymeetings.Alltherepresentativesofthestakeholdercom-mitteeswereinvitedtoparticipateinordertodiscusstheprogressoftheprojectwiththeresearchersandtoproposeinitiativestodisseminatetheresults. Plenarymeetingswere interspersedwithnationalgroupmee-tings.Inaddition,alongtheprojectmeetingswereorganisedbetweenmembers of the consortium and representative of institutions and as-sociationsexternal to theprojectbothatEuropeanandnational level.Inaddition to theofficialmeetings, the stakeholdershavebeencons-tantlyinvolvedwithrequestsforactivecollaboration,especiallyfordatacollectionanddiscussionoftheresults.Theconstantdemandforactiveparticipationwasparticularlyappreciatedbystakeholders:

“The request for participation was perfect, neither too much nor too little. We were asked to give our opinion on several points, but these re-quests were not concentrated in specific moments of the project develop-ment, as happened in previous experiences.”8 (Stakeholder)

LANGUAGE

Languageharmonisationwasoneofthemostsignificantanddifficultresult to achieve, theone thatproduced themost recognisable socialimpact.

Firsttheexchangeofinformationbetweenresearchersandstakehol-dersovertheprojectdurationwascrucial.Asreportedbyseveralinter-views,thesetwogroupsstartedfromdifferentdefinitionof“disability”andthedifferenceindefinitioninvolvedaseriesofdivisionsthatcouldgeneratemisunderstandings; theconsequenceofwhichwouldbe thefailureofresearchintermsofsocialandpoliticaleffects.DISCITactionshelpedtodisentanglethesedifferences,promptingresearcherstoassi-milate the languageofassociations inorder to increase the likelihoodof results tobe implemented inotherareas thanresearch.Within theproject, the interactions between stakeholders and researchers werealsousefultoovercomethedifferencesinlanguagebetweendifferentcountries.

Asecondimportant interactionwaswithorganisationsandinstitu-tions external to the project. In these occasions, a common languagewas agreed in order to avoid misunderstanding between researchers,associationsandinstitutionswhendisabilitywasrepresented,andthisresult improved substantially translation of research findings into ap-propriablegoods.Infact,themostinterestingelementwasovercomingculturalgapsbetweendifferentstakeholdersastothedefinitionofdisa-bility,achangethatproducedeffectsbeyondtheaimsofDISCIT:

“A problem is what really disability means. There are two understan-dings of disability. One is the sense of disability as marker of marginalised group of population. But there is another sense of disability which is a

8 EnglishtranslationfromtheItalianoriginal:“Larichiestadipartecipazioneèstataperfetta,nétropponétroppopoco.Cièstatochiestodiesprimereilnostropareresudiversipunti,maquesterichiestenoneranoconcentratoinmomentispecificidellosviluppodelprogetto,comeaccadutoinprecedentiesperienze”.

9 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2015:0615:FIN10 http://ec.europa.eu/social/main.jsp?catId=1137&langId=en

phenomenon that people indeed experience during their life, namely some sort of limitation in functional ability. This second sense of disability is more a universal sense of disability but does not involve marginalisation of groups. People tend to define themselves in one of the two groups on the basis of a sort of self-definition, with respect to the impact that the limi-tation they experience has on their everyday life. [...] The lack of skills in a particular context does not nullify the person as a whole, so it is necessary to rethink the concept of disability, bringing it closer to the most universal sense to prevent policies to support people with disabilities become a way to marginalise a part of the population and deprive them, in fact if not legally, of some rights as human beings.” (Researcher)

OUTCOMES

DISCITsetouttopromotetheimplementationoftheConventionontheRightsofPersonswithDisabilitiesinEuropeanandnationallegisla-tion(UNCRPD).Despitetranslationofresearchfindingsintothepoliticalprocessestooklongerthanthedurationofaproject,someelementsofimpactonEuropeanandlocalmeasureshavebeenobserveddirectlydu-ringtheactivityofDISCIT.

AttheEuropeanlevel,thecomponentsoftheDISCITresearchteam“ActiveCitizenshipthroughtheuseofNewTechnology”wereinvolvedduring the drafting of the European Directive “European AccessibilityAct”9,preparedbytheDirectorateGeneralforEmployment,SocialAffairsandInclusion.Furthermore,DISCITresearcherswereinvitedtobepartoftheHigh-LevelGrouponDisability10,composedbyEuropeanandnationalexpertschosenfrompolicymakersandstakeholders,inchargeofdefinethestrategiesforimplementationoftheUNCRPD.Otherpoliticaleffectswereobservedatnationallevel,forinstancewiththeinvolvementoftheItalianresearchgroupinthepreparatoryworkofthelawoftheTuscanyRegion forsupport to familiesofdisabledpersonsand theauditionattheNationalObservatory forDisabilitiesof theMinistryofLabourandSocialPolicies. The Irish research teamparticipated inanational taskforcethatlaunchedatrialofsupportivepoliciesforthedisabledonmoreinclusivebaseswithrespecttocurrentlegislation.Finally,theSwedishresearchgroupelaboratedsomeguidelines,adoptedbyinstitutionslikethe Swedish Agency for Participation. Interviews demonstrated that anewpointofobservationwasdevelopedpreciselythroughthedialoguebetweenresearchersandstakeholdersonwhichtheprojectwasbased:

“I think that this project has broaden the research community know-ledge because it has forced the academics to discuss their approach with organisations and to consider this information.”(Stakeholder)

“During the international meetings I had the opportunity to meet res-ponsible of associations that work in community living sector from other countries, in particular I was positively impressed by the practices used in Sweden […] I proposed to use some of these ideas, in experimental way, in order to test if they fit with our social context, and some preliminary results seem to be positive.” (Stakeholder)

DISCUSSION

ISSUE 48 | JULY 2019 151

BoththeillustrativecasestudiesonprojectscarriedoutundertheEu-ropeanFrameworkProgrammesshowthatstakeholdersgenerateadiffe-rentapproachtothesocialproblemaddressed;themainfeaturesofthestakeholders’ interactionswithresearchersaresummarised inTable1.

Modalities and communications

Timing Language OutcomesModel of interactions(Muhonen et al. 2018)

DISCIT

Circularexchangeofinformation

Diffusionanddiscussionofthemethodsofanalysiswithexternalstakeholders

Intenseinvolvementinthereviewandinvestigationphases,partialdiscussionofpolicyproposals

Harmonisationbetweencountriesandareasofinterest

FormalinvolvementinpolicymakingprocessExchangeofbestpractices

Culturalimpact:newideaofdisability

Collaborationmodel

Researchengagement

Knowledge“creeps”intosocietymodel

INCLUD-ED

Dedicatedeventsfortargetedstakeholders(trainingforteachers,disseminationforscholars,politicalmeetingsforinstitutionalrepresentatives)

Continuousinvolvementofallstakeholdersalongthefiveyearsoftheproject

Communicativemethodology

Formalstakeholders’involvementinknowledgecreation

Replicabilityoftheoutputsindifferentnationalandinstitutionalcontexts

Collaborationmodel

Publicengagementmodel

Mobilitymodel

Using the Muhonen and colleagues (2018) typology, INCLUD-EDdevelopedinteractionswithstakeholdersthatmainlybelongtotheco-

creationtypology,andtheactivitiesadoptedelementsthatrelatetocol-laboration,publicengagementandmobilitymodels.DISCIThadamorehybrid structure, which belong to the co-creation typology – throughcollaboration, and driving social change typology – through activitiesthatfeaturedtheresearchengagementandtheknowledge“creepsintosociety” models11. In this respect, typologies aimed at understandingchangesproducedthroughtheinvolvementofstakeholdersinresearchactionsisahelpfultoolforcomparingdifferentconfigurationsofthere-lationshipswithinthenetworkofactorsinvolved,whichcanalsosupportamorepreciselytracingofthetranslationaleffectsgenerated.

The co-development of a new language and harmonised wordingproducedacultural impactwhichwasextremelyimportantandtookalongtime.However,itisatypeofimpactdifficulttosingleoutthroughempirical observations related to measurable items; furthermore, alsotheimpactatpolicyleveltookalongtimetoemerge(beyondtheprojecttimelimit)anditwasinbothcasesadirectconsequenceoftheculturaltransformation. In this respect, stakeholdersare keycarriers for socialimpactinSSresearch.

Thetransformativeeffectsonsocietywerelinkedtotheco-produc-tionof knowledge that isusedby societal actorsbut, in turn, the co-production of knowledge needed the development of an appropriatecommunicationtodeconstructthecontent,organisationalfeaturesandknowledgecarriers.Theformalandinformalconfrontationbetweenre-

Table 1.Comparisonofprojectsontypologiesofstakeholdersinvolvement.

searchersandstakeholders–whenitisarecurrentmechanismofnet-workingratherthananendogenouseventforthemshowedenormouspotentialforproducingtranslationaleffects.However,thesustainabilityofthetransformationsproducedthroughtheprojectsisanelementthatwentbeyondtheeffortoftheresearchteams.Thedurationoftheprojectandtheresourceshavenotbeenentirelysufficienttohavethechancethateffectscouldremainovertime,especiallywheninstitutionalbarriersappeared.

How these results are relevant for the evaluation of research pro-jects? Some general advantages of stakeholders’ participation can beoutlined.Ontheonehand, ithelpstofigureoutatcertainextentpro-blemsofattributionof impactsproducedbytheproject,andthis isanimportantsupporttofigureoutthepresenceofcausallinkagesbetweenprojectoutcomesandeffectsonsociety.Ontheotherhand,stakeholdershelpedtofolloweffectsderivedfromtheprojectforalongerperiodaftertheprojectcompletion. It ismoredifficult tounderstandhowthecha-racteristicsof theprojectorganisationsand themodesof interactionsbetweenresearchersandstakeholderscanbeassessedthroughspecificcriteriarespectivelyatex-anteandex-postlevel.Hereitisimportanttohighlighttwomainelementsincommonoftheillustrativecasestudiesanalysed.

First,inbothcasesthescientificqualityoftheoutputswasverygood.Bibliometricindicatorsandweb-basedindicatorsshowthatthescientific

11 AccordingtoMuhonenetal.2018(pp.14-16)the“Collaborationmodel”ischaracterisedbyresearcherscollaborateregularlywithstakeholders.Impactisgainedthroughopenaccessideologyandthroughinterdisciplinaryortransdisciplinaryapproach.Inthe“Publicengagementmodel”“resultsofresearcharetakenintoactionbyusingsocietyasalaboratory.Publicityisanecessityforimpact.”.Inthe“Mobilitymodel”“knowledgeandskillsofaresearcheraretakenintouseinanewcontext”.Researchengagement“increasesawarenessofthetopicathand.Targetsofthestudygetrecognitionandsenseofempowermentthroughtheresearchprocess”.Inthemodel,knowledge‘creeps’intosociety’sdailylife’sandpoliticalarenachangesareproduced“lateroninrelationtopublicopinionorlegislation”.

ISSUE 48 | JULY 2019152

valueoftheprojectsresearchoutputs(papers in international indexedjournals,books,bookchapters, etc.)werewell recognised in the fieldcommunity.Thiselementproducedapositivefeelingaboutthecapabilityoftheprojecttorealisesoundresearchresults,despitesomecriticismsemergedintheinterviewsaboutthepossibilitythattakingon-boardcon-siderations coming from the interactions with stakeholders is likely todecreasetheoriginalityoftheresearcheffort,impedingveryinnovativeresults.

Second,bothcasesareexamplesofprojectspursuingimpactusinga theory-based approach: INCLUD-ED used the Habermas’ theory ofagents of social change and the critical communication methodology;DISCIT used the Active Citizenship approach. The effect was that im-pactwas fully integrated in the theoretical frameworkof the researchprojects,drivingthesubsequentphasesofthedesignandimplementa-tionsof stakeholders’participation.Also, the interactionsbetween re-searchersandstakeholderswereimplementedaccordingtoconceptualframeworksthatincludedthestakeholders-eithertheywerepartnersoftheconsortiumorexternaltotheproject–asmainactorstoachievetheintendedobjectives.Thiselementreducedsomeverywell-knownshort-comingsgenerallylinkedtostakeholders’interactionsduetolowcom-mitmentandcontributiontoresearchactivitiesovertheprojectduration.Finally,buildingcommonharmonisedlanguagesindifferentcontextsofapplicationemergedasthemostimportantelementtogenerateimpactunderaco-creationmodel;however,thisresultcanbeachievedthroughdedicated efforts, and it cannot be considered as a taken for grantedelement.

Insum,theory-basedapproachesofstakeholders’involvement,buil-ding a common language, in combination with organisational featuresandcareful timingof the interactionsareall importantelements tobeconsideredinex-anteevaluation.Thepresenceoftheminthedesignoftheprojectshouldimprovethelikelihoodthatanimpactmightoccur.Inthesamevein,thementioneditemsshouldbeassessedovertheprojectimplementation in order to understand whether the research activitieswereproperlydevelopedtoachievetheobjectiveofproducinganimpact.Also,inanex-postassessmentthelinkagebetweenscientificoutputsandimpactisanissuethatdeserveattentioninordertoavoidatrade-offbet-weenpursuinganimpactandthequalityoftheresearchoutputs.

CONCLUSIONSStakeholders’ participation to research efforts is definitely an im-

portantelement to reachsocial impact.For research insocialscience,stakeholdersarekeycarriersfortranslatingresearchresultsintoculturalchanges,whicharelikelytoenabletransformativeeffectsofsociety.Fur-thermore,stakeholdersrepresenttheinterestsofsocietyandthismightempower them tomediate researchoutcomes topolicymakersbetterthanresearchersthemselves.

In thispaperwedeepened twocases related toaspecificcontextofapplication, that is thedevelopmentof researchprojectsunder thefundingofEuropeanFrameworkProgrammes; theanalysisshows thatorganisationandcommunication,timingandlanguagearekeyitemstorealisefruitfulinteractionsthatcanproduce–orcontributetoproduce–animpact,translatingscientificknowledgeintoappropriablegoods.

We also pointed out some items that should be considered in theevaluationofresearchprojects,bothatex-anteandex-postlevel,chan-

ging tosomeextentcriteriaandmethodsof impactassessment inSSresearch.However,howthiscouldberealisedinconcretetermsisdefi-nitelyanopenquestionthatneedsmoreresearcheffort.

REFERENCES Bell, S., Morse, S. and Shah, R. A.(2012).UnderstandingstakeholderparticipationinresearchaspartofsustainableDevelopment.Journal of Environmental Management101,pp.13-22

de Jong, S., Barker, K., Cox, D., Sveinsdottir, T. and Van den Besselaar, P. (2013). Understanding societal impact through studying productiveinteractions,RathenauInstituutWorkingpapern.1304.

EC-EESC (2012). Active Citizenship. For a Better European Society. EC,EuropeanEconomicandSocialCommittee,Brussels.

Edelenbos, J., van Buuren, A. and van Schie, N.(2011).Co-producingknowledge: joint knowledge production between experts, bureaucratsandstakeholders inDutchwatermanagementprojects.Environmental science & policy14(2011):675–684.

Flecha, R. and Soler, M. (2014)CommunicativeMethodology:Success-fulactionsanddialogicdemocracy.CurrentSociologyMonograph,62(2):232–242

IMPACT-EV (2017a). Descriptive report of the case study on project“DISCIT – Making persons with disabilities full citizens”; IMPACT-EV,IRCrES–CNR

IMPACT-EV (2017b). Descriptive report of the case study on project“INCLUD-ED:StrategiesforinclusionandsocialcohesioninEuropefromeducation”.IMPACT-EVConsortium,IRCRESCNR.

Joly, B., Gaunand, A., Colinet, L., Laredo, P., Lemarié, S., Matt, M. (2015).ASIRPA:A comprehensive theory-basedapproach toassessingthesocietalimpactsofaresearchorganization.Research Evaluation,24,440-453

Lang, D.J. and Wiek, A., Bergmann, M. et al.(2012).TransdisciplinaryResearchinSustainabilityScience:practices,principlesandchallenges.SustainSci,7(Suppl1):25-43

Mayne, J. (2012). Contribution Analysis: Coming of Age? Evaluation,18(3),270-280

Molas-Gallart, J. and Tang, P.(2011).Tracing“productiveinteractions”toidentifysocialimpacts:Anexamplefromthesocialsciences.ResearchEvaluation,20(3):219–226

Muhonen, R., Benneworth, P. and Olmos-Peñuela, J. (2018). Fromproductive interactionsto impactpathways.Understandingthekeydi-mensions indevelopingSSHresearchsocietal impact.CHEPSWorkingPapers02/2018availableat:http://www.utwente.nl/bms/cheps/Ozanne, J. L., Davis, B., Murray, J. B., Grier, S., Benmecheddal, A., Downey, H. and Veer, E. (2016).AssessingtheSocietal ImpactofRe-

ISSUE 48 | JULY 2019 153

search:TheRelationalEngagementApproach.JournalofPublicPolicyandMarketing.DOI:10.1509/jppm.14.121

Penfield, T., Baker, M.J., Scoble, R. and Wykes, M.C.(2014).Assess-ment,evaluations,anddefinitionsofresearchimpact:Areview.Research Evaluation23,pp.21–32doi:10.1093/reseval/rvt021

Reale, E., et al.(2017).Areviewofliteratureonevaluatingthescientific,socialandpolitical impactofsocialsciencesandhumanitiesresearch.Research Evaluation,11(1)SpecIssue,onlinefirst,https://academic.oup.com/rev/article/doi/10.1093/reseval/rvx025/3978693/A-review-of-lite-rature-on-evaluating-the?guestAccessKey=ea3f8277-caee-4e3b-854a-06d9d3e939de

Reale, E., Primeri, E. and Fabrizio, S.(2017).Assessingsocialresearchimpact: exploiting productive interactions and dialogic learning. PaperpresentedattheCHER30thConference,Javaskyla,28-30August.

Spaapen, J. and van Drooge, L.(2011).Introducing‘productiveinterac-tions’insocialimpactassessment.Research Evaluation20(3):211–218.

Talwar, S., Wiek, A. and Robinson, J.(2011).Userengagementinsus-tainabilityresearch. Science and Public Policy,38(5),June2011,pages379–390, DOI: 10.3152/030234211X12960315267615; http://www.in-gentaconnect.com/content/beech/spp

Walker T., Maredia, M., Kelley, T., Rovere, R., Templeton, D., Thiele, G., Douthwaite, B.(2008).Strategic Guidance for Ex Post Impact Assess-ment of Agricultural Research.ScienceCouncilSecretariat,Rome,Italy.Report prepared for the Standing Panelof Impact Assessment, CGIARScienceCouncil

Wiek A., Talwar, S., O’Shea, M. and Robinson, J.(2014).Towardame-thodologicalschemeforcapturingsocietaleffectsofparticipatorysus-tainabilityresearch.Research Evaluation,23,pp.117–132doi:10.1093/reseval/rvt031

Zardo P, Barnett, A.G., Suzor, N. and Cahill, T.(2018).Doesengage-mentpredictresearchuse?AnanalysisofTheConversationAnnualSur-vey 2016. PLoS ONE 13(2): e0192290. https://doi.org/10.1371/journal.pone.0192290.

AUTHORS

EMANUELA REALE IRCRES – Research Institute for Sustainable Economic Growth, National Research Council19ViadeiTaurini,Roma,00139(Italy)E:[email protected]

SERENA FABRIZIOIRCRES – Research Institute for Sustainable Economic Growth, National Research Council19ViadeiTaurini,Roma,00139(Italy)E:[email protected]

LUCIO MORETTINI IRCRES – Research Institute for Sustainable Economic Growth, National Research Council19ViadeiTaurini,Roma,00139(Italy)E:[email protected]

ISSUE 48 | JULY 2019154

TheHubITproject,fundedunderthetopic“BoostinginclusivenessofICT-enabled research and innovation” (REV-INEQUL-09-2017) is part oftheoverallSSH-RRIapproach.ItaimstobringtogetherICTdevelopers,SSHresearchersandotherstakeholders(NGOs,citizensandusers)ac-rossH2020ICT-relatedprojectsandbeyond,inordertoattuneICTdeve-lopmentwithsocietalneedsandfostertheSSH-RRIapproach.

THE CONCEPT OF “RESPONSIBLE RESEARCH AND INNOVATION” (RRI)

OneofthemorewidelyaccepteddefinitionsofRRIthatemphasisestheroleofSSHresearchers,wasdevelopedbyVonSchomberg(2013).Accordingtothisdefinition“ResponsibleResearchandInnovationisatransparent,interactiveprocessbywhichsocietalactorsandinnovatorsbecomemutuallyresponsivetoeachotherwithaviewtothe(ethical)acceptability, sustainability and societal desirability of the innovationprocessanditsmarketableproducts.”(VonSchomberg,2013,p.19).

FurtherelaborationoftheseideasbytheappointedEuropeanCom-mission (EC) expert group described six major dimensions of RRI thatsignifytheimportanceofkeepingtothenormsofresponsibleresearchandinnovationthatconsidersdifferentsocietalneeds.Amongthemare:public engagement, gender equality, science education, open access,ethics,governance.Twoadditionaldimensions,sustainabilityandsocialjustice,overlapwiththepreviouslynamedones(Strandetal.,2015).AllthesedimensionsrequiretheinvolvementofSSHexpertsintheprocessofICTdevelopment.

Embedding SSH researchers into ICT research and innovation is achallenge.TheintegrationoftheSSH-RRIperspectiveintoICTresearchand development is accompanied by specific problems. Jirotka (2017)identified the following: First, the difficulty to predict potential usesof ICT research outcomes since uncertainties in this field are sociallyshapedandfixedratherthanscientificandnotfixed.Aseconddifficultystemsfromthedifference in thequicker“rhythm”of ICTdevelopmentcomparedtootherfields,assoftwaremaybedevelopedandpotentiallygoviralinthesameday.Third,thereisaproblemstemmingfromdiffe-rentdisciplinarylanguagesinvolveinICTresearch,thatmakesinterdisci-plinaryworkmoredifficult.

ABSTRACT

The development of information and communication technolo-gies (ICT) introduces radical changes inour lives. These tech-nologiesprovideanswerstoamultitudeofpeopleneeds,butat

thesametimetheyincreasetheconcernsabouttheiractualthreatsandsocietalimpacts.Thiscallsforadoptingaresponsibleresearchandinno-vationperspectiveintheprocessofdevelopingICTsolutions.Thispaperpresents preliminary results of the “Social Impact Assessment” (SIA)plan and tools that were developed within the EU-funded HubIT pro-ject.Thestudyemployedbothquantitativeandqualitativeethnographictools (e.g. surveyquestionnaireandobservations), inorder toaddressthechallengeofconductinga“ResponsibleResearchand Innovation”(RRI)assessmentofaEuropeanproject,focusingonpromotingRRI.Theprojectaimsatcreatinganecosystemthatencouragesinteractionsbet-ween ICT developers and “Social Sciences and Humanities” (SSH) re-searcherstoensureresponsibility inICTresearch.Firstresults indicateanincreaseinunderstandingandawarenessoftheSSH-RRIapproachamongSSHandICTresearchersandanincreaseoffutureplansforcol-laborationsbetweenthesetwogroups.ConclusionsaremadeastohowtheseresultscanbefedbackintotheHubITproject,aswellasserveasabasisforthepolicyrecommendationstoEuropeanandnationalbodies.

INTRODUCTIONThedevelopmentofICTintroducesradicalchangesinourlives.These

technologiesprovideanswerstoamultitudeofpeopleneeds,whileatthesametime increasingconcernabout their threatsandsocietal im-pacts.Thiscallsforadoptinga“ResponsibleResearchandInnovation”(RRI) perspective in the process of developing ICT solutions. The coreofthisapproach iscreatingamutualdialogbetweenSSHresearchersand ICT researchersanddevelopers. Indeed, in the year2012 theEu-ropean Commission adopted the SSH-RRI approach and defined it asacontinuousengagementofsocietalactorsduringthewholeresearchandinnovationprocessinordertobetteralignboththeprocessandtheoutcomesoftheirresearchwiththevalues,needsandexpectationsof“EuropeanSociety”(EuropeanCommission,2012).Furtheron,RRIwasintroduced as a cross-cutting political aim in the “7th Framework Pro-grammeoftheEuropeanUnion”anditcontinuestobeakeyconceptinthecurrent“Horizon2020Programme”.

TALSOFFER,RUTHZUZOVSKY,OLENANEDOZHOGINAANDEMANUELEBARDONEDOI:10.22163/fteval.2019.384

ASSESSINGTHEIMPACTOFSSH–RRIAPPROACHONICTRESEARCH&INNOVATION:THEHUBITPROJECT

ISSUE 48 | JULY 2019 155

ThesedifficultiescreatedtheneedtoconsidersocialaspectsintheprocessofICTdevelopmentandled,amongotherthings,toinitiatetheHubITproject.TheHubITproject(runtime:2017-2020)aimsatactivatingaconstructiveinteractionbetweenSSHresearchersandICTdevelopers,inorder to implementasocially responsibleapproachto researchandinnovationinICTprojects.Thisapproach–termedtheSSH-RRIapproach–isatthecentreoftheassessmentactivitiesoftheHubITproject.

ASSESSING THE SSH-RRI APPROACH

ForassessingtheSSH-RRIapproachintheHubITproject,the“SocialImpactAssessment”(SIA)methodologywasadopted.Thismethodologyisdefinedas“the process of identifying the future consequences of cur-rent or proposed actions, which are related to individuals, organizations and social macro-systems”.(Becker,2001,p.312).Beckerdescribesthismethodology as having two phases: a) An initial phase, including ananalysis of the problem. In the case of the HubIT project, identifyingsomenegativeconsequencesofICTdevelopment,systemanalysisandprojectdesign;andb)Amainphase,includingscenarioplanning,stra-tegicdesignandanassessmentofimpacts.Vanclayetal.(2015)follo-wedthisschemeandpreparedaguidetosocialimpactassessment.Theguideincluded26tasksthataredividedintofourphases:1.Understandtheissue;2.Predictingthelikelyimpact;3.Developingandimplemen-ting strategies to mitigate negative societal consequences; 4. Designand implementing monitoring programmes. Since many of the tasksspecifiedbyVanclayetal.(2015)canbefoundwithintheHubITprojectactivities,theassessmentplanfocusedonthesetasks.Theseactivitieshavespecific formats (e.g.workshops, conferences,hackathonsetc.),targetdifferentaudiencesandleadtodifferentoutputs(e.g.anonlineplatform,visualmaterials,reportsorpolicybriefs).Thevariabilityoftheactivitiesdictatesdifferenttoolsandevaluationcriterianeededfortheassessment.

THE DESIGN OF THE ASSESSMENT PLAN INCLUDES THREE STAGES:

The first stagewastomapoutthecharacteristicsofeachactivityi.e.specifyingthemainobjectives,expectedoutcomesandrelevanceoftheRRIdimensionswhicharepartofeachactivity.

The second stagefocusedontheidentificationoftherelevanttypesof indicators, measures and questions that tackle each of the six RRIdimensions.ThisstagestartedwithacomprehensivereviewoftheRRI-relatedevaluationeffortsconductedbyotherprojects,suchas“DoingItTogether-Science”(DITOs),“MonitoringtheevolutionandbenefitsofRe-sponsibleResearchandInnovationinEurope”(MoRRI),RRITools,etc.,aswellaswiththereviewofthemoretheoreticalstudies(Blonder,Rap,ZemlerandRosenfeld,2017;VonSchomberg,2011)andseveralreportsfromtheEuropeanCommission(2012,2013,2015)onRRI.Consequently,abankofassessmentmeasuresandquestionswascreated.

The third stageinvolvedaroundtablediscussion(calledthe“HubITgame”)wherethepartners,responsibleforcertaintasks,wereaskedtodiscussandselectfromthebankofassessmentmeasuresandquestionswithrespecttothosethatcovertherelevantRRIdimensionsthatappear

inthosespecifictasks.Basedontheresultsfromthediscussionsheldinthegroups,specifictoolsweredesignedforassessingtheimplementa-tionoftheHubITevents.

In addition, the SIA methodology included a qualitative evaluationpartthatfocusedonthenarrativesthataccompanytheinteractionbet-weenSSH-ICTresearchesduringtheactivities.Theneedforamorequa-litativeapproacharosealreadyattheliteraturereviewstage.ItbecameevidentthatacertaindissonancebetweenthecurrentstateoftheartinthefieldofRRIimpactassessmentandtheactualevaluationpracti-cesexist.Evaluationpractices,promotedbythefundingbodies,nationalandsupranationalauthorities,provideencouragementtobeaccounta-ble (tick theboxes),butarenotalways responsible (reflexive,orientedtowards strategic societal goals) in the meaning of being accepted intheRRIresearchcommunity.Currentformsofevaluationsmainlydonotlookattheprocess,andtheevaluationisconceptualisedassomething“outside”of theproject,while inreality it isusuallydeeplyembeddedintheprojectpracticeandisconductedbyprojectpartners.ThiscanbeconnectedtotherecentfindingsofFelt(2016),whowarnedaboutthedangeroftheemphasisonRRIandotherSSH-relatedpracticesinsci-enceandinnovationturningintoasimple“annex ritual to be perform at the beginning and at the end of the project” (Felt2016:15),encouragingaccountability,butnotreflectivity.

Inthisway,byemployingprocessorientedethnographicmethods,theevaluationeffortsbecamealsopartiallyshapedbythecommunity,surrounding the project, and partially driven by a desire to compre-hendandimprovetransdisciplinaryandresponsibilityoftheproject.Inthissense,acommunitywasformedaroundtheevaluationactivities,activelyengagingpartnersandstakeholdersintheprocessofassess-ment.

THE ASSESSMENT ACTIVITYAssessment activities that were enacted in the first two project-

specificeventsare in the focusof the followingsection.TheseeventsweremeanttobringtogethermembersoftheICTandSSHcommunities,publicsectorrepresentatives,policymakersandotherstakeholders.TheeventsintendedtopresenttheconceptofRRI,theHubITprojectandthe“EuropeanFrameworkModel”(aplatformthatwasdevelopedandpre-sentsthevariousresourcesandactivitiesoftheproject).Theeventsalsoaimedatidentifyingsocietalneedsthatareassociatedwithtechnologi-caldevelopmentsandsupportedmatchmakingbetweenICTdevelopersandSSHresearchers.

The assessment activities were conducted during a national work-shop in Slovakia in May 2018 and a triple event (annual conference,workshopforsocialscientistsandspeed-dating)inTartuinSeptember2018.Theaimsof thenationalworkshop,aswellas theTartuevents,weretoraiseawarenessandunderstandingoftheroleoftheSSH-RRIapproachandtoboostcollaborationbetweenSSHandICTresearchcom-munities.Theworkshopevent inSlovakia included27participants.20outof them responded toanonlinequestionnaire thatdealtwith theaboveexplainedaims.Intotal,64personsparticipatedintheeventsor-ganisedinTartu(AnnualConference,SSHworkshop,networkingsessi-on).Again,20participantsrespondedtothequestionnaire.

Theevaluationactivitiesmeanttoanswerthefollowingquestions:1. Towhatextentdidtheeventsucceedintargetingmembersof

theICTandSSHcommunities?

ISSUE 48 | JULY 2019156

2. Towhatextentdidtheeventcontributetomutualunderstand-ingoftheICTandSSHcommunitiesinthebenefitsofbridgingbetweenthem?

3. TowhatextentdidtheeventcontributetoidentifyingsocietalproblemsthatstemfromICTdevelopment?

4. Towhatextentdidtheeventcontributetocollaborativeteam-workofICTandSSHresearchers?

5. TowhatextentdidtheeventcontributetotheacceptanceoftheRRIapproachalongitssixdimensions?

Theassessmenttoolsthatweregeneratedatthisstageoftheprojectrepresentedtwomodesofassessment:aquantitativetool,whichinclu-desanonlinequestionnairewith20items,andaqualitativetool,whichincludesanobservationguideforoutsideobservers.Theobservationac-tivitiesfocusednotonlyontheoverallorganisationandimplementationoftheevent,butalsoonthedynamicsofinteractionbetweenSSHandICTcommunities,aswellasonthenarratives,surroundingRRI.Obser-vationsalsoincludedethnographicnotestakenbytheprojectpartnersduring the events, based on participants’ discussions (as each eventdevoteda significantamountof time toworld café stylediscussions).Themainaimofthequalitativeevaluationactivitieswastocollectandanalysethenarratives, surroundingtheconceptsofRRI,researchinclu-sivenessand,especiallySSH-ICT interaction. Thesenarrativesallowedidentifyingpossibleweakpointsoftheprojectstructureandunforeseenchallengesthattheprojectneedstoaddress,aswellasrecentdevelop-mentsinthediscourseofRRI.

RESULTSSURVEY RESULTS

Figure1presentsthedistributionoftherespondentswhoparticipatedinthetwoeventsaccordingtotheirdisciplineorfieldofactivity(N=39).

Mostoftheparticipantsrepresentedsocialsciences(41%)andhu-manities(13%),mainlybecausethesetwoeventsspecificallyfocusedonthistargetgroup.However,theamountofinvolvedICTresearchersandspecialistsisstillhigh(28%).Thenumberofpublicofficialsanddecisionmakersisrelativelysmall,andwillincreaseinfutureevents.

THE BENEFIT OF BRIDGING BETWEEN THE TWO COM-MUNITIES IN SUPPORT OF AN RRI APPROACH IN ICT DEVELOPMENT.

Figure2presentsrespondents’perceptionsregardingtheinteractionbetweenSSHandICTinsupportofRRIapproach.ThehighestlevelofsupportisrelatedtothestatementabouttheusefulnessofSSHcollabo-rationinICTdevelopment(Range:Likertscalefrom0to5;Median (M) =4.3,Standard Deviation (SD) =0.66),whilethelowestlevelofsupportisconnectedtotheperceptionthatSSHisaburdentoICTresearch(M=1.70,SD=0.983).Despite the fact that thenationalworkshopandtheeventsinEstoniahadsomewhatdifferingtargetaudiencesanddis-tribution of participants by discipline (national workshop was focusedonamorediverseaudience,whiletheTartueventsfocusedspecificallyonSSHresearchers),resultsdonotshowmajordiscrepanciesbetweenattitudesandperceptionofparticipants.

AWARENESS OF THE CONTRIBUTION OF SSH IN-VOLVEMENT IN ICT DEVELOPMENT TO THE IDENTIFI-CATION OF SOCIETAL PROBLEMS

Figure3presentstherespondents’awarenessofthecontributionofSSH-ICTcollaboration to the identificationofsocietalneedsandprob-lems,aswellastheproductionofsolutionstotheseproblems.There-spondents foundthatparticipation in theworkshophelpedthemonamediumtohighlevelintermsofthreeaspects:learningaboutsocietalneeds,identifyingsocietalproblemsthatcanbesolvedbycooperationbetweenICTandSSHcommunitiesandfindingpartnersforfuturecol-laborations.

PERCEIVED OPTIONS AND WILLINGNESS FOR ICT –SSH COLLABORATION

Basedonthetwoevents,mostoftheparticipants(80%-83%)foreseefutureengagementincooperationwithpeoplefromtheotherfields(ICTorSSH),andmostofthem(77%)foundthattheworkshopeventwasveryuseful (M=4.03).Additionally,basedon thespeed-datingeventeva-luation,77%ofparticipantsforeseeengagementwithICTresearchers,33.3%havealreadycontactedapersontheymatchedduringnetworkingand55%plantodoso.

Figure 1. Distributionofparticipantsintheeventsbydiscipline.

ISSUE 48 | JULY 2019 157

Figure 2. TheinteractionbetweenSSHandICTinsupportoftheRRIapproachduringeventsinSlovakia(SK)andEstonia(EE),onascalefrom0to5.

Figure 3. ContributionofSSHinvolvementinICTdevelopment.

ISSUE 48 | JULY 2019158

approachand initiating futurecollaborationbetweenactors, represen-tingSSHandICT.

Basedonthequalitativepartoftheassessmentthefollowingnarra-tiveswereidentified:

QUANTIFICATION OF “RESPONSIBILITY” AND THE TICK-BOXING LOGIC

“IfeelthatRRIisafancytermthatEuropeanCommissionhascomeupwiththatismoreoftenthannotusedasanemptysignifier(i.e.itisjust put into documents without following the principle)” (Participant,expertworkshopinRome)

OneofthefirsteventsorganisedbytheHubITproject–theexpertworkshopinRome–broughtforwardconcernsthatwouldaccompanyprojectdiscussionsfromthispointforward.Theissueofquantificationofresponsibility–thatRRIcanbereducedtoasimplelistofquantifi-ablekeyperformance indicators–wasdiscussedat length.Later, thisnarrativewasechoedduringtheSSHworkshopinTartu:fearthatRRIisjustsomethingthatneedsto“checked”,butnotfollowedinspirit,wasbrought forward from thecomment sectionof theevent report to thequestionnaire(partofeventevaluation).

However,weinterpretedthisnarrativeasanopportunitytoimproveourownHubITpractices,whichpromptedeffortstoaddaqualitative/

UNDERSTANDING AND ACCEPTING THE CONCEPT OF RRI

ConcerningunderstandingoftheconceptoftheSSH-RRIapproach,mostof the respondents (62%) indicated that they improved theirun-derstandingof thisapproachtoahighorveryhighextent.Significantdifferenceswerefoundbetweenthetwoevents:forthenationalwork-shopM=4.17,SD=0.85andfortheannualconference/SSHeventM=3.00,SD=1.6.

The participants’ agreement with statements reflecting attitudestowards the RRI various dimensions ranged from a medium to a highlevel(seefigure4below).Specifically,thoserelatedtoopenscienceandethicsdimensions,whichfocusedontheneedforofficialethicscommit-teesinorganisationsandmandatorytrainingonresearchethics.

AsconcernsthenationalworkshopinSlovakia,therespondentsin-dicatedthatthesixRRIdimensionswereaddressedexceptionallywellduringtheworkshop,especiallythedimensionsof:publicengagement(M=4.00,SD=0.78),genderequality(M =4.33,SD=0.9),openaccess(M=4.00,SD =1.1),andgovernance(M= 4.06,SD=0.97).TheannualconferenceeventinTartuhadmoremoderatescores(M=3.31,SD=1.6toM=3.77,SD=1.16)forthedifferentdimensions.

Tosumup,theresultsfromthetwoeventsindicatethesuccessofthistypeofeventinraisingunderstandingandawarenessofanSSH-RRI

Figure 4. PerceptionregardingthesixdimensionsofRRI.

ISSUE 48 | JULY 2019 159

THREATS VERSUS OPPORTUNITIES

ThefollowingquotationhighlightsthetensionbetweenSSHandICTresearchers:

“ICT representatives talked rarely to the SSH people (experience based on one table)…ICT people seemed to be more involved (engaged) in the threats discussion, while SSH more in the opportunities.” (Observation, national workshop in Slovakia)

WhilethereisageneralpresuppositionthatSSHresearchersempha-siseresponsibility,risksandthreatswhendiscussinginnovation,andICTresearcherslookmoreintoopportunities,thisparticularexampleshowedanoppositepicture.Itmightsimplyreflectthecurrentstateofthegene-raldiscourseoninnovationandglobaldevelopment:whilethebacklashagainst “irresponsible” ICT innovationhasmademore ICT researchersawareoftherisksandpitfallstheymightface,thestrengtheningnarra-tiveof“SSHinclusion”hasencouragedsocialscientiststoapproachtheissueofICT/SSHcooperationmoreproactively.

CONCLUDING REMARKSTheassessmentactivitiesthatwerecarriedoutinthefirstquarterof

theproject’slife-timeweremostlyfocusedontheVanclayetal.(2015)firstandsecondSocialimpactAssessment(SIA)evaluationphases(e.g.learningandunderstanding theproject).Even in these relativelyearlystagesof theHubITproject somespecificcharacteristicsof theHubITassessment activities emerged. First, due to a predefined responsibleassessmentstrategydecidedbyallpartners, theassessmentactivitieswerefoundtobedeeplyembeddedintheproject.Fullengagementofallpartnerswasthereforeachieved.Second,theassessmentactivitiesareanongoingprocesswhichwillevolveinaccordancewiththeprogressoftheproject.Thisallowsforcontinuousadjustmentoftheprojectactivi-ties.Third, inthecourseoftheevaluationactivities,theimportanceofinteractionswith the transdisciplinarycommunityofexperts fromSSHandICTbecameevident.Thissuitstheproject’sgoaltoformacommuni-tyaroundtheevaluationactivities,activelyengagingpartnersandstake-holdersfromdifferentfieldsintheprocessofassessment.

GeneraleventdynamicshintedthatindiscussingRRItwomainas-sociations appear: societal good (e.g. challenges of privacy, ArtificialIntelligence,robotics,etc.)andinclusion(especiallygendertopics).TheanalysednarrativessuggestthatsomeRRIdimensionsmightcarrymore“pressure”thanothers,andthatRRIinitself,shouldnotbetreatedasaneutralconcept.Conversely,itcanbepresupposedthatthereisapowerstruggleinvolved–eveninthelightoftheincreasingpressuretoensureresponsibilityof researchand innovation.Especially in thefieldof ICT,thediscourseofRRI is sometimes interpretedasadiscourseofdomi-nance,exertedbythesocialsciencesoverotherdisciplines.Animpor-tanttake-awayistoensurethatthe“responsibility”andethicsarenotseenasspecialdimensions,monopolisedbythesocialsciences.Rather,the discourse of RRI should be a space for reflection, where multipleideasandperspectivesarewelcome.

Duringtheevaluationprocessaneedwasidentifiedtocomplementthe survey type of assessment with a more ethnographic type of as-sessmentthroughobservations.Thiswasdonethroughintroducingandemphasising open-ended questions in surveys, discussion note-takingduringeventsandethnographicobservations.BasedonthenarrativesextracteditwasconcludedthattheinclusionofSSHperspectives into

ethnographic component to the HubIT evaluation activities – to con-tinuously engage with the qualitative data, to collect as much obser-vationsaspossibleandtobeflexible in the implementationofprojectactivities.Additionally,itwasdecidedthateachactivitywithintheHubITprojectwillundergoanethnographicqualitativeprocess,especiallytheplannedevents,which constitute the coreof theHubITproject. Thus,evaluationeffortsshiftedthefocustonarratives,open-endedquestionsinsurveys,discussionnote-taking,“ethnographic”analysisofeventar-tefactssuchaspostersandobservationsnotes.

“SSH BRINGS IDEOLOGY AND POLITICS INTO SCIENCE.”

Anunexploredtopicemergedfromtheparticipants:“I became aware that PC [political correctness culture] would creep

into ICT research” (Participant, expert workshop in Rome)“I am aware of the EC research ideology…” (Participant, expert work-

shop in Rome)Weprovideaninterpretationoftheexamplesaboveintwoways:on

the one hand, ethics sometimes is perceived as a complicating factorforresearch(oftenitsbureaucraticandforcefulnatureiscited).Ontheotherhand, itcanbespeculatedthatnorepresentativeofthemodernresearchcommunitywouldargueagainstthefollowingethicalguideli-nesandthespiritofresponsibilityattheiruniversities.RRIthough,asarelativelynewtermcoming fromtheEuropeanUnion,asupranationalstructure,doesnot carry the samedegreeof legitimacy,whichwouldexplaintheconceptuallinkagethatrespondentsmadebetweenRRIandtheideologyofpoliticalcorrectness(avoidanceofexpressionsthatmightnegatively impact marginalised groups), currently associated with thepolitical left.Additionally,theprocessesofglobalisationof informationflowsandmediatisation(dominantroleof(digital)mediainframingthediscourse)havedefinitelycontributedtothepolarisationofsocietiesandriseofpopulismworldwide.RRIideallyshouldnotbeseenasaright/leftissue,butasanobjectiveneedtoconsidersocietalconsiderationsinfor-mulatingandimplementingresearchideas.Howeverthissuggeststhatwemightbefacedwitharealityofpoliticisation(attributionofpoliticalagenda)oftheterm.

GENDER EQUALITY

Moreover, some additional insights came from event observations(whichcomplementedopen-endedquestionsofthesurveys):

“ICT representatives were mainly men while the SSH area was repre-sented mainly by women. This provides the feeling that SSH is something that women fight for. For instance, in the conference panel men (ICT) for-mulated their messages softer, while the woman speaker (SSH) was more a “right-fighter”.” (Observation, Slovakia national workshop)

Observationsofbotheventsprovedthatoftensomeaspectsofeventimplementationgounnoticedbyorganisers.Genderequality is thedi-mensionofRRIthat,intheexperienceoftheHubITproject,ismostvisib-leandcausesthemostdebate.Externalobservers`feedbackpointedouttheimbalanceinthepresenters(maleoverfemale),whilealsoempha-sisingthatfemaleparticipantsmakeupanactivepartoftheaudience,oftenbringingup thevalueof thediverseperspectives in ICTproductdevelopmentandtheimportanceofconsideringgenderaspectsinsomeresearchproblems.

ISSUE 48 | JULY 2019160

Jirotka, M., Grimpe, B., Stahl, B., Eden, G. and Hartswood, H.(2017).ResponsibleresearchaninnovationintheDigitalAge.Communication of the ACHVol60N05(pp62-68).

Owen, R., Macnaghten, P. and Stilgoe, J.(2012).Responsibleresearchandinnovation:Fromscienceinsocietytoscienceforsociety,withsoci-ety.Science and public policy,39(6),(pp751-760).

Strand, R., Spaapen, J., Bauer, M. W., Hogan, E., Revuelta, G., Stagl, S. and Guimaraes Pereira, A.(2015).Indicatorsforpromotingandmo-nitoring Responsible Research and Innovation: Report from the Expert Group on Policy Indicators for Responsible Research and Innovation.Lu-xemburg.

Vanclay, F., Esteves, A. M., Aucamp, I. and Franks, D. M.(2015).Soci-alImpactAssessment:Guidanceforassessingandmanagingthesocialimpactsofprojects.

Von Schomberg, R. (2011).Towardsresponsibleresearchand innova-tion in the information and communication technologies and securitytechnologiesfields.

Von Schomberg, R. (2013).Avisionofresponsibleresearchandinno-vation.Responsible innovation: Managing the responsible emergence of science and innovation in society,51-74.

AUTHORS

TAL SOFFERUnit for Technology and Society Foresight, School of EducationTel-AvivUniversity,Tel-Aviv,69978(Israel)E:[email protected]

RUTH ZUZOVSKYUnit for Technology and Society Foresight, School of EducationTel-AvivUniversity,Tel-Aviv,69978(Israel)E:[email protected]

OLENA NEDOZHOGINAFaculty of Social Sciences and Humanities, University of TartuUlikooli14,Tartu,51003(Estonia)E:[email protected]

EMANUELE BARDONEFaculty of Social Sciences and Humanities, University of TartuUlikooli14,Tartu,51003(Estonia)E:[email protected]

KEYWORDS:SSH-RRIapproach,SocialImpactAssessment,ICT

ICT research, as well as encouraging ICT-SSH cooperation has gainedmomentumatthebackdropofsocietalcallsformoreresponsibilityandreflexivityinhandlingICTinnovation–challengesofdatasecurity,algo-rithms,andinformationflowsareoneveryone`smind.However,themainhurdlestotransdisciplinarycooperationhavetodowiththefollowing:socialsciencesand“responsibilityinresearch”seemtobetightlylinked,tothepointwherethere isariskthatthevalueofengagingSSHper-spectiveinICTisnotseenbeyondtheareasofRRIandethics.Moreover,there isa risk that forcing“responsibility” intosomedisciplinesmightonlyleadtofurtherquantificationofRRIandanescapefromtheneedtoreflect.FurtherinteractionswiththeICT/SSHcommunityinthecontextofHubITendeavourstobuildtransdisciplinarityareexpectedtooutlinedirectionsoffuturework.Thereisaneedtodevelopandcommunicatenewevaluationpractices,andthispresupposesanewviewofRRIandtheroleofsocialsciences,aswellasthewaytheyarepresentedandpromotedbynationalandEuropeanbodies.

TheHubITevaluationactivitiesarestillinprogress.Differentevalua-tionactivitieswilltakeplaceandmoreinsightwillbeavailableinthefu-ture.FurtherinteractionswiththeSSHandICTcommunityareexpectedtocontributetowardsthedirectionsoffuturework.

REFERENCESBecker, H.A. (2001). Social Impact Assessment. European Journal of Operational Research,128(2),311-321.

Blonder, R., Rap. S., Zemler, E. and Rosenfeld, S. (2017).Assessingattitudesaboutresponsibleresearchandinnovation(RRI):thedevelop-mentanduseofaquestionnaire.SISYPHUS Journal of Education, 5(3)(pp122-156).

DITOs Consortium (2016b).Doing It Together science: Phase 2 project evaluation.UCL,London

Felt, U. (2016).“Response-ablePractices”or“NewBureaucraciesofVir-tue”: The Challenges of Making RRI Work in Academic Environments.10.1007/978-3-319-64834-7_4.

European Commission (2010). Eurobarometer 340 annexes. http://ec.europa.eu/commfrontoffice/publicopinion/archives/ebs/ebs_340_anx_en.pdfAccessed14.11.2017.

European Commission (2012). Responsible Research and Innovation: Europe’s ability to respond to societal challenges,DGResearchand In-novation.

Van den Hoven, J. and Jacob, K.(2013).Optionsforstrengthening Res-ponsible Research and Innovation – Report of the Expert Group on the Sta-te of Art in Europe on Responsible Research and Innovation,DGResearchandInnovation,EuropeanUnion.

European Commission(2015).Monitoring the Evolution and Benefits of Responsible Research and Innovation (MoRRI):AnalyticalReportontheDimensionsofResearchandInnovationGovernance.Sub-task2.5,deli-verableD2.4.2.

ISSUE 48 | JULY 2019 161

SERGIOMANRIQUE,MARTANATALIAWRÓBLEWSKAANDBRADLEYGOODDOI:10.22163/fteval.2019.385

RETHINKINGRESEARCHIMPACTASSESSMENT:AMULTIDIMENSIONALAPPROACH

CONTENT1.Introduction:Thechallengesofresearchimpactevaluation 1602.LiteratureReview 1603.Context:Existingsystemsofresearchimpactevaluation 162 3.1UnitedKingdom 162 3.2. TheNetherlands 162 3.3. Norway 163 3.4. EuropeanUnion 1634.“MultidimensionalApproachforResearchImpactAssessment”(MARIAmodel) 163 4.1.Dimensionsofresearchimpact 164 Responsiveness 164 Accessibility 164 Reflexivity 165 Ecology 165 Adaptability 166 4.2.Themodelinpractice 1665. Self-assessmentexamples 1676. Discussion 1677. Conclusion 168Acknowledgements 1688. References 168Ex-anteresearchimpactself-assessmentexample(Good,2018) 172Mid-termresearchimpactself-assessmentexample(Manrique,2018) 173Ex-postresearchimpactself-assessmentexample(Wróblewska,2018) 174

ABSTRACT

An interest in the evaluation of research impact – or the in-fluence of scientific research beyond academia – has beenobservableworldwide.Severalcountrieshaveintroducedna-

tionalresearchassessmentsystemswhichconsiderthisnewelementofevaluation.Sofar,researchonthispracticehasfocusedmainlyonthepracticalitiesofthedifferentexistingpolicies:thedefinitionoftheterm‘research impact’, different approaches to measuring it, their relativechallengesandthepossibleuseofsuchevaluations.Buttheintroduc-tionofanewelementofevaluationgivesrisenotonlytochallengesofapracticalnature,butalsotoimportantethicalconsequencesintermsofacademicidentity,reflexivity,powerstructures,distributionoflabourintermsofworkloadsetc.Inordertoaddressthesequestionsandtherele-vantneedsofresearchersinthispaper,weproposeamultidimensionalmodelthatconsidersdifferentattributesofresearchimpact:Responsi-veness,Accessibility,Reflexivity,EcologyandAdaptability.Thisholistic,multidimensionalmodelofevaluation,designedparticularlyforself-as-sessmentorinternalassessment,recognisesthequalitiesaprojecthasonthesedifferentscalesinabroaderperspective,ratherthanofferinga simpleandsinglenumericalevaluation.Thismodeladdressesmanyof theethicaldilemmas thataccompanyconducting impact-producingresearch.Toexemplifytheusefulnessoftheproposedmodel,theauthorsprovidereal-liferesearchprojectassessmentexamplesconductedwiththeuseoftheMultidimensionalApproachforResearchImpactAssess-ment(MARIAModel).

ISSUE 48 | JULY 2019162

accountfortheprogressofworkandtheemergentchallenges,4)recog-nisesthestrengthsofaresearchprojectintermsofimpactandpointstoitsweaknesses,ratherthanofferingasinglescore,5)isalight-touchassessment,whichcanbeas shortasonesheetofpaper.Ourmodelaims at widening the currently prevalent measurement-oriented andmetrics-orientedperspectivebypromotinga critical andcomprehensi-veassessmentofresearchimpact,bothindividuallyandinstitutionally.Throughourcontribution,wehopetoadvancethecauseofbuildingre-searchimpactliteracy(BayleyandPhipps,2017).

Themodelweputforwardhasbeendesignedparticularlywithself-assessmentorinternalassessmentinmind.Wedonotproposeamodelforassessmentofresearchethics,butratheramodelfor‘ethicalassess-ment of research impact’. The criteria of assessment we propose are:Responsiveness, Accessibility, Reflexivity, EcologyandAdaptability,whichwerecogniseasattributesof impactfulresearchinallscientificdisciplines inour“Multidimensional Approach for Research Impact Assessment” (MARIA Model).

2. LITERATURE REVIEWTheintroductionofexercisesofimpactevaluationcanbeplacedina

widerperspectiveofchangesaffectingacademia,andthusthetopiccanberelatedtoseveralbodiesofliterature,drawingfromfieldsasdifferentasphilosophyandsociologyofscience,economicsandmanagementaswellas thespecialised fieldsofscienceand technologystudies,high-ereducationstudies,valuationandevaluationstudies,etc.Below,webrieflypresent the relevantmainstrandsof research for theproposedmodel,signallinghowourproposalcomplimentstheexistingliteratureandaddressesgaps.

Inaverybroadcontext,theintroductionoftheimpactassessmentaspartofnationalorinternationalresearchevaluationsystemscanbeper-ceivedaspartofawiderchange affecting the position of universities and scholars in societies.Universitieshavealwaysbeenembeddedintheirlocalcontextswhileatthesametimeguardingtheirautonomy–asituationofperforming ‘balancingacts’between ‘pure’autonomyand‘impure’social relevance(HamannandGengnagel,2014).Against thisbackground,individualresearchersandacademicenvironmentshaveta-kenvariouspositionstowardswhatisnowcalled‘outreachandengage-ment’.Wecanrecallthe‘publicintellectuals’ofthepost-warera(Baert,2015),technocraticexpertsandentrepreneurswhoputtheirknowledgeat the service of market-oriented and governmental activities (SpielandStrohmeier,2012;Ritter,2015),aswellasresearchersfunctioninginacriticalcapacityasactivistsandsocialengineers,questioningandsubvertingexistingsocialandeconomicrelations(Maxey,1999;Pereira,2016).

Inrecentdecades,therelationshipbetweenacademiaandthesur-roundingenvironmenthasseenatransformation,partlyinresponsetobroad political and economic initiatives targeting universities’ involve-mentwithsociety,suchas the riseof theso-called ‘knowledge-basedeconomy’ (Jessop, Fairclough, and Wodak, 2008) which sees the uni-versitiesasstrategic‘knowledge-brokers’(LightowlerandKnight,2013).Hencesocial,political,oreconomicengagement,previouslyperceivedasanadditionalactivitytothe‘corebusiness’ofresearch,becameincorpo-ratedintothedefinitionofwhatitmeansto‘do’science.Inconsequence,therehasbeenanobservableincreaseinthesymbolicimportanceofap-pliedscientificdisciplinesandcollaborationsofscholarswhichtheirsoci-

1. INTRODUCTIONTHE CHALLENGES OF RESEARCH IMPACT EVALUATION

Aninterestintheevaluationofresearchimpact–ortheinfluenceofscientificresearchbeyondacademia–hasbeenobservableworldwide(Grant,Brutscher,Kirk,ButlerandWooding,2009;Wróblewska,2017a,p.162).Severalcountrieshaveintroducednationalresearchassessmentsystemswhichconsiderthisnewelement,suchastheUK,theNether-lands,Norway,Australia(AustralianResearchCouncil,2018),HongKong(HongKongUniversity Grants Committee, 2018) andJapan (NIAD-QE,2018),amongothers.Theelementof‘impact’isalsopresentintheeva-luation of research projects in international contexts, such as certainEUprogrammes,andseveralothercountriesarecurrentlydebatingthepossibilityofintroducinganimpactcomponentintotheirresearchevalu-ationsystems.Theappearanceofanewelementofacademicevaluationhasinspiredmuchscholarshipwhichfocusesonthepracticalitiesofthepolicyitself.However,thisintroductiongivesrisetopracticalchallengesaswellasethicalconsequences.Morequalitatively-orientedstudiesandreportshavepointedtoimpactevaluationimplicationsintermsofacade-micidentityandethos,emotion,academicvalues,andpowerstructures.(Bacevic,2017;Chubb,2017).Presently,itseemsthatmanyresearchersare ill-equippedfordealingwith thesenewandcomplex issues,oftenresultinginfeelingsoffrustration,confusionorresentmenttowardstheassessment exercise or impact-related activities (Chubb, WatermeyerandWakeling,2016).

Existingsystemsofevaluationseemtosufferfromseveralshortco-mings. Firstly, theymostly takea top-downapproach,whichdoesnotaccountforthenuancesofacademicknowledgeproduction.Secondly,theydonotalwaysofferaspacetoreflectontheethicalsideofimpactgeneration,oftenleavingthoseassessedfeelingalienated.Thirdly,theydonotattendtotheprocessualnatureofimpactevaluations,focusingjustonthefinaleffectofresearchintheformof‘changeorbenefittothesociety’.Fourthly,theyoftentendtowardsa‘onesizefitsall’modelaimedafinalnumericalassessmentproducingmeasurable,quantifiablescoreswhichcanlaterbeoperationalisedandranked,oftenforfundingconsiderations.Fifthly,theyareoftentime-consumingandcumbersomefortheassessedacademic.

We believe this quantitatively-oriented, ‘numerocratic’ perspecti-veon researchassessmentcan result indisregarding lessmeasurableimplications of research. To account for the reality of research in itsbreadthanddepth,evaluationsystemsshouldrecognisethesequalita-tivefeaturesandtheirrelativechallenges.Thelackofrecognitionofthiscomplexnatureof impact-lendingscienceleadstoanoverlysimplifiedvisionofresearchandcontributestofrustrationwiththeexercise,whichisseenasnotadequatelyrepresentingtherealityofimpactfulscientificwork.Toaddressthesequestionsandtherelevantneedsofresearcherswhoconduct impactfulwork,aswellas individualswhoare inchargeofresearchevaluation(policy-makers,academicmanagers),weproposea multidimensional model of research impact. A holistic model of as-sessmentenablesrecognisingthequalitiesagivenprojectmighthaveindifferentareas,ratherthanofferingasimplenumericalassessment.Toaddresstheabove-mentionedissues,weproposeamultidimensionalapproach,which1) iscreatedwithself-assessment inmind,2)shouldstimulate a reflection on the ethical aspects of achieving impact, 3)wouldideallybeconductedatdifferentstagesoftheresearchprojectto

ISSUE 48 | JULY 2019 163

cusing on how “researchers manage to connect to themes in that en-vironment, and on the ways in which this environment absorbs (‘uses’) and further develops the results of the research” (p. 89).Secondly, the‘productive interactions’concept (SpaapenandvanDrooge,2011)ari-sesasanalternative forovercoming thedifficultiesofmeasuringandevaluatingthesocialimpactofresearch,focusingonthepersonal,indi-rect(throughtextsorartefacts)andfinancial(throughmoneyor‘inkind’contributions) interactions between researchers and other actors as atransparentproxyoftheprocessfromresearchtoimpact.Suchaconcepthasbeenfurtherdeveloped,bringingmoreattentiontothegovernance,evaluation and monitoring of “Transdisciplinary Collaborations” (TDCs)addressingsocietalchallenges,asafruitful–bottom-uporstakeholderoriented–approachforvalorisingsociallyrobustknowledge(vanDroo-geandSpaapen,2017).Thirdly,andinlinewiththeapproachesprevi-ouslymentioned, theneed foramoreholisticview in theobservationandmonitoringofinterdisciplinaryresearch(Anzaiet al.,2012)hasbeenaddressedinJapanasanattempttowardsresearchvalorisation.Finally,afairertreatmentofSSHinresearchimpactassessment(Benneworthet al.,2016)hasbeenpointedoutasanecessityinthediscussiononthevalue,impactandbenefitofpublicly-fundedresearch.

Therearealso representativecasesof researchmovementsorpro-jects attempting to influence research policy in Europe, specifically intermsofresearchimpactevaluation.Since2006,all themajorsciencepolicy organisations in the Netherlands joined the project “EvaluatingResearch in Context” (ERiC), aimed at addressing the debate and themethodologicaldevelopmentofresearchevaluationinawiderperspec-tivethatincludesEuropeanandinternationalparticipation(Spaapenet al.,2007).ERiCprojectpromotesabroaderdiscussionandapproachforconducting a comprehensive research evaluation in terms of societalqualityandvalorisation.Thissocietalorientationofresearchhasbroughttogether the major organisation in Dutch science policy around theneed formethodologicalprogressand (inter)national attentionon thisissue.Withastakeholders’approach,theevaluationmethodconsiderstheconstructionofa“ResearchEmbedmentandPerformanceProfile”(REPP) that provides a wider societal reference group for a scientificproject(embedment)andthedegreeinwhichthisprojectservesthein-terestsofawiderreferencegroup(performance),consideringacontext-basedresearchimpact.Inanalysingthepossibilitiesofimpactevaluati-on,itisimportanttoreflectontheroletheproposedevaluationsystemwillhaveinthiswiderpanoramaofrathertenseandpolarisedattitudesandonhowtheresultsofevaluationmaybeusedformanagerialaims.

Withtheproposedapproach (MARIA model)wedonotseektocrea-teyetanothertoolaimedatfine-tuningacademics’performancethroughtop-down,number-drivenassessments.On thecontrary, in linewithagrowingrequestforresponsibleevaluation(Hickset al.,2015),wewishtoofferanalternativebyarguingforaresearcher-centred,multi-dimen-sionalmodelofself-evaluation,whichcouldnotonlyoffera‘profile’ofanassessedresearchproject,butmightalsoserveasan iterativetoolforfosteringethicalreflectioninthenewandoftenchallengingfieldofgenerating‘researchimpact’(ChubbandWatermeyer,2017).Whiletheuseofsuchamodelmightberatherlimitedintheframeworkof largeperformance-based research funding systems (Hicks, 2012), we arguethat it couldbevaluableasanadditionalwayof reflectingon the re-searchofindividuals,researchteams,departmentsetc.,inaniterative,qualitativeway, ineffectadvancing thecase for responsible, reflexiveresearchimpact.

alandeconomicenvironment(E3M,2012;EuropeanCommission,2003),oftendubbed–particularly inaregionalcontext–astheuniversities’‘Third Mission’ (BrundeniusandGöransson,2011).Numerousinitiativesaimedatlinkinguniversitieswithexternalpartnershavebeenlaunched,focusing on two areas: firstly, enhancing individual academics’ auto-nomyandresponsibilityinconductingentrepreneurialactivities(forananalysisofthisprocessintheBritishcontextsee:McGettigan,2013)andsecondly,valorisingthegrowingroleofuniversitiesasbusinessunderta-kingsaswellasinstrumentsinnationalpolicyagendas,cruciallyincon-tributing to thenationaleconomy (GornitzkaandMaassen,2007).TheemergenttendencyofrequiringtangibleeffectsofresearchconductedwithinuniversitiescanbecomeespeciallyproblematicinSocialSciencesandHumanities(SSH),wheremeasurablemonetaryeffectsbeyondaca-demia,suchaspatentsandlicenses,areuncommonresearchoutputs.InthecontextofagrowingtensionbetweenSSHandScience,Technology,EngineeringandMathematics(STEM)disciplines,oftenexacerbatedbythedemandsoftheperformative,metrics-drivenacademy,ourproposaloffersamorenuanced,process-orientedevaluationmodelwhichwouldstill preserve ‘entrepreneurial’ research impact, while recognising thespecificcontributionandpublicvalueofSSHdisciplines (Benneworth,Gulbrandsen,andHazelkorn,2016).

Withagrowing focuson incentivisinguniversityengagement,out-reachandimpactcamea demand to measure suchfactors,muchinlinewiththemanagerialapproachtogoverninghighereducationinstitutions– sometimes dubbed ‘academic capitalism’ – which has been on theriseinthelastfewdecades(Münch,2014;SlaughterandLeslie,1997;SlaughterandRhoades,2004).Numericalindicators–both‘traditional’bibliometrics and metric-based rankings (Hood and Wilson, 2001), aswellasnewerformsofscientometricsoralt-metrics(Priemet al.,2012;GalliganandDyas-Correia,2013)–havebeeneagerlyimplementedbytheadministrationofmanyuniversities,grantdistributorsandgovern-ments,promptingthemetaphorofagrowing ‘metric tide’ (Wilsdonet al.,2015)andacademic‘numerocracy’(Angermuller,2013).Atthesametimeanunproblematicrelianceonmetricsandrankingscontinuestobewidelycontestedbyresearchersinthefieldofhighereducationandeva-luation (Szadkowski,2015)andacademiccommunitiesworldwide (seefor instance the DORA declaration: American Society for Cell Biology,2012).

Whenreflectingonthe emergence of ‘research impact’ as a new academic value, onecandrawimportantlessonsfromevaluationandvaluationstudies.Scholarsinthisareahavearguedthatnewpracticesofvaluation(forinstancenewsportsorculinarypractices)arelikelytogiveriseto‘heterarchies’or‘plurarchies’ofvalues,astatewhereseve-ralvaluescanpersistandbeappreciatedatthesametime,ratherthantheoftenreductionist‘hierarchies’,characterisedbyonescale(Lamont,2012,p.212).Giventhatimpactevaluationisanewareaofvaluation,andthatresearchimpactconstitutesacomplexactivitywhichcanbeas-sessedfromvariedperspectives(theeconomic,thedevelopmental,theethical,and theaxiological,amongothers),weput forwardourmulti-dimensional model as an attempt to promote an open, multi-levelledapproachtoresearchimpactrecognition.

Therehavebeenvaluable contributions in the literature towardsabetter understanding and assessment of research impact. Firstly, thecontext-based perspective of research assessment (Spaapen et al.,2007)portraysamorecomprehensivemethodforassessingthequalityandrelevanceofscientificresearch,basedontherelationship(mutualtransactions) between researchers and their relevant environment, fo-

ISSUE 48 | JULY 2019164

3. CONTEXT: EXISTING SYSTEMS OF RESEARCH IMPACT EVALUATION

Creatinganapproachtoresearchimpactevaluationisachallenge,giventhattheassessmentofacademicworkhaslongrestedonfactorsinternaltoacademia:aboveallthequality(orquantity)ofresearchout-putsbut also thequality of graduate teaching, research environment,grant funding, international mobility of scholars etc. There certainlyseems to be a tension between more qualitative and quantitative ap-proaches to impactevaluation (Donovan,2017)anddependingon thestrategicgoalshandeddowntoeducationbythegovernment,academictraditions,prevailingpoliticaloptions,andoftenseveralcontingentfac-tors(Wróblewska,2018)impactevaluationstrategiesvarygreatlyfromcountrytocountry.

Belowwepresentthemostimportantpointsofreferenceforresearchimpact evaluation. While research agencies in several countries haveintroduced elements of impact evaluation, particularly in the areas oftechnology,engineeringandmedicine(BuxtonandHanney,1996;Cana-dianAcademyofHealthSciences,2009),wefocushereparticularlyontheexamplesoftheUK,NetherlandsandNorwayasthesystemswhichtakeamostcomprehensiveapproachinassessingimpactacrossallthedisciplinesaccordingtothesamecriteria.Apartfromtheapproachesim-plementedbyparticularstatesororganisationsthereexistvariousframe-worksputforwardbyscholars.Inthiscontext,themostinfluentialandnoteworthy,alsoforourownproposal,isthatof‘productiveinteractions’(SpaapenandvanDrooge,2011)whichadvocatesaprocess-orientedap-proachtoimpactvaluation,inlinewiththeapproachesofcontext-basedassessment(Spaapenet al.,2007)andTDC’s(vanDroogeandSpaapen,2017)introducedintheprevioussection.Suchapproachesvalorisethepathwayfromresearchtopractice(impact),transcendingthefocusonresearch outputs themselves by considering the different sources andexpressionsofimpactduringthewholeresearchprocess.

3.1 UNITED KINGDOM

UK’s“ResearchExcellenceFramework”(REF)withits“ImpactAgen-da” is perhaps the most well-known and influential system of impactevaluation(KhazraguiandHudson,2015),andsurelythefirsttoimple-mentimpactevaluationonsuchalargescaleandwitharigorousme-thodology.TheREFwasintroducedin2014toreplacethe“ResearchAs-sessmentExercise”(RAE)which,sinceitsintroductioninthe1980s,hadgrownintoacumbersome,time-consumingexercise.ThedebatewhichproceededtheintroductionoftheREFneatlyillustratesthetensionbet-ween qualitative (peer-review-based) and quantitative (metrics-based)approaches, which we have pointed to above. The REF was initiallyconceived as a light-touch, metrics-lead exercise which would reducetheburdentoassesseddepartments,whileprovidingevidenceastothereturn on the government’s investment in science. However, this con-ceptwasabandonedafterthefailureofthepilotofthemetrics-basedapproach(HEFCE,2009)andthe“ImpactAgenda”wasputforwardasareplacementformetrics(Sayer,2015).Initsfinalshape,theREF,runby the British research councils every 5-6 years (the first edition took

placein2014andthefollowingonewasannouncedfor2021),includes‘impact’asoneofthethreeassessedelements,alongsideoutputsandenvironment. Impact is defined as “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the envi-ronment or quality of life, beyond academia”,assessedonits‘reach’and‘significance’andaccountsfor20%(in2014)or25%(in2021)ofthefinalresult of the assessed unit (HEFCE, 2011, p. 26; HEFCE, 2016). Expertpanels evaluate impact in a process of peer review based on ‘impactcasestudies’ submittedbyuniversitydepartments,but the resultsareonlypublishedinanaggregatedmanner,i.e.forentiresubmissions,notforindividualcasestudies.

TheREFhasbeeninstrumentalinincreasingawarenessofresearchimpactintheUK(Donovan,2017)andbeyond,indeedbecomingthemo-delforimpactevaluationinothercountiessuchasSweden,NorwayorPoland (Wróblewska,2017b).Advantagesof thesystem includebeingbasedonandaccompaniedbyseveralthoroughcommissionedreports(King’sCollegeLondonandDigitalScience,2015;Manvilleet al.,2015;Manvilleet al.,2014),theuseofabroaddefinitionofimpact,whichislikely to be broadened still (Stern, 2016) and the accessibility of bothimpactcasestudiesand(aggregated)resultsoftheevaluationthroughonline resources. Weaknesses of the REF approach to impact, in ourview,includeafocusonthe‘effects’ofimpact-relatedactivities,ratherthanontheprocessualaspectandintermediateconsequencesthereof–asadvocatedbytheproductive interactionsapproach(SpaapenandvanDrooge,2011).Furthermore,theimpactcasestudytemplatedidnotencourageareflectionontheethicalaspectofimpactgeneration,whiletheperformance-orientedcharacteroftheevaluation,aswelltheonusplacedontheresultsleadacademicstopresentoftenunrealistic,idea-lisedandexaggeratedaccountsof impact(Derrick,2018;Wróblewska,2018).Theseareall shortcomingswhichwewish toaddresswithourmulti-dimensionalmodel.

3.2 THE NETHERLANDS

The“StandardEvaluationProtocol”(SEP)–asystemofresearcheva-luation adopted by the Association of Universities in the Netherlands(VSNU), the Netherlands Organisation of Scientific Research” (NWO)andtheRoyalAcademyofArtsandSciences(KNAW)in2015–incor-porates‘relevancetosociety’asoneoftheevaluationcriteria(alongsideresearchqualityandviability.‘Relevancetosociety’isdefinedas“con-tributions to economic, social and cultural groups and to public debate” (VSNU,NWOandKNAW,2016,p.7).ResearchconductedinDutchhig-hereducationinstitutionsisevaluatedbyexternalassessmentcommit-teesforeachunitorinstituteonceeverysixyearsonarollingschedule.Thisassessmentconcernstheresearchthattheunithasconducted intheevaluatedperiodaswellasthestrategytheunitwillpursueinthenextperiod.Eachresearchunitconductsaself-assessmentandprovidesadditional documents (including a report of indicators and strengths,weaknesses,opportunitiesandthreatsanalysis(SWOT)andbenchmar-kinganalyses),whichare considered, togetherwith interviewsby theunit’s representatives, theexternalcommittee,basing its judgmentoninternationaltrendsanddevelopmentsinscienceandsociety.Theexer-ciseconcludeswithareportinwhichtheexternalcommitteeoffersanassessment both in text (qualitative) and in four possible quantitativecategories(excellent,verygood,goodandunsatisfactory),accompaniedbyrecommendationsforthefuture.PhDprogrammes,researchintegrity

ISSUE 48 | JULY 2019 165

and diversity are also considered in the assessment. The assessmentreport,togetherwitharesponsepositiondocumentbytheuniversity,ispublishedintheend.

ERiCproject,referredpreviously,hastargetedsomeofthepossibleflawsoftheDutchSEP,which,similartotheREF,ignorestosomeextenttheprocessualnatureandintermediateachievementsofresearchacti-vity.A ‘onesize fitsall’modelgroupsdiverse research tobeassessedwithinthesamebasket–orresearchunit–andbythesamecommitteecanignorethevarietyofinteractionsamongresearchers,theirenviron-ment and other stakeholders, which are valuable sources of impact.Additionally, the scale ‘unsatisfactory-good-very good- excellent’ mayneglectanumberofmoderate–butstillrelevant–impactstudies.

3.3 NORWAY

TheResearchCouncilofNorwayhas introducedanelementofas-sessment very closely modelled on the British REF in its cyclical eva-luationof scientificdisciplines. The firstdisciplines tobeevaluated intermsofimpactweretheHumanitiesin2015-2017(ResearchCouncilofNorway,2017,pp.36-37),followedbytheSocialSciencesin2017-2018(ResearchCouncilofNorway,2018).TheNorwegianevaluationadoptedthedefinitionof impact, thepeer-reviewapproachand indeedthe im-pactcasestudytemplatefromtheREF,henceitmight inheritsomeoftheREFweaknessesportrayedinsection3.1.TheNorwegianapproachdiffersfromtheBritishmodelinthatitisnottiedtodistributionoffun-dingand,at least in thecaseof theexercisescarriedout todate, theexactscoresattributedtoimpactcaseswerenotmadepublic,eveninanaggregatedmanner.Insteaddescriptivefeedbackwasgivenontheoverall‘impactculture’ofasubmittingfaculty,insomecasesreferringtoindividualcasesfields(e.g.fortheHumanitiesseeResearchCouncilofNorway,2017,p.36-41).Whilethischoicepromotesamorelight-touchapproach to impact, without generating excessive anxiety about theexercise,itmaybelessconducivetoimprovementintheareaofimpactcreation.Furthermore,thesubject-specificevaluationscarriedoutbytheResearchCouncilofNorwaycaneithertangleorneglecttheassessmentof transdisciplinary research, affecting the valorisation of ‘productiveinteractions’andtransdisciplinarycollaborations,relevantaspectsofre-searchimpactintroducedinsection2(butnotethatsubmissionscouldpointoutanadditional,secondarypanelforreferences).

3.4 EUROPEAN UNION

Horizon2020(H2020),theEU’sresearchandinnovationframeworkprogramme, includeex ante andexpostassessmentsof researchandinnovationprojects,whereimpactonregionsisarelevantcriterion.Ap-plications for funding in theEU’s researchand innovationprogramme(H2020until2020,andHorizonEuropeinthenextbudgetaryperiod)setssomeexpected impactsat individual, institutionalandsystemic levels.MarieSkłodowska-Curieactions, for instance,assess impact, togetherwithexcellenceand implementation, as criteria for awarding funding.Impact assessment, with a weight of 30% (2017 call), considers theimpact on researchers’ future career as well as the strengthening ofhuman resources regionally,nationallyand internationally. Italsocon-sidersandpromotestransdisciplinarycollaborationsbetweenacademicandregionalpartners,aswellasthecommunicationanddissemination

ofresearchinsociety.Additionally,andbeyondH2020,theEUsupportsprojectsrelatedtoresearchandinnovationwithsocietalimpactsthrough“CohesionPolicy”(CP)andits“ResearchandInnovationStrategiesforSmartSpecialisation”(RIS3).CPisthecoreofEU’sstrategyforterrito-rialdevelopmentofregions,especiallylessfavouredregions(EuropeanCommission, 2014). The impact criterion has entered the research as-sessmentexercisesconductedbytheEuropeanCommissioninordertofundandmonitorresearchprojects.Thereisawholerangeoftypesofprojectsandfundingcallstacklingdifferentaspectsandthemesinso-ciety,encouragingcollaborationsbetweenacademicandnon-academicregionalpartners.TheEU iscoveringdifferentexpressionsof researchimpact through theirvarietyof fundedprogrammes, forwhich theas-sessmentprotocolsvarytoo.However,thereseemstobeawiderfocusonex-anteassessmentsforallocatingfunds,andthetrackingofresearchimpactattheresearchprojectsimplementationmightnotbereceivingenoughattention.

Other countries inwhich impacthasbeen introducedsomehow intheresearchassessmentexerciseinclude:

• Australia: “Engagement and impact assessment” (EI) in theframeworkof“ExcellenceinResearchAustralia”(ERA)(Austral-ianResearchCouncil,2018).

• Canada:“PaybackSystem”(BuxtonandHanney,1996;“Cana-dianAcademyofHealthSciences,2009).

• HongKong:“ResearchAssessmentExercise2020”(HongKongUniversityGrantsCommittee,2018)

• Sweden: Swedish Agency for Innovation Systems (VINNOVA)(Jacob,2006;LundequistandWaxell,2010).

• Japan:National InstitutionforAcademicDegreesandQualityEnhancementofHigherEducation(NIAD-QE,2018)

In addition to the above, also some research institutions have int-roduced their own approaches to research impact evaluation (for anoverviewofapproachestakenbythreeEuropeanresearchinstitutessee:GulbrandsenandSivertsen,2018,pp.36-42).

Peer-reviewingseemstobethemostcommonmethodologyforas-sessing the societal impactsof research, especially inex ante assess-ments (Holbrook and Frodeman, 2011), which puts in evidence theimportanceofqualitativeconsiderationinexercisesofresearchimpactassessment.Nevertheless,thedifferentassessmentsystemsdescribedaboveignore–todifferentextents–themultidimensionalnatureofre-search impactanddonotpaysufficientattentiontocertainattributesof impactful research, which this paper takes charge of in the modeldescribedinthenextsection.

4. “MULTIDIMENSIONAL APPROACH FOR RESEARCH IMPACT ASSESSMENT” (MARIA MODEL)

Giventheincreasingpressureonconsideringresearchimpactwhenassessingresearchactivity,itisimportanttoputforwardsystemswhichachieve this in broader and accurate way, going beyond (but withoutdismissing) the measurable effects of research. In alignment with 1)

ISSUE 48 | JULY 2019166

thecontext-basedperspectiveofresearchassessment(Spaapenetal.,2007), 2) the transdisciplinary collaborations and ‘productive interac-tions’conceptsinresearchevaluationandmonitoring(SpaapenandvanDrooge,2011;vanDroogeandSpaapen,2017),3)theneedforamoreholistic view in the observation and monitoring of interdisciplinary re-search(Anzaietal.,2012),and4)theneedforafairertreatmentofSSHinresearchimpactassessment(Benneworthetal.,2016),thispaperisan effort for joining and contributing to the ongoing learning processinresearchimpactagenda,byproposingamultidimensionalandflexi-bleapproachtowardsthisissue.TheMARIA modelisdescribedinthissection.

4.1 DIMENSIONS OF RESEARCH IMPACT

Weproposeamodelwhichindicatessixmaindimensionsofimpact-fulresearch.Thesedimensionsareattributesofresearchwhichmaybeconsideredintheassessmentprocessofanyresearchprojectatanysta-ge:exante,mid-termandexpost.Theorderinwhichthesedimensionsare presented does not represent their relevance or weighting withinthemodel.Thismodel isspecificallydesignedwithself-assessment inmind.Webelievecarryingoutsuchexercisewouldbeusefulforscholarswantingtoreflectonthe‘impact’aspectoftheirworkinconsideringtheadvantagesandpossibledrawbacks,as indeedithasbeenforus(seesection5).

RESPONSIVENESS

“Authentic thinking, thinking that is concerned about reality, does not take place in ivory tower isolation, but only in communication.”Paolo Freire (2000)

Impactfulresearchshouldberesponsivetorealproblemsandissuesinsociety.Theisolationofacademiafromsocietyleadstoresearchwhichisnotrootedinreal-worldchallenges.Hence,researchshouldtargetso-cietalneedsand face theseproblems indialoguewithaffectedstake-holders. Following Owen’s et al. (2012) idea of policy responsiveness,impactfulresearchshouldaimat:1)anticipation,foreseeingtopicsandissuesworthstudyingfortheirimportanceinsociety’sfuture,2)reflec-tion, considering the realproblem insteadofwhataudienceswant tohearorreadabout,and3)deliberation,planningconscientiousactionstorespondtorealneedsthroughresearch.Allthreeoftheseissuescanbesummarisedintheconceptofdialogueandexternalmediation,whichhaveacriticalroletoplay,especiallyinanacademicenvironment,whereinternalthoughtprocessesareoftenprioritisedoverexternalresponsi-veness.PaoloFreire,inPedagogyoftheOppressed,takesthisonestepfurther indiscussingtheimportanceofdialogiceducationasawaytocreatemeaningful,equitable,andtransformativeeducationalexperien-ces(Freire,2000);weextendthisparadigmtoresearchpractice,bypo-sitioningresponsivenessasthemainrequirementfordialogicresearch.

Impactfulresponsiveresearchshouldberealisticallyambitioustoo,byaspiringtomakeclear,specificandvaluablecontributionstocurrentpublicdebatesand/or to theresolutionofneeds insocietyand indus-try. Ambitious research tackles issues at different levels in terms ofgeography,disciplinesoractors,amongothers.Thepursuitofambitiousresearchcan takeplace indifferentways:byengagingwithglobalor

long-term issues, involvingstakeholdersmore integrally,embracing in-terdisciplinary,implementingcollaborationwithactorsoutsideacademia(e.g.industry,citizens),andingeneral,performingactionstogenerateagreater impact.Researchshouldbeambitiousandopen-mindedwhileremainingrealisticandtestable.Responsiveness,asadimensionofim-pactful research, must contribute to achieving “Responsible Researchand Innovation” (RRI). Therefore, responsive research should also beresponsible“inthecontextofresearchandinnovationascollectiveac-tivitieswithuncertainandunpredictableconsequences” (Owenetal.,2012).Exante,mid-termandexpostassessmentsofresearchrespon-sivenesscanrevisehowtheresearchersargue,considerortakecareofcurrentneedsand/orrealproblemsinsocietyandhowthisis–plannedtobe–achieved.

Responsiveness example:Thebodyofknowledgeonenvironmen-tal sustainability and clean energies responds to the global warmingandpollutionproblem that threatens societyandwhichhasbeenontheincreaseduringthelasttwodecades(Ostrom,2009);thisgrowingresearchstreamisresponsivetoarelevantissueincurrentsociety.Theenvironmental problem that society faces has been studied by seve-ral researchers from different disciplines within natural sciences andengineeringbutalsowithin social sciencesandhumanities, trying tocontribute to the understanding and solution of global warming andpollutiongenerationfromdifferentbodiesofknowledgeandwithdif-ferentperspectives.

ACCESSIBILITY

“Making research results more accessible to all societal actors contributes to better and more efficient science, and to innovation in the public and private sectors.”

European Commission (2018)

Impactfulresearchshouldbeaccessibletostakeholdersandsocietyingeneral,withinthelimitsoffeasibility.This includesitscommunica-tion and dissemination both within and outside the academy, ideallyallowingallstakeholderstoaccessandengageintheresearch.Accessi-bilityamongthegeneralpublicisalsoimportantbutmaybelimited,de-pendingonresearchscope.Thedimensionofaccessibilityassesseshowtheresearchisplannedtoinvolveorbecommunicatedtoacademicandnon-academicstakeholdersandthegeneralpublic(ex-anteassessment)andhowitendsupinvolvingorbeingeffectivelycommunicatedtobothgroups(ex-postassessment).

Oneexampleof accessibility includespublic academics.UsingMi-chael Burawoy’s definition, public academics are communicative inknowledgeproduction,derivelegitimacyfromtheirrelevance,areheldaccountablebythedesignatedpublics they interactwith,andengageinpublicpoliticaldialogue(Burawoy,2004).However,thechallengeofbeingapublicacademicistoalsoensurethatresearchisreliableandconsistentwithallethicalstandards.TherecentcaseofBrianWansinkatCornellUniversityillustratesthedamagethatcanbedonewhenaccessi-bilityisvaluedtooheavily.Wansinkledtheprestigious“CornellFoodandBrandLab”,whichwasknownforitsrevolutionaryandhighlyaccessiblestudies on the intersection of food consumption and psychology. ThisresearchlabregularlygrabbednewspaperheadlinesintheUnitedStateswitheasily reportableheadlines,mostly focusedonwayshumanscanbepsychologicallyqueuedtoeatmoreorlessfood.Thesefindingswere

ISSUE 48 | JULY 2019 167

regularly reported inmagazines,newspapers,onthe“FoodandBrandLab’s”website,andinWansink’smassmarketpaperbackbooks.How-ever in 2017, four early career researchers poured through Wansink’spublicationsandcreatedtheWansinkDossier:alistofover50publica-tionswith“minortoveryseriousissues”(Zee,2017,website–nopagenumberavailable)thateventuallyresultedinaninvestigativejournalistreport (Lee,2018)andWansink’seventual resignationfromCornell fordatamanipulationandtampering(RosenbergandWong,2018).Hiscasemakesclearhowanextremedriveforaccessibilitywhileneglectingethi-calstandardscansignificantlydamageresearchaims.For this reason,our overall model is holistic and includes other elements of researchimpact.

AccessibilitymayalsolinktotheOpen Science movement,a“move-menttomakescientificresearchanddataaccessibletoall”(UNESCO,website–nopagenumberavailable).ThismovementhasmostrecentlybeentypifiedbyTheAmsterdamCallforActiononOpenSciencewhichcallsforopenaccessforscientificpublications,datasharingandreuse,alignmentofbestpracticesandpolicies,and,mostnotably,“newassess-ment,reward,andevaluationsystems”(MinistryofEducation,Culture,andScience,TheNetherlands,2016,pg.3).Accessibility refers to thistypeoffocus,whichdoesnotjustencourageopennessinresearchcom-munication/disseminationbutproactivelypursuesit.

Accessibility example:“WhyWePost–SocialMediathroughtheEyesoftheWorld”isacollaborativeeffortfromnineanthropologists“re-searchingtheroleofsocialmediainpeople’severydaylives”(UniversityCollegeLondon,website–nopagenumberavailable).Themostextraor-dinarypartoftheirresearchwashowtheycommunicatedfindings.TheresearcherscreatedmultiplefreeeBooks,madeanentirelyfreeMOOC(MassiveOpenOnlineCourse)throughthedigitaleducationplatformFu-tureLearn,keptablogthroughoutthecourseoftheresearch,hadsocialmediapresencesonFacebook,Twitter,andYouTube,andcreatedatho-roughly interactivewebsitewithsimplifieddiscoveries, stories, videos,andinteractivemaps.Thisisinadditiontothebookchaptersandjournalarticlespublished.Furthermore,“WhyWePost”alsoensuredthatthesematerialswereaccessibleinthelanguagesofthecountrieswheretheyconducted research, ensuring translation in English, Portuguese, Spa-nish, Italian, Turkish, Chinese, Tamil, and Hindi. “Why We Post” is anextremebutalsoimportantexampleofaccessibility.

REFLEXIVITY

“Train PhD students to be thinkers not just specialists…put the philosophy back into the doctorate of philosophy.”

Gundula Bosch (2018)

MostofthepeoplewhoconductresearchinacademiaarePhDstu-dentsorgraduates.Inthissense,itisimportanttorememberthatPhDstandsforDoctorofPhilosophy,andbeyondbeingexpertsorspecialistsinagivenfield,researchersshouldbe,bydefinition,thinkersandthe-orisers(Bosch,2018).Tothisend,reflexivity isconcernedwithcritical reflection.Inthisdimension,theresearchermayask:‘hastheprocessof theorising and research design been comprehensive, well-planned,ethical,andcritical?’,‘havetheresearchtheoriesandconclusionsbeenthoroughlybrokendown,evaluated,andcritiqued?’.Impactfulresearchshould incorporate conscious and deep reasoning on the conductedresearch’s objective, methodology and results, in order to understand

howitcontributestocertainbodyofscientificknowledgeandtopublicdebates. In thissense, thebuildingof theoryandanalysisof researchresultsisespeciallyrelevantforunderstandingthegapbetweeninten-tionandwhathasreallybeenachieved(implications)intheconductedresearch.

Whileanalysisandreflectionareimportant,thereisalsoaneedtoreflect critically. Brookfield (2000) points out that critical reflection in-volvesapoweranalysis of the situationor context. This typeof refle-xivity isnecessary fromanethicalandevenecologicalperspective, toensure that the research itself is not contributing to inequality. Whilecriticalreflectionisimportant,itisalsonecessarytothenactuponthatreflection,nottreatingitsimplyasanacademicexercisebutonewhichencouragestruechangeintheresearchdesignandotherwise.Criticalreflection without social action can be seen as a “self-indulgent formofspeculationthatmakesnorealdifferences”(Cranton,2006,pg.45).Thisleadsresearchimpactbacktotheexternalfocusofresponsiveness,the firstdimension in thismodel.Researchactivitycanbecriticalandreflexivewithoutdiminishingitsscientificvalue.

Reflexivity examples:Withinthepaper“Designsand(Co)Incidents:CulturesofScholarshipandPublicPolicyonForeigners/MinoritiesintheNetherlands” (Essed and Nimako, 2006), the authors argue for an in-creasedlevelofreflexivityon“RaceCriticalPerspectives”intheDutchacademiccommunity.Theycontendthattheseframeworksonraceandpowerhierarchieshavebeendisregarded in favourofwhat they term‘minorityresearch’.Duethisfocusonethnicminorities,aninstitutionalcultureofproblematisationofthe‘other’hasdeveloped.Thisexampleofmeta-analysisismostprevalentwithin“CriticalTheory”perspectivesbutcanbeincorporatedintoanydiscipline.

ECOLOGY

“What can be studied is always a relationship oran infinite regress of relationships. Never a ‘thing’.”

Gregory Bateson (2000)

Webelieveimpactfulresearchshouldbeecological,notonlyinitsenvironmental conception, but also socially, culturally and economic-ally (Scoones, 1999). An ecological approach to research is a holisticandintersectionalonethatconsidersandisawareoftherelationshipsamongdifferenttypesofagentsintheresearchactivity,inthepathwayfromresearchtopracticeandintheimplicationsforresearchersthem-selves.Intermsofimpact,ecologicalresearchshouldconsidernotjustthepossiblebenefitsfortheaffectedcommunity,butalsothepossibledisadvantageswhichtheymaysufferinashortandlongrun.Inabroa-derperspective,ecologicalresearchwouldfavouraholisticorientation,whichDeshlerandSelener(1991)seeasoneoftheprimaryindicatorsthattheconductedresearchwillbetransformativeorhaveimpact.Whileresearchersareoftenencouragedtofocusonthemicroorminutiaeofa topic, a larger understanding of the overall research landscape in aparticularfieldandoftheinterconnectivityamongacademicdisciplinesisessentialforresearchtobedeemed‘ecological’.

Anecologicalmind-setinresearchshouldalsoencouragecollegiality,bearing in mind its effect on researchers and research stakeholders.Being collegial refers to being open to other researchers, supportingmorejuniorcolleagues,treatingpeopleinanon-instrumentalway,andingeneral,consideringthewell-beingofothers,enablingandstrengthe-

ISSUE 48 | JULY 2019168

ning‘learningonthejob’(Little,1982)foracademicsatuniversities.Inthissense,wethinkthatimpactfulresearchshouldsupportthepositionofthefieldanditsimpactonexternalcommunities,andideallyitshouldencourageand/orbetheresultofcollaborativework.

Finally, the concept of ecological impact may refer to the positionofoutreachanddisseminationintheresearcher’sowncareerplanandbroader life perspective. We see increasingly how academic activitiesaimedatcomplyingwithgovernmentalpoliciesorpreparationtoevalu-ationexercisestakesawayvaluabletimefromresearchitself.Thiscanleadtoimpactactivitiesbecoming‘instrumental’–i.e.theimpactitselfissecondarytotheadvantageitgeneratesintermsofresearchfunding,assessmentscoresetc.Therefore,itisalwaysworthreflectingonwhe-therthepaperworkconnectedtodocumentingimpactisnotdrivingusawayfromthe‘corebusiness’ofacademicwork,andifitisnotaffectinginanegativewayourrelationshipswiththestakeholders.

Ecology example: An impact case study submitted to the BritishREF (CS1698, Electropalatography (EPG) to Support Speech PathologyAssessment, Diagnosis and Intervention, Queen Margaret University)described a situation in which scholars working on a speech therapydevicehadtoomanyvolunteersfortheexperimentaltreatment.Inordernottodisappointpotentialpatientswhowouldhavetobeturnedaway,thescholarsdecidednottopublicisetheexperimentaltreatmentatthecurrentstage,despitethefactthatthiscouldlimittheir‘claimtoimpact’,possiblyresultingina lowerscore intheREFevaluation(Wróblewska,2018).This illustrateshowdimensionswhicharenotaccounted for inexistingmodelsofevaluationcanbereflected inthemultidimensionalmodelwepropose.

ADAPTABILITY

“Being open to the possibility that our understanding or definitionof a research problem may be inappropriate or partial.”

Maureen G. Reed and Evelyn J. Peters (2014)

Wearguethatimpactfulresearchisadaptable todifferentcontextsandstakeholders.Thisdimensionofresearchimpactreferstotheusa-bilityofthedifferentresearchcomponents,suchasmethodsanddata,infurtherstudiesoracrossdifferentsamples(Hilletal.,1997),lookingforpossibilitiesforresearchimpact.Inviewofthepermanentdevelop-mentof research infrastructures (RibesandPolk,2014), togetherwiththeevolutionof researchobjectsand researchers themselves, there isaneedforresearchactivitytobemoreadaptive andresilient.Adaptiveand resilient researchmethods“embrace the uncertainty and partiality of knowledge creation as well as the dynamism of the research process”(ReedandPeters,2014,pg.19).Accordingly,researchresilienceshouldbe understood as its ability to absorb perturbations (anticipation) andadapttochange(planforchange),inlinewiththe responsivenessdi-mension.Adaptableresearchmusttakecareofrecordingandreportingmethods and data appropriately (Mesirov, 2010). Potential for adapta-tionsofresearchcanbeassessed1)exante,byascertaininghowthethesis,hypothesis,methodsandanalysismeanttobeusedhavethepo-tentialtobeappliedindifferentcontextsandhowdataandmethodsareplannedtobetrackedandrecorded,and2)expost,byrevisingexecutedorplannedadaptationsof theresearch,andwatchingtheaccuracy inthetrackandrecordofextantdataandmethods.

Theadaptabilityof researchcanbepurposefulandserviceable,asit allows keeping research relevant and strengthening research-policy

dialogueinthefaceofthechangingneedsofdecision-makersindiffe-rentscenarios.Impactfulresearchcanbereusedoradaptedinnumerousoccasions,achievingvariousimpacts,oritcanbringquestionsthatmustbeansweredseveraltimesindifferentcontexts,withdifferentstakehol-ders,servingdifferentaudiences.Consequently,wethinkthatimpactfulresearchshouldbeclearaboutitslimitations,potentialfutureresearchopportunities(includingadaptations/reproductions)andunansweredoremergingquestionsthatcanleadtofurtherresearchimpactelsewhere.Impactfulresearchcanbestimulatingbothinthequestionsitanswersandinthenewquestionsitrises.

Adaptability example:The“Blue Ocean Strategy”,formulatedbyKimandMauborgne(2004), isamarketingtheorythattranscendedacade-myandhasbeenfollowedbymanyfirmsandentrepreneursaroundtheworld. Such strategy proposes, in general terms, that firms aiming atdeveloping strongcompetitiveadvantages should look forunexploitedmarket spaces, avoiding competition and focusing on new innovativeapplicationsthatgeneratenewcustomers.Thisworkhasalsoinspiredmany research pieces including empirical applications or studies andfurthertheoreticaldevelopmentsonorganisationalstrategy.

4.2 THE MODEL IN PRACTICE

TheMARIAmodelwhichweputforwardhereisprimarilydesignedforqualitativeself-assessmentbyresearchers.Whilethispaperdiscus-sesother typesofnationalassessmentmodels, it is important tonotethatthisproposalisnotmeanttobeusedbythirdparties,specificallyinrelationtofundingdecisions.Whilequalitativeassessmentisimportant,fora simpler visualisation toassist researchers, thesedimensionscanbeoperationalised,andtheassessmentquantifiedifnecessary.Again,themeaningofthesenumericalvaluescanandshouldbeassignedinawaythatismostmeaningfultotheindividualresearcher.Hence,wehavenotprovidedanyrecommendationsforscalemeaning,beyondthebasic focus on a numerical (1-5) scale. Having looked at the differentresearchimpactdimensionsseparately,anyresearchcanberepresentedthroughapentagonal figure– the “MARIApentagon”– showing thegradesgiventotheresearchinthedifferentdimensions,asexemplifiedinFigure 1.NotethatasimilarradarrepresentationhasbeenusedintheimpactassessmentoftheFrenchNationalInstituteforAgriculturalResearch(INRA,2018),althoughthereitrepresenteddifferentareasofimpact(e.g.health,economyetc.).

ISSUE 48 | JULY 2019 169

Figure 1–MARIAPentagon

TheMARIApentagonoftheleftrepresentsahypotheticalsituation

inwhichtheself-assessedresearchistotallysuccessfulinallitsdimen-sions,whilethepentagonoftherightrepresentsaresearchassessmentthatgoesinanascendingclockwiseorderfromresponsiveness.Thenextsectionprovidesrealexamplesofself-assessmentsusingthemodel.

5. SELF-ASSESSMENT EXAMPLES

We as authors of the MARIA model have put under considerationour own PhD projects using the self-assessment sheets found in thispaper’sAnnex.Exante(dissertationinformulation),mid-term(disserta-tioninprogress)andexpost(dissertationfinalised)researchimpactself-assessmentswereconductedbyBradleyGood(2018),SergioManrique(2018)andMartaN.Wróblewska(2018)respectively.

6. DISCUSSIONEachauthorofaself-assessmenthasofferedanoutlineontheexpe-

rienceusingtheMARIAmodeltoassessresearchimpact:• Bradley Good:“LastyearIunderwentamajorfundingapplica-

tionwiththeIrishResearchCouncilwhichcontainedelementsofresearchimpactandencouragedmetoreflectonthisissue.However,thetreatmentofthisaspectseemedcursoryandpri-marilyfocusedonnarrativeutilisationratherthanasystematic

treatmentofthisissue.Ifoundthatutilisingthismoreconcreteapproach gave my research planning additional focus andprovidedeasilyunderstandableways that I could improvemyproject.Specifically,accessibilitywaslowerthanIwouldhaveanticipatedwhichnowprovidesmewithextraincentivetodomoreoutreachandpromotemyresearchpublicly.Thisexercisewasincrediblyhelpful,andIplantoincorporatemyself-assess-mentasanofficialpartofmyPhDeight-monthproposal”.

• Sergio Manrique:“Ihadbeenexposedtoassessmentexercisesatproject/institutional levels,but thosereallydidn’tallowmeto reflectonmy individual research impact. Thisexercisehasbrought issues Iwasnot reallyawareofandmight routemyfuture actions towards developing the dimensions that canboost the impactofmy researchonmystakeholdersbutalsoon the general public. This self-assessment has also allowedmetorealisethatresearchimpactisn’tachievedonlythroughtheresearchoutputsthemselves(publications,reports,patents,etc.),asimpactcanbegeneratedbytakingactionsduringtheresearchprocessitself,actionsthatslippassintheday-to-dayofaresearcher”.

• Marta Wróblewska:“Intheory,everyresearcherwantstopro-duceresearchwhichisreflexive,accessible,adaptableetc.,butwerarelytakethetimetoactuallyevaluatewhatwehavedoneso far. This is also due to the continuous nature of scientificwork:thereisalwaysthatonemorearticletowrite,onemoreseminar toget to, onemoredisseminationactivitybeforewecan ‘wrapup’andevaluateourcurrentproject. In thissense,approaching the self-assessment was an incentive to take astep back and reflect on what has been achieved and whatstillrequireswork.Themostinterestingdiscoveryformewouldhavetodowiththe‘serendipity’ofimpact–theareaswheremyresearchhasbeeninfluentialarenotnecessarilytheoneswhereIplannedtohaveimpact”.

Overall,wefoundthattheutilisationofourmodeltobesimpleandeffective,withenoughdatavisualisedforresearcherstoknowwheretoimprovewhilekeepingtheprocessunencumberedbylengthynarrativeorcomplexmetrics.Withthisinitial‘fieldtest’asuccess,ournextstepis toacquire feedbackandcontinueto improveouroperationalisation,eventuallydistributingand testing itwithinabroaderdemographicofSSHresearchers.

Futureresearchopportunitieswithinthisparadigmareabundantbutofprimary importancearetheconsiderationofadditionalresearch im-pactdimensions,exploringlinksandcorrelationsbetweenthesedimen-sions,studyingtheoperationalisationofthismodelindifferentcontexts,andidentifyingpotentialdiscipline-specificweightingconfigurations.Inaddition,otherpossibilitiesincluderefiningspecificdimensionalindica-tors, providing further comparison to national systems of evaluations,and examining any differences in use and user experience betweenSTEMandSSHresearchers.Theusabilityandusefulnessofthemodelwouldideallybetestedempirically,forinstancewithinonedepartmentorresearchprojectoveraperiodoftime–theauthorsintendtopursueopportunitiesofcarryingoutsuchacasestudy.Regardless,onemustbearinmindthatthismodelisinthetheoreticalstagesofdevelopment,primarilyutilisedforself-assessmentratherthananinstitutionalfocus.Itmighthoweverbeimplementedaspartofinternalassessments(oneoftheauthorsofthisstudyintendstoimplementitinthisway–seeabove)and included as a supplementary document, even in more qualitative

ISSUE 48 | JULY 2019170

1 https://enressh.eu/2 https://runinproject.eu/

exercises(toaccountfortheethicaldimensionofimpact.Asmentionedbefore, thismodelproposal isasuggestion forbroadeningthedebateontheexistentresearchassessmentsystemsandhowtheseshouldbeenhanced, tasks inwhichmore insightsand theoretical andempiricalcontributionsareneeded.

7. CONCLUSIONTheinclusionofresearchimpactcriterionwithintheresearchassess-

mentexerciseinseveralnationalsystemsrepresentsarelevantdevelop-mentinthevaluationofresearchactivity.However,theassessmentandmeasurementofresearchimpactisanongoingprocess.Aheavyfocusonquantitativeassessment,specificallyforfundingandallocationofotheropportunities,canleadtoaneglectofimportantqualitativefactors.Toprovideanaccuratedepictionof research impact, recognitionandun-derstandingof theseattributesmustbeencouraged. To this end, thisresearch paper proposes and explains a Multidimensional Approach for Research Impact Assessment (MARIA Model), highlighting fiveimpactfulattributesofresearch:Responsiveness, Accessibility, Refle-xivity, Ecology,andAdaptability. Thesedimensionsarepresentedasattributesofimpactfulresearchconductedinanyareaordiscipline.How-ever,thismultidimensionalmodelexplicitlylooksforafairertreatmentofSSHintheassessmentofresearchimpact.Theoperationalisationofthismultidimensionalmodelhasalsobeenexplained.To thisend,asetofscalesisproposedforself-assessingeachofthedimensions,andatoolsuggestedtorepresent thegeneral impactofaresearch:The MARIA Pentagon,whichcouldbeusefulincollectiveexercisesofresearchas-sessmentwhererankingsandthresholdsarerequired.Ratherthansug-gestingafixedmodelforresearchimpactassessment,thispaperaimsatevidencingtheexistenceoffurtherimpactfulattributesthattheresearchimpactagendamighthavebeenneglecting.Theassessmentofresearchimpact can’tavoid thequalitative implicationsof science,as reducingresearchvaluetoitsmeasurableeffectswouldnotbecoherentwiththenatureofresearchpractice,andthereforeitwouldberecommendabletoconsiderabroaderperspectiveintheassessmentexercise,liketheoneproposedinthiswork.

Whilethereareseveraldevelopedsystemsforexternalassessmentofimpact,webelievethatwhatislackinginthepanoramaofresearchevaluation is1)a frameworktosystematically reflectonthe impactofone’s own work (self-assessment) 2) a multi-levelled model which re-cognisesthecomplexityofanyimpactfulwork,3)amodelwhichexpli-citlyrecognisestheethicalaspectofconductingimpactfulresearchandoffersaclear framework for reflectionon these issues. Themodelweproposeaimstoaddresstheabove-mentionedgaps.Finally,ourmodelconsiderstheserendipitousnatureofresearchimpactgeneration(Der-rickandSamuel,2016).ItcouldbearguedthataresearchprojectcouldfareveryhighlyintheMARIAmodelscale,withoutactuallyrealisinga‘changeorbenefit’tosociety(astheREFdefinitionofimpacthasit),forinstanceduetolackofuptakeofapotentiallyimpactfulinnovation,lackoffinancingforimplementationormanyotherfactorswhicharebeyondtheacademics’control.Whilethisisarealpossibility,wewouldstressthattheMARIAmodellooksattheprocessofgeneratingimpact,rather

thanthefinaleffectsthereof.Wewouldarguethataprojectwhichcon-sidersthefivedimensionsisverylikelytoproduceresearchimpact,doingsoinasustainableandethically-awareway.

Ourproposalcontributestotheongoinglearningprocessofresearchimpact,inalignmentwiththecontext-basedperspectiveofresearchas-sessment (Spaapen et al., 2007) and in recognition of the need for amoreholisticviewintheobservationandmonitoringofinterdisciplinaryresearch(Anzaietal.,2012).Ratherthansuggestingafixedmodelforresearchimpactassessment,thispaperaimsatevidencingtheexistenceofadditionalaspectsofconductingimpactfulresearchthatexistingre-searchassessmentsystemsdonotfullyrecogniseorrepresent.

ACKNOWLEDGEMENTS• ENRESSH Winter School: The authors thank EU COST action

ENRESSH1(EuropeanNetworkforResearchEvaluationintheSocial Sciences and the Humanities) for the opportunity toparticipateinitsfirsttrainingschool,heldinFebruary2018inZagreb,Croatia,wherethispaper’sproposalemerged.Specialthanks are due to ENRESSH working group 2 leader Dr PaulBenneworth (Universityof Twente,NL), and toDr LeonieVanDrooge (Rathenau Instituut, NL), Xeni Kechagioglou (Univer-sitàdegli StudidiCagliari, IT) andEloïseGermain-Alamartine(LinköpingUniversity,SE)fortheircontributiontotheteamworkduringthistrainingschool,whichhelpedtostartshapingthispaper’scontribution.

• PhDfunding:SergioManriqueisaPhDfellowofRUNINproject2,aEuropeanTrainingNetworkforEarly-StageResearchers,fund-ed by EU’s Horizon 2020 research and innovation programmeunderMarieSkłodowska-Curiegrantagreement#722295.

REFERENCESAmerican Society for Cell Biology. (2012).SanFranciscoDeclarationonResearchAssessment.Puttingscienceintotheassessmentofresearch.Retrieved from: http://www.ascb.org/wp-content/uploads/2017/07/sfdora.pdf

Angermuller, J.(2013).Discoursacadémiqueetgouvernementalitéent-repreneuriale.Destextesauxchiffres.InM.Temmar,J.AngermullerandF.Lebaron(Eds.),Lesdiscourssurl'économie(p.71-84).Paris:PUF.Retrie-vedfrom:http://www.septentrion.com/fr/livre/?GCOI=27574100482170

Anzai, T., Kusama, R., Kodama, H. and Sengoku, S. (2012). Ho-listic observation and monitoring of the impact of interdisciplinaryacademicresearchprojects:AnempiricalassessmentinJapan.Tech-novation, 32(6), 345-357. DOI: https://doi.org/10.1016/j.technovati-on.2011.12.003

Association of Universities in the Netherlands (VSNU), the Nether-lands Organisation for Scientific Research (NWO) and the Royal

1 https://enressh.eu/2 https://runinproject.eu/

ISSUE 48 | JULY 2019 171

Netherlands Academy of Arts and Sciences (KNAW). (2016). Stan-dardEvaluationProtocol2015-2021.Voorburg:KNAW,VSNUandNWO.Retrieved from: https://www.knaw.nl/nl/actueel/publicaties/standard-evaluation-protocol-2015-2021

Australian Research Council. (2018). Engagement and Impact As-sessment. Retrievedfrom:https://www.arc.gov.au/engagement-and-impact-assessment

Bacevic, J.(2017).BeyondtheThirdMission:TowardanActor-BasedAc-countofUniversities’RelationshipwithSociety.InH.ErgülandS.Cočar(Eds.),UniversitiesintheNeoliberalEra:AcademicCulturesandCriticalPerspectives(pp.21-39).London:PalgraveMacmillanUK.DOI:https://doi.org/10.1057/978-1-137-55212-9_2

Baert, P.(2015).Theexistentialistmoment:theriseofSartreasapublicintellectual.Cambridge:PolityPress.

Bateson, G. (2000). Steps towards an ecology of mind. Collected Es-says in Anthropology, Psychiatry, Evolution and Epistemology. Chica-go: University of Chicago Press. DOI: https://doi.org/10.7208/chica-go/9780226924601.001.0001

Bayley, J. and Phipps, D. (2017). Building the Concept of ResearchImpact Literacy. Evidence and Policy: A Journal of Research, Deba-te and Practice (in press). DOI: https://doi.org/10.1332/174426417X15034894876108

Benneworth, P., Gulbrandsen, M. and Hazelkorn, E. (2016).Theimpactandfutureofartsandhumanitiesresearch.London:PalgraveMacmillan.DOI:https://doi.org/10.1057/978-1-137-40899-0Bosch,G.(2018).TrainPhDstudentstobethinkersnotjustspecialists.Science,554(277).DOI:https://doi.org/10.1038/d41586-018-01853-1

Brundenius, C. and Göransson, B. (2011).TheThreeMissionsofUni-versities:ASynthesisofUniDevProjectFindings.InB.Göransson,andC.Brundenius(Eds.),UniversitiesinTransition.TheChangingRoleandChallengesforAcademicInstitutions(pp.329-352).NewYork:Springer.DOI:https://doi.org/10.1007/978-1-4419-7509-6_16

Burawoy, M. (2004). Public Sociologies: Contradictions, Dilemmas,and Possibilities. Social Forces, 82(4), 1603–1618. DOI: https://doi.org/10.1353/sof.2004.0064

Buxton, M. and Hanney, S. (1996).HowCanPaybackfromHealthSer-vicesResearchBeAssessed?JournalofHealthServicesResearchandPolicy,1(1),35-43.DOI:https://doi.org/10.1177/135581969600100107

Canadian Academy of Health Sciences. (2009).MakinganImpact:APreferredFrameworkandIndicatorstoMeasureReturnsonInvestmentin Health Research. Ottawa: CAHS. Retrieved from: http://www.cahs-acss.ca/wp-content/uploads/2011/09/ROI_FullReport.pdf

Chubb, J. (2017).Instrumentalismandepistemicresponsibility:resear-chersandtheimpactagendaintheUKandAustralia.UniversityofYork.Retrievedfrom:http://etheses.whiterose.ac.uk/18575/

Chubb, J. and Watermeyer, R. (2017).Artificeorintegrityinthemar-ketizationofresearchimpact?Investigatingthemoraleconomyof(pa-thwaysto)impactstatementswithinresearchfundingproposalsintheUKandAustralia.StudiesinHigherEducation,42(12),2360-2372.DOI:https://doi.org/10.1080/03075079.2016.1144182

Chubb, J., Watermeyer, R. and Wakeling, P.(2016).FearandloathingintheAcademy?TheroleofemotioninresponsetoanimpactagendaintheUKandAustralia.HigherEducationResearchandDevelopment,36(3),555-568.DOI:https://doi.org/10.1080/07294360.2017.1288709

Cranton, P. (2006).UnderstandingandPromotingTransformativeLear-ning:AGuideforEducatorsofAdults.2ndEd.TheJossey-BassHigherandAdultEducationSeries.SanFrancisco:Jossey-Bass.Retrievedfrom:https://www.wiley.com/en-us/Understanding+and+Promoting+Transformative+Learning%3A+A+Guide+for+Educators+of+Adults%2C+2nd+Edition-p-9780787976682

Derrick, G. (2018). The evaluators’ eye: Impact assessment and aca-demic peer review. London: Palgrave Macmillan. DOI: https://doi.org/10.1007/978-3-319-63627-6

Derrick, G. and Samuel, G. N. (2016).TheEvaluationScale:ExploringDecisionsaboutSocietalImpactinPeerReviewPanels.Minerva,54(1),75-97.DOI:https://doi.org/10.1007/s11024-016-9290-0

Deshler, D. and Selener, D.(1991).TransformativeResearch:InSearchofaDefinition.Convergence.24(3):9.Retrievedfrom:https://search.pro-quest.com/openview/039909077fd7c8ac9743831a0de569d6

Donovan, C. (2017). For ethical ‘impactology’. Journal of ResponsibleInnovation.DOI:https://doi.org/10.1080/23299460.2017.1300756

E3M. (2012).FosteringandMeasuring‘ThirdMission‘inHigherEdu-cationInstitutions(GreenPaper).Retrievedfrom:http://e3mproject.eu/Greenpaper-p.pdf

Essed, P., and Nimako, K.(2006).Designsand(Co)Incidents:CulturesofScholarshipandPublicPolicyonImmigrants/MinoritiesintheNether-lands.InternationalJournalofComparativeSociology,47(3–4),281–312.DOI:https://doi.org/10.1177/0020715206065784

European Commission. (2003).TheRoleofUniversitiesintheEuropeofKnowledge. Retrieved from http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=LEGISSUM:c11067

European Commission.(2014).CohesionPolicyFrequentlyAskedQues-tions.Retrievedfrom:http://ec.europa.eu/regional_policy/en/faq/

European Commission.(2017).WhatisHorizon2020?Retrievedfrom:https://ec.europa.eu/programmes/horizon2020/what-horizon-2020

European Commission.(2018).Horizon2020:OpenScience.Retrievedfrom:https://ec.europa.eu/programmes/horizon2020/en/h2020-section/open-science-open-access

ISSUE 48 | JULY 2019172

Freire, P. (2000). Pedagogy of the oppressed (30th anniversary Ed.).New York: Continuum. Retrieved from: http://commons.princeton.edu/inclusivepedagogy/wp-content/uploads/sites/17/2016/07/freire_peda-gogy_of_the_oppresed_ch2-3.pdf

Galligan, F. and Dyas-Correia, S. (2013).Altmetrics:RethinkingtheWayWeMeasure.SerialsReview,39(1),56-61.DOI:https://doi.org/10.1080/00987913.2013.10765486

Good, B. (2018).Teachingcriticalperspectives:Thetransformativelear-ningpotentialofdiversitycourseswithinDutchhighereducation(Ongo-ingunpublishedPhDthesis).VrijeUniversiteitAmsterdam,Amsterdam,theNetherlands.

Gornitzka, Å. and Maassen, P. (2007). An instrument for nationalpolitical agendas: the hierarchical vision. In P. Maasen and J.P. Olsen(Eds.),UniversitydynamicsandEuropeanintegration(Vol.19,pp.81-98).Dordrecht:Springer.DOI:https://doi.org/10.1007/978-1-4020-5971-1_4

Grant, J., Brutscher, P. B., Kirk, S., Butler, L. and Wooding, S.(2009).CapturingResearchImpacts.Areviewofinternationalpractice.Retrie-ved from: http://www.hefce.ac.uk/media/hefce/content/pubs/2009/rd2309/rd23_09.pdf

Gulbrandsen, M. and Sivertsen, G. (2018).Impactianvendtforskning:begrepsavklaringogpraksis.NIFU(Nordiskinstituttforstudieravinno-vasjon, forskning og utdanning) Arbeidsnotat, 10. Retrieved from: htt-ps://brage.bibsys.no/xmlui/handle/11250/2570526

Hamann, J. and Gengnagel, V. (2014). The Making and Persistingof Modern German Humanities. Balancing Acts between Autono-my and Social Relevance. In J. Maat, R. Bod and T. Weststeijn (Ed.),The Making of the Humanities (Vol. III The Modern Humanities, pp.641-654). Amsterdam: Amsterdam University Press. DOI: https://doi.org/10.1515/9789048518449-043

HEFCE. (2009).Reportonthepilotexercisetodevelopbibliometric in-dicatorsfortheResearchExcellenceFramework.Retrievedfromhttp://webarchive.nationalarchives.gov.uk/20120716093655/http://www.hef-ce.ac.uk/pubs/year/2009/200939/

HEFCE. (2011). Assessment framework andguidanceon submissions.Retrieved from https://www.ref.ac.uk/2014/media/ref/content/pub/as-sessmentframeworkandguidanceonsubmissions/GOS%20including%20addendum.pdf

HEFCE. (2016, 6 May 2016). How we fund research. Retrieved fromhttp://www.hefce.ac.uk/rsrch/funding/

Hicks, D. (2012). Performance-based university research funding sys-tems. Research Policy, 41(2), 251-261. DOI: https://doi.org/10.1016/j.respol.2011.09.007

Hicks, D., Wouters, P., Waltman, L., de Rijcke, S. and Rafols, I. (2015).Bibliometrics:TheLeidenManifesto for researchmetrics.Nature,520,429-431.DOI:https://doi.org/10.1038/520429a

Hill, C. E., Thompson, B. J. and Williams, E. N. (1997).Aguidetocon-ductingconsensualqualitativeresearch.CounsellingPsychologist,25(4),517-572.DOI:https://doi.org/10.1177/0011000097254001

Holbrook, J. B. and Frodeman, R.(2011).Peerreviewandtheexanteassessment of societal impacts. Research Evaluation, 20(3), 239-246.DOI:https://doi.org/10.3152/095820211X12941371876788

Hong Kong University Grants Committee. (2018). Research Assess-ment Exercise 2020. Draft General Panel Guidelines. Retrieved from:http://www.ugc.edu.hk/doc/eng/ugc/rae/2020/draft_gpg_feb18.pdf

Hood, W. W. and Wilson, C. S. (2001).Theliteratureofbibliometrics,scientometrics,andinformetrics.Scientometrics,52(2),291-314.DOI:ht-tps://doi.org/10.1023/A:1017919924342

INRA.(2018).Translatingresearchintoimpacts:30casestudies.Retrie-vedfrom:http://www.inra.fr/en/Scientists-Students/Economics-and-so-cial-sciences/All-the-news/Translating-research-into-impacts-30-case-studies

Jacob, M.(2006).Utilizationofsocialscienceknowledgeinsciencepoli-cy:Systemsofinnovation,triplehelixandVINNOVA.SocialScienceInfor-mation,45(3),431-462.DOI:https://doi.org/10.1177/0539018406066535

Jessop, B., Fairclough, N. and Wodak, R. (2008).Educationandtheknowledge-basedeconomyinEurope.Rotterdam:SensePublishers.Re-trieved from: https://www.sensepublishers.com/media/303-education-and-the-knowledge-based-economy-in-europe.pdf

Khazragui, H. and Hudson, J. (2015).Measuring thebenefitsofuni-versityresearch:impactandtheREFintheUK.ResearchEvaluation,24,51-62.DOI:https://doi.org/10.1093/reseval/rvu028

Kim, W. C. and Mauborgne, R. (2004).BlueOceanStrategy.HarvardBusinessReview,82(10),76-84.Retrievedfrom:https://hbr.org/2004/10/blue-ocean-strategy

King’s College London and Digital Science. (2015). Thenature, sca-leandbeneficiariesofresearchimpact:AninitialanalysisofResearchExcellenceFramework(REF)2014 impactcasestudies.Retrievedfrom:http://dera.ioe.ac.uk/22540/1/Analysis_of_REF_impact.pdf

Lamont, M. (2012).TowardaComparativeSociologyofValuationandEvaluation.AnnualReviewofSociology, 38(21), 201-221.DOI: https://doi.org/10.1146/annurev-soc-070308-120022

Lee, S. M. (2018, February 25). Sliced and Diced: The Inside Story ofHowanIvyLeagueFoodScientistTurnedShoddyDataintoViralStudies.Retrievedfrom:https://www.buzzfeednews.com/article/stephaniemlee/brian-wansink-cornell-p-hacking

Lightowler, C. and Knight, C. (2013).SustainingKnowledgeExchangeandResearchImpactintheSocialSciencesandHumanities:Investingin Knowledge Broker Roles in UK Universities. Evidence and Policy: AJournalofResearch,DebateandPractice,9(3),317-334.DOI:https://doi.org/10.1332/174426413X662644

ISSUE 48 | JULY 2019 173

Little, J. W. (1982).NormsofCollegialityandExperimentation:WorkplaceConditionsofSchoolSuccess.AmericanEducationalResearchJournal,19(3),325-340.DOI:https://doi.org/10.3102/00028312019003325

Lundequist, P. and Waxell, A. (2010).Regionalizing"Mode2"?Thead-optionofcentresofexcellenceinSwedishresearchpolicy.GeografiskaAnnaler:SeriesB,HumanGeography,92(3), 263-279.DOI:https://doi.org/10.1111/j.1468-0467.2010.00352.x

Manrique, S. (2018).AssessingtheImpactofUniversity-FirmCollabora-tiononFirmPerformanceandRegionalDevelopment(Ongoingunpublis-hedPhDthesis).UniversitatAutònomadeBarcelona,Barcelona,Spain.

Manville, C., Guthrie, S., Henham, M.-L., Garrod, B., Sousa, S., Kirtley, A., Castle-Clarke, S. and Ling, T. (2015). Assessing impactsubmissionsforREF2014:anevaluation.Retrievedfrom:https://www.rand.org/content/dam/rand/pubs/research_reports/RR1000/RR1032/RAND_RR1032.pdf

Manville, C., Jones, M. M., Frearson, M., Castle-Clarke, S., Henham, M.-L., Gunashekar, S. and Grant, J.(2014).Preparingimpactsubmis-sionsforREF2014:Anevaluation.FindingsandobservationsRetrievedfrom:https://www.rand.org/pubs/research_reports/RR726.html

Maxey, I. (1999). Beyond boundaries? Activism, academia, re-flexivity and research. AREA, 31(3), 199-208. DOI: https://doi.org/10.1111/j.1475-4762.1999.tb00084.x

Mesirov, J. P. (2010).ComputerScience.AccessibleReproducibleRe-search. Science, 327(5964), 415-416. DOI: https://doi.org/10.1126/sci-ence.1179653

McGettigan, A. (2013). The great university gamble: money, marketsandthefutureofhighereducation.London:PlutoPress.Retrievedfrom:https://www.plutobooks.com/9780745332932/the-great-university-gamble/

Ministry of Education, Culture, and Science, The Netherlands. (2016).Amsterdam Call for Action on Open Science. Amsterdam, the Nether-lands: Government of the Netherlands. Retrieved from: https://www.government.nl/documents/reports/2016/04/04/amsterdam-call-for-ac-tion-on-open-science

Münch, R. (2014). Academic capitalism: universities in the globalstruggle for excellence. Abingdon, Oxon: Routledge. DOI: https://doi.org/10.4324/9780203768761

NIAD-QE (2018).NationalInstitutionforAcademicDegreesandQualityEnhancement of Higher Education. Tokyo, Japan. Retrieved from: htt-ps://www.niad.ac.jp/english/unive/

Ostrom, E. (2009).AGeneralFrameworkforAnalyzingSustainabilityofSocial-EcologicalSystems.Science,325(5939),419-422.DOI:https://doi.org/10.1126/science.1172133

Owen, R., Macnaghten, P. and Stilgoe, J. (2012).Responsibleresearchandinnovation:Fromscienceinsocietytoscienceforsociety,withsoci-ety.ScienceandPublicPolicy,39,751-760.DOI:https://doi.org/10.1093/scipol/scs093

Pereira, M.(2016).StrugglingwithinandbeyondthePerformativeUni-versity:Articulatingactivismandworkinan“academiawithoutwalls”.Women's Studies International Forum, 54, 100-110. DOI: https://doi.org/10.1016/j.wsif.2015.06.008

Priem, J., Groth, P. and Taraborelli, D. (2012).TheAltmetricsCollec-tion. PLoS ONE, 7(11), E48573. DOI: https://doi.org/10.1371/journal.pone.0048753

Reed, M. and Peters, E. (2014).UsinganEcologicalMetaphortoBuildAdaptiveandResilientResearchPractices.ACME:AnInternationalJour-nal forCriticalGeographies, 3(1), 18-40.Retrieved from:https://acme-journal.org/index.php/acme/article/view/724

Research Council of Norway. (2017).EvaluationoftheHumanities inNorway.ReportfromthePrincipalEvaluationCommittee.Retrievedfromhttps://www.forskningsradet.no/en/Article/Evaluation_of_humanities_research_in_Norway/1254012498362

ISSUE 48 | JULY 2019174

EX-ANTE RESEARCH IMPACT SELF-ASSESSMENT EXAMPLE (GOOD, 2018)ACCESSIBILITY

Are my research outputs accessible to different stakeholders and society in general? Do I communicate and disseminate them broadly and effec-tively?

Myresearchoutputswillprimarilyexistintheformofjournalarticlesandpo-tentialpolicydocumentswithdirectaccessavailabletoallparticipatingsta-keholders.Grade:2.0/5.0

RESEARCH DESCRIPTION

Title:Teachingcriticalperspectives–ThetransformativelearningpotentialofdiversitycourseswithinDutchhighereducation.

Type:PhDthesis.

Dates:September2018–Present.

Objective:StudyingtowhatdegreediversityeducationcoursesinTheNetherlandssuccessfullymeetcourseobjectives,incorporatecriticalperspectives,andreduceracistbehaviourswhileencoura-gingfurtherexplorationoftheseissuesbeyondtheclassroom.

Author:BradleyGood.

Institution:VrijeUniversiteitAmsterdam(NL).

Status: Formulation.

REFLEXIVITY

Do I reflect on how comprehensive, well-planned, ethical and critical my research is? Have I evaluated and critiqued my theories and analyses?

I regularlyreviseandupdatemyresearchplan inaccordancewithnewlite-rature and theories. My analysis itself is based on a theoretical frame thatencouragesdeconstructionandcriticalanalysis.Grade: 4.0/5.0

SELF-ASSESSMENT CONCLUSIONS

Research Impact Pentagon

ASSESSMENT CONCLUSIONSOverallmyresearchseemstobesuccessfullyplannedformodera-teresearch impact.However,accessibilitycouldgreatly improve,with a secondary emphasis on ecology. While adaptability doesnothaveahighscore,thisisprimarilyduetothelimitedscopeofresearch,whichisunavoidable.

ECOLOGY

Does my research consider the relationships and connections among sta-keholders and subjects? Was I collegial while conducting this research?

Myresearchsubjectsarealsosomeofmymostimportantstakeholdersasim-provingtheireducationalopportunitiesbenefitsthem,aswellastheirinstruc-torsandinstitutions.Grade:3.0/5.0

ADAPTABILITY

Is my research impact usable in different contexts and among different stakeholders? Am I aware of the limitations, and unanswered or emerging questions from my research?

WhilemyresearchfocusesonaDutchcontext,itcouldbeadaptabletootherhighereducationculturesinthefuturebutonlyaftermultiplestudies.Thisisduetothelimitedsamplesizeandtimeconstraints.Grade:3.0/5.0

ISSUE 48 | JULY 2019 175

MID-TERM RESEARCH IMPACT SELF-ASSESSMENT EXAMPLE (MANRIQUE, 2018)RESPONSIVENESS

Does my research respond to real problems and needs in society? Am I contributing to current public debates?

University-firm collaboration can be a powerful tool for the performance offirmsandforthedevelopmentofregions,whichcanindirectlyendupbene-fitting citizens. However, my research is primarily focused on the economicimpactonindustry.Grade:4.0/5.0


Title: Assessingtheimpactofuniversity-firmcollaborationonfirmperformanceandregionaldevelopment(partofahorizon2020trai-ningnetwork).

Type: PhDthesis.

Dates:February2017–Present.

Objective:Assessingtheimpactofuniversity-firmcollaborationonfirms’innovationcapacityandeconomicperformance,andexplo-ringhowsuchimpacttranslatesintoeconomicgrowthandsocialdevelopmentintheregionswheretheinteractiontakesplace.

Author: SergioManrique.

Institution: UniversitatAutònomadeBarcelona(ES)

Status:Inexecution.

More info at:https://runinproject.eu/sergio-andres-manrique-garzon/

ACCESSIBILITY

Are my research outputs accessible to different stakeholders and society in general? Do I communicate and disseminate them broadly and effec-tively?

ResearchinmyprojectismeanttobepublishedinOpenAccessoutlets.Iamactiveinattendingconferencesandworkshopstocommunicateanddissemi-natemyfindings.Work inprogressandother researchoutputs (blogposts,reports)arepubliclyavailableattheprojectwebsite.Grade:4.5/5.0

REFLEXIVITY

Do I reflect on how comprehensive, well-planned, ethical and critical my research is? Have I evaluated and critiqued my theories and analyses?

PhDtopicswithinthisHorizon2020projectweremostlyfixed.Ihave,however,spentasignificantamountof timeplanningthemethodsanddata Ishoulduse.Intheend,Iincorporatedaqualitativeapproachtoaprojectwhichwasplannedtobequantitative,andnowIamconductingmixedmethodsresearch.Grade:1.5/5.0



ASSESSMENT CONCLUSIONS

MyPhDproject’simpactisboostedbybeingpartofaHorizon2020trainingnetwork,throughwhichaccessibilityofresearchoutputsisfacilitated.Additionally,Imakepartofanestablishednetworkofacademicsandregionalstakeholders,whichcontributetoshapingmyresearchinaresponsivemanner.However,bybeingaprojectplanned in advance (before recruiting researchers), the range ofactionontheresearchdesignislimited,andtherehasn’tbeentoomuchfocusoncriticalthinking.

ECOLOGY


Imakepartofateamofjuniorandseniorresearchersaswellasregionalandnon-academicpartners.MyprojectisonepieceinthelargerRUNINproposal.Grade:3.0/5.0

ADAPTABILITY


WhatIamdoingusingSpanishdatacanbereadaptedusingdatafromothercountriesandregions,andforphenomenabeyonduniversity-firmcollaborati-on.Ialwaysstateresearchlimitationsinmypublications.Grade:3.0/5.0

ISSUE 48 | JULY 2019176

EX-POST RESEARCH IMPACT SELF-ASSESSMENT EXAMPLE (WRÓBLEWSKA, 2018)RESPONSIVENESS

Does my research respond to real problems and needs in society? Am I contributing to current public debates?

MystudyoftheImpactAgendarespondstoaneedofacademicsandpolicy-makers to tackle thequestionof impactevaluation, focusingon theunder-studied aspect of language change and self-representation. Since I startedmyPhD,systemsofimpactevaluationhavebeenadoptedinseveralcountries,generatingconsiderableinterestinmywork’spracticalimplications,particu-larlyinthelinguisticaspectofeditingimpactcasestudies.Grade: 4.0/5.0


Title: ThemakingoftheImpactAgenda–Astudyindiscourseandgovernmentality.

Type:PhDthesis.

Dates: October2014–September2018.

Objective: Examiningthechangeinacademicdiscourseengende-redbytheintroductionoftheImpactAgendaanditslinktopracti-cesofsubjectivation(workuponone’s‘self’).

Author: MartaNataliaWróblewska.

Institution(s):UniversityofWarwick(UK)

Status:Concluded(nowindisseminationphase)

More info at: https://warwick.ac.uk/mnwroblewska

ACCESSIBILITY

Are my research outputs accessible to different stakeholders and society in general? Do I communicate and disseminate them broadly and effectively?

Ihavedraftedan‘executivesummary’ofthefindingsfrommyPhDworkandshareditwiththestudy’srespondentsandstakeholders.Thereachofmyfin-dingsremainslimited,butIamseekingfundingforapractice-orientedpubli-cation,ideallyinopenaccess.Grade:2.0/5.0

REFLEXIVITY

Do I reflect on how comprehensive, well-planned, ethical and critical my research is? Have I evaluated and critiqued my theories/ analyses?

Reflexivityandethicswereat thecoreofmystudy.Still Iquestion towhatdegreemycriticalstandpointisinfluencedbymyacademicbackground–oneneedstobecriticalof‘criticaltheory’too!Grade:4.0/5.0



ASSESSMENT CONCLUSIONS

Given thatmy work focused on the rise of ‘impact evaluation’, Iwasconstantlyquestionedbyaudiencesaboutthe impactofmyownwork.Thisincentive,combinedwithresourcesofferedbymyinstitutionforfosteringresponsibleoutreach,accountforthefactthatIhavereflectedonandpursuedimpactinmyPhDproject.TheweaknessofmyprojectseemstobeaccessibilityoffindingsandsoIresolvedtofocusoncreatingopen-accesspublicationsonthepracticalelementsofmyresearchfindings,whichwouldimprovemyscoreinthisarea.

ECOLOGY


Toalargedegreemyworkwassolitaryandindividualistic.Imightnothavefullyusedthepotentialpresentinmyresearchteam.IalsoworryaboutthecontrolIhaveovertheapplicationofmyfindingsbystakeholders.Grade:3.0/5.0

ADAPTABILITY


I’veengagedwithstakeholdersinothercountries(Poland,Norway)pointingtoopportunitiesandchallengesrelatedtoadaptingimpactevaluation.Inthissensemyresearchisadaptable,butthequestionremainstowhatdegreecanascholarinfluencepolicy?Grade:4.0/5.0

ISSUE 48 | JULY 2019 177

AUTHORSSERGIO MANRIQUEDepartment of Business, Universitat Autònoma de BarcelonaPlaçaCívica,Barcelona,08193(Spain)E:[email protected]

MARTA NATALIA WRÓBLEWSKACentre for Applied Linguistics, University of WarwickS.1.74SocialSciencesBuilding,Coventry,CV47AL(UK) E:[email protected]

AND BRADLEY GOODDepartment of Sociology, Vrije Universiteit AmsterdamDeBoelelaan1105,Amsterdam,1081HV(TheNetherlands)E:[email protected]

KEYWORDS:ResearchImpact,ResearchEvaluation,Self-Assessment,ResearchEthics.

ISSUE 48 | JULY 2019178

TheevaluativeinquirywasproposedbyFochlerandDeRijcke(2017)asawaytocontributetoongoingdiscussionsaboutqualityandrelevan-ceofresearch.OurteamatCWTS(LeidenUniversity,TheNetherlands)hassincethenputtheevaluativeinquiryintopracticeinseveralprojects,andthisworkinformsthispaper.Ourambitionwiththeseexperimentsinresearchevaluationis,inessence,toenablebetterconversationsaboutacademicvalueanditsbeneficiariesandrewards,ratherthantofurtherencourage“accountingfor impact” (RushforthandDeRijcke,2015)bywayofstandardisedformatsandrankings.

1) THE DUTCH CONTEXTTosituateourapproach,afewwordsareinorderonthemaincharac-

teristicsoftheDutchhighereducationandsciencegovernancesystem.Like many other European countries, the Dutch higher education sys-temdistinguishesbetween two typesofhighereducation institutions:universities on the one hand, and institutions for higher vocationaleducation – so-called hogescholen – on the other. As a general rule,sciencegovernance instrumentssincetheearly1980shavebeenbuiltaroundaprincipleof“steeringatadistance”.TheNetherlands infactoperatewithwhatRichardWhitley(2007)hascalleda“weak”systemofresearchevaluation,meaningthatassessmentresultshavenodirectconsequencesforthedistributionoffundingtouniversities(incontrastto,forexample,theUK).Rather,theprincipalstrategyistouseformalevaluation as opportunities for self-reflection and organisational lear-ning (see also Youtie and Corley, 2011; Hansson and Monsted, 2012).Conceptually,institutionalresearchevaluationsystemscanservethreemain purposes: a distributive, an improvement, and a controlling use(Molas-Gallart,2012).IntheDutchcontext,thepurposeofevaluationisclearlyfocusedonimprovement,andanevaluationcanalsosparkorga-nisationalchange.AsMolas-Gallartputsit,“[a]nimprovementusewillfocusonderivinglessonsfromthepastexperiencetoadapttheactivitiesconducted towhatevaluation studieswill conclude isbetterpractice.Theimprovementpurposeisthereforerelyingontheexistenceoffeed-backmechanismsandtheoperational flexibilityneededtofunctionasalearningorganization.”(ibid,589)Wewouldsuggestthatanimprove-ment-orientedevaluationsystemliketheDutchoneprovidesparticularopportunitiesforexperimentingwithevaluativeinquiries.

ABSTRACT

Traditional frameworks for academic evaluation are focused onregistering the achievements of research units’ academic andsocietal achievements. These frameworks and the ways they

areusuallycarriedoutarebuiltonafewdichotomies:academicversussocietalspheres,quantitativeversusqualitativeapproaches,andrepre-sentativeversusinterveninganalyses.Wearguethatthesedichotomiescontributetoanotionofacademicachievementthat isunrealistic, inanormativeanddescriptivesense.Theconceptofthe“evaluativeinquiry,”asproposedhere,amendsthelinearandindividualisednotionofacade-micworkanditsevaluationanddiscussestheimplicationsofthesemovesfortheworkoftheanalyst.Wesuggestinsteadtounderstandacademicachievementasdistributedoverahostofacademicandnon-academicparticipantstobestudiedbymeansofaportfolioapproach.Thisapproachtoresearchevaluationrequiresamoreengagedanalystwhotakesevalu-ationseriouslyasbothananalyticalandastrategicproject.

INTRODUCTIONThispaperintroducestheevaluativeinquiry,anapproachthataims

tochallengeseveraldimensionsofthecurrentsciencesystemandtheorganisationofresearchevaluation,mostnotablyitsunderstandingsofacademicachievement, impact,andthewaystheseshouldbemeasu-red.Itcontributestoa“re-loading”ofthetermimpact,drawingonthemethodologicalandconceptualapproachesof thesocialsciencesandhumanities in particular, and all the sciences in general (König et al.,2018). We propose a distributive understanding of academic achieve-ment,therebyrecognisingthecontributionsofbothacademicsandnon-academics.Inaddition,weputforthaportfolioapproachtoevaluationin order to detect the multiple realities that go into academic qualityandinordertoinspireconversationsaboutthese.Lastly,wemovebey-ondapproachesthatclaimtoneutrallyrepresentqualityandrelevance,byofferingastyleofstrategicandcollaborativeintervention.Wehopethesethreemoveswillhelp identifypathsto reformandrevitalise thesciencesystemandthenormative,unilateral,anddichotomousidealsofexcellenceandimpact.

SARAHDERIJCKE,TJITSKEHOLTROP,WOLFGANGKALTENBRUNNER,JOCHEMZUIJDERWIJK,ANNEBEAULIEU,THOMASFRANSSEN,THEDVANLEEUWEN,PHILIPPEMONGEON,CLIFFORDTATUM,GOVERTVALKENBURGANDPAULWOUTERSDOI:10.22163/fteval.2019.386

EVALUATIVEINQUIRY:ENGAGINGRESEARCHEVALUATIONANALYTICALLYANDSTRATEGICALLY

ISSUE 48 | JULY 2019 179

TheNetherlands introducedaformalevaluationsystemasearlyas1982.All researchunitsatDutchuniversities (aswellasthe institutesoftheRoyalNetherlandsAcademyofArtsandSciencesandtheNether-landsOrganisation forScientificResearch)are required toundergoanassessmentinsix-yearlyintervals(seefigure1).Theassessmentconsistsofapeerreviewprocedurebyanexternalcommittee,involvingpersonalsitevisits, interviews,anda reviewof researchoutputandotheracti-vities. Halfway between the 6-yearly national research assessments,researchunitsaremoreoverrequiredtoconductamid-termevaluation.Theresultsofaself-evaluationserveas inputfortheassessment,andarealsomeanttoencouragecontinuousself-monitoringofindividualin-stitutions.Anotherimportantchangeintroducedin2015wasthatorga-nisationalresponsibilityforevaluationwasdecentralisedanddelegatedtoindividualinstitutions,thusprovidingthemwithagreaterdegreeofadministrativediscretion.

The exact modalities of assessment are outlined in the so-called“StandardEvaluationProtocol”(SEP).AccordingtotheSEP,institutions

arerequiredtoprovidearangeofmaterialsasinputfortheassessment,includinginteraliaaformaldocumentationofoutputand“performanceindicators”(e.g.,acompletelistofpublications,numberofsuccessful-lydefendedPhDthesesetc.),adescriptionofthefinancingofagivenresearchunit, andaqualitativenarrative summarising the resultsandsocietalrelevanceoftheresearch(seetable1below).Whereasevaluati-onhastraditionallyplacedanimportantemphasisonquantityofoutputandtheperceivedprestigeofpublicationsandresearchgrants,the2015iteration of the SEP introduces a stronger emphasis on “societal rele-vance”ofresearch,i.e.theengagementwithnon-academicaudiencesandpartners.Academicexcellenceandsocietalrelevancearehoweverkept largely separate in the evaluative framework. Research units areultimatelygradedaccordingtoafour-tieredscale(from“world-leading”to“unsatisfactory”).

Figure 1.StepsintheDutchevaluationprocess.

Descriptionofunit’sorganisationalstructure.

Mostimportant(andrelevant)performanceindicators.

Descriptionofunit’sfinancing. Resultsresearchandsocietalrelevancepast6years(latterinanarrative).

Strategypast6years. LinkresultstoSEPcriteria(quality,relevance,viability).

Targetspast6years(research,societalrelevance).

Strategyandtargetsnext5-10years.

Relevantenvironmentalfactorsanddevelopmentspastsixyears.

PhDProgramme(s)

SWOT(Strengths,WeaknessesOpportunities,Threats)analysisandbenchmarking.

Researchintegrity

Table 1: Formal requirements for self-assessment report (SEP 2015-2021).

ISSUE 48 | JULY 2019180

2) EVALUATION AS A KNOWLEDGE MAKING PROJECT

Theevaluativeinquirywasfirstintroducedasaprompttomoreena-bling rather than reductiveaccountsofassessmentbyFochleranddeRijcke (2017). The evaluative inquiry understands academic work as aprocessinwhichavarietyofactors(includingnon-academicones)arepartofthesociotechnicalnetworksthroughwhichknowledgeisgene-rated.Inemphasisingprocessandengagementratherthanratingandranking,itwantstobringtolightthewayqualityiscreatedandnego-tiatedamongmultipleparticipantsandamidstmultipleepistemiccom-mitments,ratherthanattributingittoindividuals’actionsandintentionswho are subsequently compared. As such it is aligned with scholarlywork that is interested inacademicworkandqualityas it comes intobeingininteractionsbetweenvaluesandnetworksofpeople,outputs,andresources(e.g.,BozemanandSarewitz,2011;deJongetal.,2014;Mattetal.,2016;PrinsandSpaapen,2017).

OurteamatCWTSisputtingtheevaluativeinquiry intopracticeinseveralprojects.Onthebasisofthiswork,wefurtherspecifytheinquiryin relation to three contentious issues within the current science sys-temanditsevaluation:themuch-debateddichotomybetweenacademicandsocietalrealms,thedistinctionbetweenquantitativeandqualitativeapproaches,andtheallegedlyinvisibleandneutralevaluatinganalyst.

Thefirstissueweidentifyistheunderlyingdivideinmanyevaluati-onframeworksbetweentheacademicandthesocietal.Theinclusionofbroader impacts intoframeworksthatoriginallyputmostemphasisonacademic work has done justice to the interconnections between sci-enceandsocietythatareespeciallystrongforthesocialsciencesandhumanities. However, the way this relation is often imagined is prob-lematic (cf.Calvert,2006;deJongetal.,2014;FeltandWynne,2007;Nowotnyetal.,2001).Itisoftenenvisionedasalinearmodelofscientificknowledgeproductionthatstartswithfundamentalresearchanddisco-very,andendswith innovationsthatarebeneficial forsocietythroughtranslationalandappliedresearch.Withinthismodel, the influenceofknowledge in society could be traced back to original inventions, theultimatevalueofwhichcan thenbeestablished.Aproblemwith thismodel is that the individual (scholaror research institute) remains thelocusofbothvalueandresponsibilitynownotonlyforacademicpubli-cations,butalsoforproducingsocietalrelevance(Holtrop,forthcoming).Ratherthanrealisingthatacademicworkfrequentlyentailsengagementwithsocietalactors–andthereforeonecouldarguethatbothrelevanceandqualityoriginateinthatinteraction–onenowhastowriteexcellentpapersandperform insocietally relevantwaysaswell.Theevaluativeinquiryproblematises thenotionofapassivepublicaudience that re-apsthebenefitsofacademicexpertise,andinsteadhighlightsthe“pro-ductiveinteractions”(SpaapenandvanDrooge,2011)betweendiversestakeholders,andthedistributednatureofacademicachievementmoregenerally.Regardingtheassessmentofimpact,thiswouldatleastentailthataudiencesareseennotonlyas(co)producersofknowledgeanditsimpact,butalsoas(co)producersofthecriteriabywhichsuchimpactistobeevaluated.

Theunhelpfuldividebetweenspheresandstakeholdersisperpetua-tedbyanotherunhelpfuldivide:theonebetweenquantitativeandqua-litativeevaluationmethods.We recognise theworkdone inacademicandprofessionalenvironmentstoproblematiserelianceonmetricsand

citationscoresalone,andthatarguesthatquantitativeandqualitativemethodsareimplicatedinoneanother(cfCallonandLaw,2005).Mo-reover,initiativessuchasthe“LeidenManifesto”(Hicksetal.,2015),the“MetricTide”(Wilsdonetal.,2015)and“DORA”,havepresentedcarefulresponsesandsuggestionsfornextsteps.Wefeelakintotheseinitia-tives,andwishtostayawayfromtheunproductivedichotomyofquan-titative and qualitative methods. In our contribution, we move from afixationon“gettingitright”inevaluations,toanapproachthatpresentsresearchnumerically,verbally,and/orvisuallyinwaysthatmakevisiblethecomplexityofactualpracticeanditsengagements(FochlerandDeRijcke,2017).Thismeansthatevaluativeinquirytreatsknowledgepro-ductionasheterarchical(Stark,2011):itseesphenomenaasamenabletomultipleordersofworth,ratherthanasconnectedtoonerankorderwithclearwinnersandlosers.

Ourunderstandingoftheenterpriseofacademicevaluationchangeswhilewemovefromalinearmodelofacademicachievementevidencedby individual actions and intentions to an understanding of academicvalueassituatedwithinmultipleepistemiccommitmentsandrelationsbetween many actors. Evaluations are now no projects that look intoacademicworldsfromtheoutsidewhiletakingstockofthevaluables.Instead they are themselves knowledge producing endeavors, trans-formingevaluatorsandanalysts intocollaboratorsalongsideevaluees.Thisisthethirddichotomythattheevaluativeinquirywantstounsettle:theonebetweenadetachedanalystdoingrepresentationsobjectivelyontheoutsideandanengagedanalyst locatedwithin.Webuildonapreviouswork thatproblematises theclaimtodetachment,objectivity,andneutralitythathascharacteriseddominantmodesofresearchevalu-ation(Candeaetal.,2015;DastonandGalison,2007).Instead,wetakeseriouslythattheactofrepresentingqualityisalsoanintervention(DeRijckeandRushforth,2015).

3) THE INQUIRYCentral to the evaluative inquiry is an understanding of academic

achievementasdistributedoverahostofacademicandnon-academicparticipants.Theseachievementsare tobestudiedbymeansofaco-produced portfolio approach, tailored to specific research units andevaluation purposes. Each method has its own strong points when itcomestodetectingandamplifyingreality(Law,2004).Ratherthanad-vocatingacombinationofmethodswiththepurposeofcomingtomoreaccuraterepresentationsofacademicwork,wearguethatco-productionandmultiplicationofmethodsallowsformoreinterestingconversationsaboutacademicquality,andofferspointsofdepartureforstrategicallyaddressingalltooreal issuesofpower,moneyandreputationthatarepartofacademicevaluation.Thoughtheinquiryremainsconcernedwithreachinganadequateunderstandingofacademicachievement(orqua-lity) in the analysis, the approach actively seeks to avoid reproducingthefamiliarroleoftheanalystasadetachedaccountant.Recognisingevaluationitselfasbothananalyticalandastrategicproject,theanalystthusmovesfromobjectiveobserverintotheroleofanengagedevalua-tionexpert,notonlyengagingintheanalysisofqualitybutalsointheanalysis of the broader political projects of accountability with whichit is intertwined.Morethanworkingtowardsadefinitivereportwhereresearchunitsareassessedonthebasisofapredefinedsetofcharacte-ristics,theinquiryissetuptostudy,mapandtracetheresearchthemes,pathwaysandproductiveinteractionsaroundtheresearchunitthrough

ISSUE 48 | JULY 2019 181

a portfolio ofmethods. Practically, the inquiry takes the formof threeconsecutivephases(seetable2).

Phases Approach

1.Exploration Whatisatstakeintheassessment?Questionsaddressedindocumentanalysesandinterviewswithmanagement.

Designofresearchapproach,choiceofcombinationofmethods.

2.DatacollectionandAnalysis #(e.g.)“ContextualResponseAnalysis”(Prins,n.d.);contextualscientometrics(WaltmanandvanEck,2016);bibliographiccoupling;co-citationanalyses;“AreaBasedConnectedness”(Noyons,2018)#Interviewswithresearchersandstakeholdersregardingorganisationalandacademicthemes,operationalisation,outputsandimpacts.

Workshops–totesthypotheses,presentthemesandpathways,collectmoreinputforSWOT-analysis.

3.Reporting Analysisintermsoforganisationalissuesandacademicambitionsandthemes.SWOT.Suggestionsastohowtowritetheself-evaluation.

Table 2: Phasesinanevaluativeinquiry.

Anevaluativeinquiryisgearedtowardsdetectinglivelyinteractionsand outcomes, which can be enveloped in themes and ‘pathways’(looselybasedonMattetal.,2016),therebyconnectingacademicandsocietaldomains.Inpastprojects,wehavebuiltthesethemesandpa-thwaysbyusingacombinationofquantitativemethods,interviewsandworkshopstocollectinformationaboutthethemespeopleworkon,theresourcesandpeoplethataremobilisedintheirresearchprojects,thehighlyvariedoutputsthataregeneratedasaresult,andthewaytheseoutcomestravelelsewhereintootheracademic,professionalorsocietalrealms(seetable2).Thoughtheinquiryallowsfordifferentcombinationsofmethods,oneformthiscombinationcantakeisthatofagenerativedialogue. Interviewsandquantitativeanalysesarefirstusedtogatherinformation, forexamplewith regards to themes that researchersandstakeholders consider central to the work and the variable audiencesthatitreaches(orfailstoreach).Theoutcomesofthisfirstroundofinqui-ryareusedtoidentifyafirstsetofpossiblethemesandpathways,whicharethenpresentedbacktotheresearchunit,forexampleinacollectiveworkshop.Thepresentationismeanttoelicitfurtherresponse,andal-lows those involved to thinkwithandelaborateon the first results inacollectivesettingwithcolleagues,stakeholdersandtheanalysts.Thefinalreportissubsequentlywrittenintermsoftheorganisationalissuesandtheacademicambitionsandthemesthathaveemerged.Thiscouldforexampleincludeaninteractiveanalysisandvisualisationofprevalentthemesandambitions,theiroperationalisation,thepeopleandresour-ces that are mobilised, theoutputs thisgenerates and theway thesearecited,used,andtravelfurtherintotheworld.Theoutcomesofthistypeofdetailedinteractionwithindividualresearchers,researchleadersandtheirworkcanbeaddedtotheinformationthatisgatheredintheanalysisoforganisationalissuesrepresentedinaSWOT.Organisationaldocumentsanddatacanbecombinedwithinsightsgainedinindividualinterviewsorworkshops,addingadditionaldepthandpossiblyroomforcreative synergy between people and data. Crucially, these processesandrolesarescriptedtogether,soastoenablebothahighlyrigorousandahighlygroundedanalysis.

The subsequent self-assessment document is authored by the re-searchunit itself.Our report iswritten insuchaway that it canbeaconversationpieceandoffersopeningsfordiscussion–internally,andwithotheracademicinstitutes,sciencepolicyenvironmentsandstake-holdersinterestedinacademicquality.Theinquirycan,butnotnecessa-rilydoes,fixthestateoftheobjectofevaluationinadefinitiveaccount.Moreover, theoutcomesof the inquiryare in thissensenot limited tothe report. Individualelementsof theevaluative inquiry itself, like theworkshop,areexcellenttoolstobringtheorganisationand/orstakehol-derstogetherandcollaborativelyidentifyproblems,maketoughdecisi-ons,workonsolutions,orplanforthefuture.Theytriggermeaningfulconversationsabouthowtodealwithpressingchallengessuchastheincreasingrolesanddemandsofpeercommunities,professionalandso-cietalpartners,governmentorindustrywhilebuildingonindividualandinstitutionalstrengths.Thepluralityofactorsinvolvedcantakeonavari-etyofrolesthroughouttheinquiry.Staffmembersandstakeholderscanbeconsciouslydrawnintotheproductionprocess,beinginsomewaystheexpertsandauthoritativeanalystsonthevaluesand interrelationsofthework.Theanalysts,inturn,aremorethanoutsiderswhomerely“run” pre-set quantitative or qualitative analyses: they become activeco-producersoftheinquiry.

DISCUSSION: EVALUATION AS KNOWLEDGE PRODUCTION

Thenotionof“impact”suggestsaninteractionofwhichthesource,target and content are clearly identifiable. Traditional frameworks forevaluationtendtofocusonwhetherandtowhatextentsuchimpactisachieved.Theconceptofevaluativeinquiry,asdevelopedinthispaper,revisesthislinearnotionofimpactasthecentralpreceptofresearcheva-luation.Theframeworkbuildsonworkbye.g.Mattetal.(2016),SpaapenandVanDrooge(2011),andPrinsandSpaapen(2017),byconceptuali-singscholarlyworknotintermsofalineardiffusionofknowledge,butratherasanemergenteffectofanunfolding,multidirectionalresearchprocess.Evaluativeinquiryrevealstheepistemiccommitmentsandcom-munity values of local practices. It thus essentially approaches evalu-ationasaknowledgeproductionprocess.Fromthisstartingpoint,ourapproachtoevaluationseestherelevanceofscientificworkasanunfol-dingprocess,inwhichavarietyofacademicandnon-academicactorsareinvolved.Thisapproachemphasisesprocessandengagementratherthanaccountingandranking.Crucially,evaluativeinquiryidentifiesva-lues,networksofpeople,andresourcesascollectives.Ithelpsarticulatehow“worlds”arecreatedandnegotiatedinrelationtothesevalues.

Withevaluativeinquiry,wethusmoveawayfromevaluationsasde-tached,cleardelineationsofacademicvalue.Researchersobviouslydonotjustproduceexcellentresearchorarticlesinjournals,orevenknow-ledgethatisofuseforsocietyatlarge.Morethanthat,theyarepromi-nentworld-makers,andtheirknowledgehasconsequencesfortheworldtheyandothersinhabitaswellastheirexperiencesinit.Thissuggeststhatdiverse(relational,communicative,organisational)values,activitiesandoutcomeshavetobetakenintoaccountinevaluations.Conventio-nalapproachestoresearchassessmenttreatthesevalueshierarchically.Thisworkswellwithinaccountabilitysystemsthatembedactorsandac-tionsinfixedandcalculablevalueregimes.Bothbibliometricallyframedassessmentsandassessmentsofsocietalimpactoperatethroughaquite

ISSUE 48 | JULY 2019182

similarlogicof“return-on-investment”.Anevaluativeinquiry,incontrast,isnotmeanttoresultinonedefinitivedocumentthat“makesuptheac-count”.Inthemodeofevaluativeinquiry,standardisationislessrelevantthanstayingclose to theepistemicmissions, frictionsand resonancesof academic work. The approach understands academic performanceor impactasaneffectoftranslationswithinandbetweennetworksofactorsthatmakeupacademicresearchanditsenvironments(cf.Mattetal.,2016).Theaimistofindoutwhatarethecentralissuesorambitions,how they are operationalised, what kind of outcomes this yields andwhere theoutcomes travel to.Evaluative inquiriesbroadenourunder-standingofwhatcountsasacademicachievement.Theyaffordagreaterinclusivityofresearchsettings,whichideallymeansamoremeaningfultreatmentofthesocialsciencesandhumanities.Assuch,ourcontributi-onismeanttogivespaceto,andreinforcethegreaterroletobeplayedby,thesocialsciencesandhumanities,includingthefieldsofresearchevaluationandscientometrics.Thesocialsciencesandhumanitieshavethe conceptual tools to enrich the methodological portfolio for gathe-ringinformationabouttheworldsthatacademicsinhabitandcontributeto.Apluralisationofperspectivesandmethodsenrichestheinquirybyopeningupwhatcanbetalkedaboutinevaluations.Thispluralisationshouldnothappeninsecludedspacessuchasscientometric labs,butwiththeparticipationofstakeholders,soastotakeinconsiderationtheircontexts (cf. Rafols, 2018). Furthermore, evaluative inquiry is sensitivetohowvaluesystemsmightdifferacrossteamsanorganisations,andevolveovertime.Thisapproachmakesitpossibletoarticulatepositions,rolesandvaluesthataresubordinatetodominantcurrentsinacademicpracticeandthatareoftensilencedintraditionalevaluations.

Inconclusion,withtheevaluative inquirywefullysubscribetothecall for“re-loading” thenotionof impact (Königetal.,2018),anotionwithproblematicballisticconnotations.Wehopeourcontributionfeedsongoingdiscussionsamongacademics,policy-makers,andotherstake-holdersaboutthefaultlinesbetweenformsofvalue,theuncertaintiesinevaluating,andthepoliticsofformats,protocolsandendings.Ourcon-tributionisastrongpleatocreatemoreroomforexperimentsinresearchevaluation(anditisclearthatwearenotdoneexperimentingourselves).Wethinkthissimultaneouslyentails:1)advocatingandconductingrigo-rousanalyticalwork;2)awillingnessofthoseunderassessmenttobeopentomoreengagedmodesofassessment;3)usingthefullpotentialoftheform(s)evaluationcantake;and4)usingquantitativemethodsinmuchmore interestingways.Rather than takingana-priori, reductiveapproachtowhatcounts in researchevaluation–thinkof theprolife-ration of publication lists and performance metrics – it is much moreuseful toproduceandpresentthemultiplemeaningsandpurposesofresearch.Evaluativeinquirytakesevaluationitselfasadeliberative,ge-nerativeprocessofknowledgeproductioninitsownright.Indoingso,itopensupmorethanonewayforempiricaldata,evaluators,andotheractorstobeimplicatedintheevaluation.Thegenerativecapacityoftheinquiryispartiallybuiltonkeepingmoreopentherolesofthevariousco-producers,andtheevaluativecriteriathatmaybegeneratedfromtheirvariablepositions.Thisalsomeansthatthelegitimacyoftheevaluationisnotsolelybasedon theanalyst’scorrect implementationofcriteria,butmuchmoresoonthedegreetowhichco-producersthinkthattheprocessandresultsdojusticetotheirjointwork.

REFERENCES

Association of Universities in the Netherlands (VSNU),theNetherlandsOrganisation for Scientific Research (NWO) and the Royal NetherlandsAcademy of Arts and Sciences (KNAW). Standard Evaluation Protocol(SEP)2015-2021.ProtocolforResearchAssessmentsintheNetherlands.

Bozeman, B. and Sarewitz, D. (2011).PublicValueMappingandSci-encePolicyEvaluation.Minerva,49(1),1-23.

Calvert, J.(2006).What’sSpecialaboutBasicResearch?Science,Tech-nology&HumanValues,31(2),199-220.

Callon, M. and Law, J.(2005).OnQualculation,AgencyandOtherness.EnvironmentandPlanningD:SocietyandSpace,23(5),717–33.

Candea M., J. Cook, C. Trundle and Yarrow T. (Ed.) (2015).Detach-ment: Essays on the Limits of Relational Thinking. Manchester: Man-chesterUniversityPress.

Daston, L. and Galison, P.(2007).Objectivity.Cambridge,TheMITPress.

De Jong, S., Barker, K., Cox, D., Sveinsdottir, T. and van den Besselaar, P. (2014). Understanding societal impact through productive interac-tions:ICTresearchasacase.ResearchEvaluation,23(2):89–102.

De Rijcke, S. and Rushforth, A.(2015).Tointerveneornottointervene,isthatthequestion?Ontheroleofscientometricsinresearchevaluation.JASIST,66(9),1954-1958.

Felt, U. and Wynne, B. (2007). Taking European Knowledge SocietySeriously. Report of the Expert Group on Science and Governance totheScience,EconomyandSocietyDirectorate,Directorate-General forResearch, European Commission. Luxembourg: Office for Official Pub-licationsof theEuropeanCommunities.RetrievedNovember1st,2018from: https://ec.europa.eu/research/science-society/document_library/pdf_06/european-knowledge-society_en.pdf

Fochler, M. and De Rijcke, S.(2017).ImplicatedintheIndicatorGame?AnExperimentalDebate.EngagingScience, Technology,andSociety, 3,21-40.

Hansson, F. and Monsted, M. (2012). Changing the peer review orchangingthepeers–Recentdevelopments inassessmentof largere-searchcollaborations.HigherEducationPolicy,25(3):361-379.

Holtrop, T.J.(forthcoming).TheEvaluativeInquiry:ANewApproachtoResearchEvaluation.Blogpost,forthcomingathttp://www.cwts.nl/blog.

König, T., Nowotny, H. and Schuch, K.(2018).ImpactRe-loaded.Pa-thways to impact from SSH research. CfP SSH Impact Conference Vi-enna. Retrieved 2 November, 2018 from: https://www.ssh-impact.eu/wp-content/uploads/2018/06/Impact_Re-loaded_180614.pdf

Law, J.(2004).AfterMethod:MessinSocialScienceResearch.London:Routledge.Matt, M., Gaunand, A., Joly, P.-B. and Colinet. L. (2016).Openingtheblackboxofimpact-Ideal-typeimpactpathwaysinapublicagricultural

ISSUE 48 | JULY 2019 183

researchorganization.ResearchPolicy,46,207-218.Molas-Gallart, J.(2012).ResearchGovernanceandtheRoleofEvaluati-on:AComparativeStudy.AmericanJournalofEvaluation,33(4),583–98.

Molas-Gallart, J., D’Este, P., Llopis, O. and Rafols, I.(2015).‘Towardsanalternativeframeworkfortheevaluationoftranslationalresearchini-tiatives’,ResearchEvaluation,25(3),235-243.

Nowotny, H., Scott, P. and Gibbons, M. (2001). Rethinking science:knowledgeinanageofuncertainty.Cambridge:Polity.

Noyons, E. (2018).Monitoringhowsciencefinds itsway into society:measuring societal impact through area-based connectedness (ABC).Blogpost. Retrieved November 1st, 2018 from: https://www.cwts.nl/blog?article=n-r2u264&title=monitoring-how-science-finds-its-way-into-society-measuring-societal-impact-through-area-based-connected-ness-abc

Prins, A. A. M. and Spaapen, J.(2017).Servingvariegatedaudiences:Fromrankingorientedevaluationtomissionorientedevaluation.ftevalJournalforResearchandTechnologyPolicyEvaluation,44(September),42–49.

Prins, A. (n.d.).ContextualResponseAnalysis.RetrievedNovember1st,2018fromhttp://www.adprins.nl/index.php?id=box2

Rabinow, P. (2011).TheAccompaniment:assemblingthecontemporary.Chicago:UniversityofChicagoPress.

Rafols, I.(2018).S&Tindicators'inthewild':contextualisationandparti-cipationforresponsiblemetrics.Blogpost.RetrievedNovember3rd,2018from: https://www.cwts.nl/blog?article=n-r2u254&title=st-indicators-in-the-wild-contextualisation-and-participation-for-responsible-metrics

Rushforth, A.D. and De Rijcke, S. (2015).AccountingforImpact?TheJournal Impact Factor and the Making of Biomedical Research in theNetherlands.Minerva,53,117-139.

Spaapen, J. and Van Drooge, L.(2011).IntroducingProductiveInterac-tionsinSocialImpactAssessment.ResearchEvaluation,20(3),211-18.

Stark, D.(2011).TheSenseofDissonance:AccountsofWorthinEcono-micLife.Princeton,NJ:PrincetonUniversityPress.

Waltman, L., and Van Eck, N. J. (2016).TheNeedforContextualizedScientometricAnalysis:AnOpinionPaper.InI.Ràfols,J.Molas-Gallart,E.Castro-Martínez,andR.Woolley(Eds.),ProceedingsoftheSTIConfe-rence2016.Peripheries,Frontiersandbeyond(pp.1–9).Valencia,Spain:UniversitatPolitècnicadeValència.

Youtie, J. and Corley, E. (2011)FederallySponsoredMultidisciplinaryResearchCenters:Learning,Evaluation,andViciousCircles.EvaluationandProgramPlanning,34,13-20.

Whitley, R. (2007).ChangingGovernanceofthePublicSciences.InR.WhitleyandJ.Glaeser(Eds),TheChangingGovernanceoftheSciences.

SociologyoftheSciencesYearbook(pp.3–27).Dordrecht:Springer.

AUTHORS[Numbers]CASRAICRediTtaxonomyforcontributorroles(https://casrai.org/credit/)

SARAH DE RIJCKE [1, 6, 7, 10, 12, 13, 14]

Centre for Science and Technology Studies (CWTS), Leiden University P.O.Box905,Leiden,2300AX(TheNetherlands)E:[email protected]

TJITSKE HOLTROP [1, 5, 6, 13, 14] Centre for Science and Technology Studies (CWTS), Leiden University P.O.Box905,Leiden,2300AX(TheNetherlands)E:[email protected]

WOLFGANG KALTENBRUNNER [13, 14]


JOCHEM ZUIJDERWIJK [5, 6, 13, 14]


ANNE BEAULIEU [13]

University of Groningen, Campus Fryslan (CF)Sophialaan1,8911AE,Leeuwarden(TheNetherlands)andCWTS, Leiden UniversityP.O.Box905,Leiden,2300AX(TheNetherlands)E:[email protected]:[email protected]

THOMAS FRANSSEN [5, 6, 13]


THED VAN LEEUWEN*[5, 6, 13]


PHILIPPE MONGEON [13]

Danish Centre for Studies in Research and Research Policy, Aarhus UniversityBartholinsAllé7DK–8000,Aarhus,C,DenmarkandCWTS, Leiden UniversityP.O.Box905,Leiden,2300AX(TheNetherlands)E:[email protected]:[email protected]

CLIFFORD TATUM[1, 13]

Centre for Science and Technology Studies (CWTS), Leiden University

ISSUE 48 | JULY 2019184

P.O.Box905,Leiden,2300AX(TheNetherlands)E:[email protected]

GOVERT VALKENBURG [13]

Department of Interdisciplinary Studies of Culture, Norwegian University of Science and Technology (NTNU)NO-7491Trondheim,NorwayE:[email protected]

PAUL WOUTERS [13]


KEYWORDS:

Researchevaluation;societalrelevance;intervention;mixed-methods;engagement;heterogeneity

ISSUE 48 | JULY 2019 185

KLAUSSCHUCHDOI:10.22163/fteval.2019.387

VALUATIONOFSSHRESEARCHFORATRANSFORMATIVEEUROPEANRESEARCHAGENDA–AFEWCLOSINGWORDS

1. INTRODUCTION

AscalleduponbythepromotersoftheAustrianPresidencyoftheEUCouncilConferenceon‘ImpactofSocialSciencesandHumanities for a European Research Agenda – Valuation of

SSHinmission-orientedresearch’itistimetore-loadthenotionofim-pact of Social Sciences and Humanities (SSH) and to shift away fromthetraditionalpre-dominantdefensivestancewhichSSHresearchoftenarticulatesinthediscussionabouttheimpactofresearch.Doubtlessly,mostargumentsraisedbycriticalSSHresearchersarerelevantsuchasthetoonarrowfocusoneconomicallyrelevanttechnologiesandinnova-tion,butadefensivestancenotaccompaniedbypositivepropositionswouldinevitablyleadtoamarginalisedpositionwhichissometimesal-readynowmetwithsuspicionfrompolicy-makers,butalsofromfellowcolleagues of the so called ‘hard sciences’. Thus, instead to dwell onthe embodiment of ‘integrating’ SSH into dominantly technologically-mindedprojectstoolong,SSHresearchersshouldshiftthenotionandthepromotiontoequallyvaluatedcontributionsofSSHtotransformativeinter-disciplinaryresearchwithSSHateye-level.

ThisalsomeanstopushforwardSSHintoadriver-seatinaddressinggrandchallengesandinimplementingmission-orientedresearchinHori-zonEurope.Thechallengesaregrandbecausetheyconcernoursocietiesand cultures. Challenges can and should not only be met by providingtechnologicalfixes,butbyinvestigatingtheirsocio-economicandculturalembeddingandstructuralfundamentsandbyaimingtoidentifysolutionswhichaddress,reflect,reframeandeventuallyalsochallengeandchangetheseunderlyingstructures.Bydoingso,SSHresearchcanprovidedisrup-tivecontributionstobreak-upwithtraditionalwaysofdoingthings.Thepo-liticaleconomyinanygrandchallengecanbecomescrutinised,butitalsoneeds to be addressed in calls launched under transformative researchagendas.Thisise.g.trueforthepoliticaleconomyofclimatechange,orthe political economy of transportation or of health research. The oftenraiseddifferentiationbetweenaninstrumentalunderstandingofSSHandareflexiveunderstandingofSSHisnothelpfulinthisrespectandhastobeovercomeintransformativeresearchbecausebothaspects(‘instrumen-tal’and‘reflexive’)areimportant.Finallyitalsoneedstoberepeated,thatinnovationandvaluecreationisnotjustthescopeofR&D,salesandmar-keting(orofPillar3inHorizonEurope),butasocialprocesswithvarioussocialimplicationsthatcanbeaddressedbyfieldssuchasanthropology,culturalstudies,education,sociologyorhumanandeconomicgeography.

2. A LOOK BACK ON SSH IN HORIZON 2020

Aroundthepeakofthefinancialandeconomiccrisisaround10ye-arsago,nationalresearchbudgetswerecutduetofinancialconstraintsinseveralcountries(SchöglerandKönig,2017;EUA,2011;).Thesecutswereoftenalsoaddressing theSocialSciencesandHumanities (SSH)(Marimon et al, 2011, Papanagnou, 2011). Moreover, in the dawn ofHorizon2020, theEU’s8thFrameworkProgrammeforRTD,theroleofSocialSciencesandHumanities (SSH)within theworld’s largestcom-petitiveresearchprogrammewasdowngradedtoo (seeKönig,2019 intheseproceedings).ItwaspoliticallyintendedtomainstreamSSHacrosstheentireHorizon2020 (EuropeanParliamentandCouncil,2013)withtheconsequenceofabolishingthespecificsub-programmededicatedtoSSHtopics.Theseattempts,however,metresistancefrompartsoftheSSHcommunitiesbecause‘mainstreaming’oftenresultsin‘ceding’.AnOpenLetterwassignedbyalmost26,000people(Rammeletal.,2017),and someof the research ministers from theEUmember states weresuccessfullymobilisedtoexpresstheirconcernsagainstthis‘mainstre-aming’,whichwasfrequentlyperceivedas‘downsizing’ofSSH(seeKö-nig,2019intheseproceedings).

Theprotestwasrelativelysuccessful.SSHresearchiswithinHorizon2020nowcoveredbysixpanelsintheEuropeanResearchCouncil,hasadedicatedslotwithintheSocietalChallengesPillarofHorizon2020(how-everwithapityingmarginalbudget)andismoreorless(withemphasison‘less’)sufficientlymainstreamedacrosstheSocietalChallengesPillar.

Alsotheoverallmind-setseemstohavechangedbecausetheinclu-sionofSSHwithadedicatedtopicalnichewithinthenexteditionoftheEuropeanFrameworkProgrammeforResearchandInnovation(inaddi-tiontoitsplaceintheERCandthestillvalidmainstreamingapproach)was politically almost not contested anymore. This mind-set change,however,doesnotmaterialiseinaconsiderablelargerSSHbudgetforitstopicalniche,butmightratherbeanindicationofamoresoberexpecta-tionmanagementbyR&Ipolicymakersinthatsense,thattechnologicalfixeswithoutproperconsiderationofhumanconditionsarenotsufficientfortacklinggrandchallengesandinducingtransformationalchanges.

Theideabehindthisobservablemind-changeseemstoreferprimarilytoattributinganenhancedsupportorleveragefunctionofSSHtoamore‘society-ready’technologicaldevelopment,inordertoavoidwasteofre-

ISSUE 48 | JULY 2019186

culturesbutalsohowwechangethemandourbehaviour.BeforeaskinghowSSHcanmitigatetheeffortoftechnologicaladaptationstosocialconditions, needs and wants, thus contributing to an innovation racewhichcontinuouslyseemstopickuppace,SSHshouldalsobeemplo-yed to frameandanalyse thewickedproblemsbeforea technologicalsolutionismapproach (Morozov,2013) is taken.Unfortunately,calls forproposalsseldomaskforthis.EspeciallyinHorizon2020thescopeandtheexpectedimpactsofthetopicscalledfor,areusuallyveryspecificallydescribedindetailandoftenmoreorientedtowardsanend(i.e.aspeci-ficoutput,solutionorimpact)thanaproperproblemanalysis.Althoughitcouldbeargued,forinstance,thatanytopicaddressedunderHorizon2020 (from “A” like agriculture, “B” like bio-economy, “C” like climatechangeto“Z”likezero-waste)wouldatleastdeserveaproperanalysisofthepoliticaleconomyunderlyingthesetopics.

Instead,technologicalsolutionismpromisesquickresultsandprofitsand is positively connoted with an attractive entrepreneurial ‘hooray – let’s go for it’ image,whichhasunderminedandcaptured researchpolicy-makingsincemorethan30yearsandwhichledtothe“holy du-ality” ofresearchandinnovation.Theconceptof‘societalreadinessle-vels’isfittingthisauxiliaryunderstandingofSSHtoleveragethesocialacceptanceoftechnologies.ItshouldabsolutelynotbedeniedthatSSHcanbeveryusefulinthisrespect.Onthecontrary,usageofnoveltyandaccompanyingmarketpervasion(whichistheeconomicessenceofin-novation)isasocialprocesswithvarioussocialimplications.Innovationresearchthuscanbeasubjectofbusinesseconomics,butalsoofan-thropology,culturalstudies,politicalsciences,sociology,economicandhumangeographyandsoforth.

Innovationisbasicallyanythingnewthatcreatessomeformofvalue;andthereisnovaluecreationwithoutsomesortofuptake.Valuecanbeaneconomiconebutitshouldnotbelimitedtoit.Thinkingaboutinno-vationshouldnotbereducedtoitstechnicalsubstance,whichisoftenassociatedwiththenotionofinnovationperse,oftenincombinationwitheconomic value creation, which belittles the contribution of SSH (Bell,2019 in theseproceedings). In fact, thesedayswearemoreandmoreusedtothinkaboutdifferenttypesofinnovationsuchasbusiness-modelinnovations, organisational innovations, public sector innovations, andsocialinnovations.AndweknowsincethefundamentalworksofSchum-peter,that(some)innovationshavethepotentialtotransformthewayweliveandthethingswedo,sociallyandculturallyaswellaseconomically.

After some naive R&I policy ’gold rush years’, characterised by asimplistic understanding of the relationship between research and in-novationaslinearprocessinwhichresearchisexpectedtoleadtoeverhigherTechnologyReadinessLevels(Bell,2019),thereisalsoincreasingawarenessthattheideaofpublicsupportfor‘researchandinnovation’shouldbetosupporttheright innovationsandnot innovationsperse.What“right”meansdependsonaplethoraofviews,principlesandbe-liefsandshouldneverbedecidedinisolation.Theimportantthinghereis tounderstand, that innovation isnotonly thebusinessofbusiness,butalsothebusinessofsociety.Andasabusinessofsocietyitalsobe-comesabusinessofSSHresearch.Bell(2019)callsinthisrespectforagenuineandbroadadded-valueofSSHfortransformativeresearch,star-tingwiththe“whatif”question,constructingalternativescenariosandbyconsideringalso thenon-material featuresofhumanexistence.HefurthermoreclaimsthatSSHcanprovidestrongcontributionstomaketransformationshappen.

Theorientationongrandchallenges,energisedbythe‘missionscon-cept’inHorizonEurope(Mazzucato,2018)canberegardedasanindica-

sourcesandidlecapacities.Inthislineofargument,thefocusremainson thecooperationofSSHwith technology-orienteddisciplines ratherthanonstrengtheninggenuineSSHtopicsinHorizonEurope,theEU’snextEuropeanFrameworkProgrammeforRTD(2021-2027).ThenarrativeaboutthepotentialleveragingfunctionofSSHwasalreadypromotedinHorizon2020inlinewiththe‘integration/mainstreaming’approach.Infact,andthisshouldbeappreciated,Horizon2020wasapioneerinthisrespect,whilemostnationaltechnology-orientedprogrammesstill lackaclearcommitmenttoincludeSSHresearchstrategically,althoughonecanfrequentlyidentifySSHrelatedmethods,RRIaspects,andclaimsofsocialchallengesaswellasimpactssubcutaneouslyinindustry-orientedappliedR&DprogrammestooasevidencedbyastudyaboutSSHaspectsinprojects fundedby theAustrianResearchPromotionAgency (SturnandSchuch,2018).

ThepopularnarrativeoftheauxiliaryfunctionorcontributionofSSHtotechnology-basedinnovationprocessesisoftenframedinthecontextofinter-andtrans-disciplinarychallenges(seeGraf,2019inthesepro-ceedings).Especiallytrans-disciplinarity,whichfeaturesoutreachtoandinclusion of non-academic stakeholders and of non-formalized know-ledge,isacompetencewhichissometimescredulouslyassignedtoSSHresearchersbecauseoftheirperceivedproximitytosocialspheres.Thiscross-academicapproachisstronglyfeaturedinthepillar“GlobalChal-lengesandIndustrialCompetitiveness”ofHorizonEurope,whichshould‘encourage cross-disciplinary, cross-sectoral, cross-policy and cross-border collaboration in pursuit of the UN SDGs and the competitiveness of the Union's industries therein.’(EuropeanCommission,2018;p.17).

ThisunderstandingoftheleveragefunctionofSSHrequiresthatSSHresearchersarecapableandprofessionalinmeetingandapplyingstateoftheartinvolvementtools.ThecontributionofSSHtomoretechnolo-gical oriented projects and its peculiar value is basically perceived asaprojectsteeringandoutreachcompetence,especiallyifissuesofthenormal course of life and/or the inclusion of non-academic audiences(e.g.stakeholders,users)areconcerned.ThisunderstandingbecameapartiallysharedrealityinmanyHorizon2020projects.Incertainresearchfields(suchas“PublicHealthandSustainableDevelopment”)theuseoftransdisciplinarytoolsisdailybusiness.Oftensocialscientistsarechar-gedwithengagementprocessesbyapplyingavarietyofprocesstoolssuchasdesign-thinking,participatorytechnologydevelopmentormulti-stakeholderworkshops.

3. FROM INSTRUMENTAL TO COMPREHENSIVE SSH CONTRIBUTIONS IN MISSION-ORIENTED RESEARCH

Clearly,manySSHresearchersregardthisoverallapproachtotreatSSH research as an auxiliary (or ‘instrumental’) resource for technolo-gicalprojectswhichaddress thegrandchallenges,oftenasan impro-per reductionofSSH. This view isnot far-fetched,because thegrandchallengesaregrandsincetheyconcernhumansocietiesandcultures,thewayshowwehumans interactwitheachotherbut alsowithourenvironment,howweproduceandconsume,howweconstructmeaningandjudgementtoouractions,andhowwereproduceoursocietiesand

ISSUE 48 | JULY 2019 187

entific impactdimensionaremissing.For instance,SSHresearchdoeshardlyresultinpatentapplications.Sofar,existingassessmentsofthesocialimpactofRTIpoliticalinterventionsareoftenonlycontextualandspecificaswellasqualitativeandanecdotalinnature(VandenBesselaaretal,2018;Raua,GogginsbandFahyb,2018;Barré,2010).

Alsothejudgmentsonthevalue-for-societyvary(Realeetal.,2017).DespitethefactthatSSHscholarshipisoftencommittedtodoresearchforthegoodofsociety,theinterestofresearchersisoftennotorientedtowardsproducingusableresults,butrathertoraiseawarenessandin-fluencesociety tocreatecapabilitiesofself-understanding indifferentcontexts(Realeetal.,2017;Benneworth,2015;Nussbaum,2010).

Socialimpactmeasurement,which–andthisisimportanttonote–concernsallscientificdisciplinesandsciences-remainsanunresolvedissueintechnicaltermstoo.ThecomplexitygoesfarbeyondmonetarisedapproachessuchasSROI(SocialReturnonInvestments)orSMEV(‘So-ciallyModifiedEconomicValuation'approach).Tracing,assessmentandmeasuring(centredonthecorequestion ‘What is the evidence for ob-served effects?’)isnotonlycomplexbutalsoexacerbatedbymetricpro-blems(whichapparentlymaterialiseatthelevelofindicatorsanddatabasis).Social impactassessments thusfocusoftenmoreonprocessesthanonresults(e.g.engagementwithbusiness,government,thethirdsector,andthepublicviathemediaasaproxyforsocialimpact)(Bastow,Dunleavy,Tinkler;2014).Moreover,thereisoftenafalsifiedequationofsocial impactwithdisseminationor transfer, towhichmostof theso-called alternative metrics (altmetrics) focus. In Horizon Europe, mostlyprocessandoutputindicatorswillbeappliedtotracethesocietalimpactandpolicyimpact(VandenBesselaaretal.,2018).

Particularchallenges for thedevelopmentofappropriate indicatorstomeasuresocietalimpactincludefirstly,thatthetimetakentoachievetheactualimpactonsocietyislongerthantheachievementofconcreteresults;secondly,thattheassignmentofsocialchangesismoredifficultthantheassignmentofscientificreferencesoreconomicattributes.And,thirdly,thattheavailabilityandcomparabilityofdatatotracksocialandpoliticalimpactsisseverelylimited.Sofar,however,thetracing,assess-mentandmetricquestionofsocialandpoliticalimpactseemstobemorediscussed and forwarded by the domains of research policy and eva-luationresearchwhilemostSSHresearchersfromacademiahaveonlypartiallyadopteditastheirown.

5. CLOSING WORDSThefivemissionsannouncedbytheEuropeanCommission insum-

mer 2019 do not indicate a big change. They all relate to importantchallenges,forwhichSSHcanmakecontributions,butnotinthedriverseat.Ontheotherhand,however,SSHresearchwillalsohavetokeepitspromiseinthousandsofwaystofindanewlevelofinteractionwithsociety.This refers to theclaimmadebyKönig,NowotnyandSchuch,2019intheseproceedings)asstartingpointfororganisingtheAustrianCounciloftheEUPresidencyconferenceon‘ImpactofSocialSciencesandHumanitiesforaEuropeanResearchAgenda–ValuationofSSHinmission-orientedresearch’,that‘transformativesciencemustbetrans-formative inadoublesense:wanting toexert influence insocietybutalsoopentobeinfluencedbysocietyanditsneeds.’

Suchanexerciseisnoteasyandwilldemandalotofeffortsandcrea-tivity.Thereareforinstancestillseveralareas,wheretherelationofSSHtosocietyislessexpressedthanonewouldassume.Suchshortcomings

tionoftheupdatedemphasisonthedirectionalityofR&Ipoliciestotacklethe‘right’issues(e.g.connectedwithsustainabilityand/orinclusivenessconcerns)withtheintentionofselectingtheadequately‘right’R&Ipro-jects in service of society. The Lamy-Report (2017) stipulated the needtodevelop adequate impact orientedRTIpolicydesignsand made theclaimthatSSHshouldalsoactasdriver forsomemissionsof thenextFrameworkProgrammeforR&I(andascontributortoothers).Inlinewiththis,theEuropeanParliament(2017)arguedforabroaderandclearerde-finitionofimpactbyraisingawarenessonsocietal,culturalandlong-termimpacts,whiletheEstonianEUCouncilPresidency(2017)urgedingene-ralforamoresophisticatedanddynamicapproachtoimpactassessment.Thesepoliticalclaims(andothers)weretaken-upandaddressedduringtheAustrianCounciloftheEUPresidencyConferenceonthe‘ImpactofSSHforaEuropeanResearchAgenda–ValuationofSSHinmission-ori-entedresearch’,whichwasorganisedbytheauthorofthisarticleinVi-ennaendofNovember2018(König,NowotnyandSchuch,2019intheseproceedings;Reiter-Pazmandy,2019intheseproceedings).

Duringtheconferenceitbecameclearthatthereiswidespreadac-cordamong theSSHcommunities that the impactofSSH research ismore direct on society than from other research disciplines, althoughnotnecessarilymoreevidentortangible.Thisseemstobeabasiccont-radiction,whichshouldbesolvedtoovercomedisaccordandresistance.The impactofSSH research ismoredirectbecause thesocial subsys-tems ‘Culture’, ‘State’,and ‘Market’areveryoften inthefocusofSSHresearch.SSHresearchersthussometimesclaimthatthey,bypurpose,areclosertoissuessuchassocietalimpacts,structuringimpactsonpo-licy-makingandpolicies(i.e.politicalimpact)aswellasimpactsoninno-vationandeconomy(seeamongothersFlecha,Soler-Gallart,andSordé,2015;Brewer,2013).Forthesakeoforientation,Reale,E.etal. (2017)provideadefinitionofscientific,socialandpoliticalimpact,statingthat “SSH research generates scientific impact when it influences the produc-tion of further research outputs following new approaches for analysis or based on new results. Changes related to social impact affect the cultural, economic, and social life of individuals, organizations, and institutions. Po-litical impact incorporates the contents of research into political decisions, and motivations and rationales for political action and priority setting.”

4. TRACING AND MEASURING IMPACT OF SSH RESEARCH

Impacttracing,however,isacomplexexercise,becauseitiscontext-specificand therearedifferentunderstandingsofvaluationnarrativesandtheoriesofchangeofSSHresearch,whichbythemselvesrequestthorough understanding of processes and methods in the phases ofknowledgeproduction(e.g.co-designandco-creation;inter-andtrans-disciplinaryapproaches;citizenscience;),knowledgedissemination(e.g.tailor-madetransfermechanismsandformats;mediaengagement)andknowledgeusage (e.g.social innovation;policyadvice;evaluationandaccompanying scientific research; research integrated road mapping;[public] service engineering etc.) with all their particular challengingaspects.BeckandBonß (1989)evenclaimedthat interpretationoffersprovided by social sciences are practically most successful, when theseeminglyvanishwithouttraceintheconsciousnessofeverydaylifeandpolicy. What makes impact measurement of SSH research even morecomplexisthefactthatstandardisedindicatorsofusagebeyondthesci-

ISSUE 48 | JULY 2019188

European Commission(2018):ProposalforaRegulationoftheEuropeanParliamentandoftheCouncilEstablishingHorizonEurope(COM(2018)435final,2018/0224(COD)),17(URL:https://ec.europa.eu/commission/sites/beta-political/files/budget-may2018-horizon-europe-regulation_en.pdf,accessedon31.10.2018.

European Parliament(2017):ReportontheassessmentofHorizon2020implementationinviewofitsinterimevaluationandtheFrameworkPro-gramme9proposal(2016/2147(INI)).

European Parliament and Council (2013).Regulation(EU)No1291/2013of11December2013establishingHorizon2020,AnnexI,121(URL:htt-ps://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:347:0104:0173:EN:PDF,accessedon31.10.2018.

Flecha, R., Soler-Gallart, M. and Sordé, T. (2015):SocialImpact:Euro-peMustFundSocialScience.Nature,528:193.Doi:10.1038/528193d.

Graf, J.(2019):BringingConceptsTogether:Interdisciplinarity,Transdis-ciplinarityandSSHIntegration.FtevalJournalforResearchandTechno-logyPolicyEvaluation, Issue48,June2019,pp.32-35.DOI:10.22163/fteval.2019.364

Gustavsen, B. (2012).SocialInnovationandActionResearch.InH.-W.Franz,J.HochgernerandJ.Howaldt(Ed.),Challengesocialinnovation:Potentialsforbusiness,socialentrepreneurship,welfareandcivilsociety(pp.353-366).Berlin,NewYork:Springer.

Howaldt, J. (2019): New Pathways to Social Change – Creating Im-pact through Social Innovation Research. Fteval Journal for ResearchandTechnologyPolicyEvaluation,Issue48,June2019,pp.36-47.DOI:10.22163/fteval.2019.365

Howaldt, J., Kaletka, C., Schröder, A., Rehfeld, D. and Terstriep, J.(2016).MappingtheWorldofSocialInnovation.KeyResultsofaCompa-rativeAnalysisof1.005SocialInnovationInitiativesataGlance.SI-DriveProject.https://www.si-drive.eu/?p=2283;accessedon30June2019.

König, T. (2019):SSH-ImapctPathwaysandSSH-Integration inEURe-search Framework Programmes. Impact re-loaded. Fteval Journal forResearchandTechnologyPolicyEvaluation,Issue48,June2019,pp.7-8.DOI:10.22163/fteval.2019.362

König, T., Nowotny, H. and Schuch, K.(2019):Impactre-loaded.FtevalJournalforResearchandTechnologyPolicyEvaluation,Issue48,June2019,pp.7-8.DOI:10.22163/fteval.2019.361

Lamy, P. et al.(2017):Lab,Fab,App.InvestingintheEuropeanfuturewewant.ReportoftheindependentHighLevelGrouponmaximisingtheIm-pactofEUResearchandInnovationProgrammes.EuropeanCommission(thesocalled‘Lamy-Report’).

Marimon, R., Guardiancich, I., Mariathasan, M. and Rossi, E.(2011):SurveyonResearchFundingfortheSocialSciencesinEurope.EuropeanUniversityInstitute.

canforinstancebefoundwithrespecttocitizenscienceorsocialinno-vation (Howaldt, 2019 in these proceedings; Anderson, Domanski andHowaldt,2018;Howaldtetal.2016),althoughwefindalongtraditionofactionresearchwhichstimulatedsocialaction(Gustavsen,2012),which,however,shouldnotbeequalizedwithsocialinnovation.

Theseshortcomings,however,alsohavestructuralreasons,suchasthecomparativelylowfundingfondnesstowardsSSH-drivencitizensci-enceprojects.Alsointheareaofsocialinnovation,onecanhardlyfindmaterialorimmaterialprofessionalstructureswithinmosthighereduca-tionandnon-universityresearchorganisationsforsupportingsocialinno-vation.Exampleslikethe“6Iresearchmodel”attheUniversityofDeusto(Caro-Gonzalez,2019 in theseproceedings)or theKnowledgeTransferCentreforSSHinAustria(Russegger,2019intheseproceedings)arestilltheexceptionandnottherule.

6. LITERATUREAnderson, M. M., Domanski D. and Howaldt, J. (2018):SocialInnova-tionasachanceandachallengeforHigherEducationInstitutions.InJ.Howaldt,C.Kaletka,A.SchröderandM.Zirngiebl(Ed.),AtlasofSocialInnovation.NewPracticesforaBetterFuture(pp.50-53).Dortmund:So-zialforschungsstelle,TUDortmundUniversity.

Bastow, S., Dunleavy, P. and Tinkler, J.(2014):TheImpactoftheSoci-alSciences.LosAngeles-London-NewDehli-Singapore-WashingtonDC:SAGE.

Beck, U. / Bonß, W. (Hrsg),1989:WederSozialtechnologienochAuf-klärung? Analysen zur Verwendung sozialwissenschaftlichen Wissens.Frankfurt:Suhrkamp.

Bell, J. S. (2019):The importanceofSSHresearch inHorizonEurope.FtevalJournalforResearchandTechnologyPolicyEvaluation,Issue48,June2019,pp.48-51.DOI:10.22163/fteval.2019.366

Benneworth, P. (2015): Tracing How Arts and Humanities ResearchTranslates, Circulates and Consolidates in Society. How Have scholarsBeenReacting toDiverse ImpactandPublicValueAgendas?ArtsandHumanitiesinHigherEducation,14/1:pp.45-60.

Brewer, J.(2013):ThePublicValueofSocialSciences.AnInterpretativeEssay.Bloomsbury.

Caro-Gonzalez, A.(2019):The“6IResearchModel”:EvolutionofanIn-novativeInstitutionalSTIPolicyFrameworkattheUniversityofDeusto.FtevalJournalforResearchandTechnologyPolicyEvaluation,Issue48,June2019,pp.104-112.DOI:10.22163/fteval.2019.376.

Estonian Presidency of the Council of the EU (2017):TallinnCall forAction2017.Seizetheopportunitynow:researchandinnovationmatterfor the future of Europe. Statement of theEstonianPresidency of theCounciloftheEU.

EUA(2011):ImpactoftheeconomiccrisisonEuropeanuniversities.Eu-ropeanUniversityAssociation,January2011.

ISSUE 48 | JULY 2019 189

AUTHOR

KLAUS SCHUCH Zentrum für Soziale Innovation GmbH - Centre for Social Innovation (ZSI) LinkeWienzeile246,Vienna,1150(Austria)E:[email protected]

Mazzucato, M.(2018).Mission-OrientedResearchandInnovationintheEuropeanUnion.RetrievedOctober18,2018from:https://ec.europa.eu/info/sites/info/files/mazzucato_report_2018.pdf.

Morozov, E.(2013):ToSaveEverything,ClickHere:Technology,Solutio-nism,andtheUrgetoFixProblemsthatDon’tExist,London.

Nussbaum, M. (2010):NotforProfit.WhyDemocracyNeedstheHuma-nities.Princeton:PrincetonUniversityPress.

Papanagnou, G.(ed)(2011):SocialScienceandPolicyChallenges.De-mocracy,ValuesandCapacities.UNESCO.

Rammel, S., Hoffmann, A. and Halbmayr, B. (2017): ERA ThematicDossieronSocialSciencesandHumanities(SSH)inHorizon2020.FFG:AustrianResearchPromotionAgency.

Raua, H., Gogginsb, G. and Fahyb, F(2018):Forminvisibilitytoimpact:Recognising the scientific and societal relevance of interdisciplinarysustainabilityresearch.ResearchPolicy,47,pp266-276.DOI:10.1016/j.respol.2017.11.005.

Reale, E. et al.(2017):Areviewofliteratureonevaluatingthescientific,socialandpolitical impactofsocialsciencesandhumanitiesresearch.ResearchEvaluation2017,1-11,doi:10.1093/reseval/rvx025.

Reiter-Pázmándy, M. (2019): Foreword. Fteval Journal for Researchand Technology Policy Evaluation, Issue 48, June 2019, pp. 5-6. DOI:10.22163/fteval.2019.360.

Russegger, G. (2019):SSHA-DrivenKnowledgeTransferWithintheThirdMissionofUniversities.Foreword.FtevalJournalforResearchandTech-nologyPolicyEvaluation,Issue48,June2019,pp.61-64.DOI:10.22163/fteval.2019.369

Schögler, R. and König, T. (2017):ThematicResearchFunding in theEuropean Union: What is Expected from Social-Scientific Knowledge-making?Serendipities2.2017(1):107–130|DOI:10.25364/11.2:2017.1.7

Sturn, D. and Schuch, K. (2018): Geistes-, sozial- und kulturwissen-schaftlicheAspekteinFFGgefördertenInnovationen.ReportfortheAus-trianResearchPromotionAgency(notpublished).

Van den Besselaar, P., Flecha, R. and Radauer, A.(2018):Monitoringthe ImpactofEUFrameworkProgrammes.ExpertReport.PublicationsOfficeoftheEuropeanUnion:Luxembourg.

ISSUE 48 | JULY 2019190

SCIENTIFIC COMMITTEE OF THE CONFERENCE: IMPACT OF SOCIAL SCIENCES AND HUMANITIES FOR A EUROPEAN RESEARCH AGENDA – VALUATION OF SSH IN MISSION-ORIENTED RESEARCH

HELGA NOWOTNY Chair of the ERA Council Forum Austria and Member of the Austrian Council for Research and Technology Development, Former President of the European Research Council (Chair of the Scientific Committee)

PAUL BENNEWORTH Senior researcher at the Center for Higher Education Policy Studies at the University of Twente, Netherlands

OLIVIER BOUIN Director of the Foundation-Excellence Laboratory “Network of French Institutes for Advanced Study”

ULRIKE FELT University of Vienna, Professor of Science and Technology Studies (Vice chair of the Scientific Committee)

YVES GINGRAS Scientific Director of the Observatory of Science and Technology (OST), Montréal, Canada

POUL HOLM Professor of Environmental History, Trinity College Dublin, Ireland

JÜRGEN HOWALDTDirector of the Social Research Centre at TU Dortmund University and Professor at the Faculty of Economics, Dortmund, Germany

CARINA KESKITALOMember of High Level Group on Scientific Advisors of the EC, Professor at Umeå University, Umeå, Sweden

KIRSTEN M. LANGKILDEDirector of the University of Applied Sciences and Arts Northwestern Switzerland, Academy of Art and Design, Basel, Switzerland

STEFANIA MILANAssociate Professor of New Media and Digital Culture at the University of Amsterdam and of Media Innovation at the University of Oslo, Amsterdam/Oslo, Netherlands/Norway

ANDREA PETŐProfessor in the Department of Gender Studies at Central European University, Budapest, Hungary

CLAUDIO M. RADAELLIProfessor of Public Policy, University College London, Department of Political Science, United Kingdom

EMANUELA REALESenior Researcher at IRCRES (Research Institute on Sustainable Economic Growth) – CNR (National Research Council), Roma, Italy

SARAH DE RIJCKEProfessor of Science and Evaluation Studies and deputy director at CWTS, Leiden, Netherlands

EVELYN RUPPERTProfessor at the Goldsmiths University of London, London, United Kingdom

KLAUS SCHUCHScientific Director of the Centre for Social Innovation, Vienna, Austria

MARTA SOLERProfessor at the University of Barcelona (Spain) and Director of CREA, Community of Research on Excellence for All

JACK SPAAPENEmeritus senior policy advisor at the Royal Netherlands Academy of Arts and Science, Amsterdam, Netherlands

TEREZA STÖCKELOVAResearcher and editor in chief at the Institute of Sociology of the Czech Academy of Sciences, Prague, Czech Republic

JOHANNES VOGELGeneral Director of the Museum for Natural History Berlin and Chair of the European Citizen Science Association, Berlin, Germany

MILENA ZIC-FUCHSProfessor at the University of Zagreb, Croatia, Fellow of the Croatian Academy of Sciences and Arts, Member of Academia Europaea

ISSUE 48 | JULY 2019 191

POSTER 1/PAGE190Cultivating Europe: Democratic Europe – Social Europe – Educated EuropeAuthors: KatharinaBerghöfer,NicoleBirkle,GenyPiotti,StefaniePreuss,UrsulaSchlichter,ChristianVeldman,KristinaWiege.DOI:10.22163/fteval.2019.388

POSTER 2/PAGE192Structural Interdisciplinary Cooperation On Crime And Security ResearchAuthors:NoëlKlimaDOI:10.22163/fteval.2019.389

POSTER 3/PAGE194Inclusive Employment Of Deaf PeopleAuthors: MonikaHaiderDOI:10.22163/fteval.2019.390

POSTER 4/PAGE196Collaborative engagement on societal issuesAuthor: AlessiaSmaniottoDOI:10.22163/fteval.2019.391

POSTER 5/PAGE198Generating Impact in Transnational Humanities Research: HERA 2009-2018Authors:JoannaSofaer,WojciechSowa,TonyWhytonDOI:10.22163/fteval.2019.392

POSTER 6/PAGE200Generating Impact in Transnational Humanities Research: HERA 2009-2018Authors: JoannaSofaer,WojciechSowa,TonyWhytonDOI:10.22163/fteval.2019.393

POSTER 7/PAGE202CLARIN - Transcending Scientific Boundaries across SSH Research focusing on Parliamentary DataAuthors: FranciskadeJong,DarjaFišer,MariaEskevich,KarolinaBadzmierowskaDOI:10.22163/fteval.2019.394

POSTER 8/PAGE204Rethinking Research Impact Assessment: A multdimensional ApproachAuthors: SergioManriqe,MartaNataliaWróblewska,BradleyGoodDOI:10.22163/fteval.2019.395

POSTER 9/PAGE206SIOR Contributing to the scientific research transparency by evaluating the social impact of sciencesAuthors: JoanCabré,VladiaIonescu,GiselaRedondo,TeresaSordéDOI:10.22163/fteval.2019.396

POSTER 10/PAGE208Are Humanities failing to generate Impact?Author: DorotheaSturnDOI:10.22163/fteval.2019.397

POSTER 11/PAGE210Ethics & economics regarding access to healthcare for marginalised groupsAuthors: UrsulaTrummer,SonjaNovak-Zezula,NadavDavidovitch,NoraGottliebDOI:10.22163/fteval.2019.398

POSTERS

ISSUE 48 | JULY 2019192

ISSUE 48 | JULY 2019 193

ISSUE 48 | JULY 2019194

ISSUE 48 | JULY 2019 195

ISSUE 48 | JULY 2019196

ISSUE 48 | JULY 2019 197

ISSUE 48 | JULY 2019198

ISSUE 48 | JULY 2019 199

ISSUE 48 | JULY 2019200

ISSUE 48 | JULY 2019 201

ISSUE 48 | JULY 2019202

ISSUE 48 | JULY 2019 203

ISSUE 48 | JULY 2019204

ISSUE 48 | JULY 2019 205

ISSUE 48 | JULY 2019206

ISSUE 48 | JULY 2019 207

ISSUE 48 | JULY 2019208

ISSUE 48 | JULY 2019 209

ISSUE 48 | JULY 2019210

ISSUE 48 | JULY 2019 211

ISSUE 48 | JULY 2019212

NOWAVAILABLE!LATEST EDITION OF THE

'EVALUATION STANDARDS FOR RESEARCH, TECHNOLOGY AND INNOVATION POLICY'OFTHEAUSTRIANPLATFORMFORRESEARCHANDTECHNOLOGYPOLICYEVALUATION

Download in German: fteval.at/content/home/standards/fteval_standards/

You want print [email protected]

Download in English: fteval.at/content/home/standards/fteval_standards/index.jsp?langId=2

FTEVAL JOURNAL FOR RESEARCH AND TECHNOLOGY POLICY EVALUATION

… RTI POLICY EVALUATION AT THE INTERFACE BETWEEN PRACTICE AND ACADEMIA …

The ‘fteval Journal for Research and Technology Policy Evaluation’ contributes in a quality assured way to exchange between various stakeholder groups in the area of science, research, technology and innovation policy evaluation. It addresses policy-makers, practitioners, evaluators and the academic community. Thematic editions alternate with thematically open issues. By now 48 editions were published, which can be accessed and downloaded from the fteval-repository and the fteval homepage. The fteval Journal is open access. The journal and each published paper receive a separate DOI. All paper contributions are quality controlled, but not subject to peer-review in the strict academic sense.Researchers and practitioners from the RTI policy domain are invited to send contributions to the editor of the ‘fteval Journal for Research and Technology Policy Evaluation’ ([email protected]). The contributions can be submitted either in German or English.

If you want to receive the ‘fteval Journal for Research and Technology Policy Evaluation’ free of charge, please fill out this form and send it to [email protected] or drop it into the infobox at one of our conferences and events.

first name

last name

institution

address

postal code

city/place

country

e-mail

By filling in this information you agree to* receive the fteval Journal by mail (free of charge)* to receive the electronic newsletter of fteval, which informs about pertinent issues of RTI evaluation in Austria and the EU via email (such as calls for contributions for the next fteval Journal) and thus you entitle the fteval to process your indicated personal data for the purpose of transmission of information to you. You can revoke consent at any time by informing us via eMail ([email protected]).

EDITORIAL BOARDRebecca Allinson, Technopolis UK; Balázs Borsi, Eszterházy Károly College; Elke Dall, Centre for Social Innovation; Michael Dinges, Austrian Institute of Technology; Leonid Gokhberg, National Research University Higher School of Economics; Wolgang Polt, Joanneum Research; Andreas Reinstaller, WIFO; Klaus Schuch, fteval (chief editor); Michael Stampfer, WWTF; Lena Tsipouri, University of Athens

AUSTRIAN PLATFORM FOR RESEARCH AND TECHNOLOGY POLICY EVALUATION (fteval)c/o ZSI – Centre for Social Innovation GmbHLinke Wienzeile 246, A-1150 ViennaT +43 1 495 04 42 - 79F +43 1 495 04 42 - 40E [email protected] www.fteval.at

DESIGNW carotte.atE [email protected]

PRINTagensketterl Druckerei, Bad Vöslau

Gedruckt auf PEFCTM-zertifiziertem Papier.

AUSTRIAN PLATFORM FOR RESEARCH AND TECHNOLOGY POLICY EVALUATIONPlatform fteval members:

Austrian Federal Ministry of Education, Science and Research (bmbwf), Austrian Federal Ministry for Digital and Economic Affairs (bmdw), Austrian Federal Ministry for Transport, Innovation and Technology (bmvit), Austrian Cooperative Research (ACR), Austrian Council for Research and Technology Development, Austrian Institute of Technology (AIT), AQ Austria – Agency for Quality Assurance and Accreditation Austria, Austria Wirtschaftsservice (AWS), Christian Doppler Research Association (CDG), convelop Cooperative Knowledge Design GmbH, Austrian Research Promotion Agency (FFG), Austrian Science Fund (FWF), Institute for Advanced Studies (IHS), Industriewissenschaftliches Institut (IWI), Joanneum Research Forschungsgesellschaft mbH, Austrian Institute for SME Research (KMFA), Ludwig Boltzmann Society (LBG), Austrian Academy of Science (OEAW), Technopolis Group Austria, Vienna Business Agency – A service offered by the City of Vienna, Austrian Institute of Economic Research (WIFO), WPZ Research GmbH, Vienna Science and Technology Fund (WWTF), Centre for Social Innovation (ZSI)

WIFO – AUSTRIAN INSTITUTE OF ECONOMIC RESEARCHArsenal, Objekt 20Postfach 91, 1103 ViennaDr. Jürgen JangerE: [email protected]. Andreas ReinstallerE: [email protected]

OEAW – AUSTRIAN ACADEMY OF SCIENCEDr. Ignaz Seipel-Platz 2, 1010 ViennaNikolaus Göth, MScE: [email protected]

AUSTRIAN COUNCIL FOR RESEARCH AND TECHNOLOGY DEVELOPMENTPestalozzigasse 4/DG 1, 1010 Vienna Dr. Johannes GadnerE: [email protected]

AQ AUSTRIA – AGENCY FOR QUALITY ASSURANCE AND ACCREDITATION AUSTRIAFranz-Klein-Gasse 5, 1190 ViennaDr.in Elisabeth Froschauer-Neuhauser E: [email protected] Eva Maria FreibergerE: [email protected]

AIT – AUSTRIAN INSTITUTE OF TECHNOLOGYGiefinggasse 4, 1210 Vienna Mag. Michael DingesE: [email protected] Barbara Heller-Schuh, MASE: [email protected]

AWS – AUSTRIA WIRTSCHAFTSSERVICE GMBHWalcherstraße 11A, 1020 Vienna Mag.a Marlis BaurechtE: [email protected] Mag. Norbert KnollE: [email protected]

CDG – CHRISTIAN DOPPLER RESEARCH ASSOCIATIONBoltzmanngasse 20, 1090 ViennaDIin Maga. Brigitte MüllerE: [email protected]

WWTF – VIENNA SCIENCE AND TECHNOLOGY FUNDSchlickgasse 3/12, 1090 ViennaDr. Michael StampferE: [email protected]. Michael StrassnigE: [email protected]

VIENNA BUSINESS AGENCY. A SERVICE OFFERED BY THE CITY OF VIENNA.Mariahilfer Straße 20, 1070 ViennaRobert Mayer-UnterholzerE: [email protected]

ZSI – CENTRE FOR SOCIAL INNOVATIONLinke Wienzeile 246, 1150 Vienna Dr. Klaus SchuchE: [email protected]

LUDWIG BOLTZMANN GESELLSCHAFTNußdorfer Straße 64, 1090 ViennaMag. Patrick LehnerE: [email protected]

TECHNOLPOLIS GROUP AUSTRIARudolfsplatz 12/11, 1010 ViennaMag.a Katharina WartaE: [email protected]

WPZ RESEARCH GMBHMariahilfer Straße 115/16, 1060 ViennaDr.in Brigitte EckerE: [email protected]

JOANNEUM RESEARCH FORSCHUNGSGESELLSCHAFT MBHHaus der ForschungSensengasse 1, 1090 ViennaMag. Wolfgang PoltE: [email protected]. Jürgen StreicherE: [email protected]

KMU FORSCHUNG AUSTRIA –AUSTRIAN INSTITUTE FOR SME RESEARCH Gusshausstraße 8, 1040 Vienna Mag.a Iris FischlE: [email protected]. Peter KaufmannE: [email protected]

FEDERAL MINISTRY OF EDUCATION, SCIENCE AND RESEARCHMinoritenplatz 5, 1014 Vienna Mag.a Irene DanlerE: [email protected] Simone MesnerE: [email protected]

FEDERAL MINISTRY FOR DIGITAL AND ECONOMIC AFFAIRSStubenring 1, 1014 Vienna Mag.a Sabine Pohoryles-DrexelE: [email protected]

FEDERAL MINISTRY FOR TRANSPORT, INNOVATION AND TECHNOLOGYRadetzkystraße 2, 1030 Vienna Dr. Rupert PichlerE: [email protected]. Mario SteyerE: [email protected]

FFG – AUSTRIAN RESEARCH PROMOTION AGENCYHaus der Forschung,Sensengasse 1, 1090 ViennaDIin Dr.in Sabine MayerE: [email protected]. Leonhard JörgE: [email protected]

FWF – AUSTRIAN SCIENCE FUNDHaus der ForschungSensengasse 1, 1090 ViennaDr. Falk Reckling E: [email protected] Dr. Thomas VölkerE: [email protected]

IHS – INSTITUTE FOR ADVANCED STUDIESJosefstädter Straße 39, 1080 Vienna Dr.in Angela WroblewskiE: [email protected]. Richard Sellner E: [email protected]

CONVELOP – COOPERATIVE KNOWLEDGE DESIGN GMBHBürgergasse 8-10/I, 8010 Graz DIin Drin. Karin GrasenickE: [email protected]

Erdbergstraße 82/4, 1030 WienMag. Thomas JudE: [email protected]

ACR – AUSTRIAN COOPERATIVE RESEARCHSensengasse 1, 1010 Vienna Dr.in Sonja Sheikh E: [email protected]

INDUSTRIEWISSENSCHAFTLICHES INSTITUT – IWIMittersteig 10, 1050 WienFH-Hon.Prof. Dr. Dr. Herwig W. SchneiderE: [email protected]. Philipp BrunnerE: [email protected]

PROCEEDINGS OF THE CONFERENCE “Impact of Social Sciences

and Humanities for a European Research Agenda

Valuation of SSH in mission-oriented research”VIENNA 2018

to impact from SSH research

c/o ZSI – Centre for Social Innovation GmbHLinke Wienzeile 246, A-1150 Vienna

T +43 1 495 04 42 - 79F +43 1 495 04 42 - 40E [email protected] www.fteval.at

ZVR-Zahl: 937261837ISSN-Nr. 1726-6629© Vienna 2019

The fteval Journal for Research and Technology Policy Evaluation is a forum to discuss current evaluation practices in the field of research, technology and innovation policy.

to impact from ssh research - fteval.at · 23345678coccnte c5prf 3 content 5 preface heinz...

Documents