dr kim sweeny dr masha fridman professor bruce rasmussen · nectar virtual laboratories report to...

of 56/56
Estimating the value and impact of Nectar Virtual Laboratories Report to Nectar Dr Kim Sweeny Dr Masha Fridman Professor Bruce Rasmussen Victoria Institute of Strategic Economic Studies Victoria University September 2017

Post on 28-Oct-2019

2 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Estimatingthevalueandimpactof

    NectarVirtualLaboratories

    ReporttoNectar

    DrKimSweeny

    DrMashaFridman

    ProfessorBruceRasmussen

    VictoriaInstituteofStrategicEconomicStudies

    VictoriaUniversity

    September2017

  • 1

    Contents

    1.Introduction 2

    2.TheNectarproject 3

    3.Methodology 7

    4.TheVirtualLaboratories 13

    4.1BiodiversityandClimateChangeVirtualLaboratory 13

    4.2.CharacterisationVirtualLaboratory 15

    4.3GenomicsVirtualLaboratory 16

    4.4.HumanitiesNetworkedInfrastructure 18

    5.DataonusageofNectarVirtualLaboratories 23

    6.EstimatingthevalueandimpactofNectarVirtualLaboratories 26

    7.FurtherassessmentofthevalueandimpactoftheCharacterisationVirtualLaboratory 35

    8.QualitativevaluationofNectarVirtualLaboratories 39

    9.Summaryandconclusions 47

    References 49

    Appendices 53

    1. VirtualLaboratoriesfundedbyNectar 532. CVLimagingfacilitypartners 54

  • 2

    1.Introduction

    In2016theVictoriaInstituteofStrategicEconomicStudies(VISES)atVictoriaUniversitywascommissionedbytheNationaleResearchCollaborationToolsandResourcesproject(Nectar)toestimatethevalueandimpactofNectarVirtualLaboratories(VLs).

    NectarprovidesanonlineinfrastructurethatsupportsresearcherstoconnectwithcolleaguesinAustraliaandaroundtheworld,allowingthemtocollaborateandshareideasandresearchoutcomes,whichwillultimatelycontributetoourcollectiveknowledgeandmakeasignificantimpactonoursociety.

    Itprovidessupportforanetworkof14VLswhicharedomain-orientedonlineenvironmentsthatdrawtogetherresearchdata,models,analysistoolsandworkflowstosupportcollaborativeresearchacrossinstitutionalanddisciplineboundaries.

    GiventheresourcesavailableandthecentralroleofVLsintheNectarproject,itwasagreedthatthemostappropriateapproachtoestimatingthevalueandimpactofNectarwastoconcentrateonestimatingthevalueandimpactof4VLs,namely

    • BiodiversityandClimateChangeVirtualLaboratory(BCCVL)• CharacterisationVirtualLaboratory(CVL)• GenomicsVirtualLaboratory(GVL),and• HumanitiesNetworkedInfrastructure(HuNI)

    MostVLsfundedbyNectarhaveonlybeenactiveforafewyearsandarestillintheirgrowthstages.AnevaluationoftheiroverallimpactandvaluemightbestbedonefromtheperspectiveofsomeyearsinthefuturewhentheVLsareinamorematuregrowthphase.Thereforetheanalysisandconclusionsdrawninthisstudyshouldbetreatedaspreliminaryanddependsignificantlyontheassumptionsmadeaboutfuturegrowthpaths.

    ItbecameclearduringthecourseofthisstudythatwhileVLssharemanyfeaturesincommon,theydiffersignificantlyfromeachotherintermsoftheservicestheyprovidedtotheirtargetcommunities.Furthermoretheydifferinhoweasyitistoexpresstheirvalueandimpactintraditionaleconomicterms.Consequently,wehaveadoptedanumberofquantitativeandqualitativeapproachestorevealingvalueandimpactandhaveusedtheCharacterisationVirtualLaboratoryasacasestudytoanalysethisinmoredepth.OurapproachtoHumanitiesNetworkedInfrastructure(HuNI)hasbeenexploratoryinnaturewiththeaimofidentifyingdifferentapproachesthatmightbemorefullydevelopedinafuturestudy.

  • 3

    2.TheNectarproject

    NectarhasitsoriginsinCommonwealthgovernmentinitiativestosupportnationalresearchinfrastructure.TheNationalCollaborativeResearchInfrastructureStrategy(NCRIS)wasestablishedbytheCommonwealthGovernmentin2004andcurrentlyprovidesfundingfor27activeprojectsacross222institutionsemployingover1700technicalexperts,researchersandfacilitymanagers.NCRISfacilitiesareusedbyover35,000researchers,bothdomesticallyandinternationallyandalsoattractfundingfromorganisationsinthehighereducation,government,non-profitandindustrysectors(DepartmentofEducation2017).

    SinceitsestablishmenttherehavebeenanumberofreviewsandroadmapsoftheNCRISprogram.

    TheStrategicRoadmapforAustralianResearchInfrastructure,releasedinAugust2008,updatedasimilarroadmapin2006butplacedincreasedemphasisoneResearch

    “inrecognitionofthepervasiveandunderpinningrelevanceofICTtoresearch.Ascollaborativeresearchincreases,eResearchisprovidingthemostinfluentialandeffectivewayofenablinginstitutionstoworktogether,usingsharedinfrastructure,resourcesandpolicies.”1

    Inits2009/10BudgettheAustralianGovernmentannouncedaSuperScienceInitiativetofurthersupportresearchinfrastructurebyaddressingtheprioritiesidentifiedinthe2008StrategicRoadmap.TheSuperScienceInitiativewasfinancedthroughtheEducationInvestmentFund.Nectarwasestablishedin2009bytheAustralianGovernmentfollowinga2009/10Budgetannouncementof$47milliontosupportNectaraspartoftheSuperScienceinitiativefinancedbytheEducationInvestmentFund(EIF),andsubsequentlyreceivedNCRISfunding.Nectarhasreceived$61millioningovernmentfunding,matchedbyco-investmentof$54millionfromAustralianuniversitiesandresearchorganisations

    TheUniversityofMelbourneistheleadagentfortheadministrationofNectar.GovernanceofNectarisprovidedbytheNectarProjectBoard,chairedbyRussellYardley,with9othermembersdrawnfromparticipatingresearchinstitutions.

    TheNectardirectoratecomprisestheequivalentof6.5full-timestaffandisledbytheDirector,AssociateProfessorGlennMoloney.

    1Theroadmapfurtherstatesthat“anewcapabilityintheHumanities,ArtsandSocialSciences(HASS)hasbeenidentifiedinrecognitionofthewiderangingcontributionsthesedisciplinesmaketothenationalinterest.InvestmentinthisareawouldrelatetoaHASSeResearchinfrastructureincludingdatacreationanddigitisationofresearchmaterials”.TheHumanitiesNetworkedInfrastructure(HuNI)canbeviewedasaresponsetothis.

  • 4

    NectarisanexampleofasciencegatewaywhichShahand(2015)definesasa“web-basedenterpriseinformationsystemsthatprovidescientistswithcustomizedandeasyaccesstocommunity-specificdatacollections,computationaltoolsandcollaborativeservicesone-Infrastructures.”

    Arecentreviewofsciencegateways(Barkeretal2017)describestheirbenefitsasfollows.

    Sciencegatewaysareakeycomponentofthefuturedigitalresearchenvironment,enablingresearcherstoutilizeaglobalnetworkofinteractingdigitalplatformstoaccessandsharetheleading-edgedataandtoolsthatarecriticalfortheirresearch.Theybothfacilitate,andaresupportedby,broadermovementssuchasopenresearch,openscience,opensourcesoftwareandopendata.Consequently,sciencegatewaysarevaluabletoarangeofstakeholders:individualresearchers,researchcommunities,researchorganizationsandinstitutions(includingindustryandgovernment)andfundingagencies.

    Definingsciencegatewaysintermsofcommoncharacteristicsandfunctionalityassistsinidentifyingtheirvaluetotheirstakeholders.Sciencegatewayslowerbarriersbyhidingthecomplexityoftheunderlyingdigitalresearchinfrastructureandsimplifyingaccesstobest-practicetools,dataandresources,therebydemocratizingtheirusage.Theycanenablecollaborationandbuildcommunities,sharingdataandanalysesamongmultidisciplinaryandgeographicallydispersedresearchgroups,leadingtoincreasedopenness.Theycanenableresearchtobeundertakenmoreefficientlythroughtheprovisionofmodellingandothersoftwareandhardwareresourcesthroughasingleportal,andenableresearchtobeundertakenthatwouldnototherwisebeconducted.Researchersnolongerneedtobephysicallyco-locatedbecauseresourcescanbegloballydistributed,andthisalsoenablesinclusionoflessadvantagedresearchers/institutions.Bysharingresourcesacrossmultipleinstitutions,thecostsofsettingupandsupportingresearchinfrastructureislowered,aseachinstitutionisnolongerrequiredtosupportareplicaofdata,computeandtoolsattheirsite.Forgatewaysthatareopensource,theirverybuildingandevolutioncanbedemocratized.Anycommunitymembercandownloadandusecodeandalsocontributefeatures,forexample,viagitpullrequests.

    Nectarprovidesthemajorityofitsfundingthroughtwomaininvestmentprogramsine-researchinfrastructure:

    • VirtualLaboratories• ResearchCloud

  • 5

    VirtualLaboratories

    NectarVLsarerichdomain-orientedonlineenvironmentsthatdrawtogetherresearchdata,models,analysistoolsandworkflowstosupportcollaborativeresearchacrossinstitutionalanddisciplineboundaries.TheyarebuiltandledbytheAustralianresearchsectorandareusednationallyandinternationallybytheresearchcommunityandotherstakeholders,includingindustry.

    Dataanalysisisincreasinglyanessentialaspectofresearchinmostacademicdisciplines,withlargeamountsofdatarequiringmorestorageandcomputingpowerthanmostdesktopenvironmentscanprovide.VirtualLaboratoriescanprovideHighPerformanceComputing(HPC)resourcestoanalyseandstoresuchdataandprovidetraininginadvancedanalyticalmethodsneededtoanalyselargedatasets.

    NectarVLsarehostedonremoteserversandareaccessibleremotely,viatheinternet.Researchersnolongerneedtobephysicallyco-located:anyresources,includingpeople,canbegloballydistributed.Allthatisneededisaninternetconnectionforaccesstocollaborators,data,computationalandanalyticaltools.

    Remoteaccesstoresearchresourcesincreasestheefficiencyofresearchandexpandsitsimpact.ThecostofHPCinfrastructurecanbeshared–itnolongerneedsnotbereplicatedineveryresearchfacilitythatrequiresit.ThesavingsincludecostlyupgradesthatareneededeveryfewyearstomaintainHPC.TheVLsplatformiswellsuitedfordeliveringtraininginadvancedanalyticalmethodstoupdateanalyticalskillsandtotrainanewgenerationofresearchers.Furthermore,VLsprovideaplatformforcollaboration,enablingresearcherstoshareideasandresearchoutcomeswithcolleaguesinAustraliaandaroundtheworld,acrossinstitutionalanddisciplineboundaries,ultimatelyexpandingtoourcollectiveknowledge.

    OnceNectarwasestablisheditinitiatedtwoRequestsforProposalsforVLsinSeptember2010andApril2012.ContractswiththeinstitutionsthatproposedthesuccessfulVLsweresignedfromMay2012toJanuary2013.

    VirtualLaboratoriesfundedunderNectarrecordedover19,000usersin2017across12VLs,including,onaverage,usersfromover20internationalorganizationsand30Australianorganizations(Barkeretal2017).

    AcompletelistofVLsisgiveninAppendix1.

    Sincetheirinception,accesstotheVLsandotherservicesfundedbyNectarhasbeenfreetousers.

    ThefourVLsconsideredinthisstudyhavereceivedfundingfromNectarasshowninTable2.1.

  • 6

    Table2.1 FundingforfourVLsto2016-17,$

    BCCVL CVL GVL HUNINectarcash to2013-14 1,236,063 1,618,108 2,334,121 1,329,0002014-15 220,000 320,000 445,000 193,0002015-16 230,000 225,000 230,000 192,5002016-17 177,000 177,000 177,000 120,000Total 1,863,063 2,340,108 3,186,121 1,834,500Partners Cash 600,000 In-kind 1,315,971 1,836,740 2,917,707 2,500,000 Total 3,779,034 4,176,848 6,103,828 4,334,500NectarCloud

    NectarCloudprovidescomputinginfrastructure,softwareandservicesthatallowAustralia’sresearchcommunitytostore,access,andrundata,remotely,rapidlyandautonomously.NectarCloud’sself-servicestructureallowsuserstoaccesstheirowndataatanytimeandcollaboratewithothersfromtheirdesktopinafastandefficientway.

    TheNectarResearchCloudisasinglenationalcloudcomputinginfrastructure,comprisedofsevencollaborating“nodes”.Nodesprocurehardwarenecessaryforrunningthecloudcompute,storageandnetworkservices.

    Nectarisaworldleaderindeployinghighlyinnovativecloudcomputingtechnologyforthebenefitofresearch–providingopportunitiesforfederationwithemerginginternationalresearchclouds.TheNectarResearchCloudsupportstheNCRISmissiontodeliverworldclassresearchfacilitiessothatAustralianresearcherscansolvecomplexproblemsinAustraliaandinternationally.

  • 7

    3.Methodology

    3.1Approachestoestimatingvalueandimpact

    TheassessmentofprogramssuchasNectarVLs,whichprovidefreeaccesstoarangeofweb-basedinformationtechnologydatabases,analysissoftware,trainingandotherservices,isarelativelynewfieldandinvestigatorshaveusedavarietyofapproachestoestimatethevalueandimpactsoftheseprograms.

    InarecentstudyofthevalueandimpactoftheEuropeanBioinformaticsInstitute(EBI),acentreforresearchandservicesinbioinformatics,whichispartofEuropeanMolecularBiologyLaboratory(EMBL).,BeagrieandHoughton(2016)reviewtheapproachestakenbystudiesofsciencefacilities.Theynotethatmostoftheseareoflargescaleinfrastructurefacilitiessuchassynchrotrons,andonlyafewofdatarepositoriesandrelatedinfrastructureandservices.TheyquoteareviewofstudiesbythegroupEvaRIO(EvaluationofResearchInfrastructuresinOpeninnovationandresearchsystems)(2013)whichfoundthatcost-benefitanalysisandtechniquesforestimatingthereturnonR&Dexpenditurewererelativelyrareintheestimationofthevalueandimpactofsciencefacilities.

    BeagrieandHoughton(2016)foundthatthreemaintypesofanalysiswereusedinthesestudies:variousformsofinput-outputanalysis,casestudiesandexamples,andformsofcost-benefitanalysisusingactivitycostingand/orcontingentvaluationasthebasisoftheanalysis.

    Input-outputanalysisisbestsuitedtoanalysisofsinglesitefacilitiessuchasasynchrotronwhilecasesstudiesarelimitedintheextenttowhichtheresultscanbescaleduporgeneralised.

    BeagrieandHoughtonthereforeuseamixedmethodapproachtocost-benefitanalysisdrawingupontheirexperiencevaluingarangeofdataservicessuchastheNationalCrystallographyServiceatSouthamptonUniversityandtheUKDataArchiveattheUniversityofEssex(Beagrieetal.2010),theUKEconomicandSocialDataService(Beagrie,Houghton,Palaiologk,andWilliams2012),theUKArchaeologyDataServices(BeagrieandHoughton2013a),andtheBritishAtmosphericDataCentre(BeagrieandHoughton2013b).

    Again,inmanyrespects,theservicesanalysedinthesestudiesaresimilartoNectarVLs.

    InstudyingtheEBIandtheseotherservices,BeagrieandHoughton(2016)notethat,becauseoftheiropenaccessnature,theydonotrecordmuchinformationabouttheusersoftheservices,beyondwhatisavailablefromIPaddressesorwebdownloadswhenusersaccesstheseservices.Theynotefurtherthatmorecomprehensiveandreliabledatawouldimproveeconomicanalysisoftheseservices.

  • 8

    FollowingtheseauthorswehaveapproachedthetaskofmeasuringthevalueandimpactofNectarVLsinanumberofways.

    TheseareshowninFigure3.1.Aswithmostlargesciencefacilities,theVLsprovidesvaluebothtotheuserandtothewidersociety.TheVLsprovideaservicetoacommunityofresearcherswhichcanbevaluedinaseveralways.Thesearediscussedinmoredetailbelowbutoneisgivenbythevalueofthetimespentusingtheserviceandanother,acontingentvaluation,isanestimateofitsmarketvaluei.e.thevalueauserwouldplaceontheserviceifitwasofferedinthemarketplace.InmeasuringthecontributiontoGDP,serviceswithnomarketvalue,areoftensimplyincludedatcost.Howeverevenwherethereisnomonetarytransactionitispossibletoestimatea‘market’valueofaservicetoaconsumer.Thismarketvalueis,theamounttheconsumerwouldeitherbewillingtopay(WTP)touseitoraccept(WTA)nottouseit.

    Figure3.1 Impactvaluationmethodologies

    Source:Beagrie,N.&Houghton,J.W.(2016)TheValueandImpactoftheEuropeanBioinformaticsInstitute(www.beagrie.com/publications).

    ThiswillingnesstopayoracceptderivesfromtheefficiencygainseitherfromusingtheVL,orforgonefromnotusingit,respectively.TheseefficiencygainscanbeseparatelyestimatedfromdatacollectedfromVLusersandprovidesacredibilitycheckontheWTPandWTAestimates.

    Beyondthesesourcesofvalue,liethereturnstoR&DwhicharederivedfromthevalueoftheresearchoutputsoftheusersoftheVLs.Thesearemoredifficulttoestimatebutincludesignificantsocialreturnsarisingfromspillovers,andotherbroadercommunitybenefits,

  • 9

    fromnewinnovativeproductssourcedinpartatleastfromthebenefitsgainedfromusingtheVLs.

    1.Investmentandusevalue

    Theinvestmentvalueofaproductistheamountofresourcesspentonitsproductionanddelivery.Theusevalueontheotherhandistheamountofresourcesspentbyusersonobtainingandusingtheproduct.MeasuresoftheamountoftimeandmoneyusedbyNectarVLusersinaccessingandusingtheVLservicesisanindicatoroftheminimumamountthattheserviceisworthtothem.

    Informationabouttheresourcesinvolvedinthediscovery,access,anduseofNectarVLservicescanbeobtainedbyusersurveysandinterviews.

    2.Contingentvaluation

    Contingentvaluationinvolvestheassignmentofmonetaryvaluestonon-marketgoodsandservicesbasedonpreferences(i.e.,PreferenceTheory).Itisoftenusedinthevaluationofenvironmentalandotherassets,theservicesofwhich,incontrasttogoodsandservicestradedinthemarketplace,areprovidedfreeoratnominalpricestousers.

    TheusualmeasureofeconomicactivityisGrossDomesticProduct(GDP).Thisprovidesanaggregatevalueofgoodsandservicestradedinthemarketplaceincurrencyunits.Thevalueofthesetransactionsislargelyestablishedthroughapricesettledbydemandandsupplyoutcomes.However,therearewidelyacknowledgedshortcomingswithGDPasameasureofeconomicvalue(StiglitzCommission2008).Oneismeasuringthevalueofservices.

    Services,particularlythoseprovidedbythepublicsector,areoftenincludedatcostbecausethereisnodirectdemandandsupplyinteractionbetweenbuyersandsellers.Ineconomicterms,thereisnomechanismwherebytheconsumerisabletorevealherpreferencesofhowmuchshewouldbewillingtobuyataparticularprice.ThisoftenmeansthatservicesareundervaluedinestimatingGDP,becausetheuserswouldbewillingtopaysignificantlymorethanthecostofproductioniftheyweregiventhatopportunity.ThisislikelytobethecasefortheNectarvirtuallabs.

    ThevalueoftheservicesprovidedbytheVLswillbeincludedinGDPlargelybasedonthesalarycostsofoperatingthelabs,wheninfact;thevalueplacedontheservicesbyusersmaybesignificantlygreater.Thecontingentvaluationmethodologyprovidesonewayofestimatingthisvalue.

    Ifagoodorservicecontributestohumanwelfare,ithaseconomicvalue,andwhethersomethingcontributestoanindividual’swelfareisdeterminedbywhetherornotitsatisfiesthatindividual’spreferences.Preferencesarerevealedbywhatanindividualiswillingtopayforagoodorserviceand/orbytheamountoftimeandotherresourcesspentobtainingthe

  • 10

    preferredgoodorservice.Wherepreferencesarenotrevealedinthemarket,individualscanbeaskedwhattheywouldbewillingtopayfor,ortoacceptinreturnforbeingwithout,thegoodorserviceinahypotheticalmarketsituation(i.e.,statedpreference).Forapublicgood,thevalueisthesumof“willingnesses”,asconsumptionisnon-rivalrous(i.e.,thesameinformationcanbeconsumedmanytimes).Thekeydifferencebetweenwillingnesstopayandwillingnesstoacceptisthattheformerisconstrainedbyabilitytopay(typicallybydisposableincome),whereasthelatterisnot.Hence,willingnesstopaydirectlymeasuresthedemandcurvewithabudgetaryconstraint,whilewillingnesstoacceptmeasuresthedemandcurvewithoutabudgetaryconstraint(BritishLibrary2004).

    Thecontingentvaluationapproachinvolvesaskingwhatpeoplewouldbewillingtopay(toreceiveabenefitorremoveacost)oraccept(toforgoabenefitorincompensationforacost)ifamarketexistedfortheexternaleffect.(CommonwealthofAustralia2006)

    3.Efficiencyimpacts

    Widerbenefitsandimpactscanbeexploredbylookingattheefficiencygainsenjoyedbyusersandassigninganeconomicvaluetothem,suchasthevalueoftimesavings(productivity),andtheavoidanceofcostsforusersthatwouldotherwisebeinvolvedinthecreation/collectionofthedataforthemselvesorobtainingitelsewhere.Forthiswecombineusersurveyquestionsaboutperceivedefficiencyimpactswithactivitycosting.

    4.Returnoninvestment

    TherehavebeenanumberofstudiesbothinAustraliaandoverseasthathaveattemptedtomeasurethereturnstobothpublicandprivateinvestmentinresearchanddevelopment,andthecontributionthatR&Dandtechnologydevelopmentmorebroadlyhavemadetoeconomicgrowth.

    TheProductivityCommissionhasundertakenanumberoflargestudiesontheroleofR&DinAustralia(e.g.IndustryCommission1995,ShanksandZheng2006).

    AnanalysisofthereturnstohealthR&DbyAccessEconomics(2008)foundthathealthR&DprovidesreturnstoAustraliaof117%perannum,withabenefit-costratioof2.17.Inanextensionofthisstudyin2014.DeloitteAccessEconomics(2014)foundthateverydollarinvestedbytheMedicalResearchFutureFundwouldgeneratereturnsof$3.39infuturehealthandproductivitygains.

    InexaminingtheseandotherAustralianandinternationalstudies,BeagrieandHoughton(2016)concludedthatthereturnstoresearchanddevelopmentweretypicallyintherangeof20%to60%perannum.IntheiranalysisbasedonamethodologydevelopedbyHoughtonandSheehan(2009)andHoughtonetal(2009)basedonamodifiedSolow-Swanmodelofeconomicgrowth,theyusedaconservativevalueof40%.

  • 11

    Theseandotherstudiesofhaveconsistentlyfoundlargereturnstoinvestmentinbothpublicandprivateresearchanddevelopment.

    BalancedValueImpactModel

    Inourdiscussionswiththe4VLsthatarethefocusofthisstudybothHuNIandtheGVLdrewourattentiontotheBalancedValueImpactModelwhichisanevaluationschemadevelopedbyProfessorSimonTannerofKing’sCollegeLondon(Tanner2012).Thispublicationisamanualforuserstoundertaketheseevaluationsandthemethodologyiscomplex.Fromabriefreviewoftheliteratureitdoesnotappeartohavebeenimplementedfullyinanyevaluationtodate.

    WediscusstheBalancedValueImpactModelinmoredetailinSection7below.

    3.2Surveydata

    MostoftheapproachesoutlinedabovedependonhowusersuseVLservicesandthevaluethatcanbeputonthisuse.

    AmajorlimitationexperiencedwithstudiesofthevalueandimpactofresearchinfrastructuresuchasNectarVLsisthepaucityofdatarelevanttotheapproachesoutlinedintheprevioussection.WhencommentingonthesestudiesBeagrieandHoughton(2016)notedthevariabilityofresearchinfrastructuremetricsandtheimplicationsforeconomicanalysisandquotedfromtheirreview(BeagrieandHoughton2014)asfollows

    Itisalsoclearfromthesestudiesthatdifferentdatacentrescollectfinancialandoperationaldata,suchasuserstatistics,datadeposit,accessanddownloadstatistics,tovaryinglevelsofdetailandusingdifferentdefinitions.Moreguidanceisneededonthecollectionofsuchdata.Doingsowouldhelptoensureagreaterdegreeofstandardisationofoperationalrecordsacrossdatacentres.Thiswouldbeofgreatestbenefittofundersinvestinginarangeofdatacentres,andwouldprovidemorecomprehensiveandreliabledataforeconomicanalysis.Therewouldbeconsiderableadvantagetoprovidingguidanceregardingthecollectionofsuchdataasitisfundamentaltotheeconomicanalysisandinmakingthebusinessorfundingcase.

    Funderswillneedtoensureallowanceismadeinbudgetstoenablesuchcentrestocollectadequateinformationsothatproperevaluationscanbeundertaken.

    ToaddressthepotentiallimitationsofusagedatacollectedbyNectarVLs,andtoprovidedatarelevanttotheapproachesoutlinedabove,anonlinesurveyquestionnairewasdevelopedtomeasureaspectsoftheuserexperiencewithVLsusingtheQualtricson-linesoftwaretool.(https://www.qualtrics.com/homepage/)

    ThestartingpointforthequestionnairewasthatusedbyBeagrieandHoughton(2016)butmodifiedtoaddressthesituationofeachVL.Draftinitialsurveyquestionnaireswere

  • 12

    developedbyVISESanddiscussedwithkeyNectarandVLpersonnel.Theywerethenrefinediterativelyoveranumberofdiscussioncyclesandpilottested.

    Thequestionnaireusedarangeofstandardsurveyapproaches,includingtheuseof“criticalinstances”,suchasthelastdataaccessed/downloaded(forusers).Anumberofquestionssoughtspecificinformationon:thetimeandcostofaccessforusers;thebenefitsandefficiencyimpactsofaccess;andcontingentvaluation(i.e.,willingnesstopayoraccept)usingstatedpreferencetechniques.Answerstothesequestionswereinterpretedcarefully,inthecontextofopen-endedtextcommentsinthesurvey.ThesequantitativequestionsweresupplementedbyqualitativequestionsaskingforviewsontheimportanceandimpactoftheVLforusers,toensurethatthequantitativeandqualitativefindingswereinaccord.

    ThesurveyswereprogressivelyimplementedfromApriltoMay2017.

  • 13

    4.TheVirtualLaboratories

    4.1BiodiversityandClimateChangeVirtualLaboratory(BCCVL)

    TheBiodiversityandClimateChangeVirtualLaboratorywasformedunderanagreementinNovember2012betweenNectarandGriffithUniversity.TheothermainpartnersinBCCVLare

    • JamesCookUniversity• UniversityofNewSouthWales• MacquarieUniversity• UniversityofCanberra• AtlasofLivingAustralia(ALA),ane-infrastructurefundedbyNCRIS• TerrestrialEcosystemResearchNetwork(TERN),anationalobservatoryfor

    Australianecosystems,deliveringdatastreamsthatenableenvironmentalresearchandmanagement,withfundingfromNCRIS

    • QueenslandCyberInfrastructureFoundation(QCIF)aHPCconsortiumofQueenslanduniversities

    Undertheagreement,Nectarhasprovidedfundingof$1,863,063to2016-17andpartnershavecommittedto$600,000incashand$1,315,971in-kind.TheBCCVLwebsitewaslaunchedinAugust2014.

    TheBCCVLprojectteamof8peopleislocatedatGriffithUniversityandheadedbyMrHamishHolewa.IthasaSteeringCommitteeof17peopledrawnfromparticipatinginstitutionsandNectar,aswellasanEcologicalModellingScientificAdvisoryGroupof8people.

    TheBiodiversityandClimateChangeVirtualLaboratory(BCCVL)isa“onestopmodellingshop”thatsimplifiestheprocessofbiodiversity-climatechangemodelling.ItsmissionistoconnecttheresearchcommunitytoAustralia’snationalcomputationinfrastructurebyintegratingasuiteoftoolsinacoherentonlineenvironmentwhereresearcherscanaccessdataandperformdataanalysisandmodelling.

    Previously,thelinesofinquiryintobiodiversityandclimatechangeimpactswerestymiedduetoresearchers’inabilitytoaccessastandardisedsetoftoolsforanalysisandrequisitedatasources,andcomputationallimitations.

    ThegoaloftheBCCVListointegratethesetoolsanddatasetswithhigh-performancecomputersandmajordatastoragefacilities,therebyenablingmoreefficientinvestigationofbiologicalsystemsandfacilitatingthedevelopmentofadditionalresearchtrajectoriescurrentlynotpossibleduetocomputationallimitations.

  • 14

    CentraltothemodellingfacilitatedbyBCCVLisspeciesdistributionmodelling,alternativelyknownasenvironmentalnichemodelling,(ecological)nichemodelling,predictivehabitatdistributionmodelling,orclimateenvelopemodelling.Itreferstotheprocessofusingcomputeralgorithmstopredictthedistributionofspeciesingeographicspaceonthebasisofamathematicalrepresentationoftheirknowndistributioninenvironmentalspace(=realisedecologicalniche).Theenvironmentisinmostcasesrepresentedbyclimatedata(suchastemperature,andprecipitation),butothervariablessuchassoiltype,waterdepth,andlandcovercanalsobeused.Thesemodelsallowforinterpolatingbetweenalimitednumberofspeciesoccurrenceandtheyareusedinseveralresearchareasinconservationbiology,ecologyandevolution.

    Developingaspeciesdistributionmodelbeginswithobservationsofspeciesoccurrences:theseareplaceswhereweknowaspecieshasbeenfound.Theseoccurrencesaremostlypoint-basedandcomefromsourcessuchasmuseumrecordsandobservationsofexpertsinthefield.BCCVLuserscanuploadtheirownspeciesdistributiondataoraccessoneormoreofthemanydatasetsprovidedbyBCCVL.

    Tocalibrateacorrelativespeciesdistributionmodeltwotypesofinputdataareneeded:speciesoccurrences,andmeasurementsofasuiteofenvironmentalvariables,suchastemperatureandrainfall.Thesetwotypesofdataarethenputintoanalgorithmtofindassociationsbetweentheknownoccurrencesofaspeciesandtheenvironmentalconditionsatthosesites,toidentifytheenvironmentalconditionsthataresuitableforaspeciestosurvive.Thisprovidesinformationaboutwherespeciesoccurandsomethingabouttheenvironmentalconditionsofthoseplaces.Thealgorithmusesthesetwotypesofinformationtoestimatetheprobabilityofaspeciesoccurringinaplaceassomefunctionoftheenvironmentalconditionsofthatplace.

    BCCVLcurrentlysupports17differenttypesofspeciesdistributionalgorithms.

    BCCVLalsooffersarangeofclimatechangemodelsthatcanbeusedinconjunctionwithspeciesdistributionmodellingtoestimatetheimpactofclimatechangeonspeciesdistribution.

    TheBCCVLwebsiteoffersaKnowledgeBasewhichcanguideusersthroughthemodellingprocessbyprovidinginformationaboutthestepsinvolvedinmodellingandthedatabasesandotherresourcesavailable.

    Thewebsitealsooffersatrainingcourseof10modulesexplainingSDM,thechoiceofalgorithmsandhowtointerpretmodellingresults.

    BCCVLoffersworkshopsaimedatacademicsandindustryprofessionals,forinstanceHDRstudents,environmental/climatescientistsandresearchers,ecologists,decision-makers,membersofgovernmentandindustrygroups.Italsoprovidesworkshopsforundergraduatestudentsstudyinginareassuchasecology,environmentalscience,sustainability,climate

  • 15

    changeimpacts,biology,floraandfauna,animalbehaviour,planninganddevelopment,andconservation.

    4.2CharacterisationVirtualLaboratory(CVL)

    TheCharacterisationVirtualLaboratory(CVL)wasformedunderanagreementinMay2012betweenNectarandMonashUniversity.FoundingpartnersinCVLare

    • AustralianMicroscopyandMicroanalysisResearchFacility• AustralianNuclearScienceandTechnologyOrganisation• AustralianSynchrotron• NationalImagingFacility• AustralianNationalUniversity• UniversityofSydney• UniversityofQueensland

    Throughitslife,anumberofprojectpartnershavejoinedtheCVLandengageddirectlythroughprojectfunding:

    • CSIRO• DeakinUniversity• Intersect,anditspartners• QCIF,anditspartners• RMIT• ThePawseySupercomputingCentre• TheTerrestrialEcosystemResearchNetwork• UniversityofMelbourne• UniversityofNewSouthWales• UniversityofWesternAustralia• VicNode,anditspartners

    Undertheagreement,Nectarhasprovidedfundingof$2,340,108to2016-17andpartnershavecommittedto$1,836,740in-kind.

    TheCVLwebsitewaslaunchedinMarch2013.

    ThepredominantpractiseoftheCVLprojectistoworkwithinstrumentfacilitiestoprovidedatacapture,analysisandvisualisationservicesandtherebyunderpinthefacilityusercommunity.

    CVLhasworkedwithorisworkingwith26facilitiestointegrateover100instrumentswithatotalvalueofaround$250million..AcompletelistofthesefacilitiesisgiveninAppendix2.

  • 16

    TheCharacterisationVirtualLaboratory(CVL)aimstointegrateAustralia’simagingequipmentwithspecialisedHighPerformanceComputingcapabilitiesandwithdatacollectionnodesandprovidescientistswithacommonenvironmentforanalysisandcollaboration.Itinitiallydevelopedfourresearchapplication(‘drivers’)inmulti-modalorlarge-scaleimaginginneuroscience,structuralbiology,atomprobeandX-rayscience,andthroughpartnership,thishasbeenextendedtocytometry,neutron-beamimaging,lightmicroscopy,andbioinformatics.

    Eachdriverisledbyaworld-classresearchgroup,issupportedbyanAustralianresearchconsortiumandisinanationalresearchpriorityarea.TheresultsfromthisdevelopmentaredistributedtothecommunitythroughCVL“Workbenches”.

    CVLusershavetheoptionofusingthecomputingresourcesontheNectarresearchcloudoraccessingMASSIVE(Multi-modalAustralianScienceSImagingandVisualisationEnvironment)theHPCfacilityjointventurebetweenMonashUniversity,CSIROandtheAustralianSynchrotron

    AnimportantroleforCVLhasbeentoestablishthecapabilitywithinimagingsitesforautomaticcaptureandstorageofimagingdatatothecloud.ThisenablesuserstobeabletoaccesstheirimagingdatathroughtheCVLwebsiteandtoperformanalysisusingthesoftwaretoolsprovidedbyCVLbasedaroundtheMyTardisandStore.Monashtools(Ceguerraetal2013).

    CVLhasdevelopedanumberofapplicationsthatarereusedacrossarangeofAustralianfacilities.Theseinclude:extensionstotheMyTardisdatamanagementplatformtosupportinstrumentfacilities;aninstrumentintegrationapp,calledMyDatatomakeintegrationquicker,simplerandlessreliantonspecialistITsupport;andStrudel,atoolthatmakesaccessingremoteanalysisenvironmentseasier.

    4.3GenomicsVirtualLaboratory(GVL)

    TheGenomicsVirtualLaboratorywasformedunderanagreementinMay2012betweenNectarandtheUniversityofQueensland.TheothermainpartnerinGVListheUniversityofMelbourne.ThefollowingorganisationsarepartnersorcollaboratorswithGVL

    • TheUniversityofQueensland• QFABBioinformatics• QueenslandCyberInfrastructureFoundation• MelbourneBioinformatics(formerlyVictorianLifeSciencesComputationInitiative)

    VictorianeResearchStrategicInitiative• TheUniversityofMelbourne• BakerIDIHeartandDiabetesInstitute• PeterMacCallumCancerCentre

  • 17

    • TheGarvanInstituteofMedicalResearch• VictorChangCardiacResearchInstitute• CSIRO• BioplatformsAustralia• MonashUniversity• TheUniversityofSydney• TheUniversityofWesternAustralia• AustralianBioinformaticsNetwork• EMBLAustralia• AustralianGenomeResearchFacility• AustralianNationalDataService

    Undertheagreement,Nectarhasprovidedfundingof$3,186,121to2016-17andpartnershavecommittedto$2,917,707in-kind.

    TheGVLisadministeredbyMelbourneBioinformatics.Ithasastaffof8ledbyAssociateProfAndrewLonie,asDirectorofMelbourneBioinformatics.

    TheGenomicsVirtualLaboratoryprovidesacloud-basedsuiteofgenomicsanalysistoolsforlifescienceresearchandtraining.

    Biologistswithoutcomputersciencetrainingcangostraighttoauser-friendlyplatformwhichhostsasuiteoftestedbioinformaticstoolsandpipelinesforfast,consistent,dataanalysis.Theplatformisconstantlyupdatedtohavethelatestfeaturesinusebyexpertbioinformaticians.Adoptedbothlocallyandoverseas,theGVLhasalreadybeenrecognisedasaqualityplatformtohelpaddresstheshortageofbioinformaticsexpertisearoundtheworldandmanagethecomplex,multiple-layereddataanalysistasksconfrontinglifescientiststoday.

    Nationallyandinternationallyitisbeingusedbothbylifescientistsworkingwithgenomicdataandacademicsteachingbioinformaticsatundergraduateandpost-graduatelevels.

    LifescientistswithoutaccesstobioinformaticsexpertisearetheprimaryusersoftheGVL.Practisingbioinformaticianswhoknowhowtofindtherighttoolforthedataanalysisjob,findthattheGVLisworkingfortheminotherways.SmallbioinformaticsgroupsorlonepractitionersusetheGVLtotraintheirlocalteamstodotheirownsimplebioinformaticstasksontheGVL,freeingupcapacitytoworkonmorecomplexresearchproblemsortocollaboratemorebroadly.TheGVLwebsiteisbuiltaroundGalaxy,anopen,web-basedplatformfordataintensivebiomedicalresearch.GalaxyisaninternationalcollaborativeprojectmanagedbytheCenterforComparativeGenomicsandBioinformaticsatPennStateUniversity,andtheDepartmentofBiologyatJohnsHopkinsUniversity.

  • 18

    UsersofGVLhavetheoptionoflaunchingtheirownprivateserveronapublicorprivatecloud,withtheGVLpre-installed,oraccessingitthroughoneofthepublicGVLservicesontheGVLwebsitemanagedbyGVL.MostuserswithoutrelevantITskillsuseoneofthreemainpublicservicesnamely

    • GalaxyMelbourne• GalaxyQueensland• GalaxyTut

    GalaxyTutisusedfortrainingpurposes.Aswell,theGVLprovideson-linebasicandadvancedGalaxytutorialsandcommandlinetutorialswhichgivestepbystepinstructionsforanexampleanalysis.Protocolsoutlinetheanalysismethodsandsuggestandcomparetoolsforeachstepratherthanprescribethem.

    GVLuserstypicallyloadgenomicsequencesthatarethenmanipulatedandanalysedusingacomprehensivesuiteoftools.

    AsidefromthepublicservicesmanagedbytheGVL,anumberofotherinstitutionssuchasCSIROandLaTrobeUniversityhavedownloadedtheGVLtotheirserverstolocallymanageaccessfortheirinstitutionalusers.

    4.4HumanitiesNetworkedInfrastructureVirtualLaboratory(HuNI)

    TheHumanitiesNetworkedInfrastructureVirtualLaboratory(HuNI)wasformedunderanagreementwithDeakinUniversityinMay2012.Nectarhascontributedfundingof$1,834,500to2016-17andpartnershavecommittedto$2,500,000in-kind.

    ThepartnersinHuNIhaveincluded

    • AustralianInstituteofAboriginalandTorresStraitIslanderStudies(AIATSIS)• AustralianNationalUniversity• DeakinUniversity• FlindersUniversity• IntersectAustralia• MacquarieUniversity• RMITUniversity• UniversityofMelbourne• UniversityofNewSouthWales• UniversityofQueensland• UniversityofSydney• UniversityofWesternAustralia• V3Alliance• NationalLibraryofAustralia

  • 19

    • AustralianCentrefortheMovingImage(ACMI)

    TheProjectDirectorofHuNIisProfessorDebVerhoevenatDeakinUniversity.

    GoogleAnalyticsdataprovidedbyHuNIindicatesaround4,600usersofHuNIduring2016.

    HuNI(HumanitiesNetworkedInfrastructure)combinesdatafrommanyAustralianculturalwebsitesintoalargehumanitiesandcreativeartsdatabase.HuNIdatacoversalldisciplinesandbringstogetherinformationaboutthepeople,works,events,organisationsandplacesthatmakeupthecountry'srichculturallandscape.

    HuNIcombinesinformationfrom32ofAustralia’smostsignificantculturaldatasets.Thesedatasetscomprisemorethan17millionauthoritativerecordsrelatingtothepeople,organisations,objectsandeventsthatmakeupAustralia'srichculturalheritage.HuNIalsoenablesresearcherstoworkwithandsharethislarge-scaleaggregationofculturalinformation.HuNIhasbeendevelopedasapartnershipbetween16publicinstitutions,ledbyDeakinUniversity.

    AccesstotheHuNIwebsitewasfirstavailableinJuly2012andwasofficiallylaunchedinOctober2014.

    AsVerhoevenandBurrows2015suggeststhisplacesHuNI

    ‘somewherebetweena“datawarehouse”inwhichtheincomingdataarefirstcleanedandorganisedintoaconsistentschemaanda“datalake”inwhichtheincomingdataareingestedintheirrawformandtheresponsibilityormakingsenseofthedataliesentirelywiththeenduser’p418

    Theconstituentdatabases,havebeenreconfiguredbyHuNIsothattherecordsaremappedtosixcoreentities:Person,Organization,Event,Work,Place,andConcept.(VerhoevenandBurrows2015).

    Theuserbeginstheirsearchforanitemofinterestinoneoftheseentities.Thatcouldbeanameofapersonoraparticularorganisationoraconcept(topic)ofinterest.ThepowerofHuNIistoretainasdatawithinHuNItheconnectionsdiscoveredbetweenitemsintheentities,suchasaconnectionlinkingapersontoothersand/oranorganisationandaconceptetc.madebytheuser.Thesearesavedbytheuserandmaybeplacedinthepublicdomainandthusmadeavailableforotherusers.Subsequentusersmaydisputethelinksmadeorsupplementthelinks,creatingarichersetofnetworksconnectedtooneormoreoftheentities.

    Anexampleofsuchanetwork(aHuNIKnowledgeGraph)isshowninFigure4.1.

    Thisgraphshowslinksrelatedtothecollection:AustralianFashion1850-1950,includingrelatedpersonsandworks.InparticularitlinkstheAustralianFashionCollectionviaa1938

  • 20

    filmwithafashiontheme,DadandDaveCometoTowntoacollectiongeneratedbyadifferentHuNIuserinvestigatingworksdirectedbyKenHallandproducedbyCinesoundProductions.TherearefurtherlinksmadetootherfilmsdirectedbyKenHall.BytracingthepathwayofthesevisuallinksaHuNIuserisabletoexpandtheirdiscoveryopportunitiesthroughtheserendipitousconnectionsprovidedbyamultitudeofHuNIusers.

    Figure4.1 AnexampleofaHuNIKnowledgeGraph

    SourceVerhoeven2016,p23

    Asthefigurebelowillustrates,thisprocessconvertsisolateddataandinformationtoknowledgebycapturingtherelationshipsarisingfromtheexpertiseofHuNIusers.The

  • 21

    knowledgecreatedbytheseconnectionsisfurtherenrichedthroughthecollaborationofotherhumanitiesusersofthewebsite.

    Figure4.2 ConnectionsbetweendataandinformationusingHuNI

    SourceVerhoeven2016,p13

    Thisfacilitationofanunstructuredapproachtoknowledgecreationwithinthedatabaseencouragesaserendipitousrelationshipsearchprocess.ThisisseenasamajoraddedvalueoftheVL,onethatisparticularlyapplicabletothehumanitiesstyleofresearchandonethatisnotcateredforbyconventionalarchivaldatabases.

    ‘HuNImovesbeyondthinkingofserendipityasonlyatechnicalproblemandinsteadtreatsitasamatterofsocial,philosophical,andpoliticalsignificance’(p23Verhoeven2016).

    HuNIhasbeenestablishedwiththeideathatvaluableknowledgeandnewperspectivesareburiedintraditionalarchivaldatabases,whichcanbereleasedbythepartlyunanticipatedsearchprocessessupportedandusedbyHuNIusers(Verhoeven2016).PerhapsthiscreationofnewconnectionsisHuNI’shighestvalue.Itnotonlyfacilitatesthecreationofnewnetworksthroughlinkswithmultipledatabases,butalsohasthesoftwarethatallowstheconnectionsmadeinthesearchprocesstoberecorded,storedandmadeavailabletootherresearchers.OvertimethatarchiveoftheseHuNIKnowledgeGraphswilladdsignificantvaluetotheHuNIproject.

    Atthisstagetheresourcesavailabletotheprojecthavebeenlargelydevotedtothedevelopmentofthesoftwarethatfacilitatesthecharacterisationoftheunderlyingarchivaldatabasesintermsofsearchableentities.Thefacilitytostoreuserdevelopedconnectionsisstillinitsinfancy.Anewsystemupgrade,includinganewuserinterface,isabouttobeinstalled,whichwillbetterfacilitatethecreation,storageandsearchingofvaluableHuNIKnowledgeGraphs.

  • 22

    HuNI’svalueliesintheincreasingparticipationofitsusers.ThemorelinksandcollectionsthataremadeinHuNI,themorevaluableatoolitbecomesforeveryone.ThismeansthatearlyonthebenefitversuseffortforHuNIusersisrelativelylow,thisrisesexponentiallyasmoreusersaddtheirexpertisetotheplatform.InvestmentinthepromotionofHuNIandbuildingofthecommunityisrequiredtoensureHuNImovespastthetippingpointofusers“gettingmoreoutofHuNIthantheyputin”.

  • 23

    5.DataonusageofNectarVirtualLaboratories

    ThemethodologiesformeasuringthevalueandimpactofNectar’sVLsdependonbeingabletodescribetheextenttowhichtheVLshavebeenabletoattractusersoftheirservices.IntheevaluationswerelyondataprovidedbytheVLsonusageandontheresponsestotheon-linesurveysofusers.

    AlthoughtheVLsdifferintheextentandtypeofdatatheycaptureaboutusageofVLservices,theyreadilyprovidedusagedatabasedontheregularcollectionstheymakeformonitoringpurposesandinresponsetofurtherrequestsfordifferentormoredetaileddata.Theirabilitytoprovidethisdatawaslimitedforsomebecauseitwasnotdatathatwasusuallycollectedinthecourseoftheiroperations.

    ThetablesandfiguresbelowsummarisetheusageofVLservicesgenerallyfromwhentheybecameactivetothepresent.Forsome,veryearlydatawasnotavailableandthemostrecentdatawasforearly2017.

    ItisclearthatthenumberofusersforaparticularVLincreasesrapidlyinthefirstfewyearsandhascontinuedtogrowoverthepastyear.

    ThisstudyonthevalueandimpactofVLsisthereforeoccurringrelativelyearlyintheirlifetimes,presentingchallengesinassessingthelikelynumberofusersoverthenextfewyearsandthepotentialmaximumnumberofusers.ItisdifficulttoassesswhatpercentofthecurrentresearchbaserepresentsthemaximumachievablebyeachVLandwhenthiswilloccur.Itwouldbeexpectedthatoncethisoccurred,thefuturerateofgrowthwouldbelowerandjustreflectthenumbersofnewresearchersenteringtheresearchfieldsofrelevancetotheVL.

    BCCVL

    BCCVLprovidedmonthlyusagedatafromJune2015toMarch2017.FromthisdataweshowinTable5.1thenumbersofnewusersforeachcalendaryearandthecumulativenumberofusersatDecemberofeachyear.Tothisweaddestimatesofthenewusersandcumulativeuserscalculatedasfollows.

    Overthe11monthstoMarch2017,thenumberofusersincreasedby3.75timescomparedtotheprevious11months.Ifitisassumedthatthenumberofnewusersin2017willbe2.0timesthatin2016,andthevaluein2018willbe1.5times2017,andwiththesamenumberofnewusersin2019and2020asin2018,thenthenumberofnewusersandthecumulativenumberofusersfor2017to2020willbeasinTable5.1.

    ThesegrowthassumptionssuggestthatthecumulativenumberofusersbyDecember2020willbe11,246.

  • 24

    Table5.1 BCCVLactivitymeasures

    Newusers

    Cumulativeat

    December2015 254 2542016 916 1,1702017 1,832 3,0022018 2,748 5,7502019 2,748 8,4982020 2,748 11,246CVL

    CVLconsultedwitharangeofimagingfacilitiespartnersandprovidedestimatesofthenumberofusers.ToAugust2017thisamountedto2558,soweassumethatthenumberatDecember2016was2400.Basedonaninitialcohortof101atDecember2012,weinterpolatedthenumberofnewusersfrom2012to2016andestimatedthatthiswouldgrowby10%intheyearsto2020.Thisresultedinanestimateofthecumulativetotalof6,485bytheendofDecember2020(Table5.2).

    Table5.2 CVLactivitymeasures

    Newusers

    Cumulativeat

    December2012 101 1012013 300 4012014 500 9012015 700 1,6012016 800 2,4012017 880 3,2812018 968 4,2492019 1,065 5,3142020 1,171 6,485GVL

    GVLhasprovidedusagedataforthethreemajorpublicserviceportals.Ithasnotbeenpossibletoobtainandactivitydataforthoseorganisationsandindividualsthathavedownloadedthedataandlaunchedtheirownprivateserveronapublicorprivatecloud,withtheGVLpre-installed.

    GVLprovidedmonthlyusagedataforGalaxy–QueenslandfortheperiodDecember2013toFebruary2017.ThedataforGalaxy–MelbournecoveredSeptember2015toFebruary2017andforGalaxy–TutitwasDecember2012toFebruary2017.

  • 25

    ThenumbersofnewusersandthecumulativenumbersofusersatDecemberforthethreeportalsisshowninTable5.3.InasimilarmannertothatusedforBCCVL,weestimatethenumberornewusersandcumulativeusersto2020butrecognisethatthe3portalshavebeenoperatingfordifferentlengthsoftime.

    ForGalaxy–QueenslandandGalaxy–Tutweassumea10%growthinnewusersforeachoftheyears2017to2020.ForGalaxy–Melbournethenumberofnewusersinthe9monthstoFebruary2017was2.6timesthatoftheprevious9months.Weassumethatthenumberofnewusersin2017willbe2.5timesthatof2016,thenumbersin2018willbe2.0timesthatof2017,thenumbersin2019willbe1.5timesthatof2018andthenumbersin2020willbethesameas2019.TheestimatednewusersandcumulativeusersatDecembereachyearareshowninTable5.3.

    Table5.3 GVLactivitymeasures

    Galaxy-Queensland Galaxy-Melbourne Galaxy-Tut Newusers Cumulative

    atDecember

    Newusers Cumulativeat

    December

    Newusers Cumulativeat

    December2012 40 402013 27 27 117 1572014 285 312 1,060 1,2172015 586 898 65 65 510 1,7272016 631 1,529 408 473 587 2,3142017 694 2,223 1,020 1,493 646 2,9602018 764 2,987 2,040 3,533 710 3,6702019 840 3,826 3,060 6,593 781 4,4512020 924 4,750 3,060 9,653 859 5,311

  • 26

    6.EstimatingthevalueandimpactofNectarVirtualLaboratories

    MostofthecalculationsreportedinthissectionrelyonestimatesofthetimespentbyusersaccessingVLservicesandthesalariesoftheseusers.

    6.1Costdata

    ForacademicuserswehavecalculatedthehourlyratefordifferentacademicpositionsbyobtainingsalarydatafromMonashUniversityandcalculatingthevaluefor2017.TheannualsalariesshowninTable6.1areaveragesofthelevelswithineachacademicgradeforthesepositions.WehavecheckedtheselevelswiththoseinanumberofotheruniversitiesandothersourcesofinformationandregardtheMonashUniversityvaluesasrepresentativeofsalarieswithinAustralianuniversities.Forcomparisonpurposes,thesalariesarearound5%higherthanthoseforVictoriaUniversity.Toobtainanhourlycostsequivalentwedividetheannualsalariesbytheaveragenumberofworkingdaysinayear(225)andbythenumberofhoursworkedperday(7.35).

    FollowingBeagrieandHoughton(2016)weadd30%tothistoaccountfornon-wagelabourcosts.ThisisalsoinlinewithcurrentAustralianuniversityguidelines.TheindicativehourlycostscalculatedinthiswayareshowninTable6.1.

    Forstudentandotherusersthestartingpointisthemedianstartingsalariesofbachelordegreegraduatesin2015publishedbyGraduateCareersAustraliaLtd.Theirmostrecentdataisfor2015.Toestimatethevaluesfor2017,weaddafurther4.4%,beingtheestimatedtwoyearincreaseinaverageweeklyearningsforfull-timeadults(ABS2017).

    HourlycostsarecalculatedinthesamewayasforacademichourlycostsandtheseareshowninTable6.2.Fromthisweassumethatthehourlycostforstudentsisthehourlygraduatecostof$44.52

    Table6.1 Academicsalaries,andhourlycosts

    Level Salary,2017 Hourlysalary,2017

    Hourlycosts,2017

    $,000 $ $AcademicA 77,819 47.06 61.17AcademicB 102,469 61.96 80.55AcademicC 123,546 74.71 97.12AcademicD 145,211 87.81 114.15AcademicE 178,005 107.64 139.93

  • 27

    Table6.2 Annualgraduatesalariesandhourlycosts

    Qualification Salary,2015 Salary,2017,est.

    Hourlysalary,2017,est.

    Hourlycosts,2017,est.

    $,000 $,000 $ $AgriculturalScience 50.0 52.2 31.56 41.03BiologicalSciences 50.0 52.2 31.56 41.03ComputerScience 54.0 56.4 34.09 44.32EarthSciences 60.0 62.6 37.88 49.24Engineering 60.0 62.6 37.88 49.24Mathematics 60.0 62.6 37.88 49.24PhysicalSciences 50.0 52.2 31.56 41.03VeterinaryScience 50.0 52.2 31.56 41.03Average 54.3 56.6 34.25 44.52 TOTAL(all

    qualifications)54.0 56.4 34.09 44.32

    Source:Table1:Medianstartingsalariesofbachelordegreegraduatesinfirstfull-timeemploymentandagedlessthan25,byfieldofeducationandsectorofemployment,2015($,000,n)¤†,GraduateCareersAustraliaLtd,GraduateSalariesReport2015,GraduateCareersAustraliaLtd,2016athttp://www.graduatecareers.com.au/research/researchreports/graduatesalaries/

    Intheon-linesurveyofusers,participantswereaskedabouttheirpositionwithintheirorganisation.Allocatingbothacademicandgraduateleveltothesepositionsandusingthedistributionofanswersfromrespondents,wecalculatedameanhourlycostofusersbasedonthehourlycostsinTables6.1and6.2.

    ForBCCVL,CVLandGVLthehourlycostofuserswere$74.82(or$63,127annually),$66.39($56,015annually)and$71.02($59,925annually)respectively.

    Investmentvalue

    IntheirstudyofEBI,BeagrieandHoughtonwereunabletoestimatethetotalvalueofallthecomponentsofEBI’sinvestmentvalue,suchasdataacquisition,depositingdata,collaborationcostandaddingvaluetothedata.Theyreliedthereforeonanestimateoftheaverageannualoperatingexpenditureasaproxy.

    ThesetupandoperatingcostsofeachVLtotheyear2016-17weredescribedinTable2.2inSectionearlier.

    IfweassumethattheannualoperatingexpensesofeachVLisequaltotheaverageoftheyears2014-15to2016-17asshowninTable2.2andweapplythattoeachyearoutto2020,wecancalculatetheestimatedaccumulatedcostofeachVLto2020.Thisisshownintable6.3andistheestimatedinvestmentvalueofeachVL.

  • 28

    DividingthisbythenumberofyearsfromVLinceptionto2020,wecancalculatetheannualvalueoftheoverallinvestmentmadeineachVL.AscanbeseenfromTable6.3thisisintherange$550,000to$600,000forBCCVL,CVLandHuNIandissomewhathigheratabout$870,000forGVL.

    Table6.3 EstimatedtotalandannualcostforVirtualLaboratories

    BCCVL CVL GVL HUNITotalcoststo2016-17 3,779,034 4,176,848 6,103,828 4,334,500Averageannualoperatingcost 209,000 240,667 284,000 168,500Totalcoststo2020 4,406,034 4,898,848 6,955,828 4,840,000Annualvalueoftotalcoststo2020 550,754 612,356 869,479 605,000

    Contingentvaluation

    Thecontingentvalueofanon-marketgoodorserviceistheamountusersarewillingtopayforitand/orarewillingtoacceptinreturnforgivingitup.Forapublicgoodthevalueisthesumofwillingnesses,asconsumptionisnon-rivalrous(e.g.,thesameinformationcanbeconsumedmanytimes).Thekeydifferenceisthattheamountthatusersarewillingtoacceptinreturnforgivingupaccessistypicallyhigherthantheamounttheywouldbewillingtopay,primarilybecausethelatterisconstrainedbywhattheycanafford(e.g.,bydisposableincome,limitedresearchgrants,etc.).

    BCCVL

    ForBCCVL,themeanamountthatuserswerewillingtopayforaccesstoBCCVLserviceswas$1,154(medianvalue$250).OntheotherhandthemeanamountuserswouldbewillingtoaccepttoforegoBCCVLserviceswas$10,005(medianvalue$5,000).UsersnaturallyputahighervalueonBCCVLserviceswhentheyarenotsubjecttoanincomeconstraint.Asacomparison,BeagrieandHoughtonfoundthemeanwillingnesstopayforEBIwasaround$2,800(GBP1628).

    Usingthemeanwillingnesstopayvalueof$1,154,thecontingentvaluationofBCCVLbyuserswouldthereforehavebeen$1,350,180intheperiodtoDecember2016,oranannualequivalentof$852,745,risingto$13.0millionbyDecember2020,or$2.3millionperyear.(Table6.4).TheannualequivalentforeachyeariscalculatedbydividingthecumulativevaluetothatpointbythenumberofmonthselapsedsinceinceptionoftheVLandmultiplyingby12.AsusageofBCCVLbeganinJune2015,theannualvalueestimatedfor2015ishigherthanthecumulativevalue.

    Usingthehighervaluemeasuredbyusers’willingnesstoaccept,thecontingentvaluationofBCCVLwouldbe$112.5millionbyDecember2020or$20.1millionperyear.

  • 29

    Table6.4 BCCVLcontingentvaluation

    Cumulativeusers

    WTP WTA

    Cumulative Annual Cumulative Annual2015 254 293,116 502,485 2,540,000 4,354,2862016 1,170 1,350,180 852,745 11,700,000 7,389,4742017 3,002 3,464,308 1,341,022 30,020,000 11,620,6452018 5,750 6,635,500 1,851,767 57,500,000 16,046,5122019 8,498 9,806,692 2,139,642 84,980,000 18,541,0912020 11,246 12,977,884 2,324,397 112,460,000 20,142,090CVL

    ThemeanamountthatCVLuserswerewillingtopaywas$1,524(median$900),whiletheirmeanwillingnesstoacceptwas$14,130(median$20,000).

    Usingthemeanvalueof$1,524andtheestimatesofCVLusersabove,suggeststhatbyDecember2020thecontingentvalueofCVLwouldbe$9.9millionoranaverageannualvalueof$1.2million.Usingthewillingnesstoacceptvaluethiswouldbe$91.6millionbyDecember2020,or$11.5millionannually(Table6.5).

    Table6.5 CVLcontingentvaluation

    Cumulativeusers

    WTP WTA

    Cumulative Annual Cumulative Annual2013 301 611,124 611,124 5,666,130 5,666,1302014 601 1,373,124 686,562 12,731,130 6,365,5652015 1,051 2,439,924 813,308 22,622,130 7,540,7102016 1,550 3,659,124 914,781 33,926,130 8,481,5342017 2,100 5,000,244 1,000,050 46,360,530 9,272,1062018 2,705 6,475,476 1,079,246 60,038,370 10,006,3952019 3,371 8,098,231 1,156,890 75,083,994 10,726,2862020 4,103 9,883,262 1,235,410 91,634,180 11,454,271GVL

    ThemeanamountthatGalaxy-Melbourneuserswerewillingtopaywas$606(median$100),whiletheirmeanwillingnesstoacceptwas$1285(median$800).

    Usingthemeanvalueof$606andtheestimatesofallGVLusersabove,suggeststhatbyDecember2020thecontingentvalueofGVLwouldbe$11.9millionoranaverageannualvalueof$1.9million.Usingthewillingnesstoacceptvaluethiswouldbe$25.3millionbyDecember2020,or$4.0millionannually(Table6.6).

  • 30

    Table6.6 GVLcontingentvaluation

    Cumulativeusers

    WTP WTA

    Cumulative Annual Cumulative Annual2013 184 135,744 284,167 287,840 602,5662014 1,529 926,574 528,529 1,964,765 1,120,7262015 2,690 1,630,140 718,806 3,456,650 1,524,2002016 4,316 2,615,496 858,905 5,546,060 1,821,2762017 6,676 4,045,535 1,070,515 8,578,403 2,269,9862018 10,190 6,174,885 1,363,932 13,093,610 2,892,1662019 14,871 9,011,667 1,684,004 19,108,898 3,570,8672020 19,714 11,946,691 1,901,364 25,332,505 4,031,769

    Efficiencyimpacts

    Intheon-linesurveys,userswereaskedthefollowingquestionsabouttheamountoftimetheyspentdoingresearch,theshareofthattimeworkingwithdataandtheirestimatesofthetimesavingsfromusingtheVL.

    Overthelasttwelvemonths,onaveragehowmanyhoursperweekdidyouspendonresearch?

    Canyouestimatetheapproximateshareofyourtotalresearchworkingtimespentwithdataduringthelasttwelvemonths(e.g.creating,manipulatingandanalysingdata)?

    Alldata:

    approximatepercentofmytotalresearchworkingtime

    DatafromGVL:

    approximatepercentofmytotalresearchworkingtime

    Towhatextent,ifany,hasyouruseofGVLservicesandresourceschangedyourresearchefficiency(i.e.thetimesavedcomparedtothesituationifGVLdidnotexist?

    ThetimespentonresearchexpressedinhoursperweekisshowninTable6.7foreachVL.

    UsingtheaverageannualcostperusercalculatedinSection6.1,thevaluesinTable6.7canbeusedtocalculatetheannualcostperuserof(i)thetimespentonresearch,(ii)thetimespentworkingwithdata,(iii)thetimespentworkingwithVL,and(iv)theresultingincreaseinresearchefficiency.TheresultsofthesecalculationscanbeseeninTable6.7.TheaverageannualvalueperuseroftheefficiencyimpactofBCCVL,CVLandGVLare$11,898,$13,240and$23,431respectively.Asacomparison,BeagrieandHoughtonfoundthatEBIwouldbeworthGBP26,000or$44,828perpersonperannum.

  • 31

    ApplyingthesevaluestothecumulativenumberofusersforeachVLweestimatethattheefficiencyimprovementduetoeachVLisasshowninTable6.9.Moreconservatively,ifweapplythesevaluesjusttothenumberofnewuserseachyearforeachVL,thecalculatedvaluesareasshowninTable6.10.WhichevermethodisuseddemonstratesthattherearesubstantialbenefitsinresearchefficiencyarisingfromtheVLs.

    Table6.7 Surveyresponsesonresearchandefficiency

    BCCVL CVL GVLTimespentonresearch,hoursperweek mean 22.8 28.5 37.9 median 20.0 30.0 40.0Shareofresearchtimeworkingwithdata,% mean 65.1 59.9 66.4 median 70.0 50.0 75.0ShareofresearchtimeworkingwithVL,% mean 11.8 28.0 11.4 median 5.0 23.0 7.0Increaseinresearchefficiency,% mean 31.0 31.1 39.1 median 30.0 30.0 50.0

    Table6.8 Averageannualvalueofefficiencyimpactperuser

    BCCVL CVL GVLAnnualcosts 63,127 56,015 59,925 Annualcostoftimespentonresearch mean 38,381 42,572 59,925 median 33,668 44,812 59,925Annualcostofresearchtimeworkingwith

    datamean 24,986 25,500 39,790

    median 23,567 22,406 44,944AnnualcostofresearchtimeworkingwithVL mean 4,529 11,920 6,831 median 1,683 10,307 4,195Increaseinresearchefficiency mean 11,898 13,240 23,431 median 10,100 13,444 29,963

    Table6.9 ValueofVirtualLaboratoriesonresearchefficiency,usingcumulativeusers,$

    Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 5,309,154 4,311,244 0 5,309,154 4,311,2342014 0 11,929,046 35,825,502 0 5,964,523 17,912,7072015 3,022,137 21,196,896 63,028,516 1,007,386 7,065,632 21,009,4542016 13,920,867 31,788,724 101,126,793 3,480,241 7,947,181 25,281,6362017 35,718,327 43,439,735 156,423,186 7,143,715 8,687,947 31,284,5602018 68,414,518 56,255,846 238,758,578 11,402,498 9,375,974 39,792,9992019 101,110,708 70,353,569 348,437,568 14,444,486 10,050,510 49,776,673

  • 32

    2020 133,806,898 85,861,065 461,912,327 16,725,977 10,732,633 57,738,899

    Table6.10 ValueofVirtualLaboratoriesonresearchefficiency,usingnewusers,$

    Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 3,971,935 3,374,009 0 3,971,935 3,374,0092014 0 6,619,892 31,514,180 0 3,309,946 15,757,0902015 3,022,158 9,267,849 27,202,947 1,007,386 3,089,283 9,067,6492016 10,898,805 10,591,828 38,098,184 2,724,701 2,647,957 9,524,5462017 21,797,610 11,651,011 55,296,257 4,359,522 2,330,202 11,059,2512018 32,696,415 12,816,112 82,335,190 5,449,402 2,136,019 13,722,5322019 32,696,415 14,097,723 109,678,720 4,670,916 2,013,960 15,668,3892020 32,696,415 15,507,495 113,474,480 4,087,052 1,938,437 14,184,310

    AdditionalresearchmadepossiblebyVirtualLaboratories

    Theon-linesurveyaskedusersthefollowingquestion(oranequivalentvariant)abouthowimportanttheVLwastotheirresearch

    IfGVLhadnotexisted,wouldyouhavebeenabletoobtaintheresourcesyoulastusedfromanothersource?

    ForBCCVL,CVLandGVLtheproportionofuserswhoanswered“No”tothisquestionwas55.26%,29.03%and34.3%respectively.ItisclearthereforethattheVLsareimportantagentsenablingresearchintheirfields.Againbywayofcomparison,about45%ofEBIusersalsoindicatedthattheycouldnothaveproceededintheirresearchwithoutEBI.IfweassumethatthesepercentagestranslateintoresearchenabledbytheVLswhichwouldnototherwiseoccur,thisimpliesthatwecancalculatethevalueoftheresearchundertakenbyVLusersbymultiplyingthenumberofusersbytheirannualaveragecostandapplyingthepercentagesquotedabove.Aswiththeprevioussectionwepresenttworesults.Thefirstusersthecumulativenumberofusersforthecalculationandthesecondusesthenumberofnewusers.Tables6.11and6.12showtheresults.Aswiththevalueofresearchefficiency,thisdemonstratesthattherearesubstantialbenefitsarisingfromtheVLsinenablingresearchwhichwouldotherwisenothaveoccurred,whichevermethodisused.

  • 33

    Table6.11 ValueofVirtualLaboratoriesinenablingadditionalresearch,usingcumulativeusers,$

    Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 6,520,762 3,781,977 0 6,520,762 3,781,9772014 0 14,651,388 31,427,409 0 7,325,694 15,713,7052015 8,860,592 26,034,265 55,290,864 2,953,531 8,678,088 18,430,2882016 40,814,537 39,043,267 88,712,033 10,203,634 9,760,817 22,178,0082017 104,722,428 53,353,169 137,220,003 20,944,486 10,670,634 27,444,0012018 200,584,264 69,094,061 209,447,547 33,430,711 11,515,677 34,907,9252019 296,446,101 86,409,042 305,661,872 42,349,443 12,344,149 43,665,9822020 392,307,937 105,455,522 405,205,981 49,038,492 13,181,940 50,650,748

    Table6.12 ValueofVirtualLaboratoriesinenablingadditionalresearch,usingnewusers,$

    Cumulative Annualaverage BCCVL CVL GVL BCCVL CVL GVL2013 0 4,878,376 2,959,808 0 4,878,376 2,959,8082014 0 8,130,626 27,645,432 0 4,065,313 13,822,7162015 8,860,592 11,382,877 23,863,455 2,953,531 3,794,292 7,954,4852016 31,953,945 13,009,002 33,421,169 7,988,486 3,252,250 8,355,2922017 63,907,891 14,309,902 48,507,970 12,781,578 2,861,980 9,701,5942018 95,861,836 15,740,892 72,227,545 15,976,973 2,623,482 12,037,9242019 95,861,836 17,314,981 96,214,325 13,694,548 2,473,569 13,744,9042020 95,861,836 19,046,480 99,544,109 11,982,730 2,380,810 12,443,014

    ReturnstoresearchactivitiesmadepossiblebyVirtualLaboratories

    Becausetechnologyisimportantinthecontinuingdevelopmentofnewproductsandimprovedproductivity,theroleofresearchanddevelopmentanditscontributiontotheeconomyhasbeenstudiedwidely.AreviewofthesestudieshasbeenprovidedinSection3abovewhichconcludedthata40%annualreturnoninvestmentisaconservativeestimateofthevalueofresearch.TakingthevalueoftheadditionalresearchduetotheVLsfromTable6.12wecalculatethevalueofthisresearchfollowingtheproceduresetoutbyBeagrieandHoughton.Weassumethattheusefullifeofanyresearchis30yearsandwedepreciatethevalueat3.4%ayearsothatthevalueinyear30iszero.Foreachoftheyears2013to2020asshowninTable6.12,wecalculatetheresultingbenefitsstreambyapplyingthismethodology.Thiswillgenerate8benefitsstreamswhichwesumforeachoftheyears2017to2047.Wethendiscountfuturebenefitstreamsusinga3%discountrateandcalculateanetpresentvalue(NPV)ofthesummedbenefitsstreams.FinallywecanexpresstheNPVofthesebenefitsasanannualised

  • 34

    valuebydividingby30.BoththetotalNPVandtheannualisedvalueareshownforeachVLinTable6.13.

    Table6.13 ReturnstoadditionalresearchmadepossiblebyVirtualLaboratories$m.

    30yearbenefit AnnualisedbenefitBCCVL 2,277.8 75.9CVL 609.5 20.3GVL 2,356.0 78.5

  • 35

    7.FurtherassessmentoftheimpactandvalueoftheCharacterisationVirtualLaboratory

    DifferentVirtualLaboratories(VLs)sharethesamepurpose,whichistofacilitatedataanalysis,provideresearcherswithITresourcesandpromotecollaborativeresearch.WheretheVLsdifferisintheirscopeandimplementation.AmajorpointofdifferenceforCVLisitsmethodofoperationbecauseitworksprimarilywithinstrumentfacilitiesratherthanendusers.Bydevelopingdatacapture,analysisandvisualisationworkflowsdirectlywithinstrumentfacilities,CVLaimstounderpintheentirefacilityusercommunity.Thisintegrationenablesseamlesstransferofdatafromtheinstrumentsforstorageandanalysis.

    TheCVLmethodofoperationandeaseoftransfermeansthatusersarenotalwaysawarethattheyareusingCVL,andmanyconsiderthecapabilitiesprovidedbyCVLasacomponentoftheinstruments,ratherthanasaseparateservice.LowawarenessofCVLamongitsusersraisesconcernsabouttherepresentativenatureofresponsesfromtheCVLusersurvey.

    IncollaborationwithDrWojtekGoscinski,themanagerofCVL,thecurrentstudyusedanadditionalapproachtoevaluateCVLfromtheperspectiveofmanagersfromanumberofimagingfacilities.Imagingfacilitieshouseandmaintainadvancedimaginginstruments,andtheyrepresentconsiderablenationalinvestmentinAustralia’sresearchinfrastructure.AlistofimagingfacilitiesthatutiliseCVLservicesisgiveninAppendix2.Todate,CVLhasbeenintegratedwith69instrumentsin26imagingfacilities.

    TheinterviewsconductedwithfacilitymanagersprovidedaglimpseintodiverseaspectsofCVLvalue.MuchofthatvalueisadditionaltothedirectbenefitofCVLtoindividualusers.Theadditionalbenefitsincludedeconomiesofscale,thepotentialforscalingupCVLapplicationandtheintrinsicvalueofCVLasadigitalresource.Thesedifferenttypesofvalueareoutlinedbelow.

    (i) AcentralisedHPCcapabilitycutscostsbyreducingHPCreplicationacrossresearchorganisations

    ImaginginstrumentscangeneratevastamountsofdatathatrequiresextensiveHPCcapabilitiesforanalysis.DrGeorgRamm(CliveandVeraRamaciottiCentreforStructuralCryo-ElectronMicroscopy,MonashUniversity)managesanelectronmicroscopyfacilitythatcontainsmanysuchinstruments,mostnotablytheTitanKrioscryo-electronmicroscope,whichisusedtypicallyfor2Dand3Dcharacterizationofcellularormolecularstructuresanditisparticularlydataintensive.Therearearound100usersatthecentre,ofwhomapproximatelyhalfrelyontheHPCcapabilityofCVLonMASSIVEfordatastorage,whiletheremainderrequiretheanalyticalcapacityofCVLfordataprocessing.

    CentralisingHPCatthelevelofeachimagingfacilitywouldbeacheaperalternativetouser-fundedHPC,thoughstillconsiderablymoreexpensivethanthecostofmaintainingCVL:

  • 36

    • Asmallcomputingclusterwouldcostaround$200,000anditwouldneedtobeupdatedwithin5years.Salarycostsforpart-timePhD-levelstafftomanagetheclusterwouldaddaround$50,000peryeartotheoutlay,bringingthetotaltoaround$90,000peryear.

    • DrAndrewJanke(CentreforAdvancedImaging,UniversityofQueensland)estimatedCVLreplacementcostat$250,000.Theamountwasbasedonthecostofsettingupanin-houseplatformforthecentre.

    • ProfSampson(CentreforMicroscopy,CharacterisationandAnalysis,UWA)estimatedthatwhenimplementedatfullcapacity,thevalueofCVLservicestothecentrecouldwellbeworth$50,000peryear.Atthisstage,flowandmasscytometeristheonlyinstrumentintegratedwithCVL,althoughintegrationwithotherinstrumentsisbeingrolledout.

    • DrIanHarperfromMonashMicroImaging(MMI)estimatedthecostofreplacingtheaccesstoHPCthroughCVLat$20,000peryear.

    UsingDrHarper’smostconservativeestimate,itwouldcostover$500,000peryeartoreplaceHPCservicescurrentlyprovidedbyCVLto26facilities.ThisfigureiswellabovetheCVLrunningcostof$177,000-$320,000peryear(Table2.2).TheestimatedreplacementcostisforHPCalone;itdoesnottakeintoaccountthevalueofallotherservicesprovidedbyCVL,whichareoutlinedbelow.

    (ii) PotentialforscalinguptheCVLplatform

    CVLhasbeenadoptedwidelyinarelativelyshorttime.Inlessthanfouryearssinceitsinception,ithasbeenintegratedwith69instrumentsacross26imagingfacilities.TheexpansionofCVLisongoing.Inaddition,CVLintegrationwithcommonlyusedtechnologyhasthepotentialforscalingup.Forexample,theintegrationoftheflowcytometerattheCentreforMicroscopy,CharacterisationandAnalysisinUWA,isexpectedtobereplicatedwidely.Flowcytometryisincreasinglyusedforcellcountingandsortingnotonlyinresearch,butalsoinmajorhospitals.Thereareover20machinesandaround100usersofflowcytometryinWA,over100machinesinAustraliaandaround5,000worldwide.IntegrationwithCVLhasthepotentialtobereplicatedinothercentres,attractingmoreuserstotheplatform.

    Lightmicroscopyisanotherwellestablishedandcommontechnology.DrIanHarperfromMonashMicroImaging(MMI),outlinedplanstointegrateMMIinstrumentswithCVL.MMIisalightmicroscopyfacilitythatcontainsinstrumentsvaluedataround$50million,andwhichattractapproximately500users.MMIislocatedonthreecampuses:MonashUniversity(Clayton),AlfredHospitalandMonashMedicalCentre.AtthisstageonlytheinstrumentsattheClaytoncampusareintegratedwithCVL.Withinthecomingyear,40-48MMImicroscopesatallthreecampusesareexpectedtobeintegratedwithCVL.

  • 37

    (iii) Increasingtheefficiencyofresearchandtheimpactofresearchinvestment

    Inprovidingaccesstosecurestorageforexperimentaldata,CVLcontributestoimprovedresearchoutcomes.DrIanHarperfromMonashMicroImaging(MMI)describedproblemswithdatalosspriortointegrationwithCVL,whenUSBdatatransferwasused.DrRamm(CentreforStructuralCryo-ElectronMicroscopy)alsorecalledprolongedperiodsofdowntimewhenusersbroughttheirownharddrivestodownloaddata,makinginstrumentsvulnerabletoattackbycomputerviruses.

    Theabilitytostoreallexperimentaldatasecurelyallowsresearcherstointerrogateolddatainsubsequentyearstoanswernewresearchquestions.TheabilitytosharedatawithcollaboratorsthroughtheCVLplatformalsomeansthatthesamedatacanbeinterrogatedbyotherresearchers.Reusingexperimentaldatainthiswayandimprovestheefficiencyandimpactofeachexperiment;italsoreducestheneedtorepeatexperiments

    Increasingreproducibilityofresearchfindingsisanotherwaytoimprovetheefficiencyandimpactofexperimentalresearch.DrAndrewJanke(CentreforAdvancedImaging,UQ)emphasisedthatoneofthemainbenefitsofCVLtohisfacilityisthatitfacilitatesreproducibility.DrJankeusedtheexampleof“quarantining”supersededversionsofanalyticalsoftware,suchasFSL,apackageofimageanalysistoolsusedforMRIbrainimagingdata.AfterareleaseofupdatedFSLolderversionsarenolongeravailable.Thisoftenmeansthatresultsofearlierexperimentscannotbereplicated.Arecentstudyestimatedthatfewerthan16%ofneuroimagingresultsarereproducible(Russelletal2017).CVLpreservesallreleasesofthesoftware.

    DrCeguerra(AustralianCentreforMicroscopy&Microanalysis,UniversityofSydney)notedthatintheabsenceofCVL,thereturnonthe$5millioninvestmentintheAtomProbewouldbegreatlydiminishedintermsofresearchoutcomes.TheAtomProbeenables3Dimagingandchemicalcompositionmeasurement.Becauseofthenoveltyofthistechnology,therearefewanalyticaltoolsavailableandusersarerequiredtodeveloptheirownmethodsandsoftware.TheAtomProbeworkbenchdevelopedbyCVLactsasarepositoryofanalyticalsoftwarethatisdevelopedbyusers.Thisrepositoryrepresentsasignificantinvestmentinresearchtime–over14yearsofeffortfrommultiplepeople.AccordingtoDrCeguerra,withoutCVLsomeprojectswouldbesetbackbyasmuchasfiveyears,withtheresearchershavingtodevelopthetoolstheyneedbythemselves.CVLisalsocriticalfordisseminatingresearchoutcomesintheAtomProberesearchcommunity.

    (iv) ValueofCVLasadigitalresource

    AnotherimportantaspectofthevalueofCVListhedevelopmentofitsintrinsicworthasadigitalresourceis.DrCeguerraemphasisedtheimportanceofCVLasaresourceforthesmallcommunityofcommunityofresearchersworkingwiththeAtomProbe.Thereare

  • 38

    closetoonehundredsuchinstrumentsworldwide2witharound500-600researcherstrainedtousethem,34ofthematthefacilityinSydney.Despiteitssmallsize,thiscommunityrepresentanimportantfieldofresearchthathasproducedfar-reachingdiscoveriesinmaterialsengineering.

    Asalreadymentioned,theatomprobeisnoveltechnologywithoutacanonofwell-establishedanalyticalmethods.Ithasfallentouserstodevelopthemethodstheyneed.Therepositoryofuser-generatedsoftwareontheCVLAtomProbeworkbenchhasnoequivalentandhasbecomethekeyplatformforAtomProberesearchworldwide.CVLisalsocriticalfordisseminatingresearchoutcomesintheAtomProberesearchcommunity.ThevaluationoftheintrinsicvalueofVLsasadigitalresourceisoutlinedinmoredetailinSection8ofthisreport.

    DrAndreJankesaidthatCVLprovidednoviceuserswithaneasytonavigatemechanismfortorapidlygetuptospeedwiththeiranalysiswithouttheneedtoinstallorconfiguresoftware,somethingthatpreviouslytookalargeinvestmentoftime.DrJankeestimatedthesavingintimeasapproximatelythreemonths.Thattimeisvaluedatover$19,000juniorforapostdoctoralfellow(commencementlevelAcademiclevelAsalaryis$77,819,Table6.1)or$6,670foraPhDstudent(MonashUniversityPhDannualstipendsare$26,682)

    Interviewswithimagingfacilitiesmanagersprovidedaninsightintofar-reachingbenefitsofCVL,whichwouldnotbeapparentfromsurveyresponsesofindividualusers.ThecategoriesofvalueidentifiedinthiswaywouldbeequallyapplicabletotheotherVLs.Inasimilarway,BCCVL,GVLandHuNIoffereconomiesofscalethroughcentralisingITresources;andaimtodevelopvalueasdigitalresources,toimprovetheefficiencyandimpactofresearchandtoexpandtheirservicesoutsidethehostorganisationsandoutside

    2AtomProbefacilitiesaroundtheworld(http://www.atomprobe.com/2ndLinks/apt-internet-sites.aspx

  • 39

    8.QualitativevaluationofNectarVirtualLaboratories

    TheimpactandvalueofNectarVLscalculatedintheSection6arenecessarilyrestrictedtothoseforwhichthekeyinputstothecalculationcanbereadilyquantifiedandexpressedineconomicterms.

    ItisclearhoweverthattheactivitiesofVLsextendwellbeyondthisintoareaswhicharemoredifficulttoquantify.WeattempttocapturesomeofthequalitativebenefitsofNectarVLsusingtheBalancedValueImpactModelframeworksuggestedbyTanner(2012).Table8.1providesabriefdescriptionofthesetypesofvalue.Table8.2attheendofthissectionsummarisesimportantaspectsofthevaluesdescribedinTable8.1forBCCVL,CVLandGVL.

    Table8.1 TypesofvalueinBalancedValueImpactframework

    TypeofValue Definition

    UtilityValue Peoplevaluetheutilityaffordedthroughuseofthedigitalresourcesnoworsometimeinthefuture.

    Existenceand/orPrestigeValue

    Peoplederivevalueandbenefitfromknowingthatadigitalresourceischerishedbypersonslivinginsideandoutsidetheircommunity.Thisvalueexistswhethertheresourceispersonallyusedornot.

    EducationValue Peopleareawarethatdigitalresourcescontributetotheirownortootherpeople’ssenseofculture,education,knowledgeandheritageandthereforevalueit.

    CommunityValue Peoplebenefitfromtheexperienceofbeingpartofacommunitythatisaffordedbythedigitalresource.

    Inheritance/BequestValue Peoplederivebenefitfromtheinheritancepasseddowntothemandsatisfactionfromthefactthattheirdescendantsandothermembersofthecommunitywillinthefuturebeabletoenjoyadigitalresourceiftheysochoose.

    BasedonTanner(2012)p37

    Utilityvalue

    TheprecedingsectionhasshownthevaluethatusersputonVLservicestotheextentthatthiscanbeexpressedineconomicterms.

  • 40

    Intheon-linesurveysuserswereaskedthefollowingquestion

    TowhatextentdoyoubenefitfromusingVLinanyofthefollowingways?

    Haven'tused,Nobenefit,Lowbenefit,Mediumbenefit,Highbenefit,Veryhighbenefit

    Table8.2 ExtentofbenefitsfromVirtualLaboratories,%ofrespondents

    Services BCCVL CVL GVLMediumandhigher

    Highandveryhigh

    Mediumandhigher

    Highandveryhigh

    Mediumandhigher

    Highandveryhigh

    Data&tools 73.8 57.4 72.5 70.0 60.6 44.8

    Collaborations 23.0 11.5 50.0 30.0 48.7 25.0

    Training 37.7 27.9 27.5 15.0 51.2 32.9

    Usersupport 27.9 19.7 --- --- 36.8 29.0Other 27.9 19.7 7.5% 2.5% 36.8 29.0

    Intheon-linesurveysuserswereaskedthefollowingquestion

    WhatimpactwouldithaveonyourworkorstudyifyoucouldnotaccessVLservicesandresources?

    Noimpact,Slightimpact,Moderateimpact,Majorimpact,Severeimpact

    ForBCCVLusers18.2%answeredthatiftheycouldnotaccesstheservicesandresourcesprovidedbytheVLthiswouldhaveamajororsevereimpact.ForCVLandGVLthepercentageswerehigherat52.8%and35.7%respectively(Table8.3).

    Howevertakingmoderate,majorandsevereimpactassignificant,thepercentagesofrespondentswere41.8%,75.0%and72.9%.

    Table8.3 ImpactifcouldnotaccessVirtualLaboratoryservices,%

    BCCVL CVL GVLNoimpact 23.6 5.6 11.4Slightimpact 34.5 19.4 15.7Moderateimpact 23.6 22.2 37.1Majorimpact 12.7 27.8 27.1Severeimpact 5.5 25.0 8.6 Majorandsevereimpact 18.2 52.8 35.7Moderate,majorandsevereimpact 41.8 75.0 72.9

  • 41

    CVL

    WeconductedphoneinterviewswiththemanagersofanumberofimagingfacilitiesthathaveworkedcloselywithCVLonintegratingtheirfacilitieswiththeon-linestorageandcomputingfacilitiesofferedthroughCVL.

    TheyprovidedthefollowingdescriptionsofthevalueofCVLtotheirfacilities.

    DrKeithSchulze,ImageAnalyst,MonashMicroImaging,MonashUniversity

    MyDataandStore.MonashrepresentasignificantadvanceindatahandlingatMonashMicroImaging.Transferandstorageofuserdataisnowreliable,seamlessandsecure.Store.Monashprovidesuserswiththeabilitytosharedatawithcolleaguesandcollaboratorsinaconvenientandsecuremanner.Italsohasmechanismstocaptureandexposeimportantmetadatathatallowforbetterreuseandreproducibilityofdata.

    Next-generationimagingtechnologies,liketheLatticelight-sheetmicroscope,arecapableofproducingdataatarateofTerabytesperhour.ThetoolsforautomatedhandlingandstorageofdataprovidedbyCVLareakeyenablerforresearcherstoderivethemostbenefitfromtheselargedatasetsi.e.,theyspendlesstimestrugglingwithdatatransfersandmoretimeextractinginterestinginformationfromtheirdata.Moreover,CVLandNectarprovideacrucialplatformonwhichtoolstoanalyseandvisualthisdatacanbedevelopedanddeployed.

    DrAndrewMehnert,GroupLeader–DataManagement,AnalysisandVisualisation,CentreforMicroscopyCharacterisationandAnalysis,TheUniversityofWesternAustralia

    TheCentreforMicroscopy,CharacterisationandAnalysis(CMCA)isaUniversityfacilitythatcollaboratesinmicroscopyandcharacterisation,supportingresearchexcellencelocally,nationally,andinternationally.Itcomprises~40staff,morethan500usersandhosts~50instrumentplatformsacrosscytometry,opticalmicroscopy,micro-magneticresonanceimaging(microMRI),preclinicalbioimaging,electronmicroscopy(EM),X-raymicro-computedtomography(microCT),secondaryionmassspectrometry,bio-organicmassspectrometry,X-raydiffractionandnuclearmagneticresonance(NMR)spectroscopy.TheCMCAcollaboratesinandsupportsresearchacrossbiologicalscience,biomedicalscience,earthscienceandphysicalscience.

    CMCAusersareacquiringever-largermulti-dimensionaldatasetsandareincreasinglyusingmultipleinstrumentplatforms;e.g.microCTtogetherwithEMandRamanspectroscopy.ThispresentsaBigDatachallenge,notonlyintermsofmanaging/curatingthisdataovertheresearchlifecycle,butalsointermsof

  • 42

    analysingthisdatatofacilitatenewdiscoveries.TosolvethischallengetheCMCAisleveragingtheCharacterisationVirtualLaboratory(CVL)andCVL-supportedservicesincludingMyTardis.Overthelast12monthstheCMCAhasbeenworkingwithMonashUniversitytodeveloptheCytometryworkbench(collectionofsoftwaretools)fortheCVLtosupporttheanalysisofdatafromimage-,mass-andflow-cytometryinstruments.TheCMCAhasalsointegratedseveralEMinstrumentswithMyTardisandisintheprocessofintegratingitsNIFflagshipMRIscanner.TheCMCAplanstointegrateallofitsinstrumentswithMyTardisandtodevelopadditionalworkbenchesandworkflowsintheCVLinsupportofitsusers/researchers.TheCVLandsupportedservicesrepresenttheonlyviablesolutionacrossmultipleinstrumentplatformsandmodalities.Moreover,lookingtothefuture,theCVLoffersasolutiontotheanalysisandvisualisationofverylargedatasetsusingtoolsandcomputeresourcesnototherwiseavailablefromadesktopworkstation.

    DrAndrewJanke,InformaticsFellow,NationalImagingFacility,UniversityofQueensland

    ThevalueofCVLandmanagedworkbenchesinneuroimagingandpreclinicalimagingtotheNationalImagingFacility(NIF)ispredominatelyaroundencouragingouruserstoperformreproduciblescience.CVLprovidesatoolinwhichresearcherscanprocessandanalysetheirimagingdatasafeintheknowledgethatiftheyneedtoreproducetheanalysisin5years’timethesameversionsofsoftwarewillstillbeavailable.

    Traditionallythishasnotbeenthecaseinimagingresearchandsoftwarepackagesandoperatingsystemsareamovingtarget.CVLprovidesamechanismtofreezeintimeaparticularanalysispipeline.InadditionCVLprovidesNIFwithaneasytonavigatemechanismfornoviceuserstorapidlygetuptospeedwiththeiranalysiswithouttheneedtoinstallorconfiguresoftware,somethingthatpreviouslytookalargeinvestmentofNIFpersonneltime.

    Educationvalue

    Intheon-linesurveysBCCVLandGVLuserswereaskedthefollowingquestion

    HowdidyoulearntousetheservicesprovidedbytheVL?

    Asshowninthetablebelow,53.8%ofBCCVLusers,50.0%ofGVLusersrelyontherespectiveVLstolearntheanalyticaltoolsrequiredfordataanalysisintheirrespectivefields.

    AsisevidentfromTable8.4,animportantaspectofbothBCCVLandGVListhetrainingtheyprovidetoresearchersandstudents.

  • 43

    Table8.4 TrainingprovidedbyVirtualLaboratories,%ofrespondents

    BCCVL GVLAlreadyproficient 11.5 21.1TutorialthroughtheVLwebsite 21.3 18.4Face-to-facecourseprovidedbyVLpersonnel 32.8 9.2Face-to-facecourseprovidedbysomeoneelse 0.0 22.4Anotherway 16.4 15.8Missing 18.0 13.2Total 100.0 100.0

    OneoftheimportantbenefitsofGVListoprovidetouserstheresourcestheyneedwhichotherwisewouldrequireadvicefromexperiencedbio-informaticians.Thesupplyoftrainedbio-informaticiansisacriticalbottleneckinexpandingresearchwithinthefieldofgenomics,soenablingresearcherstobypassthisbottleneckisanimportantcontributionthatGVLmakes.

    OvertheperiodtoFebruary2017,GVLhasundertakentrainingcoursesfor313users,whogiveanaverageratingof4.5outof5forthesecourses.

    Benefitstousers

    BCCVL

    InitsSnapshotfor2016,BCCVL(2016)providesthefollowingquotesfromusers.

    ...becauseitprovidesanenvironmentforexperimentingwithmodels,andalsokeepscarefultrackoftheseexperiments,BCCVLalsoencouragesbestpracticeandtransparencyinmodelling.

    Withitseasytouseinterface,accessiblefromanywhere,itopensthefieldtoawholenewarrayofresearcherswhounderstandthesystemstheyareworkingon,butdonothavethetechnicalskill-setsorhardwaretoproperlyanswertheirquestions.

    …essentiallytheBCCVLhasenabledustoaskquestionsthatwecouldn’taskbefore–questionswemayhavewantedtoaskbutcouldn’tlogisticallyhopetoanswer,soit’sopenedupawholenewfieldofenquiry.

    CVL

    Initsannualreportfor2015-16,theMassiveprojectatMonashUniversitynotedthat

    CVLhasunderpinnedworkflowdeploymentattwoAustralianSynchrotronbeamlinesforaccesstotheMASSIVEDesktop.ResearchersvisitingboththeImagingandMedicalBeamline(IMBL)andtheX-rayFluorescenceMicroscopy(XFM)beamlineareautomaticallycreatedabeamlinevisitproject,anduseraccounts.Authenticationis

  • 44

    integratedwithAustralianSynchrotronsothatuserscanusetheirAScredentialstologintothesystem.Asetofbeamlinechangeoverscripts,developedincollaborationwithAustralianSynchrotron,controltheflowofdata.Theimpactofthisworkissignificant.Researchersareprovidedwithaccesstoaremotedesktopenvironmentforthedurationoftheirbeamlinevisitthatprovidesaccesstothedatacaptured,andtoolsfordataprocessingandvisualisation

    Additionally,CVLsoftwareisnowbeingadoptedbytwomajorinternationalsupercomputingcentres:

    >JulichSupercomputingCentre,isintheprocessofdeployingStrudeltosupportvisualisationusersandhigh-endcommercialengineeringapplications;

    >EdinburghParallelComputingCentreforindustryaccesstoHPCsystemsundertheEUFP7project,Fortissimo.

    WithCVLNectarfunding,MonashUniversityisdevelopinganinstrumentintegrationapp,calledMyDatatomakeintegrationquicker,simplerandlessreliantonspecialistITsupport.AsaresultofthisprojectMyDataisnowusedat35instruments,across10facilities,at4institutions,inaddition17furtherinstrumentsareplannedat4institutions.Inaddition,thisprojecthasmadeMyTardiseasiertouseforFacilityManagers.(p23)

    HuNI

    DescriptionsbytwoHuNIusersofitsvalue,whichincludesnewapproachestomulti-disciplinaryresearchandconnectingwithotherresearchers.‘Userscancapturerelationshipsbetweencontent,buildpathwaysandstructures,andshareanddistributedata.Itprovidesaplatformwhereserendipitouscollaborativeopportunitiescanquicklyemerge’(User2).

    User1(literarystudies)

    Iworkintheareaofliterarystudies,particularlyliteraryandculturalhistoryinthenineteenthcentury,andmyresearchcutsacrossarangeaphysicalarchivesandlibrariesaroundtheworld.WhileIuseanumberofdigitaltoolsinmyresearch(particularlyasdigitisationhasopenedupaswatheofhistoricprintedtexts),HuNIopenedupmythinkingandapproachestoresearchinwaysthattheseindividualtoolshadneverencouraged.Inparticular,HuNIinnatelyencouragesuserstoidentifytheinterdisciplinaryconnectionswithintheirresearchfield.Asourresearchmethodshavelargelybeenshapedwithinourdisciplinarysilos,IwasnotevenawareofsomeofthedatasetsthatHuNIdrawsonbutwhichprovedtobesupremelyusefultomyresearchproject.Moreover,HuNIrepresentsarealstepforwardinefficiencyinthatsimpleawarenessoftheseotherdatasetsontheirownwouldstillmeanincreasedlabourin

  • 45

    individuallylocatingandsearchingthem.PerhapsmostexcitingandunexpectedofallwastheabilityofHuNItoconnectmyideasandresearchwiththeworkofotherresearchersthroughitsvisualisationcapabilities.Thesekindsofassociationsinrelationtospecificresearchsites(particularplays,books,orpublicfigures,forexample)aresimplynotpossibleinanyotherforumapartfromtherandomnetworksweformasacademics.Inthisrespect,HuNIclearlyhasthepotentialtoradicallytransformhowweconductourresearchinrelationto—andpotentiallyintandemwith—otherscholarsandmembersofthepublic.

    User2(Archivist)

    HuNI(HumanitiesNetworkedInfrastructure)makesasignificantandvaluablecontributiontoAustralia’sinformationinfrastructure.Inbringingtogether30curated,authoritativeandscholarlydatasetsinawell-designedonlineresource,HuNIsupportsdiverseresearchandprovidesresearcherswithdiversecontentwithoutthenoiseoflargeaggregators.Throughthis,HuNIalsohelpsincreasethevisibilityanduseofthese30foundationalhumanitiesresourceswhichtogetherconstituteasubstantialinvestmentofresourcesandscholarshipspanningmorethantwodecades.

    Moreimportantly,unlikeaggregatorsand‘portals’,HuNIbringsthiscontentintoaresearchplatformwhichsupportsthetypeofrelational,non-hierarchical,iterativeengagementwithcollectionsanddatathatiscentraltocontemporaryhumanitiespractice.Userscancapturerelationshipsbetweencontent,buildpathwaysandstructures,andshareanddistributedata.Throughthis,HuNInotonlyfostersserendipitousdiscovery,itprovidesaplatformwhereserendipitouscollaborativeopportunitiescanquicklyemergeinanonlineenvironment.Astheuserbasegrows,thisaspectisincreasinglylikelytosparkbroadercollaborativeworkbyhumanitiesresearchers,contributingtothestrengthanddiversityofthesectorasawhole.

  • 46

    Table8.5 SummaryofbenefitsprovidedbyBCCVL,CVLandGVLusingBalancedValueImpactframework

    Valuedriversandindicators BCCVL CVL GVL

    1.UtilityValue• Reportedbenefitfromaccesstodataandtools1

    ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)

    45(73.8%)35(57.4%)

    29(72.5%)28(70.0%)

    46(60.6%)34(44.8%)

    2.Existenceand/orPrestigeValue• reachacrossdifferentorganisations[e-mailsuffixes]2

    NumberofUniversitiesNumberofResearch/Not-for-profitorganisationsNumberofGovernmentagencies/departmentsNumberofCompanies

    • ReachoutsideAustralia2Numberofcountries(includingAustralia)Australia(numberofusers)Othercountries(numberofusers)

    • recognitionbyprovidersofotherdigitalplatforms

    70(46.7%)20(13.3%)38(25.3%)22(14.7%)201005(94.8%)55(5.2%)Seetext

    46(49.5%)33(35.5%)7(7.5%)7(7.5%)152,388(93.4%)170(6.6%)Seetext

    24(64.9%)8(21.6%)--4(10.8%)1665NoneSeetext

    3.EducationValue• trainingtroughcoursesbyVLpersonnel/VLwebsite

    • VLtraining

    ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)

    • satisfactionsurveysforGVLcourses(meanscoreoutof5)

    33(54.1%)23(37.7%)17(27.9%)--

    --

    11(27.5%)6(15.0%)--

    21(27.6%)39(51.2%)25(32.9%)4.5

    4.CommunityValue• useofVLplatformsforcollaboration

    ofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)

    • usersupportofbenefit(numberofrespondents)ofhighbenefit(numberofrespondents)

    14(23.0%)7(11.5%)17(27.9%)12(19.7%)

    20(50.0%)12(30.0%)

    ----

    37(48.7%)19(25.0%)28(36.8%)22(29.0%)

    5.Inheritance/BequestValue• VLusersuploadnewdatasetstoaVL(numberofrespondents)• NumberofdatasetsuploadedbyDec2016• Repositoryofuser-writtenalgorithms

    14(22.9%)

    ----

    --

    35,787yes5

    22(29.0%)

    --yes

    1SurveyofVLusers2Informationderivedfromusere-mailsuffixes3Studentfeedbackfromparticipantsin31GVLtrainingcourse(informationprovidedbyVL

  • 47

    9.Summaryandconclusions

    TheNectarfundingforthreeVLsexaminedinthisstudyhashaddemonstrablebenefits,whenmeasuredinbothquantitativeandqualitativeterms.ForanexpectedexpenditurebyNectarandpartnersofaround$4.4to$7.0milliontotheyear2020(oraround$550,000to$870,000inannualterms)thethreeVLshavegeneratedeconomicvalueexceedingthis.

    Table9.1summarisesthecalculationofeconomicbenefitsexpressedinannualtermsandcomparesthistotheannualisedcostofeachVL.

    Table9.1 SummaryofeconomicbenefitsandcostsforVirtualLaboratories

    Annualisedbenefit,$

    Annualisedcost,$

    Ratio

    Contingentvaluation Willingnesstopay

    BCCVL 2,324,397 550,754 4.2CVL 1,235,410 612,356 2.0GVL 1,901,364 869,479 2.2

    Willingnesstoaccept BCCVL 20,142,090 550,754 36.6CVL 11,454,271 612,356 18.7GVL 4,031,769 869,479 4.6

    Efficiencyimpacts BCCVL 4,087,052 550,754 7.4CVL 1,938,437 612,356 3.2GVL 14,184,310 869,479 16.3

    Additionalresearchimpact BCCVL 11,982,730 550,754 21.8CVL 2,380,810 612,356 3.9GVL 12,443,014 869,479 14.3

    Returnstoadditionalresearch BCCVL 75,925,851 550,754 137.9CVL 20,318,154 612,356 33.2GVL 78,532,855 869,479 90.3

    TheratiosforallVLsarehigherthanoneindicatingthattheyallgeneratebenefitsinexcessoftheircosts.ForBCCVLandGVLtheratiosareconsistentlyhighacrossallmeasuresofvalueandarehigherthanthoseoftenachievedbyconventionalphysicalinfrastructure.ThestudyindicatesthattheVLscontributevalueindifferentways.Forinstance,theefficiencyimpactratioforGVLishigherthanthatforBCCVL,thecontingentvaluationratioandtheadditionalresearchratioarehigherforBCCVLthanforGVL.ThesevariationsreflectdifferencesinthewayinwhichtheVLsdelivervaluetotheirusers.

  • 48

    Takingawiderperspectiveonbothquantitativeandqualitativebenefits,itisclearfromthecasestudyofCVLthatithasgeneratedconsiderablebenefitsfortheimagingfacilitieswithwhichitworks.Thesefacilitieswouldotherwiseincurconsiderablecosts(discussedinSection7butnotfullycapturedbythestudy)inreplicatingtheservicestheyprovide.ThefacilitymanagershavealsoprovidedevidenceofthewiderbenefitsofworkingwithCVL.WehaveusedtheBalancedValueImpactframeworktodescribecharacteristicsofVLsthatprovidedarangeofbenefitstousersandcommunities.ThisapproachcouldbeadaptedfurtherforprovidinginsightintothevalueandimpactofotherVLs.

    WenotedintheIntroductionthatmostVLsfundedbyNectarhaveonlybeenactiveforafewyearsandarestillintheirgrowthstages.AnevaluationoftheiroverallimpactandvaluemightbestbedonefromtheperspectiveofsomeyearsinthefuturewhentheVLsareinamorematuregrowthphase.Thereforetheanalysisandconclusionsdrawninthisstudyshouldbetreatedaspreliminaryanddependsignificantlyontheassumptionsmadeaboutfuturegrowthpaths.

    AlthoughVLssharemanyfeaturesincommon,theydiffersignificantlyfromeachotherintermsoftheservicestheyprovidedtotheirtargetcommunities.ThismeansthattheapproachtotheevaluationofvalueandimpactmustbetailoredtothecircumstancesofeachoneandthatconsiderableeffortshouldbemadeinunderstandingtheservicesprovidedbyeachVLandhowtheseservicesaredeliveredtoandbenefittheirtargetcommunities.

  • 49

    References

    ABS2017,6302.0AverageWeeklyEarnings,Australia,athttp://www.abs.gov.au/AUSSTATS/[email protected]/DetailsPage/6302.0Nov%202016?OpenDocument

    AccessEconomics2008,Exceptionalreturns:thevalueofinvestinginhealthR&DinAustraliaII.Canberra(Australia):AustralianSocietyforMedicalResearch;2008.

    AfganE,SloggettC,GoonasekeraN,MakuninI,BensonD,CroweM,GladmanS,KowsarY,PheasantM,HorstR,LonieA.,GenomicsVirtualLaboratory:APracticalBioinformaticsWorkbenchfortheCloud.,PLoSOne.2015Oct26;10(10):e0140829.

    BarkerMichelle,NancyWilkins-Diehr,SandraGesing,DanielS.Katz,SilviaDelgadoOlabarriaga,ShayanShahand,ScottHenwood,TristanGlatard,KeithJeffery,BrianCorrie,AndrewTreloar,HelenGlaves,LesleyWyborn,NeilChueHong,AlessandroCosta2017,TheGlobalImpactofScienceGateways,VirtualResearchEnvironmentsandVLs,

    BarkerM2016,TheAustraliansciencegatewaysecosystem.PeerJPreprints;2016Oct6.

    BCCVL2016,Our2016Snapshot,Jan5,2017,availableathttp://www.bccvl.org.au/our-2016-snapshot/

    Beagrie,N.,Lavoie,B.,andWoollard,M.(2010)KeepingResearchDataSafe2FinalReportLondon:Jisc.http://www.jisc.ac.uk/publications/reports/2010/keepingresearchdatasafe2.aspx#downloads.

    Beagrie,N.,Houghton,J.W.,Palaiologk,A.,andWilliams,P.(2012)EconomicImpactEvaluationoftheEconomicandSocialDataService(ESDS),EconomicandSo