mdpi experience survey results introduction
TRANSCRIPT
MDPIExperienceSurveyResults
IntroductionThisreportpresentsthefindingsofashortsurveyconductedofexperiencesofMDPIjournals.ThesurveywasdisseminatedbyTwitterinearlyApril2021,andreceived1168usableresponses.ItshouldbereadinconjunctionwithsomeblogpostsandjournalarticleswrittenonMDPI’sgrowth,bymyself(here),andasecondbyChristosPetroufortheScholarlyKitchenandbyPaoloCrosetto(here)Thisreportpresentsthemethods,andthehealthwarningtheseentail(thispage)andthedataanalysisIused(thenext).Iprovideadescriptionofrespondentswhotookpartonpages3-4.TheanalysispresentsrespondentsviewsaboutMDPI’scommunicationsandcomparesresponsesaccordingtorespondents’priorandplannedengagementswithMDPI(page4-5).OnthebasisoftheseengagementsIproposefivecategoriesofexperience,rangingfromHostiletoEnthusiastic,allofwhicharewellrepresentedamongtherespondents(page7).Ithenexplorepatternsinthosegroupingswithrespecttodiscipline,careerstage,institutionandlanguage.Ithenexaminepatternsinthebrandassociationsofthesegroupings(10),andaddtotheseinsightsfromthequalitativecommentsofsomerespondents(570)whotookthetimetoprovidethem(page13).FinallyIpresentsomepreliminaryobservationsabouthowthesedatamightbeinterpreted(page15).MethodsandaHealthWarningIcreatedashortsurveyinGoogleForms(showninAppendixOne,page18)thatcouldbecompletedinlessthanthreeminutes,thatexploredrespondents’previousengagementswithMDPI,theirintentionswithrespecttofutureengagements,theirexperienceofcommunicationsfromMDPI,andtheirviewsaboutthereputationofMDPIasawhole.ThesurveywasscrutinisedbyresearchethicsreviewproceduresattheUniversityofSheffieldbeforebeingreleased.IthendisseminatedthesurveyusingTwitterfrom1stApril.Itweetedit,requestedcontactstoretweet,andrepliedtotweetswhichmentionedMDPIaskingpeopletocompletethesurvey(from25thMarchto6thAprilandon9th-10thApril).IalsopostedlinkstoitontwoblogsthatIhavewrittenaboutMDPIthatreceiveoverahundredweeklyvisits.ThesurveycirculatedwidelyonTwitter(over350retweets,Figure1)andinterestinitwassuchthatMDPIalsochosetopromotethesurveyfrom8thAprilontheirLinkedandFacebookpagesandonTwitter.Iclosedthesurveyon11thApril.Thelimitationsofthissurveyarediscussedindetailattheendofthedocument.SufficetosayherethatTwittermentionsofMDPItendeithertobecomplaintsorqueriesaboutthecompany,peoplecelebratingthepublicationofpapers,orpeopleappealingforpapersintheSpecialIssuesthattheyareediting.ThereforetargetingthesegroupsonTwitterislikelytobringpolarisedresponses.Howeverthedatasuggestthatextensiveretweetingbypeopletotheirnetworksclearlybroughtinanumberofmoreambivalentrespondents.
2
Thesemethodsplaceconstraintsonthewiderrelevanceofthesample.Inthefirstinstanceacademicswhoarenotontwitterareunlikelytohavetakenpart.Inthesecond,whileIhavestrongresponsesfromnaturalandsocialscientists(inwhichIamwell-networked),medics,engineersandthehumanitiesarelesswellrepresented.Asweshallsee,thelatterthreegroupstendtohaveadifferent,andbetter,opinionaboutMDPIthantheformertwo.Figure1:TweetImpressionsofSurveyDissemination
Ithereforecautionagainstextrapolatingfromthissurveytoconclude,forexample,thatX%ofacademicslikeordon’tlikeMDPIjournals.Mymethodsdonotallowthatsortofgeneralisationbecausewecannotreliablyweightthesample.Itisanywayunhelpful,asoneofthemainfindingsisthatMDPIjournal’sreputationsvarybetweendisciplines.Howeverthissurveycanexplorewhatdifferentsetsofviewsmightexistandwhatsortofrespondentsholdthesedifferentviews.AsIwillshowbelow,IamabletoidentifygroupsofpeoplewhoenjoyengagingwithMDPI,whohaveengagedwithMDPIinthepastandnolongerwanttodoso,andthosewhodonotwanttoengagewiththematall.Furthersurveyworkwouldberequiredtoexplorehowtypicalthesepatternsareoftheresearchcommunitymoregenerally,andhowtheyaregrowingordecliningamongdifferentdisciplines.DataAnalysisThesurveycollectedinformationonrespondents’careerstage,institutionandcountryofinstitution.ItaskedabouttheirpastengagementswithMDPI,andplannedfutureengagements.Itaskedthemaboutwhatcommunicationstheyhadreceivedfromthejournals.ItaskedthemabouttheirperceivedassociationsbetweenMDPI’sbrandand9keywords.Finallyitgaveanoptionofenteringfreetextcomments.RawanswerstothemainsurveyquestionsarepresentedinaseriesofgraphsinAppendixTwo(page21).IanalysedthedatausingExcelandSPSS.Icleanedthemtoremovesubstantiallyincomplete,andexactlyduplicated,responses.Respondents’countrieswereallocatedtoRegions.Institutionswereallocatedarank(1-100,101-200etc)accordingtothe2020QSUniversityrankings.AsignificantminorityofrespondentswerepartofinstitutionsthatwerenotpartoftheQSrankings.Thisdoesnotmeanthattheseareweakresearchinstitutions.ForexampletheZoologicalSocietyinLondon,USGeologicalSurveyand
3
CentreNationaldelaRechercheScientifiqueinFranceareallunrankedintheQSlistings,butdoexcellentresearch.IfrespondentsfromthoseinstitutionshadansweredbysurveythentheywouldappearasoutsidetheUniversitysystem.IcreatedindicesofpreviousandfutureengagementwithMDPI.Ialsoproduceda‘CommunicationAppreciationIndex’,creatingasinglescorefromrespondents’reactionstoinvitationsthattheyhavereceivedfromMDPIjournals.Thesurveydidnotenquireaboutthefrequencyofinteractions(howmanyemailsreceived,reviewsundertaken,paperssubmitted)oraboutwhentheseinteractionstookplace.Measuresofengagementandcommunicationarethereforeameasureofthevarietyofengagementsundertakenandcommunicationsreceived,nottheirintensity.FinallyIcreatedabrandscore,byconvertingrespondents’viewsabouttheassociationofMDPI’sbrandwithparticularwordsintoasinglenumber.The‘brandscores’thisgeneratesarenotageneralassessmentofMDPIbrand,astherewerediverseaspectsofthebrandwhichwerenotmentionedinmysurvey(forexampleassociationswithwordslike‘Speed’,‘Breadth’,‘Diversity’,‘Basic’or‘Responsive’etc).MysurveyfocussedonspecificdimensionsthatIbelievedtobecontestedandthatIwishedtoexplore.Datacollectedforthissurveyareavailablehere.DetailsofallindexcalculationsarepresentedinAppendixThree(page26).WhorespondedtotheSurvey1168peoplerespondedtothesurvey.MostcamefromEurope,withasizeableminorityfromNorthAmerica,reflectingconcentrationsofMDPIauthorsfromtheseregions,butveryfewfromAsia,asmysurveywasnotavailableinChina(Table1).ApluralityofrespondentswereEarlyCareerResearchers,onethirdweremid-careerandafifthwereseniorscientists.Intermsofdisciplinesthebestrepresentedcategorywasnaturalscientists,withlargenumbersofsocialscientistsalsoparticipating(
Figure2).Smallernumbersofotherdisciplinestookpart.MostrespondentsfromAnglophonecountriesworkedinUniversities,andespeciallyhigherrankedUniversities.Therewasmorediversityofinstitutionaloriginamongnon-Anglophonecountries(Table2).Table1:Respondents’RegionandCareerStage,andallpublishedAuthorsfor2020
Region Junior Mid-Career Senior Total AllAuthorsNoResponse 19 14 8 41 AustralasiaPacific 19 16 11 46 5,852Asia 25 14 10 49 91,778Europe* 275 273 178 727 103,155LatinAm&Caribb 9 13 5 27 8,701NorthAmerica 139 72 41 252 25,975AfricaandME 14 8 4 26 10,576
Total 500 410 257 1,168 246,037*OneEuropeanrespondentdidnotprovidetheircareerstage.AuthorDataprovidedbytheMDPI
4
Figure2:Respondents’Disciplines
Respondentscouldentermultipledisciplinaryidentities.Onerespondentprovidednoanswer
Table2:UniversityRankandMainCountryLanguage
QSRank Anglophone NotAnglophone Total1-100 158 32 190101-200 43 115 158201-300 49 21 70301-400 26 34 60401-500 9 13 22501-600 7 22 29601-700 11 18 29701-800 19 7 26800+ 10 16 26NoResponse 84 82 166NotinQSRankings 74 277 351Total 490 637 112741respondentsdidnotprovidecountrydetails.
5
ResultsWhenIanalysedpatterns,twoaxesofdifferenceappearedinthesurvey.ThefirstpertainstodifferencesinpreviousandintendedfutureengagementswithMDPI.FromtheseengagementsgroupingsofexperiencewithMDPIjournalsemerged.Ishalladdresseachaspectinturn.
CommunicationsfromandAppreciationofMDPI’sBrandManyrespondentstothissurveyreportedreceivingemailsfromMDPI,regardlessoftheircareerstage(Table3).952ofthe1167whoprovidedcompleteresponseshadreceivedthreeormoredifferenttypesofinvitation(toreview,submitapaper,readapaper,editaspecialissueorserveonaneditorialboard).Mysurveydidnotaskaboutfrequencyofemails,justaboutthevarietyofemailrequests,buttheresultsclearlyshowthatthegreatervarietyofrequestsreceived,thelowertheappreciationofMDPI’semails.Mostdidnotwanttoreceivetheseinvitations.ThisdoesnotmeanthattheydidnotwanttoengagewithMDPI–manydid.Itmeansthattheinvitationsweremisplaced.Respondents’appreciationofthesecommunications,asmeasuredbytheappreciationindex,waslow.OnestrikingfeatureofthedataishowfewpeoplehadreceivedalowvarietyofcommunicationsfromMDPI.Only18%hadreceivedtwosortsofcommunicationorfewer.Mostofthosewhohadreceivedtwoorfewertypesofinvitationwerejuniorresearchers.Only11%ofseniorrespondentshadreceivedasfew.Table3:AverageCommunicationAppreciationIndexandtheVarietyofInvitations
CareerStage0-2typesofInvitationto
engagewithMDPI3+typesofinvitationstoengagewithMDPI
Junior -1.3 -3.9 Mid-Career -0.3 -4.6 Senior 1.0 -3.1 CareerStage No.ofRespondents No.ofRespondents TotalJunior 141 359 500Mid-Career 46 364 410Senior 28 229 257Total 215 952 1,167ScoresintheAppreciationIndexrangefrom-10(nodesiretoreceiveanycommunication)to10(wantstoreceiveeverycommunication)DissatisfactionaboutMDPI’semailswasmentionedinthefreetextcomments.19%offreetextcomments(thejointlargestcategory)complainedaboutexcessiveand/orinappropriateemails.Ihavereproducedsomeofthesecomments,selectingonlythosesubmittedfromrespondentswhowereMDPI’smostenthusiasticsupporters(Box1,‘enthusiastic’isdefinedonpage7).ThecommentsfromMDPI’scriticsaboutthecompany’semailpracticesusedlessparliamentarylanguagethanthoseinBox1.
6
Box1:IssueswithemailraisedbyMDPIEnthusiasts
Toomanyemails,itislikespam.Itislikeapredatoryjournalinthisregard.Someinterestingjournalsandpublications,butsomepoliticsshouldchange.Ihaveworkedwithonethatisamongthestrongeroftheirjournals...But,IgetanumberofrequestsfromMDPIfromjournalsthatarenoteveninmyareaofexpertise.Sosomerequestsforourworkdonotseemwellthoughtout.They are relatively cheap and it’s easy to publish inthem . . . But I hate the nagging, the angry emailsbecause on Monday you haven’t replied to the emailtheysentonSaturday. . .Ihatebeingaskedtoreviewirrelevantandoftenincomprehensiblepapers
Waytoomanyunsolicitedemails.
Toomanyinvitationsforservingasguesteditorsonspecialissuesandsubmittingtospecialissues...WhilellikeMDPI'spublishingspeedandmodel,thesekindsofpracticeslowersitsreputationinmy(andothers')minds.Thepublisherisfine-it'sthoseannoyinge-mailsthatreallytarnishtheirreputation.Reviewedmanytimesinthepastforthem,publishedwithtoo.Butbecametoomuchofanuisance...Haveaskedthemtotakemeoffreviewerlist,whichtheyignoredandcontinuouslyhoundme.Shame-asgoodjournal'swithstrongacademicside.Theysendmelotsofemailaskingtoeditaspecialissue.Lots!
PastandFutureEngagementsPreviousengagementwithMDPIamongtherespondentscanbesplitintofourcategoriesofincreasinglevelsofengagement
1. ThosewhohavedonenothingwithMDPIjournals;2. Thosewhohaveonlyreviewedforthem;3. Thosewhohavesubmittedpapers,;4. Thosewhohaveundertakeneditorialwork,asSpecialIssueEditorsorserving
onEditorialBoardsIgroupedintendedfutureengagementsintosimilarcategories,butaddedan‘Ambivalentcategory’whichreferredtorespondentswhowerenotsureiftheyweregoingtoengagewithMDPIinthefuture.Asubstantialmajorityofrespondents(82%)hadengagedwithMDPIjournalsinsomeway,with62%havingsubmittedapaperand/orundertakeeditorialduties(Table4).65%ofrespondentswerepossiblyordefinitely,goingtoengagewithMDPIinthefuture,with32%sayingthattheywoulddefinitelysubmitapaperand/orundertakeeditorialdutiesinthefuture.TheminorityoftherespondentswhohavedonenothingforMDPIjournalstendedtohavethelowestopinionofthejournalsasawhole.TheserespondentstendednottobepleasedtoreceiveemailsfromMDPIjournals.TheyhadalowopinionoftheMDPIbrand(Table5).12%ofthosewhohadhadnoengagementwithMDPIjournalsthusfardefinitelyintendedtoengagewiththeminthefuture(Table4).RespondentswhohadonlyreviewedforMDPIdidnot,ingeneral,wanttodosoagain.57%saidthattheydidnotintendtoreviewagain.TheyalsodidnotwanttoreceiveemailsfromMDPIjournalsandhadalowopinionoftheMDPIbrand(Table5).
7
Respondentswhohadsubmittedpapersand/orundertakeneditorialwork,tendedtohaveamorepositiveviewoftheMDPIbrand,andtowelcometheircommunicationsmore(Table5).Theyweremorelikelytocontinuetoengagewiththecompany.46%reportedthattheywoulddefinitelybesubmittingpapersandortakingoneditorialwork(Table4).ThosewhowerepreparedtoserveasaneditorweremostappreciativeofemailsfromMDPIandhadthehighestopinionoftheirbrand(Table5).Table4:PriorandFutureEngagementwithMDPIJournals
PriorEngagement
FutureEngagement
NoneAmbi-valent
WillReviewOnly
WillSubmitaPaper,noEd.Work
WillserveasEditor Total
None 123 54 12 7 4 200
ReviewOnly 109 61 11 2 7 190SubmitaPaperOnly 118 123 50 100 69 460
EditorialWork 41 43 14 41 116 255
Total 391 281 87 150 196 110536Respondentsprovidedincompletedataaboutpriorengagementsandafurther27incompletedataonfutureengagementsTable5:CommunicationAppreciationandBrandScorebyPriorEngagement
PriorEngagement AverageCommIndex AverageBrandScoreNone -5.7 -7.9ReviewOnly -5.7 -7.1SubmitaPaperOnly -2.2 -2.5EditorialWork -0.5 0.8
Total -3.0 -3.5 CategoriesofEngagementBasedonthedifferencesabove,fivegroupingsofrespondentsemergedfromthissurvey:
1.Hostile:HavenothadandwillnothavedealingswithMDPI.2.Put-Off:HavehaddealingswithMDPIandwillnotdosoagain.3.Ambivalent:MayengagewithMDPIinthefuture.4.Engaged:Preparedtoreviewand/orsubmitMSes.5.Enthusiastic:Preparedtoundertakeeditingwork.
Thebrandappreciationandcommunicationappreciationscoresvaryinpredictablewaysacrossthesegroups.Communicationtoallgroupsisgenerallyhigh.Thisagainindicatesthatasignificantvarietyofrequestsarebeingmadetoresearchersinthissurveywhichareunwanted,especiallythoseinvitationsgoingtoresearchersintheyellowboxes.
8
Table6:MDPIEngagementCategoriesAverageIndexScoresandInvitationsReceived
Values Hostile Put-Off Amb’lent Engaged EnthusiasticNumberofRespondents 123 268 281 237 196PastEngagementIndex 0.0 11.3 10.3 14.9 21.1FutureEngagementIndex 0.0 0.0 2.3 5.4 9.2CommunicationsIndex -9.6 -9.3 -5.7 1.3 5.8InvitationsReceived 3.5 3.8 3.3 3.0 3.1BrandScore -7.0 -6.4 -4.4 -0.6 2.6PastEngagementscoresrangefrom0(none)to35(maximumpossible).FutureEngagementscoresrangefrom0(none)to10(maximumpossible).CommunicationsAppreciationscoresrangefrom-10(noemailswanted)to10(allemailswelcome).InvitationsReceivedrangefrom0to4typesofinvitationreceived.BrandScorerangesfrom-10(worstpossiblerating)to10(bestpossiblerating)Therelativesizeofthecategories(theNumberofRespondentsrow)cannotbetakentoberepresentativeofresearchcommunitiesasawhole.Inthesegroupingsanumberofpatternsemerged.First,withrespecttoDisciplineNaturalandSocialscientistsweremorelikelytobeinthehostileorput-offcategoriesthanotherdisciplines(Table7).RespondentsdescribingthemselvesasMedicsorEngineersweremorelikelytobeengagedorenthusiastic.RespondentsfromArtsandHumanitiesweretoofewforpatternstoemerge.IngeneralMedics,EngineersandscholarsfromtheHumanitiestendedtohavemorepositiveviewsofMDPI’sbrandthanscholarsinnaturalsciencesandsocialsciences.WecanseethisinFigure3.ThisportraysthebrandassociationswithRigourandImportanceofrespondentswhoidentifiedasonlyMedics,EngineersorfromtheArtsandHumanitiesagainstrespondentswhoidentifiedasonlyNaturalorSocialScience(orbothofthese).Theaxesshowpositiveassociationstotherightandtop,andnegativeassociationstothebottomandleftinbothcases.RespondentswhowereNaturalorSocialScientiststendtofeaturefurtherdown,andfurthertotheleftofthegraphs.
Table7:CategoriesofMDPIEngagementbyDiscipline
Hostile Put-Off Amb’lent Engaged Enthusiastic Total
ArtsandHums 5 5 7 9 8 34SocialScience 63 123 102 87 38 414NaturalScience 57 149 164 134 94 599MedicalSciences 4 19 24 36 41 125Engineering 11 27 37 32 55 163
Total 140 323 334 298 236 1335Respondentscouldindicatemorethanonedisciplinaryidentity.
9
Figure3:BrandAssociationbyDiscipline:‘Rigour’(Xaxis)and‘Important’(Yaxis)
Medics,EngineersandHumanitiesinPeach;NaturalandSocialScientistsinPlum.AxesrangefromStrongDisagreement(farleft,bottom),throughDisagreement,Neutral(centre),Agreement,andStrongAgreement(farright,top)Thelargestpeachrepresents27respondents(agreeingtobothassociations),thelargestplum150(stronglydisagreeingwithbothassociations).OtherassociationsareshowninTable8.Theseindicatethat,withrespecttoCareerStage,thereisnodiscernabledifferenceincareerstageongroupingmembership.ConcerningInstitution,therewasnodiscernabledifferenceamonggroupingsduetoUniversityrankings.RespondentsfromhigherrankedUniversities(<200)seemedtothinksimilarlytothoseinlowerranks(>200).ThereissomeindicationthatrespondentsfromoutsidetheUniversitysystemweremoreengagedandenthusiasticaboutMDPIthanthoseinUniversities.ConcerningLanguage,thereissomeindicationthatrespondentsfromAnglophonecountrieswerelesssupportiveofMDPIthanrespondentsfromnon-Anglophonecountries.Table8:CategoriesofMDPIEngagementandRespondentCharacteristics
AverageScores Hostile Put-Off Amb’lent Engaged Enthusiastic TotalCareer 19.1 18.1 16.9 17.0 19.1 34QSRankings 18.1 17.2 18.4 19.8 20.9 414OutsideQSRankings 2.4 2.4 2.8 3.8 3.6 599Language 5.4 5.0 4.6 3.5 3.5 125Careerscores:Junior=10,Med=20,Senior=30;QSRankings:<201=10,>200=30;OutsideQSRankings:NotQS=10,QSranked=0;Language:Anglophone=10,NotAnglophone=0.Exploringpatternsinindividualfactorshowevercanbemisleadingbecausewecannotseehowtheyarerelatedtoeachother.Iconductedanordinallogisticregressionto
10
examinewhataspectsweremoststronglyassociatedwithdifferentcategoriesofMDPIengagementfor739respondentsforwhichcompletedatawereavailable.Ididthisincrementally,addingfactorstoexploretheirpowerinexplainingengagementdecisionsandtheirstatisticalsignificance.FullresultsarereportedinAppendix4.Inthisanalysisthemostimportantfactorisbrandscore.Thisisnotquiteasbanalassayingthatourdecisionstopublishinajournalareinfluencedbywhatwethinkaboutit.AfterallIdidnotcaptureafullpictureofwhatpeoplethinkaboutMDPI’sbrand,Ionlyaskedabout9aspectsinwhichIwasinterested.Ratheritshowsthattheaspectsofbrandthatinterestedmewereindeedimportantinexplainingdecision-makingforalargenumberofpeople.Astrongermodelmightbeobtainedbyaskingfurtherquestionsaboutbrand,orbymakingthequestionsspecifictoparticularjournals,ratherthanthebrandasawhole.Thereisalsothepossibilitythatsomepeoplearechoosingajournaldespitetheirmisgivingsaboutit,forexampleinordertohelpoutacolleaguewhoisconveningaSpecialIssue.Thenextmostimportantfactoristhecommunicationsindex.Thisindicatesthat,forexample,excessivecommunicationmightdiscouragepeoplefromengagingeveniftheyapprovedofthejournal.Disciplinewasalsosignificantwhenaddedtothemodel(asabinaryvariable,eitherMedic/Engineer/Arts&HumsorNat/SocScience),withNaturalandSocialScientiststendingtowantlessengagementthanpeoplefromotherdisciplines.Howeverdisciplinedoesnotreallyincreasethemodelsexplanatorypowermuch.Thismayreflecttherelativelysmallnumbers(123)ofMedics,EngineersandHumanitiesscholarsinthatanalysis,asopposedtothelargernumber(616)ofNaturalandSocialScientists.Itmayreflectthefactthatattitudestobrandandcommunicationsalreadydifferbydisciplinesoaddingdisciplinedoesnotmakethemodelmorepowerful.Oftheotherfactors(Careerstage,Language,andInstitution),beinganEarlyCareerorJuniorresearcherhadaveryslightpositiveimpactonengagement,thatwasjustsignificant.Respondents’institutions,andthelanguageofthecountrywheretheseinstitutionswerebasedprovedinsignificant.BrandAssociationsThebrandassociationsbetweenengagementgroupingsfollowedsomepredictablepatterns.TheseareshowninFigure4.EachgraphfollowsthesamestructureasFigure3.Agreementwitheachwordisshownonasingleaxis,withstrongagreementtopositivebrandassociationsontheright/top,andnegativebrandassociationsonthebottom/left.Responsescanappearatfivepointsoneachaccess(StrongAgreement,Agreement,Neutral,Disagreement,StrongDisagreement).Theresponseofdifferentcategoriesofengagementareshownseparatelyineachrow,movingfromhostile(purple,ontheleft)toenthusiastic(red,ontheright).Theyshowclearpatternsofrespondentsmovingfromthebottomleftoftheboxplots(negativeassociations)totopright(positiveassociations)asengagementwithMDPIincreased.HencePurpleandGreencirclesappearinthebottomleftoftheplots,indicatingthatHostileandPut-offrespondentshavelowopinionsofthejournals.Orangecircles,
11
(Ambivalent)aremoreapproving,andEngagedrespondents(bluecircles)moreso.Redcircles,showingthemostEnthusiasticrespondents,appearonthetopright.Thesefigureshavetobecomparedwithcare.Thesizeofeachcirclevariesaccordingtootherdataineachgraph,notbetweendifferentengagementgroups.Thisinthefirstrow,thelargepurplebubbleofHostilerespondentsrepresentstheviewsof53people.ThelargegreenbubbleofPut-offrespondentsintheadjacentgraphrepresents91people.Howeverthefiguresservetomakeasimplepoint.Thefirstthreerowsshowstheconcentrationofrespondentsmovessteadilyupfromthedisapprovingbottomleftcorneruptotherightasengagementincreases.ThustheydemonstratethatBrandassociationswiththewordsRigour,Importance,ValueforMoney,ExploitationandNuisancefollowpredictablepatterns.Thereaftersomesurprisesappear.First,withrespecttoperceptionsofHasteandPrestige(thefourthrow),MDPIjournalstendedtodopoorly,evenamongthe‘Ambivalent’and‘Engaged’groups.Thisisvisibleinthelargerorangeandbluecirclestowardsthebottomleftofthegraphs.Onlythemostenthusiasticsupportersweredistributedtowardsthetopright.Thissuggeststhat‘Haste’isgenerallyassociatedwithMDPIjournals,but‘Prestige’isnot.Converselywhenwecompareattitudesto‘Convenience’adifferentpatternemerges.Themosthostilerespondentsmightthinkthatthejournalswereunprestigious(row5),orpredatory(row6),butsomewerestillpreparedtorecognisetheirconvenience.Othercategorieswereevenmoreappreciativeofthejournals’Convenience.Enthusiastsstronglydisagreedwiththeideathatthejournalswerepredatory.
Figure4:Agreement/DisagreementwithwordsassociatedwithMDPI’sbrand,dividedaccordingtoEngagementCategory
FirstwordlistedistheXaxis,secondwordtheYaxis.AxisscaleasperFigure3.Circlesinthebottomleftcornerindicatenegativebrandassociations,toprightindicatespositiveassociations.Circlesizeiscalculatedseparatelyforeachgrouping.So,forexample,purplecirclescannotbecomparedtoredorbluecircleswithineachrow,butredbubblesizesindifferentrowscanbecompared.Key:Purple–Hostile;Green–Put-Off;Orange–Ambivalent;Blue–Engaged;Red–EnthusiasticRigour:Importance
Importance:GoodValueforMoney
Exploitative:Nuisance
Haste:Prestigious
Prestigious:Convenient
Predatory:Convenient
13
VariationandChangeinEngagementasreportedinQualitativeData570respondentsaddedfreetextcommentsabouttheMDPIandtheseremarksmostlyfallintopairedcategoriesoftopics.Therearethosewhocelebratethejournals’efficiencyandconveniencewhileothersweresuspiciousofthefastturnaroundtimesorcomplainedofexpensiveAPCcharges.Somereportedgoodpapersandrespectedcolleaguesworkingwiththejournals,otherspoorqualitywork.Andsomecomplainedofpoorexperienceseithergivingorreceivingreview,andothersreportedexcellent,highqualityservicethatwasprovidedquickly.IhaveprovidedillustrativecommentsofexperiencesofgoodandbadreviewinBox2toshowthediversitythatthissurveycaptured.Box2:ExperiencesofReview,withRespondent’sEngagementCategoryshown.
PoorReviewExperience GoodReviewExperienceReviewprocesscanbesomewhatrandom,witheditorsandreviewerswithoutenoughexpertise.(Enthusiastic)Ihadagoodexperiencewithtwoofthemandtheyaredeemedtobereputableinmyfield.However,thequalityofpeerreviewandpublishedpapersisquestionableforthreeotherjournalsIwasinvitedtowriteforortodoapeer-review.(Enthusiastic)Ipublished2papers,bothhadgoodreviews,butthewholeexperienceseemedlessrigorousthanwithotherjournals.Theemphasisofpeerreviewseemedtobeonaboxcheckedratherthanimprovingthepaper.(Engaged)Iwaspleasedbytheefficiencyandsimplicityofpublishingwiththem.Ididhoweverfindthereviewprocessveryhasty,withthereviewersprovidingverylittlemeaningfulcomment.Theeditorseemedfinewiththisanddidnotraiseanyissues.(Engaged)Reviewsdonotseemtobedonebyresearchers.Isubmittedapaperand3outof5reviewerswerefocusedonthepaperformat(e.g.referencestyle).(Ambivalent)Ihavehadnormalpapersaccepted.Thereviewerswerepoorlychoseninmostcases,andthereviewswerepoor(short,lackofdetail...).(Ambivalent)
Submittedonlytwoarticlessofar.Sonotmuchtosay,exceptgotsomelengthyfourreviewsforonearticlethatwasrejectedaftermajorrevisions,astworeviewerswerenotsatisfiedwithrevisions.(Enthusiastic)Onepaperwassubmittedthereandreceived4reviews-total17pagesofcomments!Ittooklotaoftimetoreviseandrespondtoreviewerscomments.Oneofthereviewersdisclosedtheiridentityandwasanexpertinthatfield.Itwasacceptedaftertworoundofrevisions.(Enthusiastic)IhavehadgoodexperienceswithMDPI.Fastandrigorouspeerreview(fourreviewers,excellentandchallengingreviews,10daysaftermysubmission).(Engaged)Sofarinmy2firstauthorpapersI’vehad4reviewerseachtimeandtheyweregenerallycompetent.Thereisalwaysa“reviewer2”butI’vebeenhappywiththerigoursofar.(Engaged)Ipublishedwiththemasanearly-careerresearcherbecausetheyofferedtowaiveopen-accessfeeswhichforanECRisratherconvenient.Itdidgothroughafairlythoroughpeerreview(2rounds,6reviewersintotal).(Ambivalent)Ihadagoodexperiencewithmyoneandonlypaperpublishedwiththem.4reviewers,providedusefulandpromptreviews.(Ambivalent)
Reflectingthisdiversity,asignificantproportionofrespondents(19%ofcomments)saidthatthejournals’reputationswerehighlyvaried,andthatitwasdifficulttocomeupwithasingularopinionaboutthem.TheserespondentswerefoundinallcategoriesofengagementwithMDPI.
14
Inadditiontotheseratherpredictablepatterns,twootherthemesareworthnoting.Firstseveralrespondents(10%ofthoseprovidingcomments)wrotethatMDPIwasnotfundamentallydifferentfromotherpublishers.Allpublishersexploittheresearchcommunity.Therootofanyproblemslienotwiththepublisher,butwiththebrokensystemthatrewardsbehaviourfromwhichpublisherscanprofitsohandsomely.Second,therewasarepeatedconcernwiththedirectionoftravelthatMDPIappearedtobetaking.ThislendssupporttoPaoloCrosetto’sargumentthatthegrowthofMDPIjournalsisunsustainable.Theconcernisevidentinthecategoryofrespondents(24%ofmysample)thatIdescribeas‘PutOff’.Butdoubtsoverrecenttrendsandfuturetrajectorieswasexpressedacrossallcategories.AnumberofpeoplecommentedthattheyhadhadchangedtheiropinionofMDPIovertheyears,eitherbecauseofpoorexperiences,orbecausetheywereconcernedthatthecompany’spracticeswerenotmaintainingstandards.Arepeatedissuefortheserespondentswasthebewilderingabundanceofpapersandspecialissues.Box3:ChangingOpinionsaboutMDPIovertime
Inmyperception,MDPIoverallhasaneutralorevenslightlynegativereputation.MDPIjournalsareveryheterogenous,withthequalityofjournalsdependingalotontheirrespectiveEiCsandEditorialboardmembers.Iamthe[redacted]ofanMDPIjournal,trytokeeprigorousstandards,andregularlyhavetodealwithnegativespilloversarisingfromproblemswithotherMDPIjournals.(Enthusiastic)MDPIcouldimproveit'sreputationbycuttingbackonthenumberofjournalsandspecialissues(sothatspecialissuesareactuallyspecial),avoidingsendingirrelevantinvitationsandotherpredatorypractices.ThepoorreputationofMDPIasawholeisdamagingthemorereputableMDPIjournalsthataretryingtopublishqualityresearchwhilealsoofferingauthorsagoodservice.(Enthusiastic)Ihaveeditedtwospecialissuesandservedontheeditorialboardof[redacted].Isteppeddownin[redacted]becauseIfeltuncomfortableabouttheveryrapidincreaseinno.ofassociateeditors,no.ofpaperspublished,no.ofspecialissues.(Engaged)MDPI..isspammingspecialissues,someofwhichdocontainverygoodpapers.Myownexperienceasreviewerwasgoodsofar,andasauthormixed.Iwouldn'tconsideritpredatoryyet,butitismovinginthatdirection..MDPIisrapidlydegradingitsreputationandthesympathytheyhadinmyfieldasOA-publishersatthemoment.(Engaged)IstronglydisagreewiththeMDPIpolicyoflaunchinginnumerablespecialissues.Ihaverefusedmanyinvitationstoeditspecialissues(Idonotevenreplytotheseinvitations),thesearereallyanuisanceforme.(Engaged)AlthoughmyexperienceinpublishingandreviewingforMDPIhasbeengoodoverall,newsandopinionbypeopleinacademiathatitisapredatorypublisher(despitesomejournalsbeingreallygoodandsomestillbeingnotandhaveroomtoimprove)canaffecttheviewofmyinstitutionandsuperiorstoo,andmaynotallowustopublishinMDPIjournalsasitaffectsthereputationnotjustoftheinstitutionbutaffectsmyperformanceevaluationtooandsorecentlyIoptnottopublishorreviewinMDPIjournals.(Engaged)WhenIstartedmyPhDandthentransitionedtoapostdoc,myopiniononMDPIjournalwasverygood.Ilikedtheopenaccesspartofitandthatpaperswerepublishedquickly...[it]wasajustfantastic.ThenIwasinvitetoreviewapaper,thenanotherone,thenthelistgrew.ThenIbecameaguesteditor,thenamemberoftheeditorialboard.ThisallowedmetoseeMDPIfromtheinsideandIquicklyrealizedwhattheirrealprioritiesare.Everythingisdoneasquicklyaspossible,everythinghasashortduedate,andeverythingisapartofaspecialissuesothattheguesteditorsdoallthework.Forfree,ofcourse.IdonotregrethavingmypaperspublishedinMDPIbefore,anddonotsayIwillnotsubmitmypaperstoMDPIagain.However,theirjournalswillneverbemyfirstchoice.Further,everytimeIreadapaperpublishedbyMDPIIamextracarefulandsuspiciousofthequalityofthepresentedresearch.(Ambivalent)
15
DiscussionResearchcommunitiesarediverse.TherearesignsthatdifferentdisciplinesrespondindifferentwaystoMDPI.SocialandNaturalScientistsaremorecritical,MedicsandEngineersmorewelcoming.ItisnotclearifthesedifferentdisciplinarystoriesarisebecauseresearchersfromdifferentdisciplinesgotodifferentMDPIjournals.Thiswouldbethecase,forexample,ifEngineersencounteradifferentsetofjournalsthandoSocialScientists.Oritcouldbebecausebehavioursandnormsindifferentdisciplinesvarywithrespecttothesamejournal.Thiswouldhappenif,forexample,everyoneisworkingwiththejournalSustainabilitybutdifferentdisciplinesthinkaboutitspracticesindifferentways.BecauseofthelimitationsofmysampleitisdifficulttomakepredictionsaboutwhatEngineers,MedicsandresearchersfromtheArtsandHumanitiesthinkaboutMDPIjournals.Morespecificresearch,withlargersamplesfromthosedisciplinesandaboutdisciplinespecificjournals,wouldberequired.IcanspeakwithmoreconfidencewithrespecttoSocialscientistsandNaturalscientists.Butevenherethereisawiderangeofviews.TherearesomewhoneverhaveandneverwillengagewithMDPI;therearethosewhowillnotdosoagain.Othersareuncertain.Somehaveengagedandarekeentointhefuture.Buthowwillmembershipofthesedifferentgroupschangeovertime?IftheengagedgroupremainkeenonMDPIjournals,and/oriftherearelargenumbersofauthorswhohavenotyettriedtopublishinMDPIandwantto,thenMDPI’ssuccess(intermsofthenumbersofarticlespublished)willcontinue.Butifscepticismmounts,ifMDPIpapersarenothighlyrated,ifengagementwithMDPIistakentobeablotonaCV,thenitwillbecomeharderforthejournalstorecruitauthors,reviewersandeditors.Twosalientfindingsfromthissurveywillbeimportantasthesepossibilitiesunfold.First,MDPIstaffwritetotoomanypeople,toofrequentlyandwithinappropriaterequests.Theywritetopeopleabouttopicswhicharenotpartoftheirexpertise.Theywritedespitehavingbeenaskednotto.Theychivvyauthorsandreviewers.Theydonotalldothis.Butenoughstaffdosoforthispracticetocauseresentment,evenamongthejournals’supporters.
Second,perceptionsofMDPI’sbrandshowsomeweaknesses,exceptamongitsmoreardentsupporters.Itsjournalsarenotgenerallyconsideredprestigious.Itsperceivedhastedetractsfromitsreputation.Theproblemsseeninsomejournalsmaybeaffectingtheothers.ConsideragainforexampletheproblemsofpeerreviewthatIreportedabove(Box1).Peerreviewis,afterall,notmeanttoworkwellonaverage.Itismeanttoworkwellallthetime.Deficienciesinpeerreviewinsomejournalsmayerodesupportintheothers.ItmightbepossibleforMDPItosustainitsgrowth,regardlessoftheseissues,becauseitappealstodeep-rootedveinsdesiresintheresearchcommunitytopublish,toconvenepublicationsandtosupporttheircolleaguesastheytrytodoso.Thelackofcheapandappealingalternativeswithinthebrokenpublishingmodelofacademiamayprovideacontinualstreamofcustomerstryingtopublishtheirwork.Itsroleinprovidingaroutearounddisciplinarygatekeepersisalsoimportant.
16
AlternativelyitispossiblethatenoughpeoplehavepositiveexperiencesofMDPI’spublicationandspecialissuesthattheywillcontinuetoreturntothepublisher.ThiswasVolkerBeckmann’sargumentinhiscommentrespondingtoPaoloCrosetto’sblog:
‘AsaGuestEditorforMDPIjournalsyougetextraordinarysupportfromthein-houseAssistantEditors,butalsofromEditorialBoardMembers.FrommyexperienceIwouldsaythat’sthemainreasonwhySpecialIssuesatMDPIjournalsaresopopularanddevelopedintothegrowthengineofmanyMDPIjournals.’
Source:Commentpostedon13thAprilhereThisimpliesthatthegrowthofSpecialIssuescomesfromreturningcustomers,ratherthanneweditors,whichisempiricallytestable.AnditisalsopossiblethatMDPIitmayneedtoalteritsbehaviourtocounteradverseperceptionsthatmaydetractfromthebrand.Thismightentail:
1. Sendingfeweremails.Itwouldprobablyrequiremonitoringthe‘hitrate’ofreviewandSpecialIssuerequestscarefully.Thisislikelytorequireachangeintheincentivesandmanagementofjournalsupportstafftoencouragethemtosendfeweremails.
2. LongerMSprocessingtimes.Thismightmeanofferingauthorsmoretimetorevisepapersand/orlongerreviewingtimesforthepaperthattheyhavesubmitted.Itmightmeanhigherrejectionrates,especiallyformorepopularjournals.Againthisislikelytorequirechangestothetargetsthatjournalsupportstaffhavetomeet.
3. FewerpapersandSpecialIssues.Thisislikelytomeanreducingthecompany’srevenues.Itwillalsorequirechangestothetargetsthatjournalsupportstaffhavetomeet.
ThesefindingspromptanumberofrecommendationsandchallengesforMDPIstaff,andfortheresearchcommunitywhoarewriting,reviewingandediting.HoweverfirstIwouldlikefirsttoinvitefurthercomment,reflectionsandquestionsontheanalysisthusfar.PleasesendyourcommentsbeloworjoinintheTwitterdebates.Afinal,thirdblogwillfollowinduecourse.
DanBrockington,
Glossop,April2021
17
MethodsandDatainmoredetailThedatawhichIusedintheanalysisaboveareavailableonthispage.IcreatedashortsurveyusingGoogleforms,andcommunicateditintwoways.First,IpostedlinkstotheformonmyblogsaboutMDPI,whichregularlyreceiveover100visitsperweek.SecondIdisseminateditusingTwitter.Ididthispartlybeaskingpeopleinmynetworktotweettheinvitation.Itaggedinand‘DM’edpeoplewhohadengagedwithmeonthistopicandaskedthemtoretweet.Ialsosystematicallycontactedanyonewhohadused‘mdpi’intheirtweetsbetween25thMarchand5thApril,andon9th–10thApril.ThisdrewthesurveytotheattentionofpeoplecomplainingaboutMDPIontwitter,peoplewhowerecelebratingpublishingpapersinMDPIjournals,andpeoplewhowereannouncingspecialissuelaunches.SomewhattomysurprisethetwitteroutreachblewupovertheEasterWeekendasfartoomanyresearcherswereusingholidaytimetochecktheirfeeds.Asaresultmyrequeststotakepartwereretweetedover350times,andsometimesrepeatedlybyenthusiasts.Theseretweetsreachedover900ktwitterfollowers.BecauseofthisinterestIwasthencontactedbytheCEOofMDPI,DeliaMihaila,withwhomIhadcorrespondedonblogsIhavepreviouslywrittenabouttheMDPI.SheobservedthatMDPIsupporterscouldbequietonTwitter,andcriticsmorevocal.SheofferedtouseMDPI’splatformstopromotethesurveytoresearcherswhopublishedwithMDPI.Ididnotexpressapreferenceeitherway.InsteadIsuggestedthat,ifMDPIdidpromotethesurvey,thattheyshouldnotuseemail,andthattheywaituntiltheinitialTwitterinteresthaddieddown.MDPIpromotedthesurveyusingdiverseplatforms(Twitter,FacebookandLinkedin)on8thApril.Thetimingofdifferentrespondents’participationsuggeststhatmorepeoplehostiletoMDPItookpartinthefirstdaysofthesurvey.Butthereaftermypracticeofreplyingtothetweetsofpeoplewhowerecelebratingtheirengagementwiththejournalsbroughtinalargenumber(hundreds)ofresponsesfrommorerespondentswhoweremorepositiveaboutthejournals.MDPI’spromotionofthesurveydidnotresultinaflurryofpositiverespondents.Indeeditseemedtohaveincreasednumbersengagingbyonlytwentyorsoonthatday(8thApril).Iclosedthesurveyon11thAprilbecauseIhadalimitedtimetoworkonpromotingit,andbythenthedifferentsortsofgroupswhoweretakingpartwereobvious,andwellpopulated.Giventhelimitationsofthedisseminationmethodenoughrespondentshadtakenpartforpatternstobecomevisible.Therearemultiplelimitationstothismethod.First,mychoiceofsurveysoftwareanddisseminationtechniquemeantthatnoonebasedinChinacouldaccessthesurvey.Second,itispossibleforpeopletoreturnmultipleresponses.IdidnotuseGooglesign-ins,orCaptcha’storeducethissortofabusivebehaviour.Iwashoweverabletotrackhowmanypeoplefromdifferentcountrieswerevisitingthewebpageonwhichtheformwasposted,andcomparethattothecountryoftherespondents.Iwasalsoabletotrackmentionofthissurveyontwitterandmatchthesetoresponserates.NothingintherateoftheresponsesIreceived,ortheirorigin,suggesteddeliberatesurveyabuse.Third,thesurveyismostobvioustopeoplewhoareonTwitter,orwhoaresearching
18
forinformationaboutMDPIandcomeacrossmyblogs.ItisnoteasilyfoundbypeoplewhoarenotonTwitter,andwhoarenotaskingquestionsaboutMDPI.Table9:TimingofRespondents’Participation
Date HostilePut-Off
Ambiv-alent Engaged
Enthu-siastic
Med/Eng/Hum
Nat/Soc
Inter-disc
April1 20 35 28 27 5 10 104 4April2 21 52 50 32 13 13 138 25April3 24 42 38 32 23 20 129 21April4 10 24 43 23 18 24 92 16April5 21 48 58 47 33 36 160 28April6 11 27 28 28 28 9 93 23April7 4 8 7 6 9 5 26 4April8 4 7 10 12 21 19 30 6April9 3 13 12 24 22 26 42 8April10 5 12 7 6 24 17 32 8Total 123 268 281 237 196 179 846 143
IhavenotputresponsestoquestionsaboutInstitution,noranyfreetextcomments,inthepublicdomaintoensurerespondents’completeanonymity.AfterconsultingwiththeethicalreviewcommitteeIhavealsonotputrespondents’institutionscountriesinthepublicdomain,reportingjusttheregioninwhichtheirinstitutionsarefound.Ihaveincludedcareerstageinformation,againafterconsultingwiththeethicalreviewcommitteebecausethisinformationcannotcompromiserespondents’anonymity.AcknowledgementsManythankstoallwhotookpartinthesurvey.ThankstocolleaguesinGeographyattheUniversityofSheffieldforapromptreviewofthisresearch,toJevgeniyBluwsteinandJensLundforcommentsonthesurvey,JohanOldekopandCalebGallemoreforstatisticaladvice,DeliaMihaila,StefanTochevandFacundoSantoméfordataonMDPIauthorshipandfordisseminatingthesurveyandtoPaoloCrosettoforinsightsandchallengesthroughout.
Appendix1:TheMDPIExperienceSurvey
20
21
ThelistofwordsinQuestion8wasrandomlyorderedforeachrespondent.
Appendix2:AnswerstotheMDPIExperienceSurvey
PreviousengagementwithMDPIjournals:A–ReviewedaPaper;B–EditedaSpecialIssue;C–SubmittedaPaper;D–HadaPaperRejected;E–PublishedaPaper;F–ServedonanEditorialBoard.Blue=Yes;Red=No.
23
FutureplanswithMDPIjournals:A–ReviewaPaper;B–EditaSpecialIssue;C–PublishaPaper;D–ServeonanEditorialBoard.Blue=Yes;Red=No;Green–Uncertain
24
EmailInvitationsReceived:A–ToReview;B–ToEditaSpecialIssue;C–ToSubmitaPaper;D–ToReadaPaper;E–ToJoinanEditorialBoard.Orange:IhavebeeninvitedanddonotmindifIreceiveanotherinvitation;BlueIhavebeeninvitedandIwanttobeinvitedagain;GreenInvitationreceivedandIdonotwanttoreceiveanother;BlackNoinvitationreceived;YellowNoinvitationreceived,andIwouldlikeone;RedNotinvitedandIdonotwanttobe
25
Agreement/DisagreementwithWordsAssociatedwithMDPI’sbrand
Appendix3:CalculationofIndices
PreviousEngagement:TotalofthefollowingscoresEngagement Yes NoReview 5 0SubmitaPaper 10 0EditaSpecialIssue 10 0ServeonanEditorialBoard 10 0Respondentswhoindicatedthattheyhadnotengagedinsomeactivities,butleftactivitiesblank,wereexcludedfromtheanalysis.Whererespondentsindicatedthattheyhadundertakensomeactivities,butleftothersblankIassumedthatnon-responsesmeant‘No’andcodedaccordingly.Scoresrangedfrom0(nopreviousengagement)to35(everypreviousengagement).Thedistributionofscoresforrespondentswithcompletedataisshownbelow.Score Total0 2045 19410 19215 31620 725 14530 135 73Total 1132FutureEngagement:AverageofthefollowingscoresFutureAction Yes No MaybeReview 10 0 5SubmitaPaper 10 0 5EditaSpecialIssue 10 0 5ServeonanEditorialBoard 10 0 5Respondentswhodidnotprovidefourresponseswereexcludedfromtheanalysis.Scoresrangedfrom0(nofurtherengagement)to10(maximumfurtherengagement).Thedistributionofscoresforrespondentswithcompletedataisshownbelow.Score Total0 404
28
1.3 1352.5 1093.8 875.0 936.3 527.5 928.8 4910 118Total 1139Communications:Averageofthefollowingscores
Invitation No
invitationreceived,andIwouldlikeone
NotinvitedandIdonotwanttobe
IhavebeeninvitedanddonotmindifIreceiveanotherinvitation
IhavebeeninvitedandIwanttobeinvitedagain
InvitationreceivedandIdonotwanttoreceiveanother
Review 10 -10 5 10 -10SubmitaPaper 10 -10 5 10 -10EditaSpecialIssue 10 -10 5 10 -10
Noinvitationreceivedregisteredasblank.Iomittedfromthisindexresponsesconcerninginvitationstoreadapaperandserveonaneditorialboardbecauseoftherelativelyhighproportionofpeoplewhohadreceivednoinvitation.Scorerangefrom-10(allcommunicationsareunwelcome)to10(allcommunicationswelcome).Thedistributionofscoresforrespondentswithcompletedataisshownbelow.IcreatedcategoriesofCommunicationsAppreciationfortheOrdinalLogisticRegressionasfollows:-10StronglyNegative,>-10to<0Negative,0Neutral,>0-5Positive,>5StronglyPositive.Score Total-10 61-9.4 28-8.8 44-8.1 32-7.5 45-6.9 66-6.3 55-5.6 60-5.0 71-4.4 63-3.8 64
29
-3.1 47-2.5 53-1.9 38-1.3 46-0.6 440 510.6 291.3 321.9 232.5 213.1 233.8 274.4 185.0 145.6 96.3 136.9 137.5 148.1 48.8 310 1Total 1112Brand:AverageofthefollowingscoresWord StronglyAgree Agree Neutral Disagree StronglyDisagreeRigour 10 5 0 -5 -10Important 10 5 0 -5 -10Prestigious 10 5 0 -5 -10GoodVal.ForMoney 10 5 0 -5 -10Haste -10 -5 0 5 10Nuisance -10 -5 0 5 10Predatory -10 -5 0 5 10Exploitation -10 -5 0 5 10AssociationswithConveniencedidnothelpdistinguishbetweenrespondentsandwereexcludedfromthebrandscore.Incompleteresponses,whichdidnotscoreallthewordsabovewereexcluded.Thedistributionofscoresforrespondentswithcompletedataisshownbelow.Scoresrangefrom-10(allassociationsstronglynegative)to10(allassociationspositive).IcreatedcategoriesofCommunicationsAppreciationfortheOrdinalLogisticRegressionasfollows:StronglyNegative(<-5),Negative(<-0.7to-5),Neutral(-0.6to0.6),Positive(0.601to5),StronglyPositive(>5).Score Total
30
-10 61-9.4 28-8.8 44-8.1 32-7.5 45-6.9 66-6.3 55-5.6 60-5.0 71-4.4 63-3.8 64-3.1 47-2.5 53-1.9 38-1.3 46-0.6 440 510.6 291.3 321.9 232.5 213.1 233.8 274.4 185.0 145.6 96.3 136.9 137.5 148.1 48.8 310 1Total 1112OtherchangesAfewrespondentsindicatedambiguouscareerstages(bothJuniorandMid,orbothMidandSenior).Iallocatedallsuchrespondentstothemorejuniorcategory.
31
Appendix4.ResultsoftheOrdinalLogisiticRegression
1.PLUM-OrdinalRegression.BrandAlone
Notes
OutputCreated 12-APR-202110:58:16Comments Input ActiveDataset DataSet14
Filter <none>Weight <none>SplitFile <none>NofRowsinWorkingDataFile
739
MissingValueHandling
DefinitionofMissing User-definedmissingvaluesaretreatedasmissing.
CasesUsed Statisticsarebasedonallcaseswithvaliddataforallvariablesinthemodel.
Syntax PLUMGroupingBYBrandScoreCategory/CRITERIA=CIN(95)DELTA(0)LCONVERGE(0)MXITER(100)MXSTEP(5)PCONVERGE(1.0E-6)SINGULAR(1.0E-8)/LINK=LOGIT/PRINT=FITPARAMETERSUMMARY.
Resources ProcessorTime 00:00:00.02ElapsedTime 00:00:00.00
32
CaseProcessingSummary
NMarginalPercentage
Grouping 1Hostile 91 12.3%2PutOff 191 25.8%3Ambivalent 187 25.3%4Engaged 159 21.5%5Enthusiastic 111 15.0%
BrandScoreCategory
1StronglyNegative
291 39.4%
2Negative 254 34.4%3Neutral 74 10.0%4Positive 94 12.7%5StronglyPostive 26 3.5%
Valid 739 100.0%Missing 0 Total 739
ModelFittingInformation
Model-2Log
Likelihood Chi-Square df Sig.InterceptOnly
607.664
Final 89.383 518.281 4 .000Linkfunction:Logit.
Goodness-of-Fit Chi-Square df Sig.Pearson 42.887 12 .000Deviance 24.371 12 .018Linkfunction:Logit.
PseudoR-SquareCoxandSnell .504Nagelkerke .527McFadden .223Linkfunction:Logit.
33
ParameterEstimates
Estimate
Std.Error Wald df Sig.
95%ConfidenceInterval
LowerBound
UpperBound
Threshold
[Grouping=1Hostile]
-6.551 .468 195.593
1 .000 -7.469 -5.633
[Grouping=2PutOff]
-4.684 .454 106.410
1 .000 -5.574 -3.794
[Grouping=3Ambivalent]
-2.899 .440 43.336 1 .000 -3.763 -2.036
[Grouping=4Engaged]
-.649 .409 2.522 1 .112 -1.450 .152
Location
[BrandScoreCategory=1StronglyNegative]
-5.471 .467 137.310
1 .000 -6.387 -4.556
[BrandScoreCategory=2Negative]
-3.764 .455 68.326 1 .000 -4.656 -2.871
[BrandScoreCategory=3Neutral]
-1.481 .467 10.050 1 .002 -2.396 -.565
[BrandScoreCategory=4Positive]
-.262 .458 .329 1 .566 -1.159 .634
[BrandScoreCategory=5StronglyPostive]
0a . . 0 . . .
Linkfunction:Logit.a.Thisparameterissettozerobecauseitisredundant.
34
2.PLUM-OrdinalRegression.BrandandCommunicationsIndex
NotesOutputCreated 12-APR-202110:58:38Comments Input ActiveDataset DataSet14
Filter <none>Weight <none>SplitFile <none>NofRowsinWorkingDataFile
739
MissingValueHandling
DefinitionofMissing User-definedmissingvaluesaretreatedasmissing.
CasesUsed Statisticsarebasedonallcaseswithvaliddataforallvariablesinthemodel.
Syntax PLUMGroupingBYBrandScoreCategoryCommsIndexcategory/CRITERIA=CIN(95)DELTA(0)LCONVERGE(0)MXITER(100)MXSTEP(5)PCONVERGE(1.0E-6)SINGULAR(1.0E-8)/LINK=LOGIT/PRINT=FITPARAMETERSUMMARY.
Resources ProcessorTime 00:00:00.01ElapsedTime 00:00:00.00
35
CaseProcessingSummary
NMarginalPercentage
Grouping 1Hostile 91 12.3%2PutOff 191 25.8%3Ambivalent 187 25.3%4Engaged 159 21.5%5Enthusiastic 111 15.0%
BrandScoreCategory 1StronglyNegative
291 39.4%
2Negative 254 34.4%3Neutral 74 10.0%4Positive 94 12.7%5StronglyPostive 26 3.5%
CommsIndexcategory
1StronglyNegative
389 52.6%
2Negative 101 13.7%3Neutral 48 6.5%4Positive 114 15.4%5StronglyPositive
87 11.8%
Valid 739 100.0%Missing 0 Total 739
ModelFittingInformation
Model-2Log
Likelihood Chi-Square df Sig.InterceptOnly
952.582
Final 202.257 750.325 8 .000Linkfunction:Logit.
Goodness-of-Fit Chi-Square df Sig.Pearson 148.782 84 .000Deviance 76.215 84 .715Linkfunction:Logit.
PseudoR-Square
36
CoxandSnell .638Nagelkerke .667McFadden .323Linkfunction:Logit.
ParameterEstimates
Estimate
Std.Error Wald df Sig.
95%ConfidenceInterval
LowerBound
UpperBound
Threshold
[Grouping=1Hostile]
-8.332 .530 247.334
1 .000 -9.371 -7.294
[Grouping=2PutOff]
-6.217 .513 146.950
1 .000 -7.223 -5.212
[Grouping=3Ambivalent]
-3.583 .473 57.282 1 .000 -4.511 -2.655
[Grouping=4Engaged]
-.895 .428 4.365 1 .037 -1.735 -.055
Location
[BrandScoreCategory=1StronglyNegative]
-3.265 .501 42.483 1 .000 -4.247 -2.283
[BrandScoreCategory=2Negative]
-2.259 .482 21.979 1 .000 -3.203 -1.315
[BrandScoreCategory=3Neutral]
-.882 .486 3.292 1 .070 -1.836 .071
[BrandScoreCategory=4Positive]
-.129 .472 .075 1 .785 -1.053 .795
[BrandScoreCategory=5StronglyPostive]
0a . . 0 . . .
[CommsIndexcategory=1StronglyNegative]
-4.147 .363 130.329
1 .000 -4.859 -3.435
[CommsIndexcategory=2Negative]
-1.523 .346 19.383 1 .000 -2.202 -.845
[CommsIndexcategory=3Neutral]
-1.162 .387 9.044 1 .003 -1.920 -.405
[CommsIndexcategory=4Positive]
-.469 .297 2.502 1 .114 -1.051 .112
[CommsIndexcategory=5StronglyPositive]
0a . . 0 . . .
Linkfunction:Logit.a.Thisparameterissettozerobecauseitisredundant.
37
3.PLUM-OrdinalRegression:Brand,CommsIndexandDiscipline
NotesOutputCreated 12-APR-202110:59:12Comments Input ActiveDataset DataSet14
Filter <none>Weight <none>SplitFile <none>NofRowsinWorkingDataFile
739
MissingValueHandling
DefinitionofMissing User-definedmissingvaluesaretreatedasmissing.
CasesUsed Statisticsarebasedonallcaseswithvaliddataforallvariablesinthemodel.
Syntax PLUMGroupingBYBrandScoreCategoryCommsIndexcategoryPureDiscRedone/CRITERIA=CIN(95)DELTA(0)LCONVERGE(0)MXITER(100)MXSTEP(5)PCONVERGE(1.0E-6)SINGULAR(1.0E-8)/LINK=LOGIT/PRINT=FITPARAMETERSUMMARY.
Resources ProcessorTime 00:00:00.02ElapsedTime 00:00:00.00
38
CaseProcessingSummary
NMarginalPercentage
Grouping 1Hostile 91 12.3%2PutOff 191 25.8%3Ambivalent 187 25.3%4Engaged 159 21.5%5Enthusiastic 111 15.0%
BrandScoreCategory 1StronglyNegative
291 39.4%
2Negative 254 34.4%3Neutral 74 10.0%4Positive 94 12.7%5StronglyPostive 26 3.5%
CommsIndexcategory
1StronglyNegative
389 52.6%
2Negative 101 13.7%3Neutral 48 6.5%4Positive 114 15.4%5StronglyPositive
87 11.8%
PureDiscRedone Med/End/Hum 123 16.6%Nat/Soc 616 83.4%
Valid 739 100.0%Missing 0 Total 739
ModelFittingInformation
Model-2Log
Likelihood Chi-Square df Sig.InterceptOnly
1027.499
Final 271.064 756.435 9 .000Linkfunction:Logit.
Goodness-of-Fit Chi-Square df Sig.Pearson 177.305 155 .106Deviance 115.177 155 .993Linkfunction:Logit.
39
PseudoR-SquareCoxandSnell .641Nagelkerke .670McFadden .326Linkfunction:Logit.
ParameterEstimates
Estimate
Std.Error Wald df Sig.
95%ConfidenceInterval
LowerBound
UpperBound
Threshold
[Grouping=1Hostile]
-8.169 .534 234.216
1 .000 -9.216 -7.123
[Grouping=2PutOff]
-6.045 .518 136.397
1 .000 -7.059 -5.030
[Grouping=3Ambivalent]
-3.407 .479 50.501 1 .000 -4.346 -2.467
[Grouping=4Engaged]
-.699 .437 2.561 1 .110 -1.555 .157
Location
[BrandScoreCategory=1StronglyNegative]
-3.192 .504 40.048 1 .000 -4.181 -2.203
[BrandScoreCategory=2Negative]
-2.193 .485 20.413 1 .000 -3.144 -1.242
[BrandScoreCategory=3Neutral]
-.850 .490 3.016 1 .082 -1.809 .109
[BrandScoreCategory=4Positive]
-.092 .475 .037 1 .847 -1.023 .840
[BrandScoreCategory=5StronglyPostive]
0a . . 0 . . .
[CommsIndexcategory=1StronglyNegative]
-4.095 .364 126.430
1 .000 -4.808 -3.381
[CommsIndexcategory=2Negative]
-1.496 .347 18.589 1 .000 -2.176 -.816
[CommsIndexcategory=3Neutral]
-1.093 .388 7.923 1 .005 -1.853 -.332
[CommsIndexcategory=4Positive]
-.462 .298 2.398 1 .121 -1.046 .123
40
[CommsIndexcategory=5StronglyPositive]
0a . . 0 . . .
[PureDiscRedone=Med/End/Hum]
.496 .202 6.035 1 .014 .100 .892
[PureDiscRedone=Nat/Soc]
0a . . 0 . . .
Linkfunction:Logit.a.Thisparameterissettozerobecauseitisredundant.
41
4.PLUM-OrdinalRegression:AllFactors
NotesOutputCreated 12-APR-202111:00:00Comments Input ActiveDataset DataSet14
Filter <none>Weight <none>SplitFile <none>NofRowsinWorkingDataFile
739
MissingValueHandling
DefinitionofMissing User-definedmissingvaluesaretreatedasmissing.
CasesUsed Statisticsarebasedonallcaseswithvaliddataforallvariablesinthemodel.
Syntax PLUMGroupingBYBrandScoreCategoryCommsIndexcategoryPureDiscRedoneCareerstagecleansedTop200QSRankingLanguage/CRITERIA=CIN(95)DELTA(0)LCONVERGE(0)MXITER(100)MXSTEP(5)PCONVERGE(1.0E-6)SINGULAR(1.0E-8)/LINK=LOGIT/PRINT=FITPARAMETERSUMMARY.
Resources ProcessorTime 00:00:00.03ElapsedTime 00:00:00.00
42
CaseProcessingSummary
NMarginalPercentage
Grouping 1Hostile 91 12.3%2PutOff 191 25.8%3Ambivalent 187 25.3%4Engaged 159 21.5%5Enthusiastic 111 15.0%
BrandScoreCategory 1StronglyNegative
291 39.4%
2Negative 254 34.4%3Neutral 74 10.0%4Positive 94 12.7%5StronglyPostive 26 3.5%
CommsIndexcategory
1StronglyNegative
389 52.6%
2Negative 101 13.7%3Neutral 48 6.5%4Positive 114 15.4%5StronglyPositive
87 11.8%
PureDiscRedone Med/End/Hum 123 16.6%Nat/Soc 616 83.4%
Careerstagecleansed Junior 309 41.8%Mid-Career 267 36.1%Senior 163 22.1%
Top200QSRanking >200 207 28.0%1-200 274 37.1%NA 258 34.9%
Language Anglophone 333 45.1%NotAnglophone 406 54.9%
Valid 739 100.0%Missing 0 Total 739
ModelFittingInformation
Model-2Log
Likelihood Chi-Square df Sig.InterceptOnly
1664.095
Final 895.473 768.623 14 .000Linkfunction:Logit.
43
Goodness-of-Fit
Chi-Square df Sig.Pearson 986.664 1086 .986Deviance 632.359 1086 1.000Linkfunction:Logit.
PseudoR-SquareCoxandSnell .647Nagelkerke .676McFadden .331Linkfunction:Logit.
ParameterEstimates
Estimate
Std.Error Wald df Sig.
95%ConfidenceInterval
LowerBound
UpperBound
Threshold
[Grouping=1Hostile]
-8.350 .552 228.630
1 .000 -9.432 -7.268
[Grouping=2PutOff]
-6.196 .535 134.080
1 .000 -7.245 -5.148
[Grouping=3Ambivalent]
-3.528 .497 50.371 1 .000 -4.503 -2.554
[Grouping=4Engaged]
-.804 .454 3.134 1 .077 -1.693 .086
Location
[BrandScoreCategory=1StronglyNegative]
-3.332 .509 42.794 1 .000 -4.330 -2.333
[BrandScoreCategory=2Negative]
-2.326 .489 22.587 1 .000 -3.286 -1.367
[BrandScoreCategory=3Neutral]
-.935 .493 3.601 1 .058 -1.902 .031
[BrandScoreCategory=4Positive]
-.135 .478 .080 1 .778 -1.071 .802
[BrandScoreCategory=5StronglyPostive]
0a . . 0 . . .
[CommsIndexcategory=1StronglyNegative]
-3.963 .366 117.346
1 .000 -4.680 -3.246
[CommsIndexcategory=2Negative]
-1.343 .350 14.726 1 .000 -2.029 -.657
44
[CommsIndexcategory=3Neutral]
-.965 .392 6.070 1 .014 -1.733 -.197
[CommsIndexcategory=4Positive]
-.395 .301 1.725 1 .189 -.985 .195
[CommsIndexcategory=5StronglyPositive]
0a . . 0 . . .
[PureDiscRedone=Med/End/Hum]
.492 .204 5.844 1 .016 .093 .892
[PureDiscRedone=Nat/Soc]
0a . . 0 . . .
[Careerstagecleansed=Junior]
.378 .192 3.897 1 .048 .003 .754
[Careerstagecleansed=Mid-Career]
-.046 .194 .057 1 .811 -.426 .333
[Careerstagecleansed=Senior]
0a . . 0 . . .
[Top200QSRanking=>200]
-.323 .189 2.936 1 .087 -.693 .046
[Top200QSRanking=1-200]
-.338 .179 3.549 1 .060 -.689 .014
[Top200QSRanking=NA]
0a . . 0 . . .
[Language=Anglophone]
-.131 .155 .723 1 .395 -.435 .172
[Language=NotAnglophone]
0a . . 0 . . .
Linkfunction:Logit.a. Thisparameterissettozerobecauseitisredundant.