mellor osf for managing and sharing research workflows
TRANSCRIPT
AnOpenScienceFrameworkforManagingandSharingResearchWorkflows
DavidMellorJournalandFunderIni1ates
h5ps://cos.io|h5ps://osf.io|@OSFramework
Mission:Increasetheopenness,integrity,andreproducibilityofscien1ficresearch.
Strategy:• Evidence to encourage change. • Training and incentives to embrace
change. • Technology to enable change.
Infrastructure Metascience
Community
Rootoftheproblem:
Incen1vesforindividualsuccessarefocusedongeHngit
published,notgeHngitright
Nosek,Spies,&Motyl,2012
Problemsinresearchandpublishing:
FlexibilityinanalysisSelec1verepor1ng
IgnoringnullsLackofreplica1on
Sterling,1959;Cohen,1962;Lykken,1968;Tukey,1969;Greenwald,1975;Meehl,1978;Rosenthal,1979
Technology to enable change
h5p://osf.io/free,opensource
Collaboration Documentation
Archiving
Putdata,materials,andcodeontheOSF
Manageaccessandpermissions
Version Control
Registration
Merges Public-Private
Workflows
Incentives to embrace change
Persistent Citable
Identifiers
File downloads
Incentives for
Openness
File downloads
Forks
Visits
Signals:MakingBehaviorsVisiblePromotesAdop1on
0%
10%
20%
30%
1st Half 2012 2nd Half 2012 1st Half 2013 2nd Half 2013 1st Half 2014 2nd Half 2014 1st Half 2015
Perc
enta
ge o
f Arti
cles
with
Ope
n D
ata
Independent Repository, BadgeIndependent Repository, No BadgePSCI Supplement, BadgePSCI Supplement, No BadgeIndependent Website, BadgeIndependent Website, No BadgePersonal website, No Badge
DataAvailabilityinPsychologicalScience
ThePreregChallenge$1,000,000inprizesforresearcherstopublishtheresultsoftheirpreregisteredwork.
Makesdis1nc1onbetweenexploratoryandconfirmatoryresearchmoreclear.
Exploratoryresearch:
• Findsunexpectedtrends
• Pushesknowledgeintonewareas
• Resultsinatestablehypothesis
Confirmatoryresearch:
• Putsahypothesistothetest
• Doesnotallowdatatoinfluencethehypothesis
• Resultsareheldtothehigheststandardofrigor
https://cos.io/prereg
Researchques1ons
Datacollec1onmethods
Variables
Sta1s1caltests
Outliers
https://cos.io/prereg
Connecting the workflow is critical to enabling change
Connects Services Researchers Use
OpenSesame
OpenSesame
API Docs
https://api.osf.io/v2/docs/
LetExpertsbeExperts
OSF Application Framework
Workflow Authentication File Storage File Rendering Meta-database Integrations Search SHARE Data
osf.io
OSF Application Framework
Workflow Authentication File Storage File Rendering Meta-database Integrations Search SHARE Data
osf.io
journals
registries
preprint servers
grants management
curation, annotation
peer review services
LetExpertsbeExperts
Substan1veexpertsshouldn’thavetodealwith…
• Workflow integration • Authentication • Permissions • File storage • File rendering • Database • Metadata/Annotations/Commenting • Persistence • External service integrations • Search • SHARE Data
Substan1veexpertsshouldfocuson…
• Defining community standards
• Changing community values and incentives
• Determining specific community requirements
• Doing scholarship
Society-brandedPreprintServers
OSFforInsJtuJons
Thank you!
David Mellor, [email protected] Find me online: https://osf.io/qthsf/ Find this presentation:
https://osf.io/6zcw6