software sustainability institute software attribution can we improve the reusability and...

Software Sustainability Institute

www.software.ac.uk

Software Attributioncan we improve the reusability and sustainability of scientific software?

http://dx.doi.org/10.6084/m9.figshare.942289

NSF SI2 PIs Meeting, 24-25 February 2014Neil Chue Hong (@npch), Software Sustainability InstituteORCID: 0000-0002-8876-7606 | [email protected]

Unless otherwise indicatedslides licensed under

Supported by Project funding from


www.software.ac.uk

The Research Cycle

Create

Test

Interpret

PublishRevise Paper

Data

Software

Research Outputs Research is a continuous cycle.

When we publish we are contributing to the body of knowledge.



www.software.ac.uk

Research/Reuse/Reward Cycle

Index

Identify

CiteRewardCreate

Test

Interpret

PublishRevise

Research Reuse Reuse is also a cycle. We build our research on the work of others.

Reward mechanisms should encourage reuse.



www.software.ac.uk

The current process

Startresearch

Writesoftware

Usesoftware

Produceresults

Publishresearch

paper

Releasedata

Releasesoftware

Which mentions software and data

This process is simple but does not reward production orreuse of good software and data.

It also has a long contribution cycle.



www.software.ac.uk

Writesoftware

A better process?

Startresearch

Identifyexisting

software

Usesoftware

Produceresults

Publishresearch

paper

Adapt/extend

software

Releasedata

Releasesoftware

Publishsoftware

paper Publishdata

paper

Which references

softw

are and data papers

Software and data papers are needed as proxies for rewarding reuse.

But it enables a shorter contribution cycle for data and software.



www.software.ac.uk

What do we choose to identify:- Workflow?- Software that runs workflow?- Software referenced by workflow?- Software dependencies? What’s the minimum citable part?

Boundary


www.software.ac.uk

Algorithm

Function

Prog

ram

Library / Suite / Package

…

Granularity


www.software.ac.uk

Versioning

Personalv1

Personal v2

Personalv3

Personal v2a

Public v1

Personal v3a

Personal v2a

Public v2

Public v3

Why do we version?- To indicate a change- To allow sharing- To confer special status


www.software.ac.uk

AuthorshipAuthorship• Which authors have had what impact on each version of the software?• Who had the largest contribution to the scientific results in a paper?

http://beyond-impact.org/?p=175

OGSA-DAI projects statistics from Ohloh


www.software.ac.uk

Software Journals

http://openresearchsoftware.metajnl.com



www.software.ac.uk

Peer review of software?

• Can the aspects of peer review be decoupled? Novelty and acceptability Validity and quality

• Accurate metadata helps sustainability But excessive metadata requirements are a barrier

• Essentially, for reuse and sustainability Where is it? Who wrote it? How do I run it? How do I find out more?

• Software Papers: Improving the reusability and sustainability of scientific software http://dx.doi.org/10.6084/m9.figshare.795303




www.software.ac.uk

JORS software metapaper


www.software.ac.uk

ReferencesReuse

ScreenshotsIntroduction

Implementation + Usage

Anatomy of a

software meta-paper

Metadata

Metadata

QualityControl


www.software.ac.uk

F1000Research Web Tool

Other journals you can publish software in:http://bit.ly/softwarejournals


www.software.ac.uk

Code as a Research Object

• What if you could assign DOIs to code easily?

• Could we make software more reusable?• http://mozillascience.org/code-as-a-research-object-a-new-project/• https://github.com/mozillascience/code-research-object


http://mozillascience.org/code-as-a-research-object-a-new-project/

https://github.com/mozillascience/code-research-object


www.software.ac.uk

Alternative Metrics


www.software.ac.uk

I can get credit for everything

Automatically generated from GitHub Repository

Starring as a means of recommendation

Forking analogous to citing for software

… but not necessarily rewardhttp://dx.doi.org/10.6084/m9.figshare.942289


www.software.ac.uk

Career Paths in UKCareers outside academic sector

Non-universityResearch (industry,government etc.)

ProfessorPermanentResearch Staff

Early CareerResearch

PhD

stud

ents

Source: The Scientific Century, Royal Society, 2010 (revised to reflect first stage clarification from “What Do PhD’s Do?” study)

UK STEM graduate

career paths


www.software.ac.uk

Where we are now

• We must describe and cite software otherwise we cannot benefit from and reward reuse and refinement Software papers are a citation mechanism that

works with existing infrastructure and norms Direct citation of code + metadata might be better

• But we still need to fix the reward mechanism for non-traditional research outputs And this is entirely in our hands as scientists



www.software.ac.uk

Further Information

• Software Papers: Improving the reusability and sustainability of scientific software http://dx.doi.org/10.6084/m9.figshare.795303

• Journals in which you can publish software: http://bit.ly/softwarejournals

• Journal of Open Research Software http://openresearchsoftware.metajnl.com/

• Discussion: what is the minimum metadata required to describe a code object for scientific reuse? https://github.com/mozillascience/code-research-object/issues

• Contribute: Code as a research object: https://github.com/mozillascience/code-research-object

• The DOI for this presentation: 10.6084/m9.figshare.942289• The Software Sustainabilty Institute is a collaboration between universities of Edinburgh, Manchester, Oxford and

Southampton. Supported by EPSRC Grant EP/H043160/1.


http://bit.ly/softwarejournals

http://openresearchsoftware.metajnl.com/

https://github.com/mozillascience/code-research-object/issues



software sustainability institute software attribution can we improve the reusability and...

Documents

software attributioncan

software dependencies

data papers software

workflow references

shorter contribution

long contribution cycle

reward mechanisms

ukthis process