Practical Benefits (and Some Annoyances) of Sharing Data

Daniel Lakens Human-Technology Interaction Group Eindhoven University of Technology

Presenter

Presentation Notes

Er zijn mensen uit de medische hoek die inschatten dat de meeste gepubliceerde bevindingen niet waar zijn. We weten niet of dit klopt of niet, omdat we vaak niet kunnen kwantificeren hoe waarschijnlijk het is dat onderzoek waar is. Daar zijn een paar redenen voor.

Presenter

Presentation Notes

Andere onderzoekers reageerden hierop door te laten zien hoe je met flexibiliteit in hoe je onderzoek analyseert bijna alles wel statistisch significant kan krijgen.

Presenter

Presentation Notes

Het besef van hoe onzeker conclusies van een enkele significante studie zijn, zeker als data flexibel geanalyseerd word, was een flinke schok voor veel wetenschappers. Je kunt wel zeggen dat de schellen van hun ogen vielen.

Presenter

Presentation Notes

Er zijn inmiddels bladen, zoals Psychological Science, die stimuleren at mensen hun data en materiaal delen, en confirmatief onderzoek aangeven wanneer dat ge-preregistreerd is.

Archie

• Internal solution • Data stored 7 years • Researchers can

upload, but not modify, research data

• Limited meta-data (sample size, age youngest participant, description of files)

Archie is an Integrated (Internal)Solution

1 •Submit Proposal to Ethical Board

2 •Get OK, Perform Study

3 •Upload Data & Materials

I still haven’t uploaded the data, even though if I do, I get the participant payment money back (I advanced the money). It interesting that 300-500 euro’s is not enough to get the typical researcher to hurry up with paperwork, or uploading data.

External Solution

• I do always upload data to an external solution (the Open Science Framework) before I submit an article. That’s a much better motivation than money.

External Solution

Sharing Data is Useful

• I recently co-edited a special issue on replications. One author of the original replicated work argued online the replication study was flawed due to a ceiling effect.

• All the data from the replication was online (as part of the special issue). The original researcher also made her data available.

• 5 researchers performed ‘post-publication peer review’ using the raw data.

The Bright Future: Better Meta-Data

• Having access to data for re-analysis is great. But what we want is: – Online databases which store relevant data for as

the sample size, effect size, test statistics – This allows us to meta-analytically evaluate

research lines.

• A new initiative CurateScience.org is doing this

Main Team

Presenter

Presentation Notes

We’ll have an innovative search results page where you can filter by several criteria, including whether an article has any available replications, available data, experimental materials, pre-registration protocols, and disclosure information.

Presenter

Presentation Notes

Here’s an example using the Schnall et al. article part of the recent replication controversy in psychology. NOTE: Unsuccessful replications are beginnings not endings.

Presenter

Presentation Notes

Another example that’s more relevant to society, and might be of broader interest to social scientists

Thanks for Your Attention

Daniel Lakens Human-Technology Interaction Group Eindhoven University of Technology