dealing with noise in science
DESCRIPTION
With increasing number of science journals and articles how can we build a post-publication evaluation system ?TRANSCRIPT
Dealing with Noise
How to build a post-publication evaluation system
PDF downloads
Nature Genetics (N = 168) for 2005.
R = 0.71
From:Turning web traffic into citationsAction Potential blog
Connotea tag:Evolution
Journal Articles CitationsMBE (2003/2004) 480 6771
Nature (2003/2004) 5193 121932
Science (2003/2004) 5306 124377
Connotea Evolution (2003-2004) 86 3243
Connotea Evolution (2005-2006) 149 5059
November 2006 - Papers in Connotea with tag "evolution” published 2003 or 2004Early 2008 - Papers in Connotea with tag "evolution” published 2005 or 2006
Bloggers talk about interesting papers
There are 519 papers published in 170 journals in the period of 2003-2004 that were mentioned in blogs tracked by Postgenomic.
Quotes
“(…)what is obvious to everyone but bone-headed bean counters. The only way to assess the merit of a paper is to ask a selection of experts in the field. “ David Colquhoun Improbable Science
“In order to shortcut the work of looking up actual (real) citation counts for investigators the journal impact factor is used as a surrogate to estimate the count. I have always warned against this use”. Eugene Garfield (1998)
“Any measurement system will produce gamesmanship to increase success. Lawrence yearns for the one system that obviates it: face-to-face conversations between peers that allows a truly deep and well focused estimate of scientific merit. “ Philip Campbell
Dealing with Noise
Peer-review
Nature, Science, Cell
Lab notebooks, unpublished manuscripts
Jour
nal i
mpa
ct fa
ctor
Perc
eive
d im
pact
Dealing with NoisePe
rcei
ved
impa
ct
Peer-review
Highly recommended papers by your peers
Online lab notebooksPre-print servers
Aggr
egat
ed u
sage
dat
a
Post-publication journals
(…) nobody wants to have to wade through a morass of papers of hugely mixed quality, so how will the more interesting papers in such an archive get noticed as such? Philip Campbell
Post-publication journals
Automatic
Algorithms
Editors
Hubs/Portals