introducing the open citation experiment - jisc digifest 2016
TRANSCRIPT
![Page 1: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/1.jpg)
/15
Introducing the open citation experiment
Petr Knoth (@petrknoth) & Drahomira Herrmannova (@damirah)Knowledge Media institute, The Open University
Verena Weigert, Jisc
1
Towards full-text based research metrics: exploring Semantometrics
![Page 2: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/2.jpg)
/15
What are Semantometrics?• A new class of metrics for evaluating research• Build on the premise that full-text is needed to
asses the value of a publication• Make use of the full-text features of a given
resource rather than relying on outside evidence
2
![Page 3: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/3.jpg)
/15
Semantometric contribution measure• Based on the idea of measuring the progress
of scholarly discussion• The hypothesis states that the added value of
publication p can be estimated based on the semantic distance from the publications cited by p to the publications citing p
• Demonstrator at http://semantometrics.org
3
![Page 4: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/4.jpg)
/15
Comparative study• Analysis carried out to investigate the
properties of the contribution measure• Experiments carried out on a dataset obtained
by merging data from the Connecting Repositories (CORE) system, the Microsoft Academic Graph (MAG) and Mendeley
• After merging the datasets over 1.6 million publications
4
![Page 5: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/5.jpg)
/15
ApproachPremise: Full-text needed to assess publication’s research contribution.Hypothesis: Added value of publication p can be estimated based on the semantic distance from the publications cited by p to publications citing p.
5
![Page 6: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/6.jpg)
/15
ApproachPremise: Full-text needed to assess publication’s research contribution.Hypothesis: Added value of publication p can be estimated based on the semantic distance from the publications cited by p to publications citing p.
5
![Page 7: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/7.jpg)
/15
ApproachPremise: Full-text needed to assess publication’s research contribution.Hypothesis: Added value of publication p can be estimated based on the semantic distance from the publications cited by p to publications citing p.
5
![Page 8: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/8.jpg)
/15
Contribution measure
6
![Page 9: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/9.jpg)
/15
Contribution measure
p
6
![Page 10: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/10.jpg)
/15
Contribution measure
p
6
![Page 11: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/11.jpg)
/15
Contribution measure
p
6
![Page 12: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/12.jpg)
/15
Contribution measure
p
A
6
![Page 13: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/13.jpg)
/15
Contribution measure
p
A B
6
![Page 14: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/14.jpg)
/15
Contribution measure
p
A B
6
![Page 15: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/15.jpg)
/15
Contribution measure
p
A B
6
![Page 16: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/16.jpg)
/15
Contribution measure
p
A Bdist(a,b)
6
![Page 17: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/17.jpg)
/15
Contribution measure
p
A Bdist(a,b)
6
![Page 18: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/18.jpg)
/15
Contribution measure
p
A Bdist(a,b)
6
![Page 19: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/19.jpg)
/15
Contribution measure
p
A Bdist(a,b)
6
![Page 20: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/20.jpg)
/15
Contribution measure
p
A Bdist(a,b)
6
![Page 21: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/21.jpg)
/15
Contribution measure
p
A Bdist(a,b)
6
![Page 22: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/22.jpg)
/15
Contribution measure
p
A Bdist(a,b)
6
![Page 23: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/23.jpg)
/15
Contribution measure
p
A Bdist(a,b)
Average distance of the set members
6
![Page 24: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/24.jpg)
/15
Contribution measure
p
A Bdist(a,b)
Average distance of the set members
6
![Page 25: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/25.jpg)
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
![Page 26: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/26.jpg)
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
![Page 27: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/27.jpg)
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
![Page 28: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/28.jpg)
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
![Page 29: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/29.jpg)
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
![Page 30: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/30.jpg)
/15
Real examples• Contribution 0.8452
![Page 31: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/31.jpg)
/15
Real examples• Contribution 0.9220
![Page 32: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/32.jpg)
/15
Real examples• Contribution 0.8296
![Page 33: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/33.jpg)
/15
Real examples• Contribution 0.9348
![Page 34: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/34.jpg)
/15
Real examples• Contribution distribution
![Page 35: Introducing the open citation experiment - Jisc Digifest 2016](https://reader035.vdocuments.mx/reader035/viewer/2022070516/587065191a28ab48378b4c9d/html5/thumbnails/35.jpg)
/15
Conclusions• First large scale analysis of the semantometric
contribution measure• As semantometrics should exhibit a number of
advantages over existing research metrics, we should continue studying this field to better understand which facets of research quality they can capture and how they can be applied
13