the pbrf and bibliometric measures
DESCRIPTION
The PBRF and bibliometric measures. Introduction. Use of bibliometrics by the Ministry of Education Part 1 – presentation of latest bibliometric results Part 2 – what if bibliometrics was used to allocate PBRF funding? – a case study. Bibliometrics. Thomson Reuters bibliometrics dataset - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/1.jpg)
The PBRF and bibliometric measures
![Page 2: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/2.jpg)
Introduction
• Use of bibliometrics by the Ministry of Education
• Part 1 – presentation of latest bibliometric results
• Part 2 – what if bibliometrics was used to allocate PBRF funding? – a case study
![Page 3: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/3.jpg)
Bibliometrics• Thomson Reuters bibliometrics dataset
– Reasonably stable– Long time series– External to the PBRF system – can triangulate and calibrate– International benchmarking– Limitations of bibliometrics
• Useful monitoring tool– But don’t see bibliometrics as a way of dispensing research
funding at this time
![Page 4: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/4.jpg)
Bibliometrics measures – NZ universities
– Share of world publications– Share of world citations– Relative impact:
(citations per paperNZ universities)
(citations per paperworld)
![Page 5: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/5.jpg)
Share of world publications and citations
Source: Thomson Reuters
0.0%
0.1%
0.2%
0.3%
0.4%
0.5%
81-85 83-87 85-89 87-91 89-93 91-95 93-97 95-99 97-01 99-03 01-05 03-07
Sha
re o
f w
orld
Publications Citations
![Page 6: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/6.jpg)
Relative impact – NZ vs Australia
Source: Thomson Reuters
0.0
0.2
0.4
0.6
0.8
1.0
1.2
81-85 83-87 85-89 87-91 89-93 91-95 93-97 95-99 97-01 99-03 01-05 03-07
Rel
ativ
e im
pact
(wor
ld a
vera
ge =
1)
NZ G8 Non-G8
![Page 7: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/7.jpg)
Relative impact – by panel
Source: Thomson Reuters
0.0
0.2
0.4
0.6
0.8
1.0
1.2
81-85 83-87 85-89 87-91 89-93 91-95 93-97 95-99 97-01 99-03 01-05 03-07
Rel
ativ
e im
pact
(wor
ld a
vera
ge =
1)
Engineering Maths Physical science
![Page 8: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/8.jpg)
Relative impact – by panel
Source: Thomson Reuters
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
81-85 83-87 85-89 87-91 89-93 91-95 93-97 95-99 97-01 99-03 01-05 03-07
Rel
ativ
e im
pact
(wor
ld a
vera
ge =
1)
Biological sciences Medicine & public heath Health
![Page 9: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/9.jpg)
Relative impact – by panel
Source: Thomson Reuters
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
81-85 83-87 85-89 87-91 89-93 91-95 93-97 95-99 97-01 99-03 01-05 03-07
Rel
ativ
e im
pact
(wor
ld a
vera
ge =
1)
Business Education
![Page 10: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/10.jpg)
Relative impact – by panel
Source: Thomson Reuters
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
81-85 83-87 85-89 87-91 89-93 91-95 93-97 95-99 97-01 99-03 01-05 03-07
Rel
ativ
e im
pact
(wor
ld a
vera
ge =
1)
Social sciences Humanities & law
![Page 11: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/11.jpg)
Relative impact – by panel
Source: Thomson Reuters
Engineering
Maths
Physical science
Biological sciences
Medicine & public heath
Health
Business
Education
Social sciences
Humanities & law
World 1998-2002 2003-2007
![Page 12: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/12.jpg)
NZ vs Australian universities 2003-2007
Engineering
Maths
Physical science
Biological sciences
Medicine & public heath
Health
Business
Education
Social sciences
Humanities & law
World NZ G8 Non-G8Source: Thomson Reuters
![Page 13: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/13.jpg)
What if citations were used to allocate PBRF funding?
• Case study – Earth Science
• Method– Compare funding allocated to each university in
2007 via the PBRF quality evaluation with what would have been allocated if based on citations
![Page 14: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/14.jpg)
Funding via the PBRF 2007University PBRF score FTE PBRF funding
($000s)
Lincoln 3.0 8.0 $146
Massey 4.7 15.7 $453
Auckland 4.6 26.5 $742
Canterbury 5.4 21.0 $695
Otago 5.5 17.7 $598
Waikato 5.0 18.3 $555
VUW 4.9 25.8 $763
Total $3,952
![Page 15: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/15.jpg)
Funding via relative impact in 2007
• Relative impact in five year overlapping time periods
• Use three year weighted average– 1999-2003 (15%)– 2000-2004 (35%)– 2001-2005 (50%)
• Then adjust by PBRF FTE
![Page 16: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/16.jpg)
Relative impact
University 99-03 00-04 01-05 Weighted
Lincoln 1.04 0.56 1.04 0.87
Massey 0.73 0.50 0.60 0.59
Auckland 0.77 0.82 0.84 0.82
Canterbury 0.73 0.78 0.64 0.70
Otago 1.68 1.63 1.28 1.46
Waikato 0.97 1.06 1.05 1.04
VUW 1.10 0.96 1.02 1.01
Source: Thomson Reuters
![Page 17: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/17.jpg)
Comparing PBRF scores and weighted relative impact
0.0
0.5
1.0
1.5
0 1 2 3 4 5 6
PBRF average quality score
Rel
ativ
e im
pact
OU
AU
VUWWU
CU
LU
MU
![Page 18: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/18.jpg)
Comparing funding allocations 2007
$0.0
$0.2
$0.4
$0.6
$0.8
$1.0
VUW Auckland Canterbury Otago Waikato Massey Lincoln
$mill
ion
PBRF Relative impact
![Page 19: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/19.jpg)
Issues
• Capturing all indexed research vs 4 nominated research outputs
• Size of NZ subject areas
![Page 20: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/20.jpg)
University staffing trends
0
500
1,000
1,500
2,000
2,500
3,000
Professors AssociateProfessors
Senior Lecturers Lecturers Research OnlyStaff
2001 2002 2003 2004 2005 2006 2007
![Page 21: The PBRF and bibliometric measures](https://reader034.vdocuments.mx/reader034/viewer/2022051517/56814686550346895db3a8a6/html5/thumbnails/21.jpg)
Ministry of Education monitoring
• Ministry of Education reports assessing research performance can be found at the Education Counts website:
www.educationcounts.govt.nz