| 0

Alexander van Servellen Consultant, Research Management (South East Asia)

Nattaphol Sisuruk Training Consultant, Elsevier Research Solutions

SciVal Evidence Based Evaluation

Prince of Songkla University, Thailand

August 9th, 2016

| 1

Agenda

1. Introductions

2. Understanding your needs

3. Quick Introduction to SciVal and Evidence Based Evaluation

4. Setting up SciVal

5. Analysis and Reporting

6. Non-Performance Variables

7. Ongoing Engagement Plan

| 2

Introductions

| 3

Introduction to SciVal &

Evidence Based Evaluation

| 4

4

SciVal offers quick, easy access to the research performance of 220 nations and

7500 research institutions worldwide. You can easily create teams, departments,

and specific research topics, and evaluate them.

SciVal is an online tool for evaluating research.

Data updated weekly from Scopus (same data source as the University Rankings use

and used in many studies about research performance)

| 5

Real World Applications

Government Ministry

Determine National Research Strengths

Identify top Institutions

Funding Body

Determine best researchers to give

funding to

Measure the impact of funding given in the past

University

Set KPI’s for research units and

researchers

Showcase unique strengths to

establish reputation

Researcher

Determine TOP researchers in their field (competitor /

potential collaborator

Show case unique strengths to

establish reputation

| 6

Reporting at various levels

Researc

h O

ffic

e Top

Management

Faculties

Department

Researchers

| 7

The data in SciVal comes from Scopus

Over 21,900 titles from more than 5,000 international publishers and 105

different countries

Over 54 million records, 23 million patents from 5 patent offices worldwide

All content is vigorously vetted by an independent, 15-person, international

board of experts called the Content Selection and Advisory Board (CSAB)

More than 3,000 customers worldwide in all geographic regions

Scopus is the largest abstract and citation database of peer-reviewed

research literature from around the world. Its the core data source of

Elsevier Research Intelligence solutions, and used by academics,

government researchers and corporate R&D professionals who need a

comprehensive and efficient place to search, discover and analyze research.

| 8

| 9

Elsevier’s metrics approach in a nutshell

Figure 1 of http://www.elsevier.com/online-tools/research-intelligence/resource-library/resources/response-to-hefces-call-for-evidence-independent-review-of-the-role-of-metrics-in-research-assessment

| 10

Levels of

aggregation

| 11

Everything

| 12

Region

| 13

Country

| 14

University

| 15

Faculty, Department or Research Centre

(Example)

| 16

Individual

| 17

Setting Up SciVal

| 18

Log-in at www.scival.com

You may need to

register to access

SciVal.

Register now

| 19

My SciVal – Researchers and Groups

• Creating Researchers

• Creating Groups of researchers (departments etc.)

| 20

Master List

| 21

First create a Master List of researcher names

| 22

We start with the authors for who you do not know the

profile, or have more than one profile in Scopus

| 23

Go to My SciVal and “Define a new Researcher”

| 24

Do this only for the authors who have more than one

Scopus Author ID!!

add the Scopus

author ID field

twice

Fill in the

Author IDs

of one

author

If you do not

know the profile

number, then

search by name

| 25

SciVal will ask you to confirm merging of profiles, in

cases where one researcher has more than one profile.

You can save the

researcher, or

validate the

publications first

| 26

Add a tag to the researcher indicating which

department he/she belongs to

Here I added a tag, so that

once there are hundreds of

researchers in SciVal, each

one is clearly tagged.

| 27

Live Demonstration

| 28

Notes

| 29

Bulk Import Researchers that

have one Scopus Author Profile

| 30

The researchers with one Scopus Author ID can be

bulk imported into SciVal

You can paste the

author IDs belonging

to one department in

the box. Past just the

ID, not the author name etc.

| 31

SciVal will show you if any Author ID is not found

If an author ID is not

found, then you

should search for

this author manually

by name in SciVal.

| 32

Create the Department and give it a name. Remember

to tag the researchers.

Check this box

to create a

group for these

researchers

Give the

group a

name

Tag the

researchers

| 33

Now you have created a department in SciVal which

you can easily share with other SciVal users.

Use the

sharing

function to

give others

access to this department

| 34

Live Demonstration

| 35

My SciVal – Research Areas

• Creating Researcher Areas using keywords

• Creating Research Areas by grouping entities

| 36

Create Research Area

| 37

Use Keywords to define a very specific area of science

Useful

search

tips

| 38

Refine & Validate

Validate that

publications are

representative of

topic

| 39

Prepare your Research Areas in advance. Areas larger

than 5000 publications can take 48 hours

| 40

Live Demonstration

| 41

Share entities you have created, with other SciVal users

Share entities with other SciVal users

| 42

Grant editing rights or change ownership of entities

you created, but have shared with others

By default, you are the only one who can edit entities which you created, and

have shared with others. You can grant editing rights to others, or even transfer

ownership to someone else.

| 43

Live Demonstration

| 44

15 minute break

| 45

Analysis & Reporting

| 46

Now that you have customized SciVal – you can

explore and analyze using SciVal

Overview module

provides dashboard

style reports based on a

3 or 5 year period. You

can analyze countries,

institutions, any

researcher, group of

researchers and

research areas you

have created. It gives a

comprehensive

overview of one entity at

a time.

Benchmarking module is

flexible, it allows you to

select a time period

anywhere from 1996 to

one month ago. You can

choose from over 17

metrics, and compare

across different types of

entities (i.e. you directly

can compare researchers

with departments,

institutions, countries in

one analysis)

Collaboration module

looks at specifically at co-

publication trends. It

allows you to identify the

top collaboration partners

of an institution or

country, as well as

identify potential

collaboration partners.

Research areas can be

applied here as a subject

filter.

| 47

20 sets of metrics at your disposal

Productivity metrics

Scholarly Output

h-indices (h, g, m)

Citation Impact metrics

Citation Count

Citations per Publication

Cited Publications

h-indices (h, g, m) Field-Weighted Citation Impact

Publications in Top Percentiles

Publications in Top Journal Percentiles

Collaboration Impact (geographical)

Academic-Corporate Collaboration Impact

Usage metrics

Views

Views per publication

Field-Weighed Views Impact

Collaboration metrics

Authorship Count

Number of Citing Countries

Collaboration (geographical)

Academic-Corporate Collaboration

Snowball Metric; www.snowballmetrics.com/metrics

Slice and dice your data from multiple angles to identify your core strengths and

weaknesses

Disciplinarity metrics

Journal count

Journal category count

| 48

Live Demonstration

Non-Performance Variables

| 50

When making comparisons, we must take into

consideration “Non-performance variables”

Size

Discipline

Publication-type

Database coverage

Manipulation

Time

In some cases the difference in a metric value between two entities

(authors for example) might not reflect a difference in performance.

Instead the difference can be caused by non-performance variables.

| 51

Publications types receive differing levels of citations

Because some publication types are cited more often than others, we

should not compare different types without applying normalization.

| 52

Citation practices differ between disciplines

Because some subject areas are cited more often than others, we should

not compare papers from different fields without applying normalization.

| 53

Size and Time

Citation count is

directly related to

volume

CPP normalizes for

differences in

publication volume

FWCI normalizes for

subject area,

document type, and

time

Do you think the declining

trend is indicative of a decrease in quality or is it

caused by a non-

performance variable?

Engagement Plan

| 55

Engagement Plan

• Introductory Meeting (today)

• Setup & Customization (next few weeks)

• Follow-up meeting (let’s set a date now)

• Super-users

• Help & Support

55

| 56

Any last questions? You can reach me by email.

56

Alexander van Servellen

a.vanservellen@elsevier.com

| 57

www.elsevier.com/research-intelligence

Thank you very much Alexander van Servellen

a.vanservellen@elsevier.com