plum analytics - clinical citations: telling the story of impact
TRANSCRIPT
1
Clinical Citations:Telling the Storyof Clinical ImpactMay 26, 2016
Andrea MichalekPresident and Founder, Plum Analytics
[email protected]@amichalek | @plumanalytics
2
Agenda• Our approach to analytics• Clinical Citations• Q&A
TELL THE STORY.
4Over 250 PlumX Customers Worldwide
5
6
?
7
What impact has our research had in the past 12 months?
8
How do we help our early career researchers secure grants.
9
How do we increase the visibility of our research?
10
How do we assess and evaluate our research?
11
What clinical impact does our research have?
12
Our Approach• Categorize
– Define the unique properties to measure research output by
– Group research outputs your way• Visualize
– See impact at a glance• Analyze
– Discover your research stories
13
The universe of metrics is expanding
14
15
Journal citations are the traditional form of
Scholarly Measure15
JournalImpact Factor
3.161
16
17
Researchers have Moved Online
18
Examples: Scholarly Data Exhaust
TECHNOLOGY HAS FINALLY CAUGHT UP
Measuring data exhaust to tell the story…
Make visible
what was invisible
Makemeasurablewhat was
immeasurable
22
Defining altmetrics• Altmetrics = Alternative ways of measuring
the use and impact of scholarship• Altmetrics combine traditional impact
measures (citation counts) with non-traditional measures
• Altmetrics = ALL METRICS
“Altmetrics are measures of scholarly impact mined from activity in online tools and environments” – Jason Priem, Co-Founder, ImpactStory
Using Altmetrics to Illustrate the Impact of Open Access on Graduate Student Research (Barnett, Collister, Chan 2014)
23
Benefits of Altmetrics• Broadness
– Measure the broader impact of research—beyond science
• Diversity– Measure the impact of scholarly products beyond just
the journal article. • Speed
– Outputs can be measured immediately.• Openness
– Obtaining and viewing the data is easier.
Woulters & Costas, 2012
24
MetricsCategories
USAGE(clicks, downloads, views,
library holdings, video plays)
CAPTURES(bookmarks, code forks, favorites,
readers, watchers)
MENTIONS(blog posts, comments, reviews,
Wikipedia links)
SOCIAL MEDIA(+1s, likes, shares, tweets)
CITATIONS(citation indexes, patent
citations, clinical citations)
25
USAGE
• Is anyone reading our work?
• Did anyone watch our videos?
• Usage is the #1 stat researchers want to know after Citation counts
• PlumX is the only product that includes Usage
clicks, downloads, views, library holdings,
video plays
26
• Captures indicate that someone wants to come back to the work
• Early indicator of future citations
bookmarks, favorites, readers, watchers
CAPTURES
27
• This category measures people truly engaging with your research
• Automatically uncover the conversations about your research
• Discover feedback, opinions, etc.
blog posts, comments, reviews,
Wikipedia linksMENTIONS
28
• Social media measures how well a researcher is promoting their work
• This is especially important for early career researchers to measure and understand
• Track the buzz and attention around your research output
+1s, likes, shares, tweets
SOCIAL MEDIA
29
• Citations still a standard of long-term impact
• Including citations enables side-by-side analysis with other metrics categories
• PlumX is expanding the definition of citations
– Patent citations, Clinical Citations, etc.
Citation Indexes,Patent Citations, Clinical Citations
CITATIONS
30
5 Categories in ActionArticle level
metrics are the building blocks
for PlumX
31
The Plum Print• Includes 5 categories
of metrics
• Designed to communicate engagement without a score
32
33
LIST OF GRANTS
for the Institution Example
of Analytics Social Media
by Grant
34
ClinicalCitations
35
Clinical Citations• Showcase your researchers and institution• Help your researchers
– Get credit for their work– Tell their stories for funding
• Understand the value of clinical and translational research
36
…clinical medicine draws heavily on basic science, but not vice versa. The result is that basic medicine is cited three to five times more than clinical medicine, and this is reflected in journal impact factors.”
Clinical Research & Citations
“
37
Researchers have difficulty getting funding.
38
Early-career researchers opt out of translational medicine.
39
Hospitals cannot showcase their talents.
40
41
Clinical Citations
Clinical Alerting Services
Clinical Guidelines
Systematic Reviews Clinical Trials
42
DynaMed Plus Topics• Clinical reference tool• Designed for point-of-care• Content: Extensive network of clinical
experts– Review 500 journals and 120 guideline
organizations– select the best and most appropriate evidence– confirm the clinical applicability of content– peer-review topics
43Examples of guideline organizations include:
• American Academy of Neurology (AAN)• American Academy of Pediatrics (AAP)• American Academy of Clinical Endocrinologists
(AACE)• American Association for the Study of Liver
Diseases (AASLD)• American College of Cardiology (ACC)• American College of Chest Physicians (ACCP)• American College of Obstetricians and
Gynecologists (ACOG)• American College of Physicians (ACP)• American Diabetes Association (ADA)• British Thoracic Society (BTS)• Canadian Cardiovascular Society (CCS)• Canadian Medical Association (CMA)• Canadian Task Force of Preventive Health
(CTFPHC)• Centers for Disease Control and Prevention
(CDC)
• European Respiratory Society (ERS)• European Society for Medical Oncology
(ESMO)• European Society of Cardiology (ESC)• Global Initiative for Chronic Obstructive Lung
Disease (GOLD)• Infectious Diseases Society of America (IDSA)• International Society on Thrombosis and
Haemostasis (ISTH)• National Comprehensive Cancer Network
(NCCN)• National Health & Medical Research Council
(NHMRC) [Australian guidelines]• National Institute for Health and Care
Excellence (NICE)• Pan American Health Organization (PAHO)• Royal College of Physicians (RCP)• Scottish Intercollegiate Guidelines Network
(SIGN)• World Gastroenterology Organisation (WGO)• World Health Organization (WHO)
44Showing when research makes its way into clinical practice
45
Different artifact types reveal different impacts
46A research study used for clinical advice
47Meta analysis of studies used to determine clinical practice
48Researchers contributing to clinical guidelines
49
Compare like with like
50
ANALYZE
• By metrics category• By type of output• By person• By grant• By department• By institution
51
PlumX: Built for Big Data• Built for Scale – Our goal is to track all the
world’s researchers, all of their output, and metrics for each.
51
>18 Million Researchers Worldwide
>120 Million Scholarly Articles
Map artifacts people groups
52
People not Papers
52
53
PlumX Benchmarks• Gather research output associated with
NIH grants• Calculate metrics across all 5 categories• Compare institutions using altmetrics
2012 - 2015
54
PlumX Benchmarks Features
• Select specific institutions to benchmark• Filter
– By Spending Category– By Granting Institute and Center– By Geography
• Rank institutions– For each category of metrics
• Analyze
55See Your Clinical Impact Compared to Peer Institutions
57
Online users expect metrics
58
59
60
Plum Printshelp
showcaseresearchers
61
62
Using metrics to gain an advantage in finding grants.
63
Big Swings in NIH Funding• $3 billion
no longer being spent on HIV research
• $65M refocused on ending AIDS epidemic
64
State of the Union Address: January 12, 2016“Let’s make America the country that cures cancer once and for all.”
65
Vice President Biden’s Memo“Over the next year, I will lead a dedicated, combined effort by governments, private industry, researchers, physicians, patients, and philanthropies to target investment, coordinate across silos, and increase access to information for everyone in the cancer community,”
66
http://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=20&catId=2
4 in 5people doNOT
get funded51,073 Applications 9,241 Awards
18% success rate
NIH: The Competition for Research $$$ in 2014
67
Finding Funding Opportunities can leverage metrics
68
69
Search for HIV FundingOpportunities
in Health
70
Link directly to thegrant detail
71
FundingOpportunity
In PlumXDashboard
Funding Opps Integrates with PlumX Dashboards
72
Where we’re heading…Automatically
TargetingFunding
Opportunities
XXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
73
Feedback changes behavior
74
Metrics that update in real time provide a Feedback Loop
Feedback loops help
change behavior.
75
Tools to analyze your stars• Established researchers
– What are your key researchers doing well?– How do you measure that?
• Early career researchers– How are they successfully establishing
themselves?• Build archetypes by discipline
– What types of outputs are important?– How should these be measured?
76
Summary• Altmetrics can be used to uncover and tell
the story of your research.• We are defining (and expanding)
– Traditional categories of metrics• Usage• Citations
– As well as modern metrics• Captures• Social Media• Mentions
77
Summary• Clinical citation counts are the latest
innovation by Plum Analytics.• They can help to tell the story about
clinical and translational research.
78
Answer the questions and tell the stories about your research
Find out more information at:http://www.plumanalytics.com
79
The PlumX SuitePlumX MetricsGet more value from your institutional repository.
PlumX DashboardsKnow what is going on with your research. Without waiting years.
PlumX +GrantsGain insight about your grants.
PlumX BenchmarksSee how you compare. Now.
PlumX Funding OpportunitiesFind the right funding opportunities.