Personal Strategies for Improving your REF Publications

Andrew Booth, Director of Information, ScHARR

Outline

• ScHARR REF Audit• Personal Publication Strategies

– Ian De Pendent– Matt O'Dologhy– Ann Hahnze– Di Kotomos– Enid Chiate

• Profiles and Practical Suggestions• REF Impact Case Studies and Individual

Researchers

ScHARR REF Audit• REF Census – Publications (2008-2009)

and Potential Impact Stories (2009- backwards)

• 116 staff returned forms• 18 staff (maternity leave, sick leave, non-

compliant) estimated returns (Mendeley database of 1201 ScHARR publications) at

• 24 queries regarding eligibility/statusTotal = 158 individuals

How Are We Doing?

• 67/158 [42%] had 4+ publications

• 9/158 [6%] had 3 publications

• 13/158 [8%] on track for 4 publications

Total 89/158

• Caveat: No consideration of Eligibility↓, no consideration of Quality of Journals↓, no consideration of Congruence↓ but also no concession for WTE↑

That’s Good Isn’t It?

• Yes, but……

……ScHARR research typically conducted as multidisciplinary teams

……Individual papers can only be submitted once

……Therefore we are most interested in First Authored Papers and ScHARR-

First Authored Papers

Because….?

• First Authored Papers – Not affected by collaborators from other institutions or by staff joining ScHARR (11 staff have 4 first authored papers; 41 have 2+)

• ScHARR-First Authored Papers – Not affected by ScHARR staff higher up “pecking order” (35 staff have 4 papers as first or only ScHARR author)

• Also interested in congruence between papers and any Unit of Assessment, but particularly HSR.

And What About Citations?

• Average number of citations was just under 2/3 of a citation (Mean = 0.63; median = 0; mode = 0)

• Highest total was 59 citations from 6 papers (including 33 for one pre-ScHARR paper)

• Most higher impact papers had clinical leadership/involvement and published in clinical journals

Current Verdict

• We work hard, we are prolific, we have a good environment and perform well for esteem (as for RAE)

• However we appear to privilege quantity over quality and have very few high impact papers (hence Power Rating for RAE)

• Is our work good enough? • Do we aim high enough? ?• Are we in our comfort zone? ?

To Illustrate

4/6 articles

?

+1 in 31st Journal; 1 with No Impact Factor

What Can We Do Collectively?

• Target “better” journals

• More strategic discussion re: Authorship

• More partnerships with Clinicians

• More synthesis of Methodology across similar projects

• More follow up work/secondary data analysis looking at Impact of Primary Work (c.p. DEC evaluation)

ISI Health Care Sciences and Services [1-10]

Only 7 of these would get into Top 20 for Medicine

ISI Health Care Sciences and Services [11-20]

MEDICINE, GENERAL & INTERNAL (Top 20)

Health Technology Assessment would appear here

ScHARR’s Top 20 Papers during REF Period so far

• BMJ (2)• Lancet (2)• Quality & Safety in Health

Care---------------------------• Medical Decision Making• Pharmacoeconomics (2)• Value in Health---------------------------• 8 with ScHARR 1st Author• 3 RCTs, 2 SRs, 3

Outcomes, 4 Cost Effect.

• Acad Emerg Med• Age and Ageing• Am J Kidney Diseases• Arthritis & Rheumatism• Breast Cancer Research &

Treatment • Diabetic Medicine• Eye• European Journal of Cancer• J Clin Endocr & Metabol• Nephrology Dialysis

Transplantation

Any Questions/Comments?

Profile – Ian De Pendent

• Typically PI• Typically 2 or More

Projects over period• 4 or More ScHARR

First Authored Publications

Personal Publication Strategy?

No!

Personal Publication Strategy?

• Aim to Improve One or More of Your Journals (from Citation List) or Papers

• Write with Clinician

• Counts Double!: Better Paper for You, Releases One to ScHARR Colleague

• Look at Co-Authoring as Second Author with ScHARR Colleague

• Mentoring

Profile – Matt O’Dologhy

• Typically Health Economist, Modeller, Statistician or Info Scientist

• Involved in Many Projects, some Clinician-led

• 3-4 Publications, none as First Author

Personal Publication Strategy?

• Second-Authored Papers with Non-ScHARR Clinician

• First-Authored Papers (Perhaps as PI/Project Manager)

• 1-2 Methodology Papers Synthesising Findings from Multiple Projects

• Aim to get Methodology Papers Open Access (BMC/White Rose)

Profile – Ann Hahnze

• Two or Three Papers in Peer Reviewed journals

• May be PI on One or More Projects

• Spending More Time “Doing” than Writing?

Personal Publication Strategy?

• Aim to Produce More Papers [1 or 1.5 per Writing Week)

• Try to “Trump” Existing Papers by Publishing in Higher Impact Journals

• Protected Time

• Team Writing

• Peer Support

Profile – Di Kotomos

• Quite new to HSR• Mixed portfolio –

disciplinary (e.g. Psychology, IS, or Social Science) and topical publications

• E.g. Reviewer with previous early stage research or Info Specialist also involved in reviews

• Potentially split across Units of Assessment

Personal Publication Strategy?

• Need to Prioritise One Half of Portfolio• Discuss with Line Manager and Unit of

Assessment Leads• Then Focus on Replacing Current Papers

as First ScHARR Author with those that fit Target UoA

• May Release Papers to Other Co-Authors• Also Contribute to Team Writing for

General Career Progression

Profile – Enid Chiate

• Recent entrant to HSR

• Mainly working on one project at a time

• One or two papers as member of research team

Personal Publication Strategy?

• Identify Sub-Project or Re-analysis as Potential First Author Paper

• Target a Mid-to-Higher Range Journal

• Take advantage of Mentoring, Writing Club and Publications Training Courses

• Continue to Participate in Team Writing

• Discuss Timescale for REF Eligibility with Line Manager

Any Questions/Comments?

Practical Suggestions/Reminders

Targeting Journals and Monitoring Impact

The Journal Finder

• Research Gate's The Journal Finder.

Research Gate - scientific network to connect researchers, find research partners, collaborate with scientists and explore journal articles.

• With more than 20,000 print Journals, Journal Finder helps discover which journals are most relevant to your research.

• Copy and Paste your article's abstract into their semantic search algorithm, and it will identify relevant journals.

• Includes information on publication restrictions as well as more information (e.g. impact factor) about journal.

Demonstration

Tip - Look through your list of references

Journal titles that appear several times are candidates for your article.

http://mulford.meduohio.edu/instr/

http://mulford.meduohio.edu/instr/

Web of Science analysis

http://wok.mimas.ac.uk/

Analyze Results

Demonstration

If at first you don’t succeed…planning for contingencies

Use the “stable” approachE.g. BMJ - Annals of Rheumatic Diseases

Sexually Transmitted Infections Archives of Disease in Childhood Thorax British Journal of Ophthalmology British Journal of Sports Medicine Injury Prevention Journal of Epidemiology & Community Health Emergency Medicine Journal Occupational and Environmental Medicine Gut Tobacco Control Heart Journal of Clinical Pathology Journal of Medical Ethics Journal of Medical Genetics Journal of Neurology, Neurosurgery and Psychiatry Medical Humanities Postgraduate Medical Journal Quality & Safety in Health Care

Recycle, recycle, recycle

• Three strikes and you’re out! – try up to three times

• Take on comments and suggestions (especially suggested target journals)

• Consider slicing or repackaging.

Researcher ID

• Accessed through University Library -> Web of Science -> Additional Resources

• Creates Unique Identifier for you• You can link yourself to your own Web of

Science Publications• You then have permanent profile for your

outputs• Updated for all stored publications• Add each year’s publications as you go

Increasing your Impact through White Rose

Research Online

WRRO: go for it

• Increase visibility & citation impact

• URLs for long term links

• Supports Open Access– Required by funders– Free to all

• WRRO FAQs at http://eprints.whiterose.ac.uk/docs/faq.html

Contacts

• Marion TattersallResearch Development LibrarianGeneric research support issues m.tattersall@sheffield.ac.uk phone: 27281 Mon-Thurs

• Anthea TuckerScience Engineering and Medicine Liaison Team assistantWRRO practicalities a.l.tucker@sheffield.ac.uk phone:27318

• Rachel ProudfootWhite Rose Repository Officer WRRO policy and development issues r.e.proudfoot@leeds.ac.uk tel: 0113 343 7067

Final Reminders

• Target Appropriate Journals• Aim to Trump Existing Journal Portfolio• Plan Strategically – Your “liberated”

publications may swell someone else’s portfolio!

• Peer Support/Mentoring etcetera • Remember: It is not Authorship (or even

necessarily First Authorship) But It Is First ScHARR Authorship that is Critical

Brief Word About Impact

Overall excellence

Outputs Impact Environment

60%?? 25%??

↓15%??

NB. Impact replaces Esteem; Some Esteem subsumed within Environment

Assessing impact

• “Significant additional recognition where institutions and researchers have built on excellent research to deliver demonstrable benefits to the economy, society, public policy, culture or quality of life”.

Principles of Impact

• Impact of submitting unit as whole, not impact of individual researchers.

• Submissions to provide examples of research-driven impact from unit’s broad portfolio of work.

• Impacts to be underpinned by high-quality research. (Focus of REF on research excellence, with additional recognition for strong impact built on that excellence).

• Because of time-lags, impact must be evident during REF assessment period, but research may have been undertaken earlier (up to 10-15 years earlier).

Assessment of Impact

• Qualitative information informed by appropriate indicators.• Submissions to include:

– Impact statement, generic template for submitted unit as a whole. Breadth of interactions with research users and overview of positive impacts that became evident during assessment period.

– Case studies, generic template to illustrate specific examples of impact and how unit contributed to them.

• One case study/5-10 members of staff.• Case studies and impact statement to include appropriate

indicators of impact, to support narrative evidence.

Draft template for impact case studies

1. Title of case study.2. Describe/provide evidence of specific benefit or impact (maximum

500 words), including:– nature of impact; how far-reaching and significant benefits are– appropriate indicators of impact (from “common menu”)

3. Explain how unit’s research activity contributed or led to impact (maximum 500 words), including:– outline of underpinning research, when undertaken and by whom– efforts made by staff to exploit or apply findings or secure impact through

research expertise– any other significant factors or contributions to impact.

4. Provide references to:– key research outputs that underpin the impact– external reports or documents, or contact details of a user, to corroborate impact

or unit’s contribution

How Impact Assessed

• Assessed by REF expert sub-panels, comprising people who understand research in discipline and its wider use and benefits supplemented with research users to assess impacts.

• Sub-panels to assess impact against criteria of:– reach (how widely impacts have been felt);– significance (how transformative impacts have

been). • Subprofile - proportion of impacts meeting each

level on five-point scale. Four-star (‘exceptional’) rating - ‘ground-breaking, transformative or of major value, relevant to a range of situations’.

Sample Indicators: Improved patient care or health outcomes

• Research income from NHS and medical research charities• Measures of improved health outcomes (for example, lives saved,

reduced infection rates)• Measures of improved health services (for example, reduced

treatment times or costs, equal access to services)• Changes to clinical or healthcare training, practice or guidelines

(including refs in relevant docs such as National Institute for Health and Clinical Excellence guidelines)

• Devt of new/improved drugs, treatments or other medical interventions; nos. of advanced phase clinical trials

• Participation on health policy/advisory committees• Changes to public behaviour (for example, reductions in smoking)

Impact in the Context of the REF

• King’s College, Friday 25th June 2010• Report of Impact Pilots• All Powerpoints Available from:

http://www.kcl.ac.uk/iss/support/ref/june2010

HEADLINES:• Measuring Impact is feasible but time-

consuming• Final weighting not decided yet• Requires skills in marketing, media, journalism• Can’t really handle “negative impact”

The Way Forward

• We have asked Staff to Nominate Impact Stories• Combined List shared with Research Committee• At RC Meeting tomorrow initial list of 8 Case

Studies will be compiled• Approximately 3 Case Studies will be worked up

over Summer (Pending fuller guidance)• We will be working with PIs and Team Members• We will probably need to develop 2:1 but

valuable for other purposes e.g. Marketing

Any Last Questions or Comments?