Shortening the distance to research evidence

The librarian as knowledge translation activist

Raechel Damarell

Outline• Search filters

– Research– Implementation– Dissemination– Evaluation

• Flinders Filters: a new research initiative and opportunity for librarians

What is a search filter?• Tool to retrieve specific subset of literature • Much more than an expert search

– Developed experimentally following a strict (and usually transparent) research methodology

– Has a known level of performance in the database for which it was developed

• The InterTASC Information Specialists' Sub-Group Search Filter Resource

Why use a search filter?

Searching without a filter

Why use a search filter?Searching with a filter

Types of search filters• Methodological vs subject-based

– Cochrane HSSS; PubMed Clinical Queries

• 1st, 2nd, and 3rd generation – subjectively derived/effectiveness unknown– subjectively derived/effectiveness known– objectively derived/effectiveness known

Subject-based filters• Diagnostic imaging techniques• Heart failure• Nephrology• Return to work • Alcohol-impaired driving• Sleep• Ultrasound for DVT• Knowledge translation• Cardiovascular biology• Occupational origin of disease• Nurse staffing research• Palliative care

• Patient safety• Mental health• Spanish studies• Measurement properties of

measurement instruments• Administrative data studies• Primary health care• Occupation health

interventions• Evidence based nursing

Methodological filters• Adverse effects• Diagnostic studies• Economic evaluations• Epidemiological studies• Etiology• Guidelines• Health services research• Observational studies• Prognosis• Qualitative research• Quality of life• RCTs and other trials

• Systematic review• Therapy studies

Basic filter methodology• Establish an Expert Advisory Group

– Scoping conceptual boundaries– Preliminary consensus on potential search terms– Advice on Gold Standard Set

The Gold Standard Set• Set of representative citations of known

relevance indexed in the database of interest

• Target set: all citations in it should be retrieved by final filter

• Provides means of iteratively improving a search strategy

Gold Standard Set options• Journal ‘hand search’ (e.g. Palliative Care Filter)• Included references of:

– Clinical Practice Guidelines (Heart Failure)– Systematic Reviews (Lung Cancer)– All citations from range of specialty journals on topic (PHC)

• Others:– Delphi study method (Best Supportive Care)– Set retrieved by a search on the topic

Basic filter methodology• Establish an Expert Advisory Group • Division of Gold Standard Set

– Term Identification Set – Filter Development Set– Filter validation Set(s)

Performance measures

Sensitivity (or Recall)The number of relevant citations retrieved by the search filter from the gold standard as a proportion of the total number of relevant citations (100% = all relevant citations retrieved)

PrecisionThe number of relevant records retrieved as a proportion of the total number of records retrieved (100% = all citations retrieved relevant)

Specificity Number of irrelevant citations not retrieved (correct exclusions)

Basic filter methodology• Establish an Expert Advisory Group

– Scoping conceptual boundaries– Preliminary consensus on potential search terms– Advice on Gold Standard Set

• Division of Gold Standard Set– Term Identification Set; Filter Development Set; Filter

Validation Set(s)

• Implementation: making it available

Why PubMed?• Accessibility: Largest freely available

biomedical database • Currency: PubMed’s ‘in process’ and ‘as

supplied by publisher’ citations• Immediacy: Search filter can be saved as

hyperlink for one-click searching • Implementable: Can be embedded in any

webpage

The Palliative Care Filter• Developed in 2004 based on 1999-2001

dataset • Specificity = 99% but sensitivity <50% • Combined with 60+ topics of clinical

importance to palliative care clinicians• Subject of three published studies (4th

currently being written up)

What the users don’t see((advance care planning[mh] OR attitude to death[mh] OR bereavement[mh] OR terminal care[mh] OR hospices[mh] OR life support care[mh] OR palliative care[mh] OR terminally ill[mh] OR death[mh:noexp] OR palliat*[tw] OR hospice*[tw] OR terminal care[tw] OR 1049-9091[is] OR 1472-684X[is] OR 1357-6321[is] OR 1536-0539[is] OR 0825-8597[is] OR 1557-7740[is] OR 1552-4264[is] OR 1478-9523[is] OR 1477-030X[is] OR 0749-1565[is] OR 0742-969X[is] OR 1544-6794[is] OR 0941-4355[is] OR 1873-6513[is] OR 0145-7624[is] OR 1091-7683[is] OR 0030-2228[is]) AND Medline[sb]) OR ((advance care plan*[tw] OR attitude to death[tw] OR bereavement[tw] OR terminal care[tw] OR life supportive care[tw] OR terminally ill[tw] OR palliat*[tw] OR hospice*[tw] OR 1049-9091[is] OR 1472-684X[is] OR 1357-6321[is] OR 1536-0539[is] OR 0825-8597[is] OR 1557-7740[is] OR 1552-4264[is] OR 1478-9523[is] OR 1477-030X[is] OR 0749-1565[is] OR 0742-969X[is] OR 1544-6794[is] OR 0941-4355[is] OR 1873-6513[is] OR 0145-7624[is] OR 1091-7683[is] OR 0030-2228[is]) NOT Medline[sb]) AND English[la]

The Heart Failure Filter• Developed in collaboration with Centre for

Cardiovascular & Chronic Care (UTS) • Gold standard: Evidence based guidelines• Sensitivity: 97%; Precision: 75% • Methodology article published in

BMC Medical Research Methodology• Innovative methodology for PubMed translation

(article in progress)• Combined with 40+ topics of clinical importance to

heart failure/palliative care clinicians

The Heart Failure Filter• Developed in collaboration with Centre for

Cardiovascular & Chronic Care (UTS) • Gold standard: Evidence based guidelines• Sensitivity: 97%; Precision: 75% • Methodology article published in

BMC Medical Research Methodology• Innovative methodology for PubMed translation

(article in progress)• Combined with 40+ topics of clinical importance to

heart failure/palliative care clinicians

What the users don’t see((heart failure[tw] OR ventricular dysfunction, left[mh:noexp] OR cardiomyopathy[tw] OR left ventricular ejection fraction[tw]) AND Medline[sb]) OR ((heart failure[tw] OR left ventricular dysfunction[tw] OR cardiomyopathy[tw] OR left ventricular ejection fraction[tw] OR cardiac resynchronization[tw] OR LV dysfunction[tw] OR left ventricular systolic dysfunction[tw] OR left ventricular diastolic dysfunction[tw] OR cardiac failure[tw]) NOT medline[sb]) AND english[la]

Palliative Care Filter effectiveness studies

Study 1• Compiled more recent gold standard based on

included studies in 29 palliative care systematic reviews

• Found that Palliative Care Filter performed better in more recent dataset than in original one (˃ 5%)

Palliative Care Filter effectiveness studies

Study 2• Real life PubMed searches of pc clinicians for pc

literature (n=37) analysed • Each run in gold standard set and retrieval compared

with that of PCF• Results highlighted difficulties in searching:

– Very limited number of search terms used for complex topic– Incorrect search construction– Negative correlation between stated confidence and actual

performance (i.e. overconfidence is issue)– Search filter retrieved 68% of gold standard; average

clinician search retrieved 25%

Palliative Care Filter effectiveness studies

Study 3• Were palliative care clinicians aware of topic

searches built on the PCF?• Of those aware:

– Were they using them? If yes, what did they think? If no, why not?

• If not aware: – What did they think after using search? (intervention study)

• Follow up study– Were they still using searches after 6 weeks?

Flinders Filters

Flinders Filters

• Established in 2012 as Flinders co-funded research venture

• Committed to producing over 2 years:– Two health filters – Two non-health filters– Four research papers that make a

contribution to search filter methodology

Flinders Filters

Completed projects: • Primary Health Care Search Filter (PHCRIS)

Flinders Filters

PHC Search Filter launch

All welcome

Flinders Filters

New projects: • ‘Indigenous health’ and ‘Australian Aboriginal health’ filters

(Lowitja Institute)• Best Supportive Care (Duke University)• Dementia• Sexual health

Flinders Filters

Urgent issues for FF:• Non-health filters: What? Who with? Where

appropriate? • Finding time to write for publication • More librarians needed to join team

Thank you

Questions?