assessing the quality of population size estimates of people who inject drugs (pwid)

ASSESSING THE QUALITY OFPOPULATION SIZE ESTIMATES OF PEOPLE WHO INJECT DRUGS (PWID)Waimar Tun, Population Council

20th International AIDS Conference Melbourne, AustraliaJuly 20 – 25, 2014

Background• Current PWID population size

estimations (PSE) in many countries are not based on strong data

• UNODC and World Bank requested a review of existing PWID population size estimates in 10 countries – Belarus, China, India, Libya, Myanmar,

Philippines, Kazakhstan, Kyrgyzstan, Tajikistan, & Uzbekistan

Learning objectives• Be able to critically review existing

estimates and their methodology

• Be able to identify opportunities to improve the estimates (if required)

• Understand the strengths and weaknesses of methods of PSE data collection

Where do I get my data?• Published and grey literature (including

HIV- and drug- related country reports)

• Discussions with stakeholders in-country (NAP/NAC, UNAIDS, WHO, UNODC, CDC, USAID, PWID representatives, civil society organizations)

• Stakeholder meeting with representatives from MoH, drug control agencies, civil society, and implementing partners

Comment on data sources• PSE methodologies not or poorly

specified• Reports not translated or translations

of technical terms were ambiguous• Stakeholders not always aware of

PSE activities happening in their own country

• Time-consuming

What should I consider when I review the quality of the estimates?

• Are the underlying assumptions of the method met?

• What are the potential biases and how do they impact the estimate?

• Are multiple methods used?• What is the quality of the data used?

Common methods to estimate PWID population size

Literature review/desk exercise

• Strengths:– Low cost– Little

time/resources required

• Weaknesses:– Local contexts may

be very different– Data sources may

not be based on rigorous methods

• Review published and grey literature for similar context and geographic region

• The benchmark from literature is applied to the adult male and female population

Delphi

• Strengths:– Utilizes local expert

views and experience– Does not require raw

data capture– May be only option

for countries with limited data sources

• Weaknesses:– Sometimes based

only on qualitative or anecdotal information

– Not good for identifying trends, comparing to other regions

• Systematically solicits and reviews selected experts’ estimates

• Iterative process through a series of feedback and revisions

Mapping/census and enumeration

• Strengths:– Is a real count, not

an estimate– Can produce a

credible lower limit

• Weaknesses:– PWID not always accessible,

may not be exposed to census data collectors

– Assumes you can locate all PWID

– PWID may not wish to reveal drug use due to stigma and/or legal concerns

– Highly time consuming and expensive

– Safety issues; dangerous hotspots

• Venues where PWID congregate are identified • Census counts all PWID at all hotspots• Enumeration counts PWID at a sample of sites

Capture-recapture

• Strengths:– Fairly easy to

implement in short period of time

– Usually cost-efficient to implement

– Can be done with multiple service sources

• Weaknesses:– Assumptions of

method difficult to meet in reality (independent samples)

– May be dangerous to implement (unsafe hotspots)

• PWID are counted and ‘tagged’; a 2nd count is conducted

• An estimate is obtained through a formula that includes captures and overlaps between the 2 rounds

Service multipliers

• Strengths:– Uses existing and

available data from service providers

– Easily incorporated into IBBSS with minimal additional questions

• Weaknesses:– High-quality service

data may not be available • No duplicates• Each target population

member must have chance of being included in service data

• Requires two data sources:1. Benchmark (service data such as drug treatment or HIV testing)2. Population-based survey with PWID where you obtain info on the

proportion who report that point of contact (‘benchmark)Estimate obtained from multiplying inverse of proportion to the benchmark

Wisdom of the crowd

• Strengths:– Very easy to

implement. Only one question.

– Easily incorporated into IBBSS or other size estimation methods

• Weaknesses:– May be biased if large

segment of population is not well-networked or “hidden”

– Bias if not implemented in a representative survey

• Ask PWID survey participants to estimate the number of PWID in a given location

• Assumes that the average response approximates the actual number

RDS† successive sampling size estimator‡

• Strengths:– Easily calculated with

existing RDS survey data

• Weaknesses:– Statistical validity

currently under debate– Not recommended as a an

“only” method of estimation.

– Results may be biased depending on number of people surveyed and actual population size

• In RDS-based survey, respondents indicate their network size

• Modelling is based on the assumption that those with large networks are sampled first and that the population will be depleted at a certain point

† Respondent-driven sampling; ‡ Handcock, Gile, Mar (2012)

Network scale-up method† (NSUM)

• Strengths:– Does not ask

sensitive questions directly to respondent

– National level estimate

• Weaknesses:– Average personal network

size difficult to estimate – Some PWID may not interact

much with members of the general population

– Respondent may not be aware that someone in their network engages in injection drug use

• Uses general population survey; questions about:• Number of people they know of a known population• Number of PWID they know

† Bernard, Killworth, Johnsen, and Robinson (1991)

Triangulation of multiple methods• Data points from multiple methods

are desirable when possible– Reduces bias from any single method–May provide plausible lower and upper

bounds– Informs stakeholder debate– Facilitates consensus on estimate

ranges

PWID population size estimation (2011 IBBSS, Nairobi)

STD HIV testing 1 Literature review

Drop-in HIV testing 2 WOTC0

5,000

10,000

15,000

20,000

25,000

20,833

13,250

6,562 5,869 5,0313,000

Median 6,107 Lower plausible 5,031

Upper plausible 10,937 (~0.5% adults)

Source: Population Council, UCSF, NASCOP/Kenya, CDC (2011)

RDS SS-Size Added

STD

HIV testin

g 1

RDS SS-S

ize

Litera

ture r

eview

Drop-in

HIV testin

g 2WOTC

0

5,000

10,000

15,000

20,000

25,00020,833

13,25011,463

6,562 5,869 5,0313,000

Source: Population Council, UCSF, NASCOP/Kenya, CDC (2011)

Lower plausible 5,031Median 6,107

Upper plausible 10,937 (~0.5% adults)

Cost

Scie

ntifi

c rig

or

Straw manConventional Wisdom

Borrow from thy neighborSoft modeling

Consensus

Wisdom of the crowdDelphi

Registries, police, SHC, drug treatment, unions, workplace

Discrepancies

Place, RAP, ethnography

Unique event multiplier

Truncated PoissonMultipliers, multiple multipliers

Multiple sample recaptureCapture-recapture

Network scale upPopulation-based survey

Census

Nomination counting

Unique object multiplier

Mapping, key informants, observation counting

Scientific rigor and costs of methods

Source: University of California, San Francisco

RDS – Sequential Size

Cost

Scie

ntifi

c rig

or



Consensus

Wisdom of the crowdsDelphi


Discrepancies






Census

Nomination counting



No resources or opportunity for data collection



Cost

Scie

ntifi

c rig

or



Consensus



Discrepancies






Census

Nomination counting



Data collected directly from PWID for size estimation purposes only



Cost

Scie

ntifi

c rig

or



Consensus



Discrepancies






Census

Nomination counting



Data collected from general population (DHS, AIDS Indicator Survey)



Cost

Scie

ntifi

c rig

or



Consensus

Wisdom of the crowdDelphi


Discrepancies






Census

Nomination counting



Data from PWID collected for other purposes (IBBSS, registries, service data)



IBBSS integration• Size estimation methods increasingly

being integrated worldwide– Leverages existing resources– Adds value to behavioral and seroprevalence data

already being collected• RDS increasingly used for IBBSS

recruitment– “Population-based” estimates

• Forthcoming RDS software-based estimation (SS-Size)– Provides an estimate using existing IBBSS RDS network

data– Has limitations and caveats, should not be used as a

sole estimate source

Other considerations• Ethical reviews• Administrative approvals• Safety of research assistants/study

team• Involvement local drug using

community representatives

Conclusion• Important to review how researchers

arrived at the estimate since many are not grounded in quality data

• Multiple PSE methods should be used• PWID size estimation should be a

part of routine surveillance• Stakeholder consensus on estimate

ranges critical

Acknowledgement• Scott Geibel (Population Council)• Henry Fisher Raymond (UCSF)• Abu Abdul-Quader (CDC)• Pandu Harimurti (The World Bank)• Riku Lehtovuori (UNODC)

GROUP DISCUSSION

Country A• Epidemic concentrated in PWID (account for

~90% of HIV transmission)• HIV prevalence in PWID: 15-30% (up to 87% in

one city)• Civil unrest has hindered PSE of PWID• PSE (2,000) based on government registration

of drug users† (0.05% of population nationally)• RDS-based IBBSS was conducted in 2013 in

one city; no PSE• No IBBSS planned for near future

† Registries based on treatment registers to arrest counts.

Country B• Epidemic concentrated in PWID, FSWs and their

clients; PWID HIV prevalence: 7%• PSE available for 25 out of the country’s 28

states• National PSE (177,000; 0.02% of population)

obtained through:– District-level mapping/enumeration at hotspots – Data updated regularly (by NGOs that implement

targeted intervention at hotspots (WOTC with PWID and gatekeepers at hotspots)

• IBBSS conducted every 2-3 years; currently being conducted but no PSE

Country C• HIV epidemic concentrated in PWID (account for ~60% of

HIV transmission)• Prevalence of PWID: 100,000-200,000 (~0.9% of

population)• Has extensive epi-behavioral data, including PSE;

regulated by government• Latest published PSE available from 2010/11

– Methods and quality varies across region– National PSE obtained from summing regional results

• Current/Upcoming activities:– 2014 RDS-based IBBSS with service multiplier (6 sites); 7

multipliers being used– Some regions will include cap-recap with independent databases– NSUM (2012/13)

Country D• Epidemic concentrated (PWID, MSM, FSW/clients)

– PWID HIV prevalence: 18%• Prevalence of PWID

– Range: 60,000-195,000 – Stakeholder consensus (2002): 75,000 – Estimate based on 0.5% of the population being PWID;

this may be based on registration of drug users• IBBSS completed in May 2014; includes PSE using

service and unique object multiplier– Conducted in 16 sites (14 are in high opium-growing

states and two are major urban centers)– Injection drug use is believed to be occurring outside of

these sites as well

Questions for Group Work• What are the potential

problems/biases with the current estimate?

• What kind of opportunity can you identify for improving the estimate? What are possible next steps?

assessing the quality of population size estimates of people who inject drugs (pwid)

Documents

country estimates

literature review

pwid datamethodology

pse data collectionso

existing estimates

joint pwid data review

pwid representatives

country napnac