weight trimming and weight smoothing methods in surveys · weight trimming and weight smoothing...

Weight trimming and weight smoothing methods insurveys

David Haziza

Departement de mathematiques et de statistiqueUniversite de Montreal

SAMSI Transition Workshop

May 6, 2014

David Haziza (Universite de Montreal) Weight trimming May 6, 2014 1 / 39

Basic concepts

Let U be a finite population of size N.

Goal: estimate the population total, ty =∑

i∈U yi .

A sample s is selected according to a sampling design p(s) withinclusion probabilities π1, . . . , πN .

We consider linear estimators of ty of the form

ty =∑i∈s

wiyi ,

where wi is the survey weight attached to unit i .

Basic concepts

i∈U yi .

ty =∑i∈s

wiyi ,

Basic concepts

i∈U yi .

ty =∑i∈s

wiyi ,

Weighting in surveys

Each unit i is assigned a basic (or design) weight. Basic weightingsystem: {di = π−1

i ; i ∈ s}.

Nonresponse adjustment: The basic weights of responding units areadjusted to compensate for the non-responding units.

Weighting system adjusted for nonresponse: {w∗i = di/pi ; i ∈ sr},where sr is the set of sample respondents.

Calibration adjustment: The weights w∗i are further adjusted tomatch known population counts.Final (or calibrated) weighting system: {wi = w∗i gi ; i ∈ sr}.

Steps: dinra−→ w∗i

calibration−→ wi

i ; i ∈ s}.

Classical estimators

In the absence of nonsampling errors, the basic weighting system{di = π−1

i ; i ∈ s} leads to the Horvitz-Thompson estimator:

tHT =∑i∈s

diyi .

Properties of the Horvitz-Thompson estimator:

design-unbiased for ty : Ep(tHT ) = ty .Fixed size sampling designs: if yi ∝ πi then tHT = ty for any sample s;i.e.,

Vp(tHT ) = 0.

We expect tHT to be very efficient if y is approximately proportionalto π.

tHT =∑i∈s

diyi .

Vp(tHT ) = 0.

tHT =∑i∈s

diyi .

Vp(tHT ) = 0.

Suppose that a Q-vector of auxiliary variables x = (x1, . . . , xQ)> isavailable for i ∈ s and that the vector of population totals(benchmarks)

tx = (tx1 , . . . , txQ )>

is known, where txq =∑

i∈U xqi , q = 1, . . . ,Q.

In the absence of nonsampling errors, the calibrated weighting system{wi = digi ; i ∈ s} leads to the calibration estimator:

tC =∑i∈s

wiyi ,

where the calibrated weights are such that∑i∈s

wixi = tx.

That is, the calibrated weighting system {wi ; i ∈ s} ensuresconsistency with known population totals.

tx = (tx1 , . . . , txQ )>

i∈U xqi , q = 1, . . . ,Q.In the absence of nonsampling errors, the calibrated weighting system{wi = digi ; i ∈ s} leads to the calibration estimator:

tC =∑i∈s

wiyi ,

wixi = tx.

tx = (tx1 , . . . , txQ )>

i∈U xqi , q = 1, . . . ,Q.In the absence of nonsampling errors, the calibrated weighting system{wi = digi ; i ∈ s} leads to the calibration estimator:

tC =∑i∈s

wiyi ,

wixi = tx.

Properties of calibration estimators:

asymptotically design-unbiased for ty : Ep(tC ) ≈ ty provided that thesample size n is large enough.If yi = x>i β for a vector of constants β (i.e., there exists a perfectrelationship between y and x), then

tC = ty

for any sample s; i.e.,MSEp(tC ) = 0.

We expect tC to be very efficient if there is a linear relationshipbetween y and x and the vector x explains the y -variable well.

The class of calibration estimators includes the GeneralizedREGression (GREG) estimator as a special case.

tC = ty

Simplest special case: xi = 1 for all i and tx = N. In this case, tCreduces to the Hajek estimator

tHA =N

with NHT =∑

i∈s di .

Properties of the Hajek estimator:

asymptotically design-unbiased for ty : Ep(tHA) ≈ ty provided that thesample size n is large enough.If yi = β for some constant β then tHA = ty for any sample s; i.e.,

MSEp(tHA) = 0.

We expect the Hajek estimator to perform well if yi and πi areunrelated.

tHA =N

with NHT =∑

i∈s di .

MSEp(tHA) = 0.

tHA =N

with NHT =∑

i∈s di .

MSEp(tHA) = 0.

Large variability of the weights

Large variability in the survey weights is often associated withunstable estimators

Often true but not necessarily true

Counterexamples:

Use of tHT if yi ∝ πi .Use of tHA if yi = c .

Combination of two factors can lead to inefficient estimators:

High variation in the weightsPoor correlation between the weights and the study variables.

Early references: Rao (1966) and Basu (1971).

In surveys with multiple characteristics, it is not rare to end up withhighly dispersed weights that are poorly related (or even unrelated) tosome study variables.

Counterexamples:

Large variation in the weights may result from

Unequal probability samplingNonresponse adjustmentsCalibration adjustments

At the nonresponse treatment stage, forming weighting classes mayhelp preventing an extreme variation in the weights.

At the calibration stage, it’s possible to ensure that the calibrationadjustment factors lie between pre-specified lower and upper boundsthrough an appropriate distance function → may also help preventingan extreme variation in the weights.

What can we do to limit the impact of dispersed weights?

A number of techniques has been proposed in the literature:

weight trimming methods;weight smoothing methods;

Different in nature but share a same objective:

modify the survey weights so that the resulting estimators have a lowermean square error than that of the usual estimators (e.g., theHorvitz-Thompson estimator)Reduction of the mean square error is generally achieved at the expenseof introducing a bias.Treatment of survey weights: can be viewed as a compromise betweenbias and variance.

What can we do to limit the impact of dispersed weights?

A number of techniques has been proposed in the literature:

weight trimming methods;weight smoothing methods;

Different in nature but share a same objective:

modify the survey weights so that the resulting estimators have a lowermean square error than that of the usual estimators (e.g., theHorvitz-Thompson estimator)Reduction of the mean square error is generally achieved at the expenseof introducing a bias.Treatment of survey weights: can be viewed as a compromise betweenbias and variance.

What is weight trimming?

Weight trimming consists of reducing the weight of units that areidentified as influential.

Influential units are those responsible for the high variance of classicalestimators; i.e., the units that contribute significantly to theinstability of the estimator.

The concept of influential units is not always clearly defined inpractice:

Units with a large weight wi?Units with large weighted value wiyi?Later, we define an appropriate influence measure called the conditionalbias of a unit.

We distinguish between two types of weight trimming methods:

the methods based only on the distribution of the survey weightsthe methods based on the distribution of the weights but that alsoaccount for the study variables.

Methods based only on the distribution of the weights

Idea: units with an extreme weight wi are influential and are”harmful”.

Let w0 be a cutpoint.

The weights after trimming are defined as

w0 if wi ≥ w0

γwi if wi < w0

where γ is a rescaling factor that ensures that∑i∈s

wi =∑i∈s

The estimator of ty after trimming is given by

ttrim(w0) =∑i∈s

wiyi .

w0 if wi ≥ w0

γwi if wi < w0

wi =∑i∈s

ttrim(w0) =∑i∈s

wiyi .

w0 if wi ≥ w0

γwi if wi < w0

wi =∑i∈s

ttrim(w0) =∑i∈s

wiyi .

w0 if wi ≥ w0

γwi if wi < w0

wi =∑i∈s

ttrim(w0) =∑i∈s

wiyi .

How determine the cutpoint value w0?

Adhoc method: w0 = cw for some constant c.

Potter (1988, 1990) provides one of the first formal treatment ofweight trimming.

Modeling the distribution of the weights: it assumes that thereciprocal of the the scaled sampling weights follows a betadistribution:

f (wi ) =nΓ(α + β)

Γ(α)Γ(β)(1/(nwi ))α−1(1− 1/(nwi ))β−1

The weights from the upper tail of the distribution, say where1− F (wi ) < .01, are trimmed to w0 such that 1− F (w0) = .01.

f (wi ) =nΓ(α + β)

Γ(α)Γ(β)(1/(nwi ))α−1(1− 1/(nwi ))β−1

f (wi ) =nΓ(α + β)

Γ(α)Γ(β)(1/(nwi ))α−1(1− 1/(nwi ))β−1

f (wi ) =nΓ(α + β)

Γ(α)Γ(β)(1/(nwi ))α−1(1− 1/(nwi ))β−1

The NAEP procedure (Potter, 1990): termed NAEP because of itsuse in the National Assessment of Educational Progress.

It consists of trimming the weights wi which are larger than

∑i∈s w

for a fixed c .

The procedure is iterated until all the weights are less or equal to thethreshold.

The excess weight is redistributed among the non-trimmed units.

Value of c : to be chosen empirically by examining the distribution ofnw2

i /∑

i∈S w2i .

∑i∈s w

for a fixed c .

i /∑

i∈S w2i .

∑i∈s w

for a fixed c .

i /∑

i∈S w2i .

∑i∈s w

for a fixed c .

i /∑

i∈S w2i .

∑i∈s w

for a fixed c .

i /∑

i∈S w2i .

Methods that account for the study variable

Potter (1990) suggested to find the cutpoint w0 that minimizes theestimated mean square error of the trimmed estimator ttrim(w0):

MSE (ttrim) = V (ttrim) +(ttrim − tHT

)2 − V (ttrim − tHT ).

Optimal trimmed estimator: ttrim(wopt0 ).

Unlike the previous ones, this method accounts for the y -variablebeing estimated.

An optimal cutpoint wopt0 for a given study variable may be far from

optimal for another study variable.

Minimizing the estimated mean square error is not an easy task, ingeneral.

Dalen-Tambay winsorization: Dalen (1987) and Tambay (1988)considered the winsorization

{yi if wiyi ≤ KKwi

(yi − Kwi

) if wiyi > K

The Dalen-Tambay winsorized estimator is given by

tDT (K ) =∑i∈s

wi yi =∑i∈s

wiyi ,

wi = 1 + (wi − 1)min

.If min

)= yi , then wi = wi .

For an influential unit: wi < wi .Nice property: wi cannot be smaller than 1.

(yi − Kwi

) if wiyi > K

tDT (K ) =∑i∈s

wi yi =∑i∈s

wiyi ,

wi = 1 + (wi − 1)min

If min(yi ,

(yi − Kwi

) if wiyi > K

tDT (K ) =∑i∈s

wi yi =∑i∈s

wiyi ,

wi = 1 + (wi − 1)min

.If min

(yi − Kwi

) if wiyi > K

tDT (K ) =∑i∈s

wi yi =∑i∈s

wiyi ,

wi = 1 + (wi − 1)min

.If min

For an influential unit: wi < wi .

Nice property: wi cannot be smaller than 1.

(yi − Kwi

) if wiyi > K

tDT (K ) =∑i∈s

wi yi =∑i∈s

wiyi ,

wi = 1 + (wi − 1)min

.If min

How to determine the cutpoint K for either the winsorized estimator?

A bad choice of K may affect the properties of the winsorizedestimators considerably ⇒ may even exhibit a mean square errorlarger than that of classical estimator (e.g., Horvitz-Thompson)!

First criterion: the value of K is set a priori

Generally, not a good idea;Should be determined using a more formal criterion.

Second criterion: Determine K that minimizes the estimated meansquare error of the winsorized estimator:

Requires historical information and/or a model for the study variable;Often requires simplifying assumptions;Complex to implement, in general;Kokic and Bell (1994), Hulliger (1995) and Rivest and Hurtubise(1995).

Third criterion: find the value of K that minimizes the maximumestimated influence:

Does not require historical information or a model;Does not require simplifying assumptions;Easy to implement in practice;see Beaumont, Haziza and Ruiz-Gazen (2013) and Favre-Martinoz,Haziza and Beaumont (2014).

What is an influential unit?

So far, units exhibiting a large weight wi (methods of Potter) or alarge weighted value wiyi (winsorized estimator) were considered asinfluential.

However, these influence measures do not seem to account for

the sampling design;the estimator used to estimate ty ;the parameter to be estimated (total, ratio, etc);

Alternative influence measure?

So far, units exhibiting a large weight wi (methods of Potter) or alarge weighted value wiyi (winsorized estimator) were considered asinfluential.

However, these influence measures do not seem to account for

the sampling design;the estimator used to estimate ty ;the parameter to be estimated (total, ratio, etc);

Alternative influence measure?

Before defining the concept of influential unit, we define that ofconfiguration.

A configuration is a quadruplet, which consists of

(1) a variable of interest;(2) a parameter of interest (to be estimated);(3) a sampling design;(4) a point estimator.

Examples of configurations:

(Revenue, total revenue, stratified simple random sampling, HTestimator);(Revenue, total revenue, stratified simple random sampling, calibrationestimator);(Revenue, total revenue, stratified Bernoulli, HT estimator);(Revenue, Portion of the annual revenue for the first quarter, stratifiedsimple random sampling, HT estimator).

(Revenue, total revenue, stratified simple random sampling, HTestimator);

(Revenue, total revenue, stratified simple random sampling, calibrationestimator);(Revenue, total revenue, stratified Bernoulli, HT estimator);(Revenue, Portion of the annual revenue for the first quarter, stratifiedsimple random sampling, HT estimator).

(Revenue, total revenue, stratified simple random sampling, HTestimator);(Revenue, total revenue, stratified simple random sampling, calibrationestimator);

(Revenue, total revenue, stratified Bernoulli, HT estimator);(Revenue, Portion of the annual revenue for the first quarter, stratifiedsimple random sampling, HT estimator).

(Revenue, total revenue, stratified simple random sampling, HTestimator);(Revenue, total revenue, stratified simple random sampling, calibrationestimator);(Revenue, total revenue, stratified Bernoulli, HT estimator);

(Revenue, Portion of the annual revenue for the first quarter, stratifiedsimple random sampling, HT estimator).

Influential unit

Let θ be a parameter to be estimated.

Let θ be an estimator of θ.

Definition

A unit is influential if, given a configuration, it has a significant impact onthe sampling error, θ − θ.

An unit may be influential with respect to a configuration and haveno influence with respect to another configuration.

Modifying a single element of the configuration may have a drasticimpact on the influence of an unit

How measure the influence of an unit?

Influential unit

Definition

Influential unit

Definition

Measuring the influence: the conditional bias

Let Ii be a sample selection indicator such that Ii = 1 if i ∈ s andIi = 0, otherwise.

Let i be a sample unit (i.e., Ii = 1).

The conditional bias attached to unit i with respect to θ is defined as

B1i = Ep(θ − θ|Ii = 1).

Interpretation of B1i : average of the sampling error taken among allsamples that contain unit i .

Conditional bias B1i : measure of influence of a sample unit.

see Moreno-Rebollo, Munoz-Reyes and Munoz-Pichardo (1999) andBeaumont, Haziza and Ruiz Gazen (2014).

B1i = Ep(θ − θ|Ii = 1).

Let i be a nonsampled unit (i.e., Ii = 0). The conditional bias of uniti with respect to θ is defined as

B0i = Ep(θ − θ|Ii = 0).

Interpretation of B0i : average of the sampling error taken among allsamples that do not contain unit i .

Conditional bias B0i : measure of influence of a nonsampled unit.

Although, a nonsample unit can have a large influence, nothing canbe done at the estimation stage because its y -value is not observed.At the estimation stage, only the influence of sample units can becurbed down.

B0i = Ep(θ − θ|Ii = 0).

Conditional bias: Horvitz-Thompson estimator

Goal: estimate ty =∑

i∈U yi

Horvitz-Thompson estimator of ty :

tHT =∑i∈s

diyi .

Conditional bias of sample unit i with respect to tHT :

BHT1i = Ep(tHT − ty |Ii = 1)

=∑j∈U

(πij − πiπjπiπj

where πij = P(i ∈ s & j ∈ s) denotes the joint inclusion probabilityof units i and j in the sample.

Conditional bias: Horvitz-Thompson estimator

Goal: estimate ty =∑

i∈U yi

Horvitz-Thompson estimator of ty :

tHT =∑i∈s

diyi .

Conditional bias of sample unit i with respect to tHT :

BHT1i = Ep(tHT − ty |Ii = 1)

=∑j∈U

where πij = P(i ∈ s & j ∈ s) denotes the joint inclusion probabilityof units i and j in the sample.

Remarks

The conditional bias generally depends on the second-order inclusionprobabilities, πij 7→ take the sampling design into account.

The conditional bias is, in general, unknown → it must be estimated

The influence of unit depends on the y -variable being estimated aswell as the sampling design.

If πi = 1 thenBHT

1i = 0.

Remarks

If πi = 1 thenBHT

1i = 0.

Remarks

If πi = 1 thenBHT

1i = 0.

Remarks

If πi = 1 thenBHT

1i = 0.

Conditional bias: special cases

Simple random sampling without replacement: πi = n/N for all i andπij = n(n − 1)/N(N − 1) for all i 6= j .

BHT1i =

(N − 1)

n− 1

)(yi − Y ),

where Y = Y /N is the population mean.

Poisson sampling: πij = πiπj , i 6= j

BHT1i = (di − 1)yi = (di − 1)(yi − 0)

BHT1i =

(N − 1)

n− 1

)(yi − Y ),

BHT1i = (di − 1)yi = (di − 1)(yi − 0)

BHT1i =

(N − 1)

n− 1

)(yi − Y ),

BHT1i = (di − 1)yi = (di − 1)(yi − 0)

Example

Consider a population of size N = 100 such that

y1 = 0, y2 = · · · = y99 = 500 and y100 = 1000⇒ Y = 500.

Simple random sampling without replacement: both units 1 and 100 have alarge influence because they are both far from the population mean Y = 500.

BHT1i =

(N − 1)

n− 1

)(yi − Y )

Poisson sampling: unit 1 has no influence, whereas unit 100 has a largeinfluence.

BHT1i = (di − 1) (yi − 0)

Example

y1 = 0, y2 = · · · = y99 = 500 and y100 = 1000⇒ Y = 500.

BHT1i =

(N − 1)

n− 1

)(yi − Y )

BHT1i = (di − 1) (yi − 0)

Example

y1 = 0, y2 = · · · = y99 = 500 and y100 = 1000⇒ Y = 500.

BHT1i =

(N − 1)

n− 1

)(yi − Y )

BHT1i = (di − 1) (yi − 0)

Poisson sampling vs. simple random sampling withoutreplacement

00 0 1000

Link between the conditional bias and the sampling error

For Poisson sampling, the sampling error can be written as:

tHT − ty =∑i∈s

BHT1i +

∑i∈U−s

BHT0i .

Interpretation: the conditional bias attached to unit i (sampled ornot) is a measure of its contribution to the sampling error.

For simple random sampling without replacement and high entropysampling designs:

tHT − ty ≈∑i∈s

BHT1i +

∑i∈U−s

BHT0i .

provided that the population size N is large.

BHT1i +

∑i∈U−s

BHT0i .

BHT1i +

∑i∈U−s

BHT0i .

BHT1i +

∑i∈U−s

BHT0i .

BHT1i +

∑i∈U−s

BHT0i .

Link between the conditional bias and the variance of tHT

For an arbitrary sampling design, the sampling variance of tHT isgiven by:

∑i∈U

∑j∈U

=∑i∈U

BHT1i yi

Calibration estimators

Conditional bias of tC attached to sample unit i :

BC1i = Ep(tC − ty |Ii = 1)

≈∑j∈U

where Ei = yi − x′iB denotes the census residual attached to unit i .

Recall: For tC , the conditional bias of unit i was given by the previousexpression with yi instead of Ei .

Conditional bias of tC attached to sample unit i :

BC1i = Ep(tC − ty |Ii = 1)

≈∑j∈U

where Ei = yi − x′iB denotes the census residual attached to unit i .

Recall: For tC , the conditional bias of unit i was given by the previousexpression with yi instead of Ei .

Poisson sampling:BC

1i ≈ (di − 1)Ei .

Simple random sampling without replacement:

BCi (Ii = 1) ≈ N

(N − 1)

n− 1

)(Ei − E )

Often, we have E = 0.

Conclusion: The conditional bias takes the type of estimator intoaccount.

Poisson sampling:BC

1i ≈ (di − 1)Ei .

BCi (Ii = 1) ≈ N

(N − 1)

n− 1

)(Ei − E )

Poisson sampling:BC

1i ≈ (di − 1)Ei .

BCi (Ii = 1) ≈ N

(N − 1)

n− 1

)(Ei − E )

Method based on the conditional bias

Proposed by Beaumont, Haziza and Ruiz-Gazen (2013).

Consider a robust estimator of tHT :

tBHR(K ) = tHT −∑i∈s

BHT1i +

∑i∈s

)where ψK (·) is the Huber function given by

ψK (x) =

K if x > Kx if|x | ≤ K−K if x < −K

Role of the ψ-function: reduce the influence of units that have a largeinfluence.

The tuning constant K may be determined so that it minimizes theestimated mean square error of tBHR(K ) 7→ usually complex withoutsimplifying assumptions.

Alternative choice of K : determine K which minimizesmaxi∈s

tBHR(Kopt) = tHT −1

min + BHTmax).

Easy to implement.

tBHR(Kopt) is design-consistent for ty .

Huber function ψ3(x) (with K = 3)

Remark: 0 ≤ ψK (x)x ≤ 1.

-10 -8 -6 -4 -2 0 2 4 6 8 10

The BHR estimator can be written as

tBHR(Kopt) =∑i∈s

diyi ,

wheredi = di −

whereαi = 1− ψK (BHT

1i )/BHT1i .

Poisson sampling: BHT1i = (di − 1)yi .

di = αi + (1− αi )di ⇒ 1 ≤ di ≤ di

Simple random sampling without replacement (assumingN/(N − 1) = 1):

di = φi + {1− φi}N

where φi = αi

(yi−Yyi

)If αi = 0 ⇒ φi = 0 ⇒ di = N/nIf αi = 1

⇒ di = 1 +

n− 1

di = αi + (1− αi )di ⇒ 1 ≤ di ≤ di

di = φi + {1− φi}N

where φi = αi

(yi−Yyi

)If αi = 0 ⇒ φi = 0 ⇒ di = N/nIf αi = 1

⇒ di = 1 +

n− 1

di = αi + (1− αi )di ⇒ 1 ≤ di ≤ di

di = φi + {1− φi}N

where φi = αi

(yi−Yyi

)If αi = 0 ⇒ φi = 0 ⇒ di = N/nIf αi = 1

⇒ di = 1 +

n− 1

di = αi + (1− αi )di ⇒ 1 ≤ di ≤ di

di = φi + {1− φi}N

where φi = αi

(yi−Yyi

)If αi = 0 ⇒ φi = 0 ⇒ di = N/nIf αi = 1

⇒ di = 1 +

n− 1

Weight smoothing

Weight smoothing: introduced by Beaumont (2008).

Idea: model the weights as a function of the study variablesy1, . . . , yp. That is,

wi = f (y1, . . . , yp;β) + εi .

Obtain the fitted weights:

wi = f (y1, . . . , yp; β).

Smoothed estimator:tsmooth =

∑i∈s

wiyi .

Weight smoothing

wi = f (y1, . . . , yp;β) + εi .

wi = f (y1, . . . , yp; β).

∑i∈s

wiyi .

Weight smoothing

wi = f (y1, . . . , yp;β) + εi .

wi = f (y1, . . . , yp; β).

∑i∈s

wiyi .

Weight smoothing

wi = f (y1, . . . , yp;β) + εi .

wi = f (y1, . . . , yp; β).

∑i∈s

wiyi .

Weight smoothing

wi = f (y1, . . . , yp;β) + εi .

wi = f (y1, . . . , yp; β).

∑i∈s

wiyi .

Weight trimming vs.weight smoothing

Weight trimming:trims only the weights of units that are identified asinfluential, whereas weight smoothing potentially modifies all theweights.

Weight trimming does not have to rely on the validity of a model tofind the optimal cutpoint, whereas weight smoothing requires a modelexpressing the relationship between the weights and the studyvariables. If the model is wrong, the smoothed estimator may bebiased.

Weight smoothing does require an optimal cutpoint, whereas weighttrimming requires an optimal cutpoint for study variable.

weight trimming and weight smoothing methods in surveys · weight trimming and weight smoothing...

Documents