Under-Five Mortality Estimation

Assessing Summary Birth History Methods

with Microsimulation

Resumen ampliado para el VI Congreso ALAP

Andrea Verhulst1 and Bruno Masquelier2

Université catholique de Louvain

March 20, 2014

Abstract

In order to track progress on the under-five mortality rate for monitoring Millennium Development Goal 4, summary birth histories collected in surveys and censuses are a key source of data in many developing countries. Two kinds of methods for analyzing summary birth histories are now available to users: first, model-based methods derived from the pioneer work of Brass, second, empirically-based methods developed more recently by researchers at the Institute for Health Metrics and Evaluation. The performance of both approaches has not been evaluated extensively and needs further scrutiny. In this paper, we assess them comparatively against gold standard estimates generated by demographic microsimulations. This allows controlling for the data quality in order to focus on the modeling assumptions of the summary birth history methods.

1 F.R.S.-FNRS Research Fellow, contact: andrea.verhulst@uclouvain.be.

2 F.R.S.-FNRS Postdoctoral Researcher.

Estimad@ evaluador(a), el siguiente resumen ampliado es redactado en inglés, pero nos comprometemos en hacer una presentación oral en español.

Introduction

Millennium Development Goal 4 aims to reduce the under-five mortality rate (U5MR) by two thirds between 1990 and 2015. However, in many developing countries, tracking progress on the U5MR is impaired by the lack of comprehensive vital registration systems.

Most countries overcome this lack of well-functioning vital registration systems with two types of alternative data: full birth histories (FBHs) collected in surveys, and summary birth histories (SBHs) collected in surveys and censuses, both from women in reproductive age. In FBHs, mothers list all their live-born children and report their date of birth and, for those who died, their age at death. This information allows estimating the U5MR directly through an occurrence/exposure procedure. By contrast, SBHs only contain the number of children ever born (CEB) and children dead (CD) without location of births and deaths in time. An indirect procedure has to be used to approximate the length of exposure to the risks of dying and to derive estimates of U5MR.

Although direct estimates derived from FBHs are deemed more reliable (Silva, 2012), SBHs are easier and cheaper to collect, and they are still the only source of information in several countries for which census data exist but no recent survey has been conducted. In addition, surveys are of limited utility for making subnational estimates or estimates across many socio-economic strata (Dwyer-Lindgren et al., 2013). Thus, SBHs collected in censuses remain indispensable for this purpose.

Two kinds of indirect methods for analyzing SBHs are now available to users: first, model-based methods derived from the pioneer work of Brass (Brass and Coale, 1968), second, empirically-based methods developed more recently by researchers at the Institute for Health Metrics and Evaluation (IHME) (Rajaratnam et al., 2010). The performance of both approaches has not been evaluated extensively and needs further scrutiny. In this paper, we assess them comparatively against gold standard estimates generated by demographic microsimulations. This allows controlling for the data quality in order to focus on the modeling assumptions of the SBH methods.

Two approaches to derive U5MRs from SBHs

The Brass indirect method makes use of the age of mothers, or of another cohort duration function such as the time since first birth, as a proxy for the length of exposure to the risks of dying of their children. On this basis, the proportions of children who died, classified by 5-year duration group, are transformed in U5MR by using model age patterns of fertility and child mortality.

The standard variant of the Brass method uses either the Coale and Demeny or the UN mortality models (Hill, 2013a). This standard method includes a time location procedure permitting to estimate U5MR trends for the period approximately 15-20 years before data collection.

The method is known to be vulnerable to errors and biases related to (1) the data quality, (2) the violation of the modeling assumptions and (3) the selection of respon-dents. The first type includes principally recall errors in the reporting of CEB/CD. The

second type concerns biases resulting from potential violations of the following modeling assumptions: no variation of child mortality by 5-year classifying group, accuracy of the fertility and mortality models, constant fertility and gradual and unidirectional change of child mortality in the recent past. The third type proceeds from the association between the non-response of some mothers and the risk of dying of their children.

These errors and biases have already been explored in the literature (Ewbank, 1982; Arthur and Stoto, 1983; Garenne, 1984; Hill, 1984; Silva, 2012; Ward and Zaba, 2008). Estimates derived for the more recent time period, based on SBHs obtained from women aged 15-19 and 20-24, are known to be severely biased upwards by an “age effect”, that is the over-mortality related to the first births and the lower socio-economic background of younger mothers. The variant based on the time since first birth (Hill and Figueroa, 2001) which aims to reduce this kind of bias still needs to be evaluated. In contrast, the estimates associated with women aged 30-34, and referring to the period around 6 years before the data collection, is considered robust (Hill et al., 2012). For the estimates located further in the past, omissions of CEB/CD usually increase with the age of mothers, and their effects interact with increasing violations of assumptions about fertility and mortality trends. Selection biases have only been explored in the context of HIV/AIDS epidemic where the survival of mothers is highly correlated with the mortality risk of their children, and were shown to be severe (Hill, 2013b).

More recently, Rajaratnam et al. (2010) developed a new set of empirically-based summary birth history methods. Using cohort and period measures of CEB/CD, they used regression models to capture the country and regional variations in the age patterns of fertility and mortality from the available Demographic and Health Surveys (DHS). Their cohort-based method uses the same 5-year group inputs and presents a similar time location procedure than the standard method. The main innovation concerns their period-derived method that generates U5MRs for each year before the data collection up to 25 years. The period measures of CEB/CD are obtained from observed frequency distributions of births and deaths averaged by region, age of mother and number of CEB/CD.

Both cohort and period-derived methods have a maternal age and a time since first birth variants, thus totaling 4 different options. In addition, a fifth method combines the four options through a Loess local regression, and is considered to perform best.

In comparison with the Brass approach, these new methods do not require choosing model patterns, and are better able to capture changes in fertility and mortality trends. In addition, they aim to produce more reliable estimates for the recent period: the cohort-derived method by capturing the observed higher mortality of younger mothers in DHS surveys (the more recent estimates associated with women aged 15-19 are excluded however due to sampling errors), and the period-derived method by using all mother age groups to generate these recent estimates. Nonetheless, these empirically-based methods are still based on implicit modeling assumptions related to the representativeness and data quality of the DHS surveys. The authors showed that their new methods performed better than the standard method. But they only validated them on average against direct estimates from all DHS FBHs. Therefore, more research is needed on their performance for specific demographic contexts.

Method and data

We resort to demographic microsimulations to produce gold standard validation estimates. We use the Socsim program3 which generates fictitious populations at the individual level. The program uses a Monte Carlo algorithm that transforms monthly demographics rates into individual waiting time before the occurrence of competitive events (births and deaths in this case). The resulting individual records include links between mothers and children, and allow applying both direct and indirect estimation methods. We use direct estimates as our gold standard to measure the relative deviation4 of the SBH methods.

The 2012 Revision of the World Population Prospects (UN, 2013) provides the input rates to simulate a great variety of contrasted mortality and fertility regimes. We simulate fictitious populations from 1950 to 2010 so as to obtain final population of minimum 500,000 individuals. We then apply the estimations methods as if a survey or census had been taken at the mid-year 2010.

Preliminary results

Our first results explore 18 Latin American and Caribbean countries. All the selected countries are characterized by smooth but large fertility and mortality declines in recent decades. These preliminary simulations do not include any age or selection effects.

We compare three SBH methods to our gold standard estimates: (1) the standard method using the West Coale and Demeny model, (2) the new maternal age cohort-derived (MAC) and (3) period-derived methods (MAP) from the IHME team. Due to the noisiness of MAP estimates, we smooth trends in mortality with Loess local regression as recommended by the authors. The country-specific plots are presented in Figure 3.

Figure 1 shows the relative deviation associated with the standard and MAC methods by age group of mother, excluding the women aged 15-19. In the absence of age effect, the standard method is relatively reliable for the two most recent estimates, particularly for the 25-29 age group, whereas the MAC method is characterized by systematic underestimation of mortality (by 19% and 9% on average). This most likely reflects the absence of age effect in our simulated environment. In conditions of higher mortality of children born to young women, the MAC method could be closer to the reference. Going further back in time through older age groups of mothers, the standard method suffers from a growing upward bias, reaching an average of 31% deviation with women aged 45-49. In contrast, the MAC method is remarkably robust for these older estimates.

Figure 2 shows the relative deviation of the MAP method for the five 5-year covered periods, with the right-hand plot offering a closer look at the five more recent years. Estimates are on average too high (around 10% in average) for the periods prior to 2005 and too low for the most recent years (12% for the first year). This is because the declining trends tend to be too strong, which is particularly visible in country-specific plots presented in Figure 3.

3 http://lab.demog.berkeley.edu/socsim/.

4 Relative deviation = (indirect estimate -direct estimate) / direct estimate.

Discussion and further work

Our preliminary results showed first that in the absence of age effect, the standard method performed well for the more recent estimates, whereas the MAC estimates were too low. In our future work, we will introduce different levels of child mortality by age group of mother and birth order so as to explore how the MAC method can handle them. We will also evaluate to what extent the since first birth variants permit to reduce this kind of effects.

Secondly, our results showed that for older estimates, the standard method performed very poorly in contexts of strong fertility and mortality decline by overestimating strongly the rate of decline in U5MR, whilst on the contrary, the MAC method produced rather robust estimates. We will expand this comparison to other regional contexts and use other model patterns.

Thirdly, the MAP method was shown to have a tendency to reproduce too rapid declines in child mortality. This is because the method is based on birth and death distributions averaged across all past DHS surveys conducted in the Latin American and Caribbean region. Therefore, although it provides very recent estimates, the MAP method should be used with caution. In the full paper, we will assess the method for the other regions, and also examine the performance of the combined method.

Other complementary research avenues include new methods to assess the SBH

data quality, and the evaluation of the selection biases related to the correlation between child and mother survival on FBH and SBH estimates, which is crucial for the study of mortality in crisis contexts (Bergouignan, 2010).

Figure 1: Relative deviation of U5MR estimates generated from standard and MAC

methods by age group of mother.

Figure 2: Relative deviation of U5MR estimates generated from MAP method by 5 and 1-year period.

Figure 3: Comparison of U5MR estimates generated from direct and SBH methods for 18 Latin American and Caribbean countries.

Figure 3: (Second part) Comparison of U5MR estimates generated from direct and SBH methods for 18 Latin American and Caribbean countries.

References

W. B. Arthur and M. A. Stoto (1983), “An analysis of indirect mortality estimation”, Population Studies, 37(2):301–314.

C. Bergouignan (2010), “Recourir aux microsimulations pour étudier la mortalité de crise : illustration par la mortalité au Burundi en 1993”, Cahiers québécois de démographie, 40:299–329.

W. Brass and A. Coale (1968), “Methods of analysis and estimation”, In W. Brass, A. Coale, P. Demeny, D. Heisel, F. Lorimer, A. Romaniuk, and E. Van de Walle (eds.), The Demography of Tropical Africa, Princeton University Press, Princeton, pp. 88–150. L. Dwyer-Lindgren, E. Gakidou, A. Flaxman, and H. Wang (2013), “Error and bias in under-5 mortality estimates derived from birth histories with small sample sizes”, Population Health Metrics, 11(13).

D. Ewbank (1982), “The sources of error in Brass’s method for estimating child survival: the case of Bangladesh”, Population Studies, 36(3):459–474. M. Garenne (1984), “Problems in applying the Brass method in tropical Africa: a case study in rural Senegal”, Genus, 40:119–133. K. Hill (1984), “An evaluation of indirect methods for estimating mortality”, In J. Vallin, Pollard J., and L. Heligman (eds.), Methodologies for the Collection and Analysis of Mortality Data, Odina Editions, Liège, pp. 145-176. K. Hill (2013a), “Indirect estimation of child mortality”, In T. Moutrie, R. Dorrington, A. Hill, K. Hill, I. Timaeus, and B. Zaba (eds.), Tools for Demographic Estimation, Interna-tional Union for the Scientific Study of Population, Paris, pp. 148-164. K. Hill (2013b), “Introduction to child mortality analysis”, In T. Moutrie, R. Dorrington, A. Hill, K. Hill, I. Timaeus, and B. Zaba (eds.), Tools for Demographic Estimation, International Union for the Scientific Study of Population, Paris, pp. 148-164. K. Hill and M.-E. Figueroa (2001), “Child mortality by time since first birth”, In B. Zaba and J. Blacker (eds.), Brass Tracks. Essays in Medical Demography, The Athlone Press, London, pp. 9-19. K. Hill, D. You, M. Inoue, M. Oestergaard, and TAG (2012), “Child mortality estimation: Accelerated progress in reducing global child mortality, 1990-2010”, PLoS Med, 9(8): e1001303. J. Rajaratnam, L. Tran, A. Lopez, and C. Murray (2010), “Measuring under-five mortality: Validation of new low-cost methods”, PLoS Med, 7(4):e1000253. R. Silva (2012), “Child mortality estimation: Consistency of under-five mortality rate estimates using full birth histories and summary birth histories, PLoS Med, 9(8):e1001296.

UN (2013), World Population Prospects: The 2010 Revision. Extended Dataset, United Nations Publication. URL: http://esa.un.org/wpp/. P. Ward and B. Zaba (2008), “The effect of HIV on the estimation of child mortality using the children surviving/children ever born technique”, Southern African Journal of Demography, 11 (1).