demystifying the measurement of complex social constructs: assessing social capital across sectors...
TRANSCRIPT
Will Story, PhD, MPH Department of Community and
Behavioral Health University of Iowa College of Public Health
Tim Frankenberger, PhD TANGO International
CORE Group Global Health Practitioner Conference Portland, OR May 18, 2016
Reviewed the approaches to creating valid and reliable measures of complex social constructs.
Examined the design of social capital assessment tools and discovered how the tools have been used in development programs.
Compared various assessment tools for measuring social capital and discussed how to adapt them for wider use in population-based surveys.
A social mechanism, phenomenon, or category created and developed by society
A perception of an individual, group, or idea that is 'constructed' through cultural or social practice
A proposed attribute of a person (or group of people) that often cannot be measured directly, but can be assessed using a number of indicators (i.e., latent variable)
The construct must be carefully defined and delineated before it can be measured
Single measure With or without a scale
Multiple measures Additive score Factor score Continuous, ordinal, dichotomous
Validity: The accuracy of a measurement The degree to which the measures in your project
reflect the theoretical constructs on which those measures are based.
Reliability: The consistency of a measurement The degree to which a measure would give you the
same result over and over again, assuming the underlying phenomenon is not changing.
(Trochim and Donnelly, 2008)
Translation validity Focus on whether the operationalization is a good
reflection of the construct
Criterion-related validity Examine whether the operationalization behaves
the way it should given your theory of the construct
(Trochim and Donnelly, 2008)
Expert review: Assess whether the survey questions are appropriate for measuring social capital Suggestions from peer-reviewed literature (De Silva et al.,
2006); independent review by social capital expert; and final review by our research team from ICDDR,B
FGDs and cognitive interviews: Explore what people know about social capital and examine how respondents formulate their answers We conducted four FGDs and 32 cognitive interviews
consisting of 18 structured survey questions followed by scripted verbal probes
(Story et al., 2015)
Internal consistency Homogeneity of multiple items within an
instrument or specific domain
Inter-rater reliability Extent that two or more observers agree with one
another
Test-retest reliability Extent that the same results are obtained on
repeated administrations of the same instrument
(Trochim and Donnelly, 2008)
Five dimensions of community capacity were used to develop a survey instrument: leadership, resources, problem assessment, connections and networks, and community attitudes
The initial scale with 41 items had a Cronbach's alpha coefficient of 0.89 indicating good internal consistency and this was evident over all four communities
(Lovell et al., 2014)
Trochim, W. M. K., & Donnelly, J. P. (2008). The research methods knowledge base. Mason, OH: Cengage Learning or http://www.socialresearchmethods.net/kb/
Story, W.T., Taleb, F., Ahasan, S.M., & Ali, N.A. (2015). Validating the measurement of social capital in Bangladesh: A cognitive approach. Qualitative Health Research, 25(6), 806-819.
De Silva, M. J., Harpham, T., Tuan, T., Bartolini, R., Penny, M. E., & Huttly, S. R. (2006). Psychometric and cognitive validation of a social capital measurement tool in Peru and Vietnam. Social Science & Medicine, 62, 941–953.
Lovell, S.A., Gray, A.R., Boucher, S.E. (2014). Developing and validating a measure of community capacity: Why volunteers make the best neighbors. Social Science and Medicine, 133, 261-268.
What complex social constructs are you currently trying to measure?
What have you done (or can you do) to assess the validity and reliability of these constructs?
What is social capital? Household social capital and socioeconomic
inequalities in child undernutrition: Evidence from rural India
The effects of social capital on resilience: Evidence from Ethiopia
A broad term including social relationships, networks, and values that facilitate collective action for mutual benefit
Theories of social capital tend to be dichotomized into two forms: structural and cognitive Structural: What people do as measured by
individuals’ actions and behaviors Cognitive: How people feel as measured by
individuals’ attitudes and perceptions (Woolcock, 1998; Harpham et al., 2002; Krishna & Shrader, 2000)
A survey-based approach is commonly used to measure specific components of social capital
Structural social capital Group membership/involvement Informal social relationships Collective action
Cognitive social capital Trust (generalized or interpersonal) Social cohesion
(Harpham et al., 2002; Narayan & Cassidy, 2001)
Bonding: social ties that connect similar people (e.g., a religious or ethnic group), build in-group loyalty, and provide resources
Bridging: social ties that connect dissimilar people (e.g., a social movement, different demographic groups) and are more ideal for accessing external assets and information
Linking: social ties to people of power in a society (e.g., health care providers, school teachers, and government officials), who can help leverage resources, ideas, and information
(Szreter & Woolcock, 2004)
William T. Story, PhD, MPH Department of Community and
Behavioral Health University of Iowa College of Public Health
Richard M. Carpiano, PhD, MA, MPH Department of Sociology
School of Population and Public Health University of British Columbia
CORE Group Global Health Practitioner Conference Portland, OR May 18, 2016
3.1 million annual deaths among children under the age of five due to undernutrition
Strong association between child nutrition status and household wealth persists in the developing world
Social capital can potentially be used by the poor as a primary means of protection against risk and vulnerability
(Black et al. 2013; Boyle et al. 2006; Carroll 2001; Story 2013)
India is home to the largest number of under-nourished children globally
42.5 % of children below the age of 5 years are underweight for age (highest in South Asia)
Two-thirds of the population is rural, which leads to greater disparities in access to resources
Prior studies have shown unequal access to social capital and differential associations between social capital and health
(Ramachandran 2014; Story 2014; Vikram et al. 2012; De Silva et al. 2007)
1) How does household SES shape access to bonding, bridging, and linking social capital?
2) If these social capital forms are accessed, how are their influences on child underweight shaped by household SES?
SES and Possession of Social Capital H1a – Opportunity Hoarding Explanation:
Higher household SES is associated with higher odds of bonding, bridging, and linking social capital H1b – Coping Explanation: Lower household SES
is associated with higher odds of membership in bonding organizations
(Wright 2009; Cattell 2001; Altschuler, Somkin, and Adler 2004)
SES as a Moderator of Social Capital and Child Underweight H2a – Resource Substitution Explanation: Social
capital demonstrates a stronger reduction in the odds of child underweight in lower (versus higher) SES households.
H2b – Fundamental Cause Explanation: Social capital demonstrates a stronger reduction in the odds of child underweight in higher (versus lower) SES households.
(Uphoff et al. 2013; Carpiano 2006; Story 2014; Carpiano, Link, and Phelan 2008)
Data source 2005 India Human Development Survey (IHDS)
Analytic sample 9,008 children under the age of 5 6,753 households 1,347 rural villages
Unit of analysis Children within households within villages
(Desai et al. 2005)
Child underweight Below -2 standard
deviations from the median weight-for-age of the international reference population Binary variable where 1 =
Underweight
(Fishmann et al., 2004)
Household SES Index of 30 dichotomous items reflecting asset
ownership and housing quality
Bonding ties (e.g., religious or caste groups) Bridging ties (e.g., women’s groups, saving and
credit groups, youth groups) Linking ties Doctor or nurse Teacher or principal Government officer
Exploratory factor analysis was used to validate the components of social capital in the IHDS
6 factors were identified and explained 82.6% of the total variance in the 15 social capital questions
3 of those factors (bonding, bridging, and linking social capital) were used in this study
Multilevel logistic regression analysis was used to estimate: The overall association between SES, social capital
and child underweight adjusting for individual, household, and village characteristics
The variation in child underweight between households and communities
Interactions between each social capital variable and household SES were examined using the margins command in Stata 13.0
(Ai and Norton 2003)
Adjusted odds ratios (95% confidence intervals) of possessing each form of household social capital as predicted by household assets after controlling for other household and village characteristics. Organizational Ties Linking Ties
Bonding Bridging Medical School Government Household assets
1.07*** (1.04-1.10)
1.07*** (1.05-1.09)
1.10*** (1.07-1.12)
1.10*** (1.07-1.12)
1.16*** (1.12-1.20)
* p<0.05; ** p<0.01; *** p<0.001 Note: n=7,568 households within 1,379 villages; Each cell represents a separate model. Each model controls for all household-, and village-level independent variables.
H1a – Opportunity Hoarding Explanation: Higher household SES is significantly associated with greater odds of bonding, bridging, and linking social capital.
H1b – Coping Explanation: Our findings do not support the hypothesis that lower household SES is associated with greater odds of membership in a bonding organization.
H2a – Resource Substitution Explanation: Poorer households with ties to bridging organizations and school personnel are less likely to have an underweight child compared to poorer households without such ties.
H2b – Fundamental Cause Explanation: Wealthier households with a medical tie are less likely to have an underweight child compared to wealthier households without a medical tie.
One cannot assume that a social capital strengthening initiative will be equally beneficial to all members of society
Socially excluded households must be given special attention
Future research should assess equity in the delivery and utilization of social capital strengthening interventions
Interventions should continue to promote social ties to bridging organizations, especially among poorer households
Need to better understand the pathways through which bridging ties lead to better child nutrition by exploring organizational strategies related to better child feeding and treatment seeking practices
Social connections to medical professionals are insufficient to reduce disparities in child nutrition
Access to a doctor or nurse must be combined with sufficient household resources to rehabilitate malnourished children
Social ties to teachers and principals may have an impact beyond the effect of formal education
It is important to identify what aspects of the education system are helping to eliminate socioeconomic disparities in child nutrition
Are children and their parents establishing new social connections to help improve the overall health and well-being of their household?
The effects of social capital on resilience: Evidence from Ethiopia
Tim Frankenberger May 18th, 2016 Core Group Global Health Practitioner Conference
Importance of Social Capital
• Previous research demonstrates that the extent and application of social capital strongly influences community level resilience.
• Disasters may sometimes enhance social capital
because they activate or give rise to neighborhood associations and collective organizations that can be used to disseminate vital information, provide community members with a voice, and afford leverage to assist in taking control of rebuilding efforts.
*Aldrich 2012; Wilson 2012; Magis 2010; Elliott et al. 2010 Figure Source: Reproduced with permission from Aldrich (2012, p. 34) in Frankenberger, T., Mueller M., Spangler T., and Alexander S. October 2013. Community Resilience: Conceptual Framework and Measurement Feed the Future Learning Agenda. Rockville, MD: Westat.
Hypotheses to be Tested • Households with greater levels of social capital (bonding,
bridging, and linking) achieve greater levels of food security than those with less social capital, all else equal.
• Households with greater levels of social capital (bonding,
bridging, and linking) are able to recover better than those with less social capital, all else equal
• For a given level of exposure to shocks, households with
more social capital report fewer negative impacts of shocks than households with less social capital, all else equal.
Empirical Evidence
• This presentation will examine empirical evidence from several studies focused on measuring resilience
• Pastoralist Areas Resilience Improvement and Market Expansion (PRIME) program in Ethiopia
Studies: PRIME • Pastoralist Areas Resilience
Improvement through Market Expansion – USAID Ethiopia Feed the Future
– Project goals: – increase household incomes – enhance resilience – Improve climate change adaptive capacity
• Program beneficiaries – pastoralists, non-pastoralists, and other
– Geographic location – 2 areas in Ethiopia (Borena and Jijiga)
– Data – Baseline (2013) – Interim monitoring data (2014 – 2015, 6
months)
Samples from Project areas
Project area # of
households # of
communities
PRIME Jijiga 1398 32
Borena 1744 41
Methodology • In order to measure the impact social capital has on
resilience, indices were created for bonding, bridging and linking social capital.
• The bonding social capital index is based on eight yes/no
questions about whether the household would be able to give or receive help from relatives or non-relatives in their community.
• The bridging social capital index is based on eight similar
yes/no questions, but about giving and receiving help from relatives or non-relatives living outside their community.
Methodology
• The linking social capital index measures the amount of information received from government agents (i.e., rural development agents and government/political officials).
• The index also measures households’ access
to services generally provided by the government and the quality of those services
Methodology
• All three forms of social capital are composite measures based on multiple other measures.
• In many of these cases Principal Components Analysis
(PCA) or polychoric factor analysis were used to validate each form of social capital.
• Simply put, these techniques reduce a set of “input”
variables that are hypothesized to be related to one another.
• Three indices were constructed using this technique
Methodology
• The dependent variable Household food security is the inverse of an experiential indicator of food insecurity, the Household Food Insecurity Access Scale (HFIAS).
• The HFIAS is an index constructed from the
responses to nine questions regarding people’s experiences of food insecurity in the previous four weeks
Methodology
• The dependent variable, recovery takes into account households’ ability to recover from climatic, conflict, and/or economic shocks using a 4-point likert scale.
• The dependent variable HH shock impact is an
index which takes into account if a household experienced a shock within the last 12 months, how many times they experienced a shock with in the last 12 months, and how severe the impact of the shock was on income and food consumption.
Multivariate regression analysis
Multivariate regression analysis
Food Security Results for PRIME Table 1. Relationship between social capital and household food security for PRIME baseline
Indicators
Jijiga Borena
Coefficient Elasticity
Coefficient
Elasticity
n n
Social capital
Bonding social capital
0.005 0.030 1236 0.072 *** 0.732 1566
Bridging social capital
0.015 0.057 1253 0.054 *** 0.402 1624
Linking social capital
0.025 0.105 1253 -0.005 -0.029 1624
NOTES: Stars represent statistical significance at the 0.01 (***), 0.05 (**) and 0.1 (*) levels.
Food Security Summary
• Bonding and bridging social capital are significantly associated with increased food security in Borena but not in Jijiga
Recovery Results for PRIME Table 2. Relationship between social capital and recovery for PRIME baseline
Indicators
Jijiga Borena
Coefficient Elasticity
Coefficient
Elasticity
n n
Social capital
Bonding social capital
0.009 *** 0.212 1127 0.005 *** 0.152 1430
Bridging social capital
0.007 *** 0.110 1146 -0.002 * -0.041 1476
Linking social capital
0.043 *** 0.757 1146 0.004 0.073 1476
NOTES: Stars represent statistical significance at the 0.01 (***), 0.05 (**) and 0.1 (*) levels.
Recovery Summary
• In both Jijiga and Borena bonding and bridging social capital enabled households to recover
• Linking social capital was important for recovery in Jijiga but not Borena
Shock Impact Results for PRIME Table 3. Relationship between social capital, asset index, and number of shocks on shock exposure for PRIME baseline Jijiga Borena
Indicators
Only bonding
social capital
Only bridging
social capital
Only linking social capital
Only bonding
social capital
Only bridging
social capital
Only linking social capital
(A) (B) (C) (A) (B) (C)
Social capital Bonding social capital -0.011
*** -0.008
***
Bridging social capital -0.011
*** 0.012
***
Linking social capital 0.000
0.004
Asset index 0.002 -0.002 -0.006 -0.015 *
-0.032 ***
-0.020 ***
Number of shocks 3.564
*** 3.558
*** 3.563
***
3.611
*** 3.599
*** 3.592
***
Number of observations
1324 1351 1352
1618 1618 1618
NOTES: Stars represent statistical significance at the 0.01 (***), 0.05 (**) and 0.1 (*) levels.
Shock Impact Summary
• Bonding and bridging social capital help mitigate the effect of shocks in Borena and Jijiga
• Linking social capital does not have an effect in either Jijiga or Borena
Conclusions • Social capital appears to have a positive effect on food
security, helps households recover and mitigates the effect of shocks across the different data sets
• Thus social capital appears to be critical to resilience • Wealthier households appear to receive the benefits of
social capital more than poorer households • Social capital can be used up in the early phases of a
prolonged covariate shock and its downstream effects
Review the three social capital assessment tools individually.
In your groups discuss the following: How might you adapt one or more of these tools
for use in your own programs? What barriers might you face when trying to
measure social capital? How will you overcome those barriers?
