an example of cooperation & some wider issues ian plewis (bedford group, institute of education)...
TRANSCRIPT
![Page 1: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/1.jpg)
AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES
Ian Plewis (Bedford Group, Institute of Education) &
Stephen Morris (Social Research Division, Department for Work &
Pensions)
![Page 2: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/2.jpg)
OVERVIEW• Government research identifies important
methodological issues• These issues often/sometimes ignored or
assumed not to be a problem – exception ONS work on methods & quality
• Where methods research takes place no government forum for exchange exists
• Child outcomes work (IoE/DWP) – a methods project - to illustrate the problem
• We suggest why methods often not addressed & thoughts on potential solution
![Page 3: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/3.jpg)
THE RESEARCH
• A feasibility study
• Are adult outcomes for individuals (parents) linked to (or caused) by changes in childhood circumstances?
• Effects of change in household income (household deprivation & parental employment) in childhood on individual’s adult outcomes
• Pathways through which changes take effect?
![Page 4: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/4.jpg)
POLICY CONTEXT• Government child poverty targets – end child
poverty by 2020.• This research informs development of policies to
meet targets• Marginal £ - what be the most effective way of
addressing child poverty– Income transfers?– Services?
• Our focus on changes in income thus whether transfers are likely to be effective
![Page 5: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/5.jpg)
WHY NEW RESEARCH?• Not much UK research looking at change (Our
interest is in establishing causality)• Tends to focus on short-range educational
outcomes • Focus on single data sources• Most useful sources – birth cohorts – have
limitations• Can data sources be combined to better address
questions of relevance – What assumptions do we have to make?– What statistical models could be used?
![Page 6: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/6.jpg)
THE PROBLEM
• Most quantitative social research has to deal with methodological issues.
• Some of these issues are ‘cutting edge’ and need to be addressed within a research council project.
• Some are ‘standard’.• But many fall between these two extremes and
tend to get ignored in Government funded projects
![Page 7: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/7.jpg)
MEASUREMENT ERROR• Suppose we want to estimate a regression
model with a measure of income as one of possibly several explanatory variables.
• We know that measures of income are unreliable and we also know that this unreliability, if ignored, can lead to biased estimates of coefficients of interest.
• We should correct for measurement error because not to do so could lead to misleading inferences for policy.
• But such corrections are not standard and the problem is often swept under the carpet. .
![Page 8: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/8.jpg)
NON-RESPONSE
• Unit non-response and attrition are features of all longitudinal datasets.
• Certain kinds of cases tend to be lost – at the outset and over time.
• Ideally, we should carry out sensitivity analyses of some kind to establish whether losses from the sample are likely materially to affect policy conclusions.
• But these analyses take time and resources
![Page 9: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/9.jpg)
INCOME MISSING
• Data are often missing for income even when other variables are measured.
• And some surveys do not collect income data.• Imputation is a way round some of these
difficulties.• But there are a number of imputation methods –
how does their application affect our conclusions?
![Page 10: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/10.jpg)
IMPLICATIONS FOR POLICY• Measurement error/non-response = biased
estimates• Biased estimates can lead to bad policy
decisions• Example (hypothetical):
– Under-estimate of effect of change in income on child outcomes
– Assume income transfers less effective– Direct policy toward services– Sub-optimal policy response
![Page 11: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/11.jpg)
WHY MIGHT THIS HAPPEN?
• No cross-government forum for exchange of information on methods research
• Within spending departments not always easy to make the case for methods research– Benefits of methods research diffuse – Not priority for policy customers
• Researchers don’t always appreciate implications
• Time – complex problems take time to solve
![Page 12: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/12.jpg)
WHAT COULD BE DONE?• Establish a GCSRO/NCRM methods group
(representatives: depts., ONS, ESRC & NCRM)• Forum for raising methods related issues with policy
focus – disseminate existing work from ONS, depts. & academics/researchers
• Identify gaps and commission – requires funding– Policy relevance– Wider application
• Dissemination – publications, seminars and training courses
• CASE studentships
![Page 13: AN EXAMPLE OF COOPERATION & SOME WIDER ISSUES Ian Plewis (Bedford Group, Institute of Education) & Stephen Morris (Social Research Division, Department](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf7b1a28abf838c83c2a/html5/thumbnails/13.jpg)
CONTACT DETAILSIan Plewis – Institute for Education(T) 020 7612 6238(E) [email protected]
Stephen Morris – Department for Work & Pensions(T) 020 7962 8193(E) [email protected]