applications & implications of big data for official statistics - emmanuel letouzé
TRANSCRIPT
![Page 1: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/1.jpg)
Applications &
Implications of
Big Data for
Official Statistics
Emmanuel Letouzé
Director & co-Founder
Data-Pop Alliance
DfID, London, February 26, 2015
![Page 2: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/2.jpg)
1. The Emergence of Big Data &
The Statistical Tragedy
Framing and surfacing of the issue
2. Big Data and Official Statistics:
Substitute, Complement, or “It’s complicated”?
3. The Case of the SDGs
A story of fish and fishermen
![Page 3: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/3.jpg)
1. The Emergence of Big Data
vs. The Statistical Tragedy
Framing and surfacing of the issue
![Page 4: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/4.jpg)
![Page 5: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/5.jpg)
Hal Varian’s nowcasting, GDP
and light emissions paper.…
Line shows returns for “Big Data” on Google Trends between 2007 and 2014; 100=maximum value
“We are at
the beginning
of what I call
The Industrial
Revolution of Data.”
Joe Hellerstein
, November
19, 2008
Context: the Big Data rush
![Page 6: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/6.jpg)
* So
urc
e: O
xfa
m In
tern
atio
na
l, citin
g C
red
it Su
isse, J
an
. 2014
“Data is the new oil”
![Page 7: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/7.jpg)
![Page 8: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/8.jpg)
![Page 9: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/9.jpg)
Google Flu Trend: rise and fall
Hope or Hype?
![Page 10: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/10.jpg)
![Page 11: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/11.jpg)
![Page 12: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/12.jpg)
2. Big Data & Official Statistics
It’s complicated. Or complex.
![Page 13: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/13.jpg)
What is Big Data? 2010-12: the 3 Vs of big data
![Page 14: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/14.jpg)
i. Exhaust
ii. Web
iii. Sensing
Crumbs
Capacities
Communities
What is Big Data? Now: the 3 Vs of Big Data
![Page 15: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/15.jpg)
Movement of an individual in Rwanda over 4 years using CDRs (Source J. Blumenstock, 2010)
![Page 16: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/16.jpg)
The new data ecosystem
![Page 17: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/17.jpg)
1. Early warning
1. Real time awareness
1. Real-time feedback
Source: Letouzé, 2012
“What can it be used for?”—Taxonomy of applications (1)
![Page 18: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/18.jpg)
1. Descriptive-e.g. maps, clouds..
1. Predictive:-forecasting
-inference
1. Prescriptive-causal inference Source: Letouzé, Vinck and Meier, 2013
“What can it be used for?”—Taxonomy of applications (2)
![Page 19: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/19.jpg)
NationalStatistical
Institutes carryout surveys
Telefonica teamused their data to‘predict’ SELs fromCell Phone Usage
Predict the present(SELs for non-
surveyed regions)
and monitor the
future (trackchanges over time)
Survey from “a
major city in Latin America”
Source: “Prediction of Socio-Economic Levels Using Cell-Phone Records” (Telefonica research, 2011)
‘Predicting’ socioeconomic levels?
![Page 20: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/20.jpg)
Promoting a people-centered Big Data Revolution
![Page 21: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/21.jpg)
![Page 22: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/22.jpg)
![Page 23: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/23.jpg)
![Page 24: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/24.jpg)
![Page 25: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/25.jpg)
Counting people?
![Page 26: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/26.jpg)
Sample bias correction
Then:
blending of hypothesis based vs. supervised
machine learning methods to model bias
![Page 27: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/27.jpg)
Source: Letouzé, 2014, based on primary and secondary sources
What & how much do we know?
….and does it matter?
Poverty prevalence 1990-2030Fragile States vs. Non-Fragile States
![Page 28: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/28.jpg)
Are official statistics ever more than shadows in
the cave? If so what are they good for?
![Page 29: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/29.jpg)
“Official statistics assumes a key role in ensuring
democracy and fostering social
progress…[should] ”provide society with
knowledge of itself”
Enrico Giovanni—former President of Istat
Co-Chair of IEG on the Data Revolution
“Knowledge is power; statistics is democracy”Former President of Statistics Finland
![Page 30: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/30.jpg)
(2) Official
Statistics as
systems—not
reducible to
producing (1)
(1) Official
Statistics as
data—entirely
defined as
product of (2)*
* According to Fundamental Principles of Official Statistics
What is/are official statistics?
![Page 31: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/31.jpg)
Official
Statistics
(1) Ensure that
societies benefit from
“knowledge of itself”
(according to some
political and
technical standards)
(2) Ensure that
societies benefit
from the presence
of a deliberative
public space
What is/are the purpose(s)
of official statistics?
![Page 32: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/32.jpg)
(2) Ensure that societies benefit from the
presence of a deliberative public
space
It’s complex—and it’s political
(1) Ensure that societies benefit from “knowledge of itself” (according to some
political and technical standards)
Official
Statistics
Big Data
![Page 33: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/33.jpg)
Source: Letouzé, 2014, based on primary and secondary sources
(How) can data reduce poverty?
Poverty prevalence 1990-2030Fragile States vs. Non-Fragile States
![Page 34: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/34.jpg)
…by that Gary King means: it’s about the analytics
Jonathan GlemmieThe Guardian, Oct 3, 2013
![Page 35: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/35.jpg)
3. The case of the SDGs
A story of fish and fishermen
![Page 36: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/36.jpg)
SDGs adopted by
the OWG
Big data examples What is monitored How is
monitored
Country(ies) Year Advantages of using
big data
1. Poverty eradication
Satellite data to estimate povertyi Poverty Satellite images, night-lights
Global map 2009 International comparable data,
which can be
updated more
frequently
Estimating poverty maps with cell-phone recordsii
Poverty Cell phone records
Cote d’Ivoire 2013-4
Internet-based data to estimate
consumer price index and poverty
ratesiii
Price indexes Online prices at
retailers
websites
Argentina 2013 Cheaper data
available at higher
frequencies
Cell-phone records to predict socio-
economic levelsiv
Socio-economic
levels
Cell phone
records
City in Latin
America
2011 Data available more
regularly and cheaper than official data;
informal economy
better reflected
2. End hunger,
achieve food security and
improved nutrition,
and promote
sustainable
agriculture
Mining Indonesian Tweets to
understand food price crisesv
Food price crises Tweets Indonesia 2014
Uses indicators derived from mobile
phone data as a proxy for food
security indicatorsvi
Food security Cell phone data
and airtime
credit
purchases
A country in
Central Africa
2014
Use of remote-sensing data for drought assessment and monitoring
Drought Remote sensing Afghanistan, India,
Pakistanvii
2004
Chinaviii 2008
3. Health Internet-based data to identify
influenza breakoutsix
Influenza Google search
queries
US 2009 Real-time data;
captures disease cases not officially recorded;
data available earlier
than official data
Data from online searches to
monitor influenza epidemicsx
Influenza Online searches
data
China 2013
Detecting influenza epidemics using
twitterxi
Influenza Twitter Japan 2011
Monitoring influenza outbreaks using twitterxii
Influenza Twitter US 2013
Systems to monitor the activity of
influenza-like-illness with the aid of
volunteers via the internetxiii,xiv
Influenza Voluntary
reporting
through the
internet
Belgium, Italy,
Netherlands,
Portugal,
United
Kingdom, United States
ongoi
ng
Cell-phone data to model malaria
spreadxv
Malaria Cell-phone
data
Kenya 2012
Using social and news media to Cholera Social and news Haiti 2012
SDG monitoring & Big Data
![Page 37: Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé](https://reader035.vdocuments.mx/reader035/viewer/2022062406/55ab05a91a28abc4118b4676/html5/thumbnails/37.jpg)
SDG achievement & Big Data