the future of statistics filelinda is 31 years old, single, outspoken, and very bright. she majored...
TRANSCRIPT
![Page 1: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/1.jpg)
The Future of Statistics London, September, 2013 · ulrich atz · @statshero
![Page 2: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/2.jpg)
A classic example Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.
Tversky, A. and Kahneman, D. (1982) "Judgments of and by representativeness". In D. Kahneman, P. Slovic & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press
Which is more probable? • Linda is a banker. • Linda is a banker and is active in the feminist movement.
![Page 3: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/3.jpg)
What is the ODI?
![Page 4: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/4.jpg)
![Page 5: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/5.jpg)
10 Startups
![Page 6: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/6.jpg)
36 Members
![Page 7: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/7.jpg)
http://certificates.theODI.org
The first robust quality badge for open data
• Helps publishers certify their data
• Helps users find and use it
• Helps policy makers benchmark
![Page 8: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/8.jpg)
Expert An exceptional example of information infrastructure.
Standard Regularly published open data with robust support that people can rely on.
Pilot Data users receive extra support from, and can provide feedback to, the publisher.
RawA great start at the basics of publishing open data.
http://certificates.theODI.org
![Page 9: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/9.jpg)
![Page 10: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/10.jpg)
Open Data Stories
![Page 11: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/11.jpg)
A collaboration with the World Bank
![Page 12: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/12.jpg)
WHAT IS DATA?
![Page 13: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/13.jpg)
Data is the raw material of the information age.
![Page 14: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/14.jpg)
Data is the New Oil
kenhodge13 (40132991@N07) on flickr.com
![Page 15: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/15.jpg)
But data is not oil! • data is a non-rival good • marginal cost of distribution • falling cost of analysis (“refinement”)
![Page 16: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/16.jpg)
More like gold than oil?
http://www.flickr.com/photos/48806909@N00/497511638/
![Page 17: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/17.jpg)
A new gold rush?
![Page 18: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/18.jpg)
![Page 19: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/19.jpg)
Open data supports collaborative ecosystems
![Page 20: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/20.jpg)
![Page 21: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/21.jpg)
WHAT IS DATA SCIENCE?
![Page 22: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/22.jpg)
Open Data found a £200m saving in the NHS budget
![Page 23: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/23.jpg)
![Page 24: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/24.jpg)
What is data science?
“often defined in terms of attributes of the data scientist” A naming problem: • the science of data? • science with data?
More: http://theodi.github.io/presentations/2013-09-oxford-real-data-scientist.html
![Page 25: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/25.jpg)
What is Statistics?
• The science of learning from (or making sense out of) data
• The theory and methods of extracting information from observational data for solving real-world problems
• The science of uncertainty • The quintessential interdisciplinary science • The art of telling a story with [numerical]
data
A Career in Statistics: Beyond the Numbers by Gerald Hahn and Necip Doganaksoy
![Page 26: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/26.jpg)
What is Statistics?
1. Understanding uncertainty 2. Making data useful (“analysing”)
![Page 27: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/27.jpg)
Stereotypes Statistics Data Science Sampling Use all the data Slow and steady Fast and dirty “It depends” The fallacy of certainty B-W bar charts Infographics Institutions Startups
![Page 28: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/28.jpg)
Stereotypes Statistics Data Science Causal Predictive Small Big Large Report Data product Asking Doing ? HBR Sexiest job of the 21st
century
![Page 29: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/29.jpg)
“I’m looking for a data scientist…” (intentions are good)
http://www.flickr.com/photos/waitingfortheword/5546445871/
= “I’m looking for ‘God’.”
![Page 30: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/30.jpg)
Mapping £378m of peer-to-peer lending across the UK Helped convene domain-experts
+ p2p lenders + banking professionals + data analytics (ODI) + communications (ODI)
Analysed 14m records
+ all the data + anonymised and analysed + ODI research
National & international reach + Front-page Financial Times
Long-term
+ Create real-time view + Stimulate market http://smtm.labs.theodi.org/
![Page 31: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/31.jpg)
TRENDS
![Page 32: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/32.jpg)
Where are we now?
Data, data everywhere 5 trends “Predictions are difficult; especially about the future.”
![Page 33: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/33.jpg)
1. Commoditisation of dataThe cost of working with data has lowered and is continuing to fall
http://aws.amazon.com/s3/
![Page 34: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/34.jpg)
2. Participation increases – web access per capita and in ease of use
http://www.google.com/glass/start/how-it-feels/
![Page 35: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/35.jpg)
Tools are getting easier to use – without the need for coding skills
![Page 36: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/36.jpg)
3. Pervasive data collection
http://www.londoncityairport.com/News/ReadArticle/93/ http://www.villevivante.ch/#animation
Sensors and Internet of Things Mobile technologies Shopping and behaviour monitoring
![Page 37: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/37.jpg)
4. Quantifying of personal data
http://quantifiedself.com/2013/01/future-normal-quantified-self-tools-at-the-apple-store/
![Page 38: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/38.jpg)
5. Demands for greater transparency
http://www.flickr.com/photos/kcorrick/7609944890/ http://www.flickr.com/photos/brostad/5841297115
![Page 39: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/39.jpg)
ANALYSING THE ANALYSERS
![Page 40: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/40.jpg)
Analyzing the Analyzers – a recent study in the data science community
Harlan D. Harris, Sean Patrick Murphy, and Marck Vaisman A skills-sorting task
![Page 41: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/41.jpg)
![Page 42: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/42.jpg)
Again we see
Data Science defined by a set of skill.
![Page 43: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/43.jpg)
THE FUTURE OF STATISTICS
![Page 44: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/44.jpg)
A look back
19th century: large data
sets, simple questions
20th century: small data
sets, complex questions
21th century: large data
sets, complex questions
![Page 45: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/45.jpg)
Statistical Modeling: The Two Cultures
Leo Breiman: “There are two cultures in the use of statistical modeling to reach conclusions from data.”
![Page 46: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/46.jpg)
The Two Cultures
(Tukey spoke of algorithmic and algebraic models.)
![Page 47: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/47.jpg)
Convergence
![Page 48: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/48.jpg)
Why use R?
Open source / free Widely used ( > 2 million and growing) R has an incredible community ( > 4,000 packages) Used by Facebook, Google (apparently 500+ users), weather forecasts, finance industry
![Page 49: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/49.jpg)
UK GSS 2020 strategy: “Collaboration will become the norm”
![Page 50: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/50.jpg)
There are rock stars, and then there are rock bands: statisticians frequently work
in teams.
http://www.flickr.com/photos/11055761@N04/1842696307/
Marie Davidian, http://www.independent.co.uk/news/world/americas/heroes-of-zeroes-nate-silver-his-rivals-and-the-big-electoral-data-revolution-8734380.html
![Page 51: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/51.jpg)
Mapping the Census
http://www.ons.gov.uk/ons/interactive/census-map-2-1---tenure/index.html http://bit.ly/1232hJb
![Page 52: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/52.jpg)
Remember Linda, 31, bright, activist…?
Bankers
Bankers active in a feminist
movement
![Page 53: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/53.jpg)
We learn
Even statisticians are bad intuitive statisticians.
![Page 54: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/54.jpg)
Statistics is about uncertainty More biases: We seek overconfidence. But understanding uncertainty should not be our only domain – combine it with data analysis and data-driven decisions.
![Page 55: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/55.jpg)
The danger: Define our limits in terms of familiar tools and familiar problems.
![Page 56: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/56.jpg)
BE OPEN
![Page 58: The Future of Statistics fileLinda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination](https://reader033.vdocuments.mx/reader033/viewer/2022041420/5e1ec562fb9a4e00416e7298/html5/thumbnails/58.jpg)