big data lab for the canadian government
TRANSCRIPT
![Page 1: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/1.jpg)
magnara.ca
Big Data Laboratory
Creating a space for continuous improvement
![Page 2: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/2.jpg)
START POINT GOAL
![Page 3: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/3.jpg)
START POINT
GOAL
![Page 4: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/4.jpg)
START POINT
GOAL
![Page 5: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/5.jpg)
START POINT
GOAL
![Page 6: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/6.jpg)
START POINT
GOAL
![Page 7: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/7.jpg)
START POINT
GOAL
![Page 8: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/8.jpg)
START POINT
GOAL
???
![Page 9: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/9.jpg)
START POINT
GOAL
![Page 10: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/10.jpg)
But wait…
![Page 11: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/11.jpg)
GOAL
![Page 12: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/12.jpg)
GOAL
![Page 13: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/13.jpg)
GOAL
![Page 14: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/14.jpg)
GOAL
![Page 15: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/15.jpg)
NOBODY DOES.We don’t know.
![Page 16: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/16.jpg)
Dev or Lab?
Development
• Known solutions• Clear deliverables• Fixed timelines• Standard management• Minimized failures• Known skills• All requirements met
Laboratory
• Unknown solutions• Lots of mistakes• Unpredictable completion• Managed risk• Controlled failures• New skills• Options eliminated
![Page 17: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/17.jpg)
![Page 18: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/18.jpg)
![Page 19: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/19.jpg)
![Page 20: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/20.jpg)
Big Data
• What is it… exactly?• Is it the answer?• Is it only part of the answer?• Is it a magic bullet?• Is it too hyped and immature?• Is it a pipe dream?• Is it a bubble ready to pop?• Will it meet the needs of the Canadian government?• When and how will we be ready?
![Page 21: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/21.jpg)
Cloud + Hadoop
![Page 22: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/22.jpg)
MapReduce
![Page 23: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/23.jpg)
MapReduce
Hadoop
![Page 24: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/24.jpg)
IT + Business
…and the two shall become one flesh.
![Page 25: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/25.jpg)
Blueprint 2020
• “…envisions a capable and high-performing Public Service that embraces innovation, transformation and continuous renewal.”
• “An agile, efficient and effective Public Service is essential to the well-being of Canadians.”
• “…the world continues to change rapidly, putting significant stress on the existing way of doing things.”
![Page 26: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/26.jpg)
GOAL
![Page 27: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/27.jpg)
What to do about…
• Tableau Server• SAS VA
![Page 28: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/28.jpg)
NEXT STEPS?If we need to really innovate, what are the…
![Page 29: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/29.jpg)
Possible directions…
Traditional BI with a twist?
Something brand new?
![Page 30: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/30.jpg)
We need to stop…
• Making promises• Buying tools• Hiring consultants• Developing reports• Answering questions• Repeating the same mistakes• Aiming only for success
![Page 31: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/31.jpg)
We need to start…
• Re-asking old questions• Brainstorming new ideas• Experimenting in safe ways• Breaking the goal down differently• Defining lab boundaries• Leveraging lessons from others• Deliberating failure for maximum learning
![Page 32: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/32.jpg)
Buyer Beware
“Organizations are scrambling to implement new software and hardware to increase the amount of data they collect and store. However, in doing so they are unwittingly making it harder to find the needles of useful information in the rapidly growing mounds of hay. If you don't know how to differentiate signals from noise, adding more noise only makes things worse.”
Signal: Understanding What Matters in a World of Noise
![Page 33: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/33.jpg)
![Page 34: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/34.jpg)
RESOURCES
![Page 35: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/35.jpg)
7 TED Talks
• How to learn from mistakes– On being wrong– How to learn? From mistakes– There are no mistakes on the bandstand– The dangers of “willful blindness”– Doctors make mistakes. Can we talk about that?– Trial, error and the God complex– Embrace the near win
![Page 36: Big Data Lab for the Canadian government](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5885b2a71a28abd2348b6b71/html5/thumbnails/36.jpg)
The Lean Startup
• Eric Ries talks at Google• TheLeanStartup.com• Build. Measure. Learn.• Running Lean