![Page 1: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/1.jpg)
Mass Production of AI solutions
![Page 2: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/2.jpg)
I am Nate KeatingI am here because I am very passionate about bringing the benefits of AI to society
Hello!
![Page 3: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/3.jpg)
A little bit about me
◉ Currently, Product Manager, Google
Cloud AI Platform
◉ Previously Manager, Applied AI, IBM
◉ Degree in Economics, Duke University
![Page 4: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/4.jpg)
Scaling the impact of AI across societyWhat does that really mean?
1
![Page 5: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/5.jpg)
Access to AI becomes accessible to everyone and not just
a privileged few
![Page 6: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/6.jpg)
Our lives change significantly when a technology scales to society
![Page 7: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/7.jpg)
Scaling TechnologyLet’s look at a successful example
2
![Page 8: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/8.jpg)
Cellular phones
Martin CooperHandheld cellular phone inventor
Cellular phones are used in rural areas around the world
![Page 9: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/9.jpg)
Big drop in cellular phone costs
Motorola DynaTAC$3995 in 1983*
HTC Desire 550$189 in 2018*
* https://www.technology.org/2017/09/18/cell-phone-cost-comparison-timeline/** https://www.ebay.com/i/282937194657?chn=ps
![Page 10: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/10.jpg)
Cellular phone access
ITU - International Telecommunications Union
https://www.itu.int/ITU-D/ict/statistics/ict/index.html
![Page 11: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/11.jpg)
Simple Scaling Formula
Low Costof production and related
services
High Supplythrough mass-production
Wide Adoptionof the technology
![Page 12: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/12.jpg)
Mass production achieves both
Low Costof production and related
services
High Supplythrough mass-production
![Page 13: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/13.jpg)
Cellular phones are essential to our livesFrom communications, to commerce and beyond.
Smartphones bring solace and aid to refugees http://time.com/4064988/refugee-crisis-selfies/
![Page 14: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/14.jpg)
Challenges in Scaling AIWhy is it so hard?
3
![Page 15: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/15.jpg)
Cost AI talent is very expensive
Access to data and talent limited to big companies
![Page 16: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/16.jpg)
$ 300,000 / year *Salary of a recent graduate AI engineer
Average US salary is $81,000 / year
* NYTimes: https://nyti.ms/2zHn4uv
![Page 17: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/17.jpg)
300%Increase in AI usage by enterprises by 2020
200,000*AI engineers worldwide in 2017*
~20 million developers worldwide in 2013**
* Tencent report: https://bit.ly/2It8nQ6 ** Computer world: https://bit.ly/2HCBmAQ
![Page 18: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/18.jpg)
“Of course demand outweighs supply. And things are not getting better any time soon. It takes many years to train a Ph.D.” *
Yoshua BengioUniv. of Montreal
* The Seattle Times, Oct 23, 2017https://commons.wikimedia.org/wiki/File:Yoshua_Bengio_-_2017.jpg
![Page 19: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/19.jpg)
This is bad news...
Increasing Demandfor AI solutions
Low Supplyof AI Talent
High Costsof AI solutions
![Page 20: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/20.jpg)
How can we fix this?Let’s take a deeper look
4
![Page 21: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/21.jpg)
Tools scale Talent
![Page 22: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/22.jpg)
Compilers scaled Programmers
Assembly routine to add two numbers
![Page 23: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/23.jpg)
Let’s take a closer look at
AI tooling
![Page 24: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/24.jpg)
There are many AI tools
TensorFlow, Pytorch, SKLearn, XGBoost and many others…
But it seems we still face a lot of challenges...
![Page 25: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/25.jpg)
AI mass production challenges
![Page 26: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/26.jpg)
AI mass production challenges
![Page 27: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/27.jpg)
History repeating itself?
“ML Technology is at a similar stage to early digital
computers where armies of white-clad technicians labored
to keep a small handful of machines operating in
production”
Tabulating the 1954 census with a UNIVAChttp://explorepahistory.com/displayimage.php?imgId=1-2-1536
![Page 28: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/28.jpg)
Why are these tools not enough for
AI mass production?
![Page 29: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/29.jpg)
The end-to-end picture of AI
Data Ingestion Data Analysis Data
TransformationTrainer Model
EvaluationModel
Validation Serving
Necessary for mass production
What most tools focus on
![Page 30: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/30.jpg)
ML code is a fraction of a real-world ML system
![Page 31: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/31.jpg)
We still have a big gap between
AI research AI mass production
![Page 32: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/32.jpg)
Mass productionLet’s look into an example
5
![Page 33: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/33.jpg)
Mass production in the automobile industry
72,105,435*Cars manufactured in 2016
135 Cars manufactured per minute
*http://www.worldometers.info/cars/
![Page 34: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/34.jpg)
But the automobile industry was not always this productive
![Page 35: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/35.jpg)
When was the first car invented?
You might be surprised
![Page 36: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/36.jpg)
1672 - First steam car design*Designed by Verbiest, Flemish missionary for Chinese emperor.
* "1679-1681 R P Verbiest's Steam Chariot". History of the Automobile: origin to 1900. Hergé. Image:https://commons.wikimedia.org/wiki/File:SteamMachineOfVerbiestIn1678.jpg
![Page 37: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/37.jpg)
When was the first ‘electric’ car invented?
You might also be surprised
![Page 38: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/38.jpg)
1888 - First electric car*German Flocken Elektrowagen
* Hans Roth: Das erste vierrädrige Elektroauto der Welt, March 2011, S. 2–3Image: https://commons.wikimedia.org/wiki/File:1888_Flocken_Elektrowagen.jpg
![Page 39: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/39.jpg)
Mass production lagged there too...
1906Stanley Steamer set record speed at
127 miles/hour*
1915Ford Model T
First mass produced car by moving assembly line*
* https://en.wikipedia.org/wiki/History_of_the_automobile#Brass_or_Edwardian_era
![Page 40: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/40.jpg)
The Moving Assembly Line *
◉ Introduced in 1913◉ 8x speed up in production◉ So fast paint became a
bottleneck◉ Cost dropped by 35% in 4
years
* G.N. Georgano 1985
![Page 41: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/41.jpg)
How does this apply to AI?
![Page 42: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/42.jpg)
AI mass production
Our AI tools today, are a lot more suited for building prototypes. Like building concept cars.
Than the mass production of AI solutions.
Images: https://commons.wikimedia.org/wiki/File:Tesla_Model_S_clay.jpghttps://commons.wikimedia.org/wiki/File:Tesla_auto_bots.jpg
![Page 43: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/43.jpg)
The rise of AI mass production systemsExamples from the industry
6
![Page 44: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/44.jpg)
AI Mass Production System
A hypothetical AI mass-production system
Data 1
Data 2
Data 3
Data n
Credit Analysis
OCR
ASR
![Page 45: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/45.jpg)
Google TensorFlow Extended (TFX)
![Page 46: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/46.jpg)
Facebook FBLearner
![Page 47: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/47.jpg)
Uber Michelangelo
https://eng.uber.com/michelangelo/
![Page 48: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/48.jpg)
So where are we going at Google?7
![Page 49: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/49.jpg)
Wide Adoptionof AI solutionsInvest in AI mass
production
Higher Supplyof AI solutions
Lower Costof AI solutions
Scale AI impact throughMass production
![Page 50: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/50.jpg)
Google Cloud AI Platform
https://cloud.google.com/ai-platform/
![Page 51: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/51.jpg)
Ingest & Label
Pre-process Trainer Validate Serving
Orchestration
UI (Jobs, Monitoring, Debugging, Visualization)
Tuner
Kubeflow PipelinesFor orchestrated end-to-end workflows
Productionize ML in two waysML
Engineer
Batch Online
PredictionTraining
AI Platform Training & PredictionFor 1-click managed serving
AI Platform
![Page 52: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/52.jpg)
Use: Fast & Simple Implementation of AI on GCPOne-click deployment of AI pipelines via Kubeflow on GCP as the go-to platform for AI + hybrid & on premise.
Share: Enterprise-grade Internal & External SharingFoster reuse by sharing deployable AI pipelines & othercontent privately within organizations & publicly.
Discover
Use
AutoML
AI Platform Train & Predict
Share
AI HubAI Hub
AI Hub: An AI Center of Excellence for your organization
Discover: All AI Content in One PlaceQuick discovery of plug & play AI pipelines & other content built by teams across Google and by partners and customers.
![Page 53: AI solutions Mass Production of - conferences.oreilly.com · AI mass production challenges. AI mass production challenges. History repeating itself? “ML Technology is at a similar](https://reader035.vdocuments.mx/reader035/viewer/2022070216/611a444ba046f95eb9029ea6/html5/thumbnails/53.jpg)
We’re not there yet!● Mostly internal systems● Incomplete solutions● Fragmented ecosystem● Vendor lock-in
and other issues…
But I can see the lightat the end of the tunnel