Machine Intelligence at Google ScaleML APIs, TensorFlow and Cloud ML

+Kazunori Sato@kazunori_279

Kaz Sato

Staff Developer AdvocateTech Lead for Data & AnalyticsCloud Platform, Google Inc.

What we’ll cover

What is Neural Network and Deep Learning

Machine Learning use cases at Google services

Externalizing the power of ML

TensorFlow: the open source library for ML

TensorFlow in the Wild

Scalable Training with Cloud ML

What is Neural Network and Deep Learning

Neural Network is a function that can learn

How about this?

More hidden layers = More hierarchies of features

How about this?

From: mNeuron: A Matlab Plugin to Visualize Neurons from Deep Models, Donglai Wei et. al.

Machine Learning use casesat Google services

11

signalfor Search ranking,

out of hundreds

improvementto ranking quality

in 2+ years

#3 #1

Search

machine learning for search engines

RankBrain: a deep neural network for search ranking

12

[glacier]

Google Photos

12

13

Smart reply in Inbox by Gmail

10%of all responses sent on mobile

Google Translate with Neural Machine Translation

Saved Data Center cooling energy for 40%Improved Power Usage Effectiveness (PUE) for 15%

16

AndroidAppsGmailMapsPhotosSpeechSearchTranslationYouTubeand many others ...

Used across products:

2012 2013 2014 2015

Deep Learning usage at Google

Externalizing the power of ML

TensorFlow Cloud Machine Learning ML API

Easy-to-Use, for non-ML engineers

Customizable, for Data Scientists

Machine Learning products from Google

Image analysis with pre-trained models

No Machine Learning skill required

REST API: receives an image and returns a JSON

General Availability

Cloud Vision API

2020

Demo

Pre-trained models. No ML skill required

REST API: receives audio and returns texts

Supports 80+ languages

Streaming or non-streaming

Public Beta - cloud.google.com/speech

Cloud Speech API

2222

Demo

Pre-trained models. No ML skill required

REST API: receives text and returns analysis results

Supports English, Spanish and Japanese

Public Beta - cloud.google.com/natural-language

Cloud Natural Language API

2424

Demo

TensorFlow:An open source library forMachine Intelligence

Google's open source library for

machine intelligence

tensorflow.org launched in Nov 2015

Used by many production ML projects

What is TensorFlow?

Sharing our tools with researchers and developers around the world

repositoryfor “machine learning”

category on GitHub

#1

Released in Nov. 2015

From: http://deliprao.com/archives/168

# define the networkimport tensorflow as tfx = tf.placeholder(tf.float32, [None, 784])W = tf.Variable(tf.zeros([784, 10]))b = tf.Variable(tf.zeros([10]))y = tf.nn.softmax(tf.matmul(x, W) + b)

# define a training stepy_ = tf.placeholder(tf.float32, [None, 10])xent = -tf.reduce_sum(y_*tf.log(y))step = tf.train.GradientDescentOptimizer(0.01).minimize(xent)

Portable and ScalableTraining on:

Mac/Windows

GPU server

GPU cluster / Cloud

Prediction on:

Android and iOS

RasPi and TPU

TensorFlow in the Wild(or democratization of deep learning)

TensorFlow powered Cucumber Sorter

From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/

TensorFlow

powered

Fried Chicken

Nugget Server

From: http://www.rt-net.jp/karaage1/

TensorFlow＋

RasPi forsorting garbages

From: https://techcrunch.com/2016/09/13/auto-trash-sorts-garbage-automatically-at-the-techcrunch-disrupt-hackathon/

TensorFlow +

Drones

for counting trucks

From: http://www.brainpad.co.jp/news/2016/09/02/3454

Distributed Training and Prediction with Cloud ML

The Challenge: Computing Power

DNN requires large training datasets

Large models doesn't fit into a GPU

Requires try-and-errors to find the best design, configs and params

↓Need to spend a few days or

weeks to finish a training

Enterprise

Google Cloud is

The Datacenter as a Computer

Jupiter network

10 GbE x 100 K = 1 Pbps

Consolidates servers with

microsec latency

Distributed Training with TensorFlow

● CPU/GPU scheduling

● Communications

○ Local, RPC, RDMA

○ 32/16/8 bit quantization

● Cost-based optimization

● Fault tolerance

Distributed Training with TensorFlow

Tensor Processing Unit

ASIC for TensorFlow

Designed by Google

10x better perf / watt

latency and efficiency

bit quantization

"Large Scale Distributed Systems for Training Neural Networks", NIPS

2015

○ Inception / ImageNet: 40x with 50 GPUs

○ RankBrain: 300x with 500 nodes

Distributed Training with TensorFlow

Fully managed distributed training and prediction

Supports custom TensorFlow graphs

Integrated with Cloud Dataflow and Cloud Datalab

Public Beta - cloud.google.com/ml

Cloud Machine Learning (Cloud ML)

Cloud ML at Work: AUCNET

The largest real-time car auction service in Japan

For 30K used car dealers

The auction volume overs $3.7B every year

Problem: auction entry is time consuming task for dealers

Classifying parts of car for thousands of photos

Identifying the exact car model

Solution: Custom Image Classification with TensorFlow/Cloud ML

Used 5,000 training images for 500 car models

Inception v3 + Transfer Learning

Cloud ML: increased training performance for 6x

4848

Demo

4949

Ready to use Machine Learning models

Use your own data to train models

Cloud Vision API

Cloud Speech API

Cloud Translate API

Cloud Machine Learning

Develop - Model - Test

Google BigQuery

Stay Tuned….

Cloud Storage

Cloud Datalab

NEW

Alpha

GA BetaGA

AlphaGA

GA

Thank you!