TensorFlow in Context

1

Jiqiong QIU

About me

2

●Master of engineer degree in Bioinformatic and modelization at INSA de Lyon

●PhD in Color Formulation by Statistical Learning at UTC and BASF Coatings(EP 2887275A1: Method and system for determining a color formula)

●Data scientist at Sfeir

1. Deep Learning

2. TensorFlow

3. TensorFlow in Context

TensorFlow in Context

3

Deep Learning1

4

1 Deep Learning

1.1 What is Deep Learning?

1.2 Difference between academic research and industry

application

5

1.1 What is deep learning

Artificial Intelligence

MachineLearning

Logistic,Regression, SVM,Neural Network

Deep LearningCNN, LSTM,

Neural TuringMachines

6

1.1 What is deep learning

7

1.2 Difference between academic research and industry application

8

1.2 Difference between academic research and industry application

Academic Research Industry Application

Key Point Research Application

Time Investment Long term Short term

Development Environment Stand alone IDE, Compilation tools, Teamwork etc

Goal Interest/ publication Problem solving

9

TensorFlow2

10

2 TensorFlow

2.1 Key features

2.2 Comparison with others deep learning libraries

11

2.1 Key features

12

●Open source by Google●Python API●Board●Android (SDK) Mobile

application

https://github.com/tensorflow/tensorflow

However, TensorFlow is very slow...

2.1 Key features

https://github.com/soumith/convnet-benchmarkshttps://www.reddit.com/r/MachineLearning/comments/48gfop/tensorflow_speed_questions/

13

2.2 Comparison with others deep learning libraries http://deeplearning.net/software_links/

14

2.2 Comparison with others deep learning libraries

Name Language OS GPU Related Library

Theano Python Win, Lin, Mac CUDA,Opencl Lasagne, Keras

Torch Lua, C Lin, IOS, Android

CUDA

Caffe C++, Python, Matlab

Lin, Win, Mac CUDA, Opencl

TensorFlow Python Lin, Mac, Android

CUDA Keras, Skflow

mxnet Python, R, Julia

Lin, Windows, Mac

CUDA

https://github.com/zer0n/deepframeworks

15

2.2 Comparison with others deep learning librariesWhy TensorFlow?

16

TensorFlow in Context3

17

3.1 What is unique about TensorFlow?

3.2 TensorFlow with Data Science Tools

3.3 TensorFlow for Big Data

3 TensroFlow in Context

18

3.1 What is unique about TensorFlow

That would be crazy if it weren't Google

19

The author list of TensorFlow:⬡ Jeff Dean: father of MapReduce⬡ Ian Goodfellow: main contributor of Theano/PyLearn2⬡ Yangqing Jia: main contributor of Caffe⬡ and other great Google researchers and engineers.

3.1 What is unique about TensorFlow

20

3.2 TensorFlow with Data Science ToolsWhy we need deep learning in Industry application besides playing Go?

21

Avoid hand-crafted features

3.2 Tensorflow with Data Science Tools

22

No free lunch:Deep learning applications are generally applied to massive unstructured data.

MNIST 60k ImageNet 50 million

Yelp Restaurant Photo Classification

230 k

3.2 Tensorflow with Data Science Tools

23

Most used data science languages:

TensorFlow has an API in Python

Python R

Data Manipulation Pandas dplyr, data.table

Data Visualization Matplotlib ggplot2, ggvis

Machine Learning scikit-learn caret

3.2 Tensorflow with Data Science Tools

24

Deep Learning is hard:

3.2 Tensorflow with Data Science Tools

25

Deep learning library like keras, Skflow (based on TensorFlow) were developed with a focus on enabling fast experimentation.

3.2 Tensorflow with Data Science Tools

26

No free lunch:Deep learning applications are generally applied to massive unstructured data.

MNIST 60k ImageNet 50 million

Yelp Restaurant Photo Classification

230 k

3.3 Tensorflow for Big DataGPU makes the deep learning training possible

27

3.3 Tensorflow for Big Data

CPU vs GPU

28

Training on Multiple-GPU:⬡ A single GTX 580 GPU has only 3GB of memory⬡ GPU memory limits the maximum size of the networks

that can be trained⬡ Training examples may be too big to fit on on GPU

3.3 Tensorflow for Big Data

29

1 GPU vs multiple-GPU

3.3 Tensorflow for Big Data

30

In TensorFlow, the supported device types are CPU and GPU. They are represented as strings. For example:⬡ "/cpu:0": The CPU of your machine.

⬡ "/gpu:0": The GPU of your machine, if you have one.

⬡ "/gpu:1": The second GPU of your machine, etc.

Much earier than others librarieshttps://www.tensorflow.org/versions/r0.7/how_tos/using_gpu/index.html

3.3 Tensorflow for Big Data

31

3.3 Tensorflow for Big Data GPU is not enough

32

Speed

Ease of use

Generality

Runs Everywhere

3.3 Tensorflow for Big Data

33

https://databricks.com/blog/2016/01/25/deep-learning-with-spark-and-tensorflow.html

Distributed Tensor Flow on Spark is published on early 2016

3.3 Tensorflow for Big Data

34

Tensorflow in Context

Name Language OS GPU Related Library

Theano Python Win, Lin, Mac CUDA,Opencl Lasagne, Keras

Torch Lua, C Lin, IOS, Android

CUDA

Caffe C++, Python, Matlab

Lin, Win, Mac CUDA, Opencl

Tensorflow Python Lin, Mac, Android

CUDA Keras, Skflow

mxnet Python, R, Julia

Lin, Windows, Mac

CUDA

35

Thank you.

36

37

by Jiqiong QIUSFEIR - Copyright ©2016