deep learning with apache mxnet
TRANSCRIPT
Deep Learning with Apache MXNet
Julien Simon, AI Evangelist, EMEA@julsimon
What to expect
• Apache MXNet
• Demos using Jupyter notebooks
• Resources
• Q&A
Apache MXNet: Open Source library for Deep Learning
Programmable Portable High PerformanceNear linear scalingacross hundreds of
GPUs
Highly efficient models for
mobileand IoT
Simple syntax, multiple
languages
Most Open Best On AWSOptimized for
Deep Learning on AWSAccepted into theApache Incubator
CPU or GPU: your choice
mod = mx.mod.Module(lenet)
mod = mx.mod.Module(lenet, context=mx.gpu(0))
mod = mx.mod.Module(lenet, context=(mx.gpu(7), mx.gpu(8), mx.gpu(9)))
Ideal
Inception v3Resnet
Alexnet
88%Efficiency
0
64
128
192
256
1 2 4 8 16 32 64 128 256
Multi-GPU Scaling With MXNet
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Storing and accessing data in multi-dimensional arraysàNDArray API
• Building models (layers, weights, activation functions) à Symbol API
• Serving data during training and validation à Iterators
• Training and using models à Module API
The Apache MXNet API
Input Output
1 1 11 0 10 0 0
3
mx.sym.Convolution(data, kernel=(5,5), num_filter=20)
mx.sym.Pooling(data, pool_type="max", kernel=(2,2),
stride=(2,2)
lstm.lstm_unroll(num_lstm_layer, seq_len, len, num_hidden, num_embed)
4 2 2 0 4=Max
13...4
0.2-0.1...0.7
mx.sym.FullyConnected(data, num_hidden=128)
2
mx.symbol.Embedding(data, input_dim, output_dim = k)
0.2-0.1...0.7
Queen
4 2 2 0 2=Avg
Input Weights
cos(w, queen) = cos(w, king) - cos(w, man) + cos(w, woman)
mx.sym.Activation(data, act_type="xxxx")
"relu"
"tanh"
"sigmoid"
"softrelu"
Neural Art
Face Search
Image Segmentation
Image Caption
“People Riding Bikes”
Bicycle, People, Road, SportImage Labels
Image
Video
Speech
Text
“People Riding Bikes”
Machine Translation
“Οι άνθρωποι ιππασίας ποδήλατα”
Events
mx.model.FeedForward model.fit
mx.sym.SoftmaxOutput
Anatomy of a Deep Learning Model
Gluon: Deep Learning gets even easierhttps://github.com/gluon-api/
• Available now in MXNet, soon in Microsoft Cognitive Toolkit• Developer-friendly high-level API• Dynamic networks can be modified during training•No compromise on performance• Extensive model zoo
https://aws.amazon.com/blogs/aws/introducing-gluon-a-new-library-for-machine-learning-from-aws-and-microsoft/
Gluon Model Zoo
VGGResNetAlexNet
DenseNetSqueezeNet
InceptionMobileNet
https://mxnet.incubator.apache.org/versions/master/api/python/gluon/model_zoo.html
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deep Learning in practiceAWS Deep Learning AMI
MXNet
Torch
CTKKeras
TheanoCaffe
TensorFlow
Amazon EC2
AnacondaIntel MKL
CUDA+cuDNN Python 2+3
Caffe2
• One-click launch• Single node or distributed• CPU, GPU, FPGA• NVIDIA & Intel libraries• Anaconda Data Science Platform• Python w/ AI/ML/DL libraries
Demos https://github.com/juliensimon/dlnotebooks1) Synthetic data set2) Learn MNIST with a Multi-Layer Perceptron3) Learn MNIST with the LeNet CNN4) Predict handmade MNIST samples
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The MNIST data set
• 70,000 hand-written digits• 28x28 grayscale images
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• MLP does well on MNIST, but not on real-life samples.
• MLP flattens the 28x28 image into a 784-byte vectorà the 2-D relationship between pixels is lost.
• Look at the ‘5’: the top-left angle is a unique feature. No other digit exhibits this. Same thing for the intersection on the ‘8’.
• CNNs such as LeNet work on 2-D images and learn to detect these unique geometric features.
• The result is a more accurate and more robust network.
Training models on MNIST
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Resources
https://aws.amazon.com/machine-learninghttps://aws.amazon.com/blogs/ai
https://mxnet.incubator.apache.orghttps://github.com/apache/incubator-mxnethttps://github.com/gluon-api
https://medium.com/@julsimon
Thank you!
Julien Simon, AI Evangelist, EMEA@julsimon