![Page 1: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/1.jpg)
![Page 2: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/2.jpg)
![Page 3: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/3.jpg)
The Microsoft Cognitive Toolkit (CNTK)
Microsoft’s open-source deep-learning toolkit
• ease of use: what, not how
• fast
• flexible
• 1st-class on Linux and Windows
• internal=external version
![Page 4: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/4.jpg)
deep learning at Microsoft
• Microsoft Cognitive Services
• Skype Translator
• Cortana
• Bing
• HoloLens
• Microsoft Research
![Page 5: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/5.jpg)
ImageNet: Microsoft 2015 ResNet
28.225.8
16.4
11.7
7.3 6.73.5
ILSVRC2010 NECAmerica
ILSVRC2011 Xerox
ILSVRC2012
AlexNet
ILSVRC2013 Clarifi
ILSVRC2014 VGG
ILSVRC2014
GoogleNet
ILSVRC2015 ResNet
ImageNet Classification top-5 error (%)
Microsoft had all 5 entries being the 1-st places this year: ImageNet classification,
ImageNet localization, ImageNet detection, COCO detection, and COCO segmentation
![Page 6: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/6.jpg)
deep learning at Microsoft
• Microsoft Cognitive Services
• Skype Translator
• Cortana
• Bing
• HoloLens
• Microsoft Research
![Page 7: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/7.jpg)
![Page 8: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/8.jpg)
24%
14%
![Page 9: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/9.jpg)
![Page 10: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/10.jpg)
![Page 11: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/11.jpg)
Microsoft’s historicspeech breakthrough
• Microsoft 2016 research system for
conversational speech recognition
• 5.9% word-error rate
• enabled by CNTK’s multi-server scalability
[W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke,
D. Yu, G. Zweig: “Achieving Human Parity in Conversational
Speech Recognition,” https://arxiv.org/abs/1610.05256]
![Page 12: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/12.jpg)
• CNTK is Microsoft’s open-source, cross-platform toolkit for learning and evaluating deep neural networks.
• CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.
• CNTK is production-ready: State-of-the-art accuracy, efficient, and scales to multi-GPU/multi-server.
CNTK “Cognitive Toolkit”
![Page 13: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/13.jpg)
• open-source model inside and outside the company• created by Microsoft Speech researchers (Dong Yu et al.) in 2012, “Computational Network Toolkit”
• open-sourced (CodePlex) in early 2015
• on GitHub since Jan 2016 under permissive license
• Python support since Oct 2016 (beta), rebranded as “Cognitive Toolkit”
• used by Microsoft product groups; but virtually all code development is out in the open
• external contributions e.g. from MIT and Stanford
• Linux, Windows, docker, cudnn5, next: CUDA 8
• Python and C++ API (beta; C#/.Net on roadmap)
“CNTK is Microsoft’s open-source, cross-platform toolkit for learning and evaluating deep neural networks.”
![Page 14: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/14.jpg)
“CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.”
![Page 15: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/15.jpg)
“CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.”
example: 2-hidden layer feed-forward NN
h1 = s(W1 x + b1) h1 = sigmoid (x @ W1 + b1)
h2 = s(W2 h1 + b2) h2 = sigmoid (h1 @ W2 + b2)
P = softmax(Wout h2 + bout) P = softmax (h2 @ Wout + bout)
with input x RM and one-hot label L RM
and cross-entropy training criterion
ce = LT log P ce = cross_entropy (L, P)
Scorpusce = max
![Page 16: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/16.jpg)
“CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.”
example: 2-hidden layer feed-forward NN
h1 = s(W1 x + b1) h1 = sigmoid (x @ W1 + b1)
h2 = s(W2 h1 + b2) h2 = sigmoid (h1 @ W2 + b2)
P = softmax(Wout h2 + bout) P = softmax (h2 @ Wout + bout)
with input x RM and one-hot label y RJ
and cross-entropy training criterion
ce = yT log P ce = cross_entropy (L, P)
Scorpusce = max
![Page 17: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/17.jpg)
“CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.”
example: 2-hidden layer feed-forward NN
h1 = s(W1 x + b1) h1 = sigmoid (x @ W1 + b1)
h2 = s(W2 h1 + b2) h2 = sigmoid (h1 @ W2 + b2)
P = softmax(Wout h2 + bout) P = softmax (h2 @ Wout + bout)
with input x RM and one-hot label y RJ
and cross-entropy training criterion
ce = yT log P ce = cross_entropy (P, y)
Scorpusce = max
![Page 18: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/18.jpg)
“CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.”
h1 = sigmoid (x @ W1 + b1)
h2 = sigmoid (h1 @ W2 + b2)
P = softmax (h2 @ Wout + bout)
ce = cross_entropy (P, y)
![Page 19: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/19.jpg)
“CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.”
•
+
s
•
+
s
•
+
softmax
W1
b1
W2
b2
Wout
bout
cross_entropy
h1
h2
P
x y
h1 = sigmoid (x @ W1 + b1)
h2 = sigmoid (h1 @ W2 + b2)
P = softmax (h2 @ Wout + bout)
ce = cross_entropy (P, y)
ce
![Page 20: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/20.jpg)
“CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.”
•
+
s
•
+
s
•
+
softmax
W1
b1
W2
b2
Wout
bout
cross_entropy
h1
h2
P
x y
ce• nodes: functions (primitives)
• can be composed into reusable composites
• edges: values• incl. tensors, sparse
• automatic differentiation• ∂F / ∂in = ∂F / ∂out ∙ ∂out / ∂in
• deferred computation execution engine
• editable, clonable
LEGO-like composability allows CNTK to supportwide range of networks & applications
![Page 21: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/21.jpg)
Cognitive Toolkit Demo
MNIST Handwritten Digits Recognitionon Jupyter Notebook
![Page 23: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/23.jpg)
![Page 24: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/24.jpg)
![Page 25: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/25.jpg)
Benchmarking on a single server by HKBU
“CNTK is production-ready: State-of-the-art accuracy, efficient, and scales to multi-GPU/multi-server.”
FCN-8 AlexNet ResNet-50 LSTM-64
CNTK 0.037 0.040 (0.054) 0.207 (0.245) 0.122
Caffe 0.038 0.026 (0.033) 0.307 (-) -
TensorFlow 0.063 - (0.058) - (0.346) 0.144
Torch 0.048 0.033 (0.038) 0.188 (0.215) 0.194
G980
![Page 26: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/26.jpg)
“CNTK is production-ready: State-of-the-art accuracy, efficient, and scales to multi-GPU/multi-server.”
Theano only supports 1 GPU
Achieved with 1-bit gradient quantizationalgorithm
0
10000
20000
30000
40000
50000
60000
70000
80000
CNTK Theano TensorFlow Torch 7 Caffe
speed comparison (samples/second), higher = better[note: December 2015]
1 GPU 1 x 4 GPUs 2 x 4 GPUs (8 GPUs)
![Page 27: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/27.jpg)
Azure NC-Instances
NC6 NC12 NC24 NC24r
Cores 6 12 24 24
GPU1 K80 GPU (1/2 Physical Card)
2 K80 GPUs (1 Physical Card)
4 K80 GPUs (2 Physical Cards)
4 K80 GPUs (2 Physical Cards)
Memory 56 GB 112 GB 224 GB 224 GB
Disk ~380 GB SSD ~680 GB SSD ~1.5 TB SSD ~1.5 TB SSD
Network Azure Network Azure Network Azure Network InfiniBand
![Page 28: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/28.jpg)
![Page 29: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/29.jpg)
• ease of use• what, not how
• powerful stock library
• fast• optimized for NVidia GPUs & libraries
• best-in-class multi-GPU/multi-server algorithms
• flexible• powerful & composable Python and C++ API
• 1st-class on Linux and Windows
• internal=external version
CNTK: democratizing the AI tool chain
![Page 30: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/30.jpg)
• Web site: https://cntk.ai/
• Github: https://github.com/Microsoft/CNTK
• Wiki: https://github.com/Microsoft/CNTK/wiki
• Issues: https://github.com/Microsoft/CNTK/issues
CNTK: democratizing the AI tool chain
![Page 31: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/31.jpg)
Seeing AI
![Page 32: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/32.jpg)
![Page 33: [html5jロボット部 第7回勉強会] Microsoft Cognitive Toolkit (CNTK) Overview](https://reader030.vdocuments.mx/reader030/viewer/2022012406/586fd46d1a28ab18428b485b/html5/thumbnails/33.jpg)