Digital Image Processing Analysis in IndustryYogyakarta, 3-5 Dec 2016

About Me

Sofian Hadiwijaya@sofianhwwww.sofianhw.com

Intel Software Innovator Tech Advisor - Nodeflux.ioCo-Founder - Pinjam.co.id

“A computer program issaid to learn fromexperience Ewith respect to someclass of tasks Tand performancemeasure P,if its performance at tasks in T,as measured by P,improves withexperience E”

12/13/16Intel Confidential 4

What is Machine Learning

4

Machine Learning, a key tool for AI, is the development, and application of algorithms that improve their performance at some task based on experience (previous iterations)

Training: Build a mathematical model based on a data set

Scoring: Use trained model to make predictions about new data

Deep LearningAlgorithms where multiple layers of neurons learn successively complex representations

RBM …RNNCNN

Statistical/ Other Machine LearningAlgorithms based on statistical or other techniques for estimating functions from examples

GA Linear RegressionSVMNaïve

Bayes

12/13/16Intel Confidential 5

End to End Workflow

5

Things

Data AnnotationLabel & Prep Data

Model DeploymentOver-the-Air

Secure & Real Time

Model ScoringApp Dev & Runtimes

Embedded OS

Model UpdateTrack Model Drift

Manage Model Lifecycle

New DataNew Model

Data AggregationData Curation

Catalog Data Sets

Model TrainingTrain for Accuracy

Model ValidationRun SimulationsCross-Validate

Distributed SystemsData Acquisition

Model PerformanceTune for Performance

Analyze Longitudinal Effects

12/13/16Intel Confidential 6

What is DeepLearning

6

Multiple definitions, however, these definitions have in common:• Multiple layers of processing units• Supervised or unsupervised learning of feature representations in

each layer, with the layers forming a hierarchy from low level to high level features.

12/13/16Intel Confidential 7

What is CNN

7

Essentially neural networks that use convolution in place of general matrix multiplication in at least one of their layers

12/13/16Intel Confidential 8

How CNN Works

8

12/13/16Intel Confidential 9

How CNN Works

9

A toy ConvNet: X’s and O’s

X or OCNN

Says whether a picture is of an X or an O

A two-dimensionalarray of pixels

12/13/16Intel Confidential 10

How CNN Works

10

CNN X

CNN O

12/13/16Intel Confidential 11

How CNN Works

11

CNN X

CNN O

12/13/16Intel Confidential 12

How CNN Works

12

=?

12/13/16Intel Confidential 13

How CNN Works

13

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 -1 -1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 1 -1 -1 -1-1 -1 1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 1 -1 -1-1 -1 -1 1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 -1 -1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

=?

12/13/16Intel Confidential 14

How CNN Works

14

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 X -1 -1 -1 -1 X X -1-1 X X -1 -1 X X -1 -1-1 -1 X 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 X -1 -1-1 -1 X X -1 -1 X X -1-1 X X -1 -1 -1 -1 X -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 15

How CNN Works

15

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 -1 -1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 1 -1 -1 -1-1 -1 1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 1 -1 -1-1 -1 -1 1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 -1 -1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

=x

12/13/16Intel Confidential 16

How CNN Works

16

=

=

=

12/13/16Intel Confidential 17

How CNN Works

17

1 -1 -1-1 1 -1-1 -1 1

-1 -1 1-1 1 -11 -1 -1

1 -1 1-1 1 -11 -1 1

12/13/16Intel Confidential 18

How CNN Works

18

1 -1 -1-1 1 -1-1 -1 1

-1 -1 1-1 1 -11 -1 -1

1 -1 1-1 1 -11 -1 1

12/13/16Intel Confidential 19

How CNN Works

19

1 -1 -1-1 1 -1-1 -1 1

-1 -1 1-1 1 -11 -1 -1

1 -1 1-1 1 -11 -1 1

12/13/16Intel Confidential 20

How CNN Works

20

1 -1 -1-1 1 -1-1 -1 1

-1 -1 1-1 1 -11 -1 -1

1 -1 1-1 1 -11 -1 1

12/13/16Intel Confidential 21

How CNN Works

21

1 -1 -1-1 1 -1-1 -1 1

-1 -1 1-1 1 -11 -1 -1

1 -1 1-1 1 -11 -1 1

12/13/16Intel Confidential 22

How CNN Works

22

1 -1 -1-1 1 -1-1 -1 1

-1 -1 1-1 1 -11 -1 -1

1 -1 1-1 1 -11 -1 1

12/13/16Intel Confidential 23

How CNN Works

23

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

Filtering: The math behind the match

12/13/16Intel Confidential 24

How CNN Works

24

Filtering: The math behind the match

1. Line up the feature and the image patch.

2. Multiply each image pixel by the corresponding feature pixel.

3. Add them up.

4. Divide by the total number of pixels in the feature.

12/13/16Intel Confidential 25

How CNN Works

25

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 26

How CNN Works

26

11 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 27

How CNN Works

27

1 11 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 28

How CNN Works

28

1 1 11 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 29

How CNN Works

29

1 1 11

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 30

How CNN Works

30

1 1 11 1

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 31

How CNN Works

31

1 1 11 1 1

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 32

How CNN Works

32

1 1 11 1 11

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 33

How CNN Works

33

1 1 11 1 11 1

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 34

How CNN Works

34

1 1 11 1 11 1 1

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 35

How CNN Works

35

1

1 1 11 1 11 1 1

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 36

How CNN Works

36

11 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 37

How CNN Works

37

1 1 -11 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 38

How CNN Works

38

1 1 -11 1 1-1 1 1

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 39

How CNN Works

39

1

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

1 1 -11 1 1-1 1 1

.55

1 1 -11 1 1-1 1 1

12/13/16Intel Confidential 40

How CNN Works

40

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

12/13/16Intel Confidential 41

How CNN Works

41

1 -1 -1-1 1 -1-1 -1 1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

=

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

12/13/16Intel Confidential 42

How CNN Works

42

1 -1 -1-1 1 -1-1 -1 1

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

=

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-1 -1 1-1 1 -11 -1 -1

1 -1 1-1 1 -11 -1 1

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

=

=

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 43

How CNN Works

43

One image becomes a stack of filtered images

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

1 -1 -1-1 1 -1-1 -1 1

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-1 -1 1-1 1 -11 -1 -1

1 -1 1-1 1 -11 -1 1

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 44

How CNN Works

44

One image becomes a stack of filtered images

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

12/13/16Intel Confidential 45

How CNN Works

45

Pooling: Shrinking the image stack

1. Pick a window size (usually 2 or 3).

2. Pick a stride (usually 2).

3. Walk your window across your filtered images.

4. From each window, take the maximum value.

12/13/16Intel Confidential 46

How CNN Works

46

1.00

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

maximum

12/13/16Intel Confidential 47

How CNN Works

47

1.00 0.33

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

maximum

12/13/16Intel Confidential 48

How CNN Works

48

1.00 0.33 0.55

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

maximum

12/13/16Intel Confidential 49

How CNN Works

49

1.00 0.33 0.55 0.33

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

maximum

12/13/16Intel Confidential 50

How CNN Works

50

1.00 0.33 0.55 0.33

0.33

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

maximum

12/13/16Intel Confidential 51

How CNN Works

51

1.00 0.33 0.55 0.33

0.33 1.00 0.33 0.55

0.55 0.33 1.00 0.11

0.33 0.55 0.11 0.77

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

12/13/16Intel Confidential 52

How CNN Works

52

1.00 0.33 0.55 0.33

0.33 1.00 0.33 0.55

0.55 0.33 1.00 0.11

0.33 0.55 0.11 0.77

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

0.33 0.55 1.00 0.77

0.55 0.55 1.00 0.33

1.00 1.00 0.11 0.55

0.77 0.33 0.55 0.33

0.55 0.33 0.55 0.33

0.33 1.00 0.55 0.11

0.55 0.55 0.55 0.11

0.33 0.11 0.11 0.33

12/13/16Intel Confidential 53

How CNN Works

53

1.00 0.33 0.55 0.33

0.33 1.00 0.33 0.55

0.55 0.33 1.00 0.11

0.33 0.55 0.11 0.77

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

0.33 0.55 1.00 0.77

0.55 0.55 1.00 0.33

1.00 1.00 0.11 0.55

0.77 0.33 0.55 0.33

0.55 0.33 0.55 0.33

0.33 1.00 0.55 0.11

0.55 0.55 0.55 0.11

0.33 0.11 0.11 0.33

A stack of images becomes a stack of smaller images.

12/13/16Intel Confidential 54

How CNN Works

54

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

0.77

12/13/16Intel Confidential 55

How CNN Works

55

0.77 00.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

12/13/16Intel Confidential 56

How CNN Works

56

0.77 0 0.11 0.33 0.55 0 0.330.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

12/13/16Intel Confidential 57

How CNN Works

57

0.77 0 0.11 0.33 0.55 0 0.33

0 1.00 0 0.33 0 0.11 0

0.11 0 1.00 0 0.11 0 0.55

0.33 0.33 0 0.55 0 0.33 0.33

0.55 0 0.11 0 1.00 0 0.11

0 0.11 0 0.33 0 1.00 0

0.33 0 0.55 0.33 0.11 0 0.77

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

12/13/16Intel Confidential 58

How CNN Works

58

0.77 0 0.11 0.33 0.55 0 0.33

0 1.00 0 0.33 0 0.11 0

0.11 0 1.00 0 0.11 0 0.55

0.33 0.33 0 0.55 0 0.33 0.33

0.55 0 0.11 0 1.00 0 0.11

0 0.11 0 0.33 0 1.00 0

0.33 0 0.55 0.33 0.11 0 0.77

0.33 0 0.11 0 0.11 0 0.33

0 0.55 0 0.33 0 0.55 0

0.11 0 0.55 0 0.55 0 0.11

0 0.33 0 1.00 0 0.33 0

0.11 0 0.55 0 0.55 0 0.11

0 0.55 0 0.33 0 0.55 0

0.33 0 0.11 0 0.11 0 0.33

0.33 0 0.55 0.33 0.11 0 0.77

0 0.11 0 0.33 0 1.00 0

0.55 0 0.11 0 1.00 0 0.11

0.33 0.33 0 0.55 0 0.33 0.33

0.11 0 1.00 0 0.11 0 0.55

0 1.00 0 0.33 0 0.11 0

0.77 0 0.11 0.33 0.55 0 0.33

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

0.77 -0.11 0.11 0.33 0.55 -0.11 0.33

-0.11 1.00 -0.11 0.33 -0.11 0.11 -0.11

0.11 -0.11 1.00 -0.33 0.11 -0.11 0.55

0.33 0.33 -0.33 0.55 -0.33 0.33 0.33

0.55 -0.11 0.11 -0.33 1.00 -0.11 0.11

-0.11 0.11 -0.11 0.33 -0.11 1.00 -0.11

0.33 -0.11 0.55 0.33 0.11 -0.11 0.77

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.11 0.33 -0.77 1.00 -0.77 0.33 -0.11

0.11 -0.55 0.55 -0.77 0.55 -0.55 0.11

-0.55 0.55 -0.55 0.33 -0.55 0.55 -0.55

0.33 -0.55 0.11 -0.11 0.11 -0.55 0.33

A stack of images becomes a stack of images with no negative values

12/13/16Intel Confidential 59

How CNN Works

59

Conv

olut

ion

ReLU

Pool

ing-1 -1 -1 -1 -1 -1 -1 -1 -1

-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

1.00 0.33 0.55 0.33

0.33 1.00 0.33 0.55

0.55 0.33 1.00 0.11

0.33 0.55 0.11 0.77

0.33 0.55 1.00 0.77

0.55 0.55 1.00 0.33

1.00 1.00 0.11 0.55

0.77 0.33 0.55 0.33

0.55 0.33 0.55 0.33

0.33 1.00 0.55 0.11

0.55 0.55 0.55 0.11

0.33 0.11 0.11 0.33

The output of one becomes the input of the next.

12/13/16Intel Confidential 60

How CNN Works

60

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

1.00 0.55

0.55 1.00

0.55 1.00

1.00 0.55

1.00 0.55

0.55 0.55

Layers can be repeated several (or many) times.

12/13/16Intel Confidential 61

How CNN Works

61

1.00 0.55

0.55 1.00

0.55 1.00

1.00 0.55

1.00 0.55

0.55 0.55

1.00

0.55

0.55

1.00

1.00

0.55

0.55

0.55

0.55

1.00

1.00

0.55

Fully connected layerEvery value gets a vote

12/13/16Intel Confidential 62

How CNN Works

62

X

O

1.00

0.55

0.55

1.00

1.00

0.55

0.55

0.55

0.55

1.00

1.00

0.55

12/13/16Intel Confidential 63

How CNN Works

63

0.55

1.00

1.00

0.55

0.55

0.55

0.55

0.55

1.00

0.55

0.55

1.00

X

O

12/13/16Intel Confidential 64

How CNN Works

64

0.9

0.65

0.45

0.87

0.96

0.73

0.23

0.63

0.44

0.89

0.94

0.53

X

O

12/13/16Intel Confidential 65

How CNN Works

65

0.9

0.65

0.45

0.87

0.96

0.73

0.23

0.63

0.44

0.89

0.94

0.53

X

O

.92

12/13/16Intel Confidential 66

How CNN Works

66

0.9

0.65

0.45

0.87

0.96

0.73

0.23

0.63

0.44

0.89

0.94

0.53

X

O .51

12/13/16Intel Confidential 67

How CNN Works

67

0.9

0.65

0.45

0.87

0.96

0.73

0.23

0.63

0.44

0.89

0.94

0.53

X

O

.92

.51

12/13/16Intel Confidential 68

How CNN Works

68

0.9

0.65

0.45

0.87

0.96

0.73

0.23

0.63

0.44

0.89

0.94

0.53

X

O

12/13/16Intel Confidential 69

How CNN Works

69

0.9

0.65

0.45

0.87

0.96

0.73

0.23

0.63

0.44

0.89

0.94

0.53

X

O

12/13/16Intel Confidential 70

How CNN Works

70

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

Conv

olut

ion

ReLU

Pool

ing

Conv

olut

ion

ReLU

Conv

olut

ion

ReLU

Pool

ing

Fully

conn

ecte

d

Fully

conn

ecte

d XO

12/13/16Intel Confidential 71

How CNN Works

71

-1 -1 -1 -1 -1 -1 -1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 -1 -1 1 -1 -1 -1 -1-1 -1 -1 1 -1 1 -1 -1 -1-1 -1 1 -1 -1 -1 1 -1 -1-1 1 -1 -1 -1 -1 -1 1 -1-1 -1 -1 -1 -1 -1 -1 -1 -1

Conv

olut

ion

ReLU

Pool

ing

Conv

olut

ion

ReLU

Conv

olut

ion

ReLU

Pool

ing

Fully

conn

ecte

d

Fully

conn

ecte

d XO

Rightanswer Actualanswer ErrorX 1 0.92 0.08O 0 0.51 0.49

Total 0.57

12/13/16Intel Confidential 72

Workshop!!!

72

• $ git clone https://github.com/sofianhw/dsi-camp-cnn.git• $ cd dsi-camp-cnn• $ docker pull sofianhw/docker-neon-ipython• $ docker run -t -p 8888:8888 --name neon sofianhw/docker-neon-ipython• $ docker cp cifar_example.ipynb neon:/root/neon• open browser http://localhost:8888• Follow the step

12/13/16Intel Confidential 73

CNN in Real Life

73

Nodeflux

12/13/16Intel Confidential 74

Data Collection

Nodeflux

12/13/16Intel Confidential 76

Nodeflux

12/13/16Intel Confidential 77

Nodeflux

12/13/16Intel Confidential 78

Yogyakarta, 3-5 Dec 2016

Sofian Hadiwijaya@sofianhwwww.sofianhw.com