deep learning: a theoretical introduction · deep learning: a theoretical introduction –episode 2...

[1]Deep Learning: a theoretical introduction – Episode 2

Deep Learning:a theoretical introductionEpisode 2The Turning Point

Università degli Studi di Pavia

[2]Deep Learning: a theoretical introduction – Episode 2

A hard AI winter then ensued …

… then, the turning point

[3]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Networks and Autoencoders

[4]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Networks and Autoencoders

[5]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Networks and Autoencoders

[6]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Networks and Autoencoders

[7]Deep Learning: a theoretical introduction – Episode 2

An aside question:probabilities or just numbers?

[8]Deep Learning: a theoretical introduction – Episode 2

•

•

Restricted Boltzmann Machine (RBM)

[9]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[10]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)h hi

hi

hi = 1 x

W i

[11]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

xj = 1 h

[12]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

hi hj x

xi xj h

x

hi

[13]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[14]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[15]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[16]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

S

[17]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

S

[18]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

(i)

[19]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[20]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[21]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

x

[22]Deep Learning: a theoretical introduction – Episode 2

•

•

•

Restricted Boltzmann Machine (RBM)

[23]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

T q

[24]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

•

•

•

P

[25]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

x(0)

h(t+1)

x(t+1)

[26]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

x(0)

h(t+1)

x(t+1)

•

•

[27]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[28]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[29]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[30]Deep Learning: a theoretical introduction – Episode 2

Deep Boltzmann Machines (DBM)

[31]Deep Learning: a theoretical introduction – Episode 2

Deep Boltzmann Machines (DBM)

x

hi

[32]Deep Learning: a theoretical introduction – Episode 2

Deep Boltzmann Machines (DBM)

h(2)

[33]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

[34]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

[35]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

[36]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

hi = 1

[37]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

[38]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

[39]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

[40]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

[41]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Network (DBN)

[42]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[43]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[44]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[45]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[46]Deep Learning: a theoretical introduction – Episode 2

Restricted Boltzmann Machine (RBM)

[47]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Networks and Autoencoders

[48]Deep Learning: a theoretical introduction – Episode 2

Auto-encoders

[49]Deep Learning: a theoretical introduction – Episode 2

Auto-encoders

[50]Deep Learning: a theoretical introduction – Episode 2

Deep Belief Networks and Autoencoders

[51]Deep Learning: a theoretical introduction – Episode 2

The Turning Point

[52]Deep Learning: a theoretical introduction – Episode 2

The Turning Point

[53]Deep Learning: a theoretical introduction – Episode 2

What happened after that

[54]Deep Learning: a theoretical introduction – Episode 2

What happened after that