learning handwritten digits with a simple neural...
TRANSCRIPT
11/11/18
1
Learninghandwrittendigitswithasimpleneuralnetwork
sampleimage
28
28
oneoutputunitforeachdigit
selectoutputunitwithmaximumresponse
e.g.9
startwithrandominitialweightsanduseback-propagationtolearnweightstorecognizedigits
MNISTdatabase:300028x28imagesofhandwrittendigits
oneinputunitforeachpixel
25hiddenunits
“Deep”neuralnetworks• earlyworkextendedsimpleneuralnetworkstohavemultiple,highly-connectedhiddenlayers
• if suchnetworkscouldbetrained,theywouldbemuchmorepowerfulthan“shallow”neuralnets
• but genericmulti-layernetworksareextremelyhardtotrain!!
11/11/18
2
State-of-the-artrecognitionsystemsarebasedonconvolutional neuralnetworksPublicdatabasesoffaceimagesserveasbenchmarks:
LabeledFacesintheWild(LFW,http://vis-www.cs.umass.edu/lfw)>13,000imagesofcelebrities,5,749differentidentities
YouTubeFacesDatabase(YTF,http://www.cs.tau.ac.il/~wolf/ytfaces)3,425videos,1,595differentidentities
Privatefaceimagedatasets:
(Facebook)SocialFaceClassificationdataset4.4millionfacephotos,4,030differentidentities
(Google) 100-200millionfaceimages,~8milliondifferentidentities
LFW YTFFacebookDeepFace 97.4% 91.4%GoogleFaceNet 99.6% 95.1%Humanperformance 97.5% 89.7%
ConvolutionalNeuralNetworks(CNNs)Fei-Fei Li,JustinJohnson,SerenaYeung(http://cs231n.stanford.edu/)
11/11/18
3
CONV:“convolution”layerwithweightsthatarelearnedRELU:“rectifiedlinearunit”appliesanactivationfunctionPOOL:“pooling”selectsmaximumvalueinsmallneighborhoodsFC:“fully-connected”neuralnetwork
SamplestagesofaCNN
Convolutionallayerfully-connectednetwork:
locally-connectednetwork:
• earlylayersperformaconvolution oftheirinputs• multipleconvolutionoperators(e.g.red&black)• weightsinconvolutionoperatorsarelearned• convolutionoperatorsaretypicallysmall(e.g.5x5)
5x5x3operator
convolutionswithmultipleoperators(RGB)
11/11/18
4
ReLU &maxpoolinglayers
sigmoid
convolutionresult
RectifiedLinearActivation
LeCun,Bottou,Bengio,Haffner (1998)
RecognizingdigitsfromtheMNISTdatabasewithaCNN:
Addingafully-connectedneuralnetlayer