Observable Data

Model Information

Observable Data

Model Information

Speech Recognition

I am a boy

Observable Data

Model Information

ImageClassify

Cat

Observable Data

Model Information

HOW?

http://ac.els-cdn.com/S089662731200092X/1-s2.0-S089662731200092X-main.pdf?_tid=403b4f50-accf-11e4-b07c-00000aacb35d&acdnat=1423096866_b7f6fa86eb3fb1f934da4cd66bebb319

• Core visual object recognition

Feedback

• Weakness in kernel machine(SVM …):

• It does not scale well with sample size.

• Based on matching local templates.

• the training data is referenced for test data

• Local representation VS distributed representation

• N N(Neural Network) -> Kernel machine -> Deep NN

• Deep learning is all about deep neural networks

• 1949 : Hebbian learning

• Donald Hebb : the father of neural networks

• 1958 : (single layer) Perceptron

• Frank Rosenblatt

- Marvin Minsky, 1969

• 1986 : Multilayer Perceptron(Back propagation)

• David Rumelhart, Geoffrey Hinton, and Ronald Williams

• 2006 : Deep Neural Networks

• Geoffrey Hinton and Ruslan Salakhutdinov

Hand-CraftedFeatures

TrainableGenericClassifier

F(X;𝜃)

𝜃

SimpleClassifier

Layer1

SimpleClassifier

Layer2

LayerN

Layer1

SimpleClassifier

Layer2

LayerN

Layer1

SimpleClassifier

Layer2

LayerN

Layer1

SimpleClassifier

Layer2

LayerN

Layer1

SimpleClassifier

Layer2

LayerN

Trainable Generic Classifier

Hand-crafted Features

Layer1

SimpleClassifier

Layer2

LayerN

Trainable Generic Classifier

Hand-crafted Features

Layer1

SimpleClassifier

Layer2

LayerN

Shallow learning Deep learning

feature extraction by domain experts(SIFT, SURF, orb...)

automatic feature extraction from data

separate modules(feature extractor + trainable classifier)

unified model : end-to-end learning(trainable feature + trainable classifier)

https://www.cs.toronto.edu/~hinton/absps/fastnc.pdf

http://darkpgmr.tistory.com/133

i j

http://www.cs.toronto.edu/~fritz/absps/reluICML.pdf

http://www.appliedcuriosity.eu/blog/post/22/sigmoidlogistic-function-and-softmax-without-overflow

http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf

http://www.image-net.org/challenges/LSVRC/

http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks

http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks

http://arxiv.org/abs/1312.4400

http://arxiv.org/abs/1312.4400

http://arxiv.org/abs/1312.4400

http://arxiv.org/abs/1312.4400

http://arxiv.org/abs/1312.4400

http://arxiv.org/abs/1312.4400

http://arxiv.org/abs/1501.02876

https://research.facebook.com/publications/480567225376225/deepface-closing-the-gap-to-human-level-performance-in-face-verification/

http://arxiv.org/abs/1312.4659

http://arxiv.org/pdf/1411.2539.pdf

• http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html

• convolutional neural networks (popular): LeCun

• Alex Krizhevsky: Hinton (python, C++)

• https://code.google.com/p/cuda-convnet/

• Caffe: UC Berkeley (C++)

• http://caffe.berkeleyvision.org/