Embed Size (px)
Citation preview
2
OverviewPart1: TensorFlow Tutorials
Handling imagesLogistic regressionMulti-layer perceptron
Part2: Advances in convolutional neural networksCNN basics
Four CNN architectures (AlexNet, VGG, GoogLeNet, ResNet)Application1: Semantic segmentationApplication2: Object detection
Convolutional neural network
3
CNNConvolutional Neural Network
CNNs are basically layers of convolutions followed by subsampling and dense layers.
Intuitively speaking, convolutions and subsampling lay-ers works as feature extraction layers while a dense layer classifies which category current input belongs to using ex-tracted features.
4
To understand CNN,
Zero-paddingStride Channel
5
Convolution
http://deeplearning.stanford.edu/wiki/index.php/Feature_extraction_using_convolution
6
Zero-padding
What is the size of the input?
What is the size of the output?
What is the size of the filter?
What is the size of the zero-padding?
7
Stride
(Left) Stride size: 1
(Right) Stride size: 2
If stride size equals the filter size, there will be no overlapping.
8
9
10
11
12
13
14
15
CNN ArchitecturesAlexNet VGG
GoogLeNet ResNet
16
Top-5 Classification Error
AlexNet
18
AlexNet
19
ReLURectified Linear Unit
tanhReLU
Faster Convergence!
VGG
21
VGG?
GoogLeNet
23
GoogLeNet
24
GoogLeNet
22 Layers Deep Network
Efficiently utilized computing resources, “Inception Module”
Significantly outperforms previous methods on ILSVRC 2014
25
Inception module
26
One by one convolution
27
One by one convolution
28
One by one convolution
29
GoogLeNet
Network in Network!
ResNet
31
Deep residual networks
152 layers network
1st place on ILSVRM 2015 classification task
1st place on ImageNet detection
1st place on ImageNet localization
1st place on COCO detection
1st place on COCO segmentation
32
Degeneration problemCiFAR 100 Dataset
ImageNet
33
Residual learning building block
34
Residual mappingBasic residual mapping (same dim.)
Basic residual mapping (different dim.)
“But we will show by experiments that the identity mapping is sufficient for address-ing the degradation problem and is eco-nomical, and thus W is only used when match-ing dimensions.”
35
Deeper bottle architecture
Dimension reduction
Convolution
Dimension increasement
36
Experimental results
37
Experimental results
