Embed Size (px)
Citation preview
발표자 : 백인성
- 2 -Copyright ⓒ 2019, All rights reserved.
1. Introduction
- 4 -Copyright ⓒ 2019, All rights reserved.
출처: https://www.youtube.com/watch?v=6NN8kXh-TpkHMG Journal, https://news.hmgjournal.com/Tech/deep-learning-carfuture
<자율 주행 차량>→ 도로 위 객체 탐지
<아마존고 자동 재고 파악>→ 선반 내 물건 인식
- 5 -Copyright ⓒ 2019, All rights reserved.
출처: https://alexisbcook.github.io/2017/global-average-pooling-layers-for-object-localization/Cheng, Chi-Tung, et al. "Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs." European radiology (2019): 1-9.
<고관절 골절 탐지>→ 병 원인 진단
<분류 모델 원인 해석>→ 예측 모델 결과에 대한 신뢰성
- 6 -Copyright ⓒ 2019, All rights reserved.
출처: Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
2. Convolutional Neural Network (CNN)
- 8 -Copyright ⓒ 2019, All rights reserved.
[Input Image][Convolutional Layer] [Neural Network]
유해진 : 30%
박지성 : 60%
기성용: 8%
손흥민: 2%
출처: http://www.sisaweek.com/news/articleView.html?idxno=28851
- 9 -Copyright ⓒ 2019, All rights reserved.
[Input Image]
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
1 0 2
0 -1 1
-1 1 2*
11 4 3 0
3 10 6 1
7 9 -1 1
7 11 -1 4
<3x3 Filter> [Output Feature]<Input Data>
- 10 -Copyright ⓒ 2019, All rights reserved.
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
1 0 2
0 -1 1
-1 1 2
*
11 4 3 0
3 10 6 1
7 9 -1 1
7 11 -1 4
<3x3 Filter> [Output Feature]<Input Data>
[사례1]
- 11 -Copyright ⓒ 2019, All rights reserved.
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
1 0 2
0 -1 1
-1 1 2
*
11 4 3 0
3 10 6 1
7 9 -1 1
7 11 -1 4
<3x3 Filter> [Output Feature]<Input Data>
[사례1]
- 12 -Copyright ⓒ 2019, All rights reserved.
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
1 0 2
0 -1 1
-1 1 2
*
11 4 3 0
3 10 6 1
7 9 -1 1
7 11 -1 4
<3x3 Filter> [Output Feature]<Input Data>
[사례2]
- 13 -Copyright ⓒ 2019, All rights reserved.
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
1 0 2
0 -1 1
-1 1 2
*
11 4 3 0
3 10 6 1
7 9 -1 1
7 11 -1 4
<3x3 Filter> [Output Feature]<Input Data>
[사례3]
- 14 -Copyright ⓒ 2019, All rights reserved.
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
2*
<1x1 Filter> [Output Feature]<Input Data>
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
- 15 -Copyright ⓒ 2019, All rights reserved.
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
2*
<1x1 Filter> [Output Feature]<Input Data>
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
- 16 -Copyright ⓒ 2019, All rights reserved.
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
*
<1x1 Filter><Input Data>
2
3
3
[Output Feature]
- 17 -Copyright ⓒ 2019, All rights reserved.
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
*
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
<1x1 Filter> [Output Feature]<Input Data>
2
3
1
3 3
3
- 18 -Copyright ⓒ 2019, All rights reserved.
20 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
*
<1x1 Filter> [Output Feature]<Input Data>
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
1
0
-1
-2
3
3
6
3
- 19 -Copyright ⓒ 2019, All rights reserved.
6
6
32
4
4
64
3
3
32
계산 비용
3x3x32x4x4x64
294,912
<3x3 filter 예시>
1x1x32x4x4x64
32,7686
6
32
4
4
64
1
1
32
<1x1 filter 예시>
- 20 -Copyright ⓒ 2019, All rights reserved.
0 0 4 2 2 0
0 2 2 4 4 2
0 6 4 2 0 0
2 2 2 6 0 0
0 2 2 2 0 4
0 0 4 2 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
0 0 2 1 1 0
0 1 1 2 2 1
0 3 2 1 0 0
1 1 1 3 0 0
0 1 1 1 0 2
0 0 2 1 0 0
*
<1x1 Filter> [Output Feature]<Input Data>
2
3
3
3. Non-local neural networks
- 22 -Copyright ⓒ 2019, All rights reserved.
출처: Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
- 23 -Copyright ⓒ 2019, All rights reserved.
- 24 -Copyright ⓒ 2019, All rights reserved.
출처: Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
- 25 -Copyright ⓒ 2019, All rights reserved.
Input
WC
H
1x1 Conv
W C/2
H
[H x W x C]
1x1 Conv
C/2
H
W
- 26 -Copyright ⓒ 2019, All rights reserved.
Input
WC
H
1x1 Conv
W C/2
H
[H x W x C]
1x1 Conv
C/2
H
W
HW
C/2HW
C/2
- 27 -Copyright ⓒ 2019, All rights reserved.
Input
WC
H
1x1 Conv
W C/2
H
[H x W x C]
1x1 Conv
C/2
H
W
HW
HW
HW
C/2HW
C/2
[HW x C/2]
[C/2 x HW]
[HW x HW]
- 28 -Copyright ⓒ 2019, All rights reserved.
Input
WC
H
1x1 Conv
W C/2
H
[H x W x C]
1x1 Conv
WC/2
H
1x1 Conv
C/2
H
W
HW
HW
C/2HW
C/2HW
C/2
HW
- 29 -Copyright ⓒ 2019, All rights reserved.
Input
WC
H
1x1 Conv
W C/2
H
[H x W x C]
1x1 Conv
WC/2
H
1x1 Conv
C/2
H
W
HW
HW
softmax
HW
C/2
C/2
HW
HW
C/2HW
C/2
- 30 -Copyright ⓒ 2019, All rights reserved.
Input
WC
H
1x1 Conv
Output
W C
H
W C/2
H
1x1 Conv
[H x W x C]
1x1 Conv
WC/2
H
1x1 Conv
C/2
H
W
W C/2
HHW
HW
softmax
HW
C/2
C/2
W
H
C[H x W x C]
HW
HW
C/2HW
C/2
Convolution 결과
Reshape 결과
행렬 연산 결과
- 31 -Copyright ⓒ 2019, All rights reserved.
Input
WC
H
1x1 Conv
Output
W C
H
W C/2
H
1x1 Conv
[H x W x C]
1x1 Conv
WC/2
H
1x1 Conv
C/2
H
W
W C/2
HHW
HW
softmax
HW
C/2
C/2
W
H
C[H x W x C]
HW
Residual연산
𝑧𝑖 = 𝑊𝑧𝑦𝑖 + 𝒙𝒊HW
C/2HW
C/2
Convolution 결과
Reshape 결과
행렬 연산 결과
- 32 -Copyright ⓒ 2019, All rights reserved.
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝜃 𝑥𝑖 = 𝑊𝜃𝑥𝑖 𝜙 𝑥𝑗 = 𝑊𝜙𝑥𝑗
𝑊𝜃𝑥𝑖
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
-Embedded Gaussian-
𝑧𝑖 = 𝑊𝑧𝑦𝑖 + 𝑥𝑖
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 33 -Copyright ⓒ 2019, All rights reserved.
𝜃 𝑥𝑖 = 𝑊𝜃𝑥𝑖 𝜙 𝑥𝑗 = 𝑊𝜙𝑥𝑗
𝑊𝜙𝑥𝑗
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
-Embedded Gaussian-
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑧𝑖 = 𝑊𝑧𝑦𝑖 + 𝑥𝑖
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 34 -Copyright ⓒ 2019, All rights reserved.
𝜃 𝑥𝑖 = 𝑊𝜃𝑥𝑖 𝜙 𝑥𝑗 = 𝑊𝜙𝑥𝑗
𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
-Embedded Gaussian-
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑧𝑖 = 𝑊𝑧𝑦𝑖 + 𝑥𝑖
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 35 -Copyright ⓒ 2019, All rights reserved.
𝜃 𝑥𝑖 = 𝑊𝜃𝑥𝑖 𝜙 𝑥𝑗 = 𝑊𝜙𝑥𝑗
𝑊𝑔𝑥𝑗
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
-Embedded Gaussian-
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑧𝑖 = 𝑊𝑧𝑦𝑖 + 𝑥𝑖
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 36 -Copyright ⓒ 2019, All rights reserved.
𝜃 𝑥𝑖 = 𝑊𝜃𝑥𝑖 𝜙 𝑥𝑗 = 𝑊𝜙𝑥𝑗
SoftMax function
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
-Embedded Gaussian-
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑧𝑖 = 𝑊𝑧𝑦𝑖 + 𝑥𝑖
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 37 -Copyright ⓒ 2019, All rights reserved.
𝜃 𝑥𝑖 = 𝑊𝜃𝑥𝑖 𝜙 𝑥𝑗 = 𝑊𝜙𝑥𝑗
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑦𝑖
SoftMax function
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
-Embedded Gaussian-
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 38 -Copyright ⓒ 2019, All rights reserved.
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒(𝑊𝜃𝑥𝑖)𝑇𝑊𝜙𝑥𝑗
-Embedded Gaussian-
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑦𝑖
SoftMax function
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
- 39 -Copyright ⓒ 2019, All rights reserved.
𝑦𝑖 =1
𝐶(𝑥)𝑒(𝑊𝜃𝑥𝑖)
𝑇𝑊𝜙𝑥𝑗𝑊𝑔𝑥𝑗 𝑌 = 𝑠𝑜𝑓𝑡𝑚𝑎𝑥(𝑋𝑇𝑊𝜃𝑇𝑊𝜙𝑋)𝑊𝑔𝑋
출처: 강현규”Visual Attention”-DMQA세미나, http://jalammar.github.io
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 40 -Copyright ⓒ 2019, All rights reserved.
𝑌 = 𝑠𝑜𝑓𝑡𝑚𝑎𝑥(𝑿𝑻𝑾𝜽𝑻𝑾𝝓𝑿)𝑾𝒈𝑿
𝐴 𝑄,𝐾, 𝑉 = 𝑠𝑜𝑓𝑡𝑚𝑎𝑥𝑸𝑲𝑻
𝑑𝑘𝑽Self
Attention
출처: 강현규”Visual Attention”-DMQA세미나, http://jalammar.github.io
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑦𝑖 =1
𝐶(𝑥)𝑒(𝑾𝜽𝒙𝒊)
𝑻𝑾𝝓𝒙𝒋𝑾𝒈𝒙𝒋
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 41 -Copyright ⓒ 2019, All rights reserved.
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝜃 𝑥𝑖 = 𝑊𝜃𝑥𝑖 𝜙 𝑥𝑗 = 𝑊𝜙𝑥𝑗
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
-Embedded Gaussian-
𝑊𝑧𝑦𝑖
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑦𝑖𝑊𝑧
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑧𝑖 = 𝑊𝑧𝑦𝑖 + 𝑥𝑖
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 42 -Copyright ⓒ 2019, All rights reserved.
𝑓 𝑥𝑖 , 𝑥𝑗 = 𝑒𝜃(𝑥𝑖)𝑇𝜙(𝑥𝑗)
𝜃 𝑥𝑖 = 𝑊𝜃𝑥𝑖 𝜙 𝑥𝑗 = 𝑊𝜙𝑥𝑗
𝑔 𝑥𝑗 = 𝑊𝑔𝑥𝑗
-Embedded Gaussian-
𝑧𝑖-Residual 연산-
𝑊 → 1 × 1 𝐶𝑜𝑛𝑣𝑙𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝐹𝑖𝑙𝑡𝑒𝑟
𝑥𝑖 , 𝑥𝑗 → 𝐼𝑛𝑝𝑢𝑡 𝑑𝑎𝑡𝑎
𝑦𝑖 =1
𝐶(𝑥)𝑓 𝑥𝑖 , 𝑥𝑗 𝑔(𝑥𝑗)
𝑧𝑖 = 𝑊𝑧𝑦𝑖 + 𝑥𝑖
𝑠𝑒𝑡 𝐶 𝑥 =
∀𝑗
𝑓 𝑥𝑖 , 𝑥𝑗
- 43 -Copyright ⓒ 2019, All rights reserved.
- 44 -Copyright ⓒ 2019, All rights reserved.
출처: Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
- 45 -Copyright ⓒ 2019, All rights reserved.
출처: Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
- 46 -Copyright ⓒ 2019, All rights reserved.
출처: Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
- 47 -Copyright ⓒ 2019, All rights reserved.
출처: Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
- 48 -Copyright ⓒ 2019, All rights reserved.
StarCraft IIReplay
승리
패배
①주요시점추출
②주요상황추출
<승패예측모델구축>
- 49 -Copyright ⓒ 2019, All rights reserved.
Non-local block
- 50 -Copyright ⓒ 2019, All rights reserved.
Model Non-local위치학습정확도(5회평균)
학습정확도(최대)
Epoch
3D-Resnet X 88.6%(±0.001)
88.8% 23
Non-local neural networks(Batch Norm 사용)
마지막 88.4%(±0.005)
89.0% 43
Non-local neural networks(Batch Norm 제외)
마지막 89.1%(±0.004)
89.6% 20
Non-local neural networks(Batch Norm 제외)
중간과마지막
82.7%(±0.007)
83.4% 17
- 51 -Copyright ⓒ 2019, All rights reserved.
감사합니다.
➢ Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision
and pattern recognition. 2018.
➢ https://www.youtube.com/watch?v=6NN8kXh-Tpk
➢ HMG Journal, https://news.hmgjournal.com/Tech/deep-learning-carfuture
➢ https://alexisbcook.github.io/2017/global-average-pooling-layers-for-object-localization/
➢ Cheng, Chi-Tung, et al. "Application of a deep learning algorithm for detection and visualization of hip fractures
on plain pelvic radiographs." European radiology (2019): 1-9.
➢ https://hwiyong.tistory.com/45
➢ https://blog.lunit.io/2018/01/19/non-local-neural-networks/
➢ https://www.youtube.com/watch?v=Dvi5_YC8Yts
➢ https://www.youtube.com/watch?v=vcp0XvDAX68 – Andrew Ng
➢ https://www.youtube.com/watch?v=C86ZXvgpejM – Andrew Ng
Appendix
- 55 -Copyright ⓒ 2019, All rights reserved.
HW
HW
HW
C/2HW
C/2
[HW x C/2]
[C/2 x HW]
[HW x HW]
1 0 1 1
0 2 0 2
1 0 1 1
2 0 1 1
2 0 1 0
1 1 0 1
0 1 1 1 2 1
0 1 0 2 0 1
1 0 1 0 1 0
0 0 1 1 0 1
1 1 3 2 3 2
0 2 2 6 0 4
1 1 3 2 3 2
1 2 4 3 5 3
1 2 3 2 5 2
0 2 2 4 2 3
C1 C2 C3 C4
C1
C2
C3
C4
- 56 -Copyright ⓒ 2019, All rights reserved.
출처: Wang, Xiaolong, et al. "Non-local neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
- 57 -Copyright ⓒ 2019, All rights reserved.
1 0 2 3
1 0 0 0
0 1 2 0
0 0 2 1
1 0 2 3
1 0 0 0
0 1 2 0
0 0 2 1
2
1 0
0 -1
*
*
1 0 2
0 -2 0
0 -1 1
2 0 4 6
2 0 0 0
0 2 4 0
0 0 4 2
Feature 내모든 값에대한 가중치
고려
- 58 -Copyright ⓒ 2019, All rights reserved.
❖ Non-local neural networks (NLNN)
출처: https://www.youtube.com/watch?v=Dvi5_YC8Yts
- 59 -Copyright ⓒ 2019, All rights reserved.
Input
WC
H
1x1 Conv
Output
W C
H
W C/2
H
1x1 Conv
[H x W x C]
1x1 Conv
WC/2
H
1x1 Conv
C/2
H
W
W C/2
H
HW
C/2
HW
C/2
HW
HW
softmax
HW
C/2
C/2
W
H
C
- 60 -Copyright ⓒ 2019, All rights reserved.
- 61 -Copyright ⓒ 2019, All rights reserved.
<원본 미니맵> <Attention>
