第60回 日本植物生理学会年会 データベース講習会 (3.14.2019@名古屋大)

Wet研究者の視点で始める深層学習～画像診断と判断要因の可視化

赤木 剛士 (Takashi AKAGI)

深層学習は「暗黙知・経験知」を反映可能

Neural Style Transfer (Gatys et al. 2016): CNNによる「特徴抽出」アートツール

＋

Flyer作りに最適！！

＋

赤木 作 2017 京都大・浙江大 農学部 joint symposiumポスター

Outline

1. Wet研究視点の深層学習基礎：座学

2. VGG16の転移学習による画像診断と診断理由の可視化：実践編

深層学習の概要・基本構造

https://leapmind.io/blog/2017/06/16/

https://www.upwork.com/hiring/for-clients/log-analytics-deep-

learning-machine-learning/

人工知能 (AI)

機械学習

ニューラルネットワーク

深層学習

人工ニューラルネットワークを「多層化」したものが深層学習

（個人的感覚で）シロウトでも扱えそうな深層学習の代表格

畳み込みニューラルネットワーク (CNN) による画像解析

〇畳み込みモデルの特徴（位置普遍性・構成性）が画像解析に向いている➡ 必ずしも画像だけではない。〇比較的扱いやすいpackageが多い (な気がする)。GUIもある（後述）

再帰型ニューラルネットワーク (RNN) (またはLSTM)

〇フィードバック系 (前を記憶) を有しており、時系列解析に向いている〇自然言語の解析によく用いられる（ということは塩基配列にも？）

https://jp.mathworks.com/solutions/deep-

learning/convolutional-neural-network.html

LuCun et al. 2015, Nature

0

5

10

15

20

25

30

2010 2011 2012 2013 2014 2015 human

Top

-5 c

lassific

ation e

rror

ILSVRC winner models

Deep learning

AlexNet

8 lay.

8 lay.GoogleNet

22 lay. ResNet

152 lay.

CNNの精度：ImageNet large scale visual recognition challengeの推移

Model Pub. Year Nos Lays. Derivative

AlexNet 2012 8

VGG16 2014 16 VGG19

GoogLeNet 2014 22 Inception

ResNet 2015 152 ResNet50

数字認識（MNIST）においてもヒトの感覚を超える

ヒトは分かるDLは苦手

DLは分かるヒトは苦手

モデル構造：AlexNet, VGG16, InceptionV3, ResNet50

__________________________________________________________________________________________________

Layer (type) Output Shape Param # Connected to

==================================================================================================

input_1 (InputLayer) (None, 224, 224, 3) 0

__________________________________________________________________________________________________

conv1_pad (ZeroPadding2D) (None, 230, 230, 3) 0 input_1[0][0]

__________________________________________________________________________________________________

conv1 (Conv2D) (None, 112, 112, 64) 9472 conv1_pad[0][0]

__________________________________________________________________________________________________

bn_conv1 (BatchNormalization) (None, 112, 112, 64) 256 conv1[0][0]

__________________________________________________________________________________________________

activation_1 (Activation) (None, 112, 112, 64) 0 bn_conv1[0][0]

__________________________________________________________________________________________________

max_pooling2d_1 (MaxPooling2D) (None, 55, 55, 64) 0 activation_1[0][0]

__________________________________________________________________________________________________

res2a_branch2a (Conv2D) (None, 55, 55, 64) 4160 max_pooling2d_1[0][0]

__________________________________________________________________________________________________

bn2a_branch2a (BatchNormalizati (None, 55, 55, 64) 256 res2a_branch2a[0][0]

__________________________________________________________________________________________________

activation_2 (Activation) (None, 55, 55, 64) 0 bn2a_branch2a[0][0]

__________________________________________________________________________________________________

res2a_branch2b (Conv2D) (None, 55, 55, 64) 36928 activation_2[0][0]

__________________________________________________________________________________________________

bn2a_branch2b (BatchNormalizati (None, 55, 55, 64) 256 res2a_branch2b[0][0]

__________________________________________________________________________________________________

activation_3 (Activation) (None, 55, 55, 64) 0 bn2a_branch2b[0][0]

__________________________________________________________________________________________________

res2a_branch2c (Conv2D) (None, 55, 55, 256) 16640 activation_3[0][0]

__________________________________________________________________________________________________

res2a_branch1 (Conv2D) (None, 55, 55, 256) 16640 max_pooling2d_1[0][0]

__________________________________________________________________________________________________

bn2a_branch2c (BatchNormalizati (None, 55, 55, 256) 1024 res2a_branch2c[0][0]

__________________________________________________________________________________________________

bn2a_branch1 (BatchNormalizatio (None, 55, 55, 256) 1024 res2a_branch1[0][0]

__________________________________________________________________________________________________

add_1 (Add) (None, 55, 55, 256) 0 bn2a_branch2c[0][0]

bn2a_branch1[0][0]

__________________________________________________________________________________________________

activation_4 (Activation) (None, 55, 55, 256) 0 add_1[0][0]

__________________________________________________________________________________________________

res2b_branch2a (Conv2D) (None, 55, 55, 64) 16448 activation_4[0][0]

__________________________________________________________________________________________________

bn2b_branch2a (BatchNormalizati (None, 55, 55, 64) 256 res2b_branch2a[0][0]

__________________________________________________________________________________________________

activation_5 (Activation) (None, 55, 55, 64) 0 bn2b_branch2a[0][0]

__________________________________________________________________________________________________

res2b_branch2b (Conv2D) (None, 55, 55, 64) 36928 activation_5[0][0]

__________________________________________________________________________________________________

bn2b_branch2b (BatchNormalizati (None, 55, 55, 64) 256 res2b_branch2b[0][0]

__________________________________________________________________________________________________

activation_6 (Activation) (None, 55, 55, 64) 0 bn2b_branch2b[0][0]

__________________________________________________________________________________________________

res2b_branch2c (Conv2D) (None, 55, 55, 256) 16640 activation_6[0][0]

__________________________________________________________________________________________________

bn2b_branch2c (BatchNormalizati (None, 55, 55, 256) 1024 res2b_branch2c[0][0]

__________________________________________________________________________________________________

add_2 (Add) (None, 55, 55, 256) 0 bn2b_branch2c[0][0]

activation_4[0][0]

__________________________________________________________________________________________________

activation_7 (Activation) (None, 55, 55, 256) 0 add_2[0][0]

__________________________________________________________________________________________________

res2c_branch2a (Conv2D) (None, 55, 55, 64) 16448 activation_7[0][0]

__________________________________________________________________________________________________

bn2c_branch2a (BatchNormalizati (None, 55, 55, 64) 256 res2c_branch2a[0][0]

__________________________________________________________________________________________________

activation_8 (Activation) (None, 55, 55, 64) 0 bn2c_branch2a[0][0]

__________________________________________________________________________________________________

res2c_branch2b (Conv2D) (None, 55, 55, 64) 36928 activation_8[0][0]

__________________________________________________________________________________________________

bn2c_branch2b (BatchNormalizati (None, 55, 55, 64) 256 res2c_branch2b[0][0]

__________________________________________________________________________________________________

activation_9 (Activation) (None, 55, 55, 64) 0 bn2c_branch2b[0][0]

__________________________________________________________________________________________________

res2c_branch2c (Conv2D) (None, 55, 55, 256) 16640 activation_9[0][0]

__________________________________________________________________________________________________

bn2c_branch2c (BatchNormalizati (None, 55, 55, 256) 1024 res2c_branch2c[0][0]

__________________________________________________________________________________________________

add_3 (Add) (None, 55, 55, 256) 0 bn2c_branch2c[0][0]

activation_7[0][0]

__________________________________________________________________________________________________

activation_10 (Activation) (None, 55, 55, 256) 0 add_3[0][0]

__________________________________________________________________________________________________

res3a_branch2a (Conv2D) (None, 28, 28, 128) 32896 activation_10[0][0]

__________________________________________________________________________________________________

bn3a_branch2a (BatchNormalizati (None, 28, 28, 128) 512 res3a_branch2a[0][0]

__________________________________________________________________________________________________

activation_11 (Activation) (None, 28, 28, 128) 0 bn3a_branch2a[0][0]

__________________________________________________________________________________________________

res3a_branch2b (Conv2D) (None, 28, 28, 128) 147584 activation_11[0][0]

__________________________________________________________________________________________________

bn3a_branch2b (BatchNormalizati (None, 28, 28, 128) 512 res3a_branch2b[0][0]

__________________________________________________________________________________________________

activation_12 (Activation) (None, 28, 28, 128) 0 bn3a_branch2b[0][0]

__________________________________________________________________________________________________

res3a_branch2c (Conv2D) (None, 28, 28, 512) 66048 activation_12[0][0]

__________________________________________________________________________________________________

res3a_branch1 (Conv2D) (None, 28, 28, 512) 131584 activation_10[0][0]

__________________________________________________________________________________________________

bn3a_branch2c (BatchNormalizati (None, 28, 28, 512) 2048 res3a_branch2c[0][0]

__________________________________________________________________________________________________

bn3a_branch1 (BatchNormalizatio (None, 28, 28, 512) 2048 res3a_branch1[0][0]

__________________________________________________________________________________________________

add_4 (Add) (None, 28, 28, 512) 0 bn3a_branch2c[0][0]

bn3a_branch1[0][0]

__________________________________________________________________________________________________

activation_13 (Activation) (None, 28, 28, 512) 0 add_4[0][0]

__________________________________________________________________________________________________

res3b_branch2a (Conv2D) (None, 28, 28, 128) 65664 activation_13[0][0]

__________________________________________________________________________________________________

bn3b_branch2a (BatchNormalizati (None, 28, 28, 128) 512 res3b_branch2a[0][0]

__________________________________________________________________________________________________

activation_14 (Activation) (None, 28, 28, 128) 0 bn3b_branch2a[0][0]

__________________________________________________________________________________________________

res3b_branch2b (Conv2D) (None, 28, 28, 128) 147584 activation_14[0][0]

__________________________________________________________________________________________________

bn3b_branch2b (BatchNormalizati (None, 28, 28, 128) 512 res3b_branch2b[0][0]

__________________________________________________________________________________________________

activation_15 (Activation) (None, 28, 28, 128) 0 bn3b_branch2b[0][0]

__________________________________________________________________________________________________

res3b_branch2c (Conv2D) (None, 28, 28, 512) 66048 activation_15[0][0]

__________________________________________________________________________________________________

bn3b_branch2c (BatchNormalizati (None, 28, 28, 512) 2048 res3b_branch2c[0][0]

__________________________________________________________________________________________________

add_5 (Add) (None, 28, 28, 512) 0 bn3b_branch2c[0][0]

activation_13[0][0]

__________________________________________________________________________________________________

activation_16 (Activation) (None, 28, 28, 512) 0 add_5[0][0]

__________________________________________________________________________________________________

res3c_branch2a (Conv2D) (None, 28, 28, 128) 65664 activation_16[0][0]

__________________________________________________________________________________________________

bn3c_branch2a (BatchNormalizati (None, 28, 28, 128) 512 res3c_branch2a[0][0]

__________________________________________________________________________________________________

activation_17 (Activation) (None, 28, 28, 128) 0 bn3c_branch2a[0][0]

__________________________________________________________________________________________________

res3c_branch2b (Conv2D) (None, 28, 28, 128) 147584 activation_17[0][0]

__________________________________________________________________________________________________

bn3c_branch2b (BatchNormalizati (None, 28, 28, 128) 512 res3c_branch2b[0][0]

__________________________________________________________________________________________________

activation_18 (Activation) (None, 28, 28, 128) 0 bn3c_branch2b[0][0]

__________________________________________________________________________________________________

res3c_branch2c (Conv2D) (None, 28, 28, 512) 66048 activation_18[0][0]

__________________________________________________________________________________________________

bn3c_branch2c (BatchNormalizati (None, 28, 28, 512) 2048 res3c_branch2c[0][0]

__________________________________________________________________________________________________

add_6 (Add) (None, 28, 28, 512) 0 bn3c_branch2c[0][0]

activation_16[0][0]

__________________________________________________________________________________________________

activation_19 (Activation) (None, 28, 28, 512) 0 add_6[0][0]

__________________________________________________________________________________________________

res3d_branch2a (Conv2D) (None, 28, 28, 128) 65664 activation_19[0][0]

__________________________________________________________________________________________________

bn3d_branch2a (BatchNormalizati (None, 28, 28, 128) 512 res3d_branch2a[0][0]

__________________________________________________________________________________________________

activation_20 (Activation) (None, 28, 28, 128) 0 bn3d_branch2a[0][0]

__________________________________________________________________________________________________

res3d_branch2b (Conv2D) (None, 28, 28, 128) 147584 activation_20[0][0]

__________________________________________________________________________________________________

bn3d_branch2b (BatchNormalizati (None, 28, 28, 128) 512 res3d_branch2b[0][0]

__________________________________________________________________________________________________

activation_21 (Activation) (None, 28, 28, 128) 0 bn3d_branch2b[0][0]

__________________________________________________________________________________________________

res3d_branch2c (Conv2D) (None, 28, 28, 512) 66048 activation_21[0][0]

__________________________________________________________________________________________________

bn3d_branch2c (BatchNormalizati (None, 28, 28, 512) 2048 res3d_branch2c[0][0]

__________________________________________________________________________________________________

add_7 (Add) (None, 28, 28, 512) 0 bn3d_branch2c[0][0]

activation_19[0][0]

__________________________________________________________________________________________________

activation_22 (Activation) (None, 28, 28, 512) 0 add_7[0][0]

__________________________________________________________________________________________________

res4a_branch2a (Conv2D) (None, 14, 14, 256) 131328 activation_22[0][0]

__________________________________________________________________________________________________

bn4a_branch2a (BatchNormalizati (None, 14, 14, 256) 1024 res4a_branch2a[0][0]

__________________________________________________________________________________________________

activation_23 (Activation) (None, 14, 14, 256) 0 bn4a_branch2a[0][0]

__________________________________________________________________________________________________

res4a_branch2b (Conv2D) (None, 14, 14, 256) 590080 activation_23[0][0]

__________________________________________________________________________________________________

bn4a_branch2b (BatchNormalizati (None, 14, 14, 256) 1024 res4a_branch2b[0][0]

__________________________________________________________________________________________________

activation_24 (Activation) (None, 14, 14, 256) 0 bn4a_branch2b[0][0]

__________________________________________________________________________________________________

res4a_branch2c (Conv2D) (None, 14, 14, 1024) 263168 activation_24[0][0]

__________________________________________________________________________________________________

res4a_branch1 (Conv2D) (None, 14, 14, 1024) 525312 activation_22[0][0]

__________________________________________________________________________________________________

bn4a_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096 res4a_branch2c[0][0]

__________________________________________________________________________________________________

bn4a_branch1 (BatchNormalizatio (None, 14, 14, 1024) 4096 res4a_branch1[0][0]

__________________________________________________________________________________________________

add_8 (Add) (None, 14, 14, 1024) 0 bn4a_branch2c[0][0]

bn4a_branch1[0][0]

__________________________________________________________________________________________________

activation_25 (Activation) (None, 14, 14, 1024) 0 add_8[0][0]

__________________________________________________________________________________________________

res4b_branch2a (Conv2D) (None, 14, 14, 256) 262400 activation_25[0][0]

__________________________________________________________________________________________________

bn4b_branch2a (BatchNormalizati (None, 14, 14, 256) 1024 res4b_branch2a[0][0]

__________________________________________________________________________________________________

activation_26 (Activation) (None, 14, 14, 256) 0 bn4b_branch2a[0][0]

__________________________________________________________________________________________________

res4b_branch2b (Conv2D) (None, 14, 14, 256) 590080 activation_26[0][0]

__________________________________________________________________________________________________

bn4b_branch2b (BatchNormalizati (None, 14, 14, 256) 1024 res4b_branch2b[0][0]

__________________________________________________________________________________________________

activation_27 (Activation) (None, 14, 14, 256) 0 bn4b_branch2b[0][0]

__________________________________________________________________________________________________

res4b_branch2c (Conv2D) (None, 14, 14, 1024) 263168 activation_27[0][0]

__________________________________________________________________________________________________

bn4b_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096 res4b_branch2c[0][0]

__________________________________________________________________________________________________

add_9 (Add) (None, 14, 14, 1024) 0 bn4b_branch2c[0][0]

activation_25[0][0]

__________________________________________________________________________________________________

activation_28 (Activation) (None, 14, 14, 1024) 0 add_9[0][0]

__________________________________________________________________________________________________

res4c_branch2a (Conv2D) (None, 14, 14, 256) 262400 activation_28[0][0]

__________________________________________________________________________________________________

bn4c_branch2a (BatchNormalizati (None, 14, 14, 256) 1024 res4c_branch2a[0][0]

__________________________________________________________________________________________________

activation_29 (Activation) (None, 14, 14, 256) 0 bn4c_branch2a[0][0]

__________________________________________________________________________________________________

res4c_branch2b (Conv2D) (None, 14, 14, 256) 590080 activation_29[0][0]

__________________________________________________________________________________________________

bn4c_branch2b (BatchNormalizati (None, 14, 14, 256) 1024 res4c_branch2b[0][0]

__________________________________________________________________________________________________

activation_30 (Activation) (None, 14, 14, 256) 0 bn4c_branch2b[0][0]

__________________________________________________________________________________________________

res4c_branch2c (Conv2D) (None, 14, 14, 1024) 263168 activation_30[0][0]

__________________________________________________________________________________________________

bn4c_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096 res4c_branch2c[0][0]

__________________________________________________________________________________________________

add_10 (Add) (None, 14, 14, 1024) 0 bn4c_branch2c[0][0]

activation_28[0][0]

__________________________________________________________________________________________________

activation_31 (Activation) (None, 14, 14, 1024) 0 add_10[0][0]

__________________________________________________________________________________________________

res4d_branch2a (Conv2D) (None, 14, 14, 256) 262400 activation_31[0][0]

__________________________________________________________________________________________________

bn4d_branch2a (BatchNormalizati (None, 14, 14, 256) 1024 res4d_branch2a[0][0]

__________________________________________________________________________________________________

activation_32 (Activation) (None, 14, 14, 256) 0 bn4d_branch2a[0][0]

__________________________________________________________________________________________________

res4d_branch2b (Conv2D) (None, 14, 14, 256) 590080 activation_32[0][0]

__________________________________________________________________________________________________

bn4d_branch2b (BatchNormalizati (None, 14, 14, 256) 1024 res4d_branch2b[0][0]

__________________________________________________________________________________________________

activation_33 (Activation) (None, 14, 14, 256) 0 bn4d_branch2b[0][0]

__________________________________________________________________________________________________

res4d_branch2c (Conv2D) (None, 14, 14, 1024) 263168 activation_33[0][0]

__________________________________________________________________________________________________

bn4d_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096 res4d_branch2c[0][0]

__________________________________________________________________________________________________

add_11 (Add) (None, 14, 14, 1024) 0 bn4d_branch2c[0][0]

activation_31[0][0]

__________________________________________________________________________________________________

activation_34 (Activation) (None, 14, 14, 1024) 0 add_11[0][0]

__________________________________________________________________________________________________

res4e_branch2a (Conv2D) (None, 14, 14, 256) 262400 activation_34[0][0]

__________________________________________________________________________________________________

bn4e_branch2a (BatchNormalizati (None, 14, 14, 256) 1024 res4e_branch2a[0][0]

__________________________________________________________________________________________________

activation_35 (Activation) (None, 14, 14, 256) 0 bn4e_branch2a[0][0]

__________________________________________________________________________________________________

res4e_branch2b (Conv2D) (None, 14, 14, 256) 590080 activation_35[0][0]

__________________________________________________________________________________________________

bn4e_branch2b (BatchNormalizati (None, 14, 14, 256) 1024 res4e_branch2b[0][0]

__________________________________________________________________________________________________

activation_36 (Activation) (None, 14, 14, 256) 0 bn4e_branch2b[0][0]

__________________________________________________________________________________________________

res4e_branch2c (Conv2D) (None, 14, 14, 1024) 263168 activation_36[0][0]

__________________________________________________________________________________________________

bn4e_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096 res4e_branch2c[0][0]

__________________________________________________________________________________________________

add_12 (Add) (None, 14, 14, 1024) 0 bn4e_branch2c[0][0]

activation_34[0][0]

__________________________________________________________________________________________________

activation_37 (Activation) (None, 14, 14, 1024) 0 add_12[0][0]

__________________________________________________________________________________________________

res4f_branch2a (Conv2D) (None, 14, 14, 256) 262400 activation_37[0][0]

__________________________________________________________________________________________________

bn4f_branch2a (BatchNormalizati (None, 14, 14, 256) 1024 res4f_branch2a[0][0]

__________________________________________________________________________________________________

activation_38 (Activation) (None, 14, 14, 256) 0 bn4f_branch2a[0][0]

__________________________________________________________________________________________________

res4f_branch2b (Conv2D) (None, 14, 14, 256) 590080 activation_38[0][0]

__________________________________________________________________________________________________

bn4f_branch2b (BatchNormalizati (None, 14, 14, 256) 1024 res4f_branch2b[0][0]

__________________________________________________________________________________________________

activation_39 (Activation) (None, 14, 14, 256) 0 bn4f_branch2b[0][0]

__________________________________________________________________________________________________

res4f_branch2c (Conv2D) (None, 14, 14, 1024) 263168 activation_39[0][0]

__________________________________________________________________________________________________

bn4f_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096 res4f_branch2c[0][0]

__________________________________________________________________________________________________

add_13 (Add) (None, 14, 14, 1024) 0 bn4f_branch2c[0][0]

activation_37[0][0]

__________________________________________________________________________________________________

activation_40 (Activation) (None, 14, 14, 1024) 0 add_13[0][0]

__________________________________________________________________________________________________

res5a_branch2a (Conv2D) (None, 7, 7, 512) 524800 activation_40[0][0]

__________________________________________________________________________________________________

bn5a_branch2a (BatchNormalizati (None, 7, 7, 512) 2048 res5a_branch2a[0][0]

__________________________________________________________________________________________________

activation_41 (Activation) (None, 7, 7, 512) 0 bn5a_branch2a[0][0]

__________________________________________________________________________________________________

res5a_branch2b (Conv2D) (None, 7, 7, 512) 2359808 activation_41[0][0]

__________________________________________________________________________________________________

bn5a_branch2b (BatchNormalizati (None, 7, 7, 512) 2048 res5a_branch2b[0][0]

__________________________________________________________________________________________________

activation_42 (Activation) (None, 7, 7, 512) 0 bn5a_branch2b[0][0]

__________________________________________________________________________________________________

res5a_branch2c (Conv2D) (None, 7, 7, 2048) 1050624 activation_42[0][0]

__________________________________________________________________________________________________

res5a_branch1 (Conv2D) (None, 7, 7, 2048) 2099200 activation_40[0][0]

__________________________________________________________________________________________________

bn5a_branch2c (BatchNormalizati (None, 7, 7, 2048) 8192 res5a_branch2c[0][0]

__________________________________________________________________________________________________

bn5a_branch1 (BatchNormalizatio (None, 7, 7, 2048) 8192 res5a_branch1[0][0]

__________________________________________________________________________________________________

add_14 (Add) (None, 7, 7, 2048) 0 bn5a_branch2c[0][0]

bn5a_branch1[0][0]

__________________________________________________________________________________________________

activation_43 (Activation) (None, 7, 7, 2048) 0 add_14[0][0]

__________________________________________________________________________________________________

res5b_branch2a (Conv2D) (None, 7, 7, 512) 1049088 activation_43[0][0]

__________________________________________________________________________________________________

bn5b_branch2a (BatchNormalizati (None, 7, 7, 512) 2048 res5b_branch2a[0][0]

__________________________________________________________________________________________________

activation_44 (Activation) (None, 7, 7, 512) 0 bn5b_branch2a[0][0]

__________________________________________________________________________________________________

res5b_branch2b (Conv2D) (None, 7, 7, 512) 2359808 activation_44[0][0]

__________________________________________________________________________________________________

bn5b_branch2b (BatchNormalizati (None, 7, 7, 512) 2048 res5b_branch2b[0][0]

__________________________________________________________________________________________________

activation_45 (Activation) (None, 7, 7, 512) 0 bn5b_branch2b[0][0]

__________________________________________________________________________________________________

res5b_branch2c (Conv2D) (None, 7, 7, 2048) 1050624 activation_45[0][0]

__________________________________________________________________________________________________

bn5b_branch2c (BatchNormalizati (None, 7, 7, 2048) 8192 res5b_branch2c[0][0]

__________________________________________________________________________________________________

add_15 (Add) (None, 7, 7, 2048) 0 bn5b_branch2c[0][0]

activation_43[0][0]

__________________________________________________________________________________________________

activation_46 (Activation) (None, 7, 7, 2048) 0 add_15[0][0]

__________________________________________________________________________________________________

res5c_branch2a (Conv2D) (None, 7, 7, 512) 1049088 activation_46[0][0]

__________________________________________________________________________________________________

bn5c_branch2a (BatchNormalizati (None, 7, 7, 512) 2048 res5c_branch2a[0][0]

__________________________________________________________________________________________________

activation_47 (Activation) (None, 7, 7, 512) 0 bn5c_branch2a[0][0]

__________________________________________________________________________________________________

res5c_branch2b (Conv2D) (None, 7, 7, 512) 2359808 activation_47[0][0]

__________________________________________________________________________________________________

bn5c_branch2b (BatchNormalizati (None, 7, 7, 512) 2048 res5c_branch2b[0][0]

__________________________________________________________________________________________________

activation_48 (Activation) (None, 7, 7, 512) 0 bn5c_branch2b[0][0]

__________________________________________________________________________________________________

res5c_branch2c (Conv2D) (None, 7, 7, 2048) 1050624 activation_48[0][0]

__________________________________________________________________________________________________

bn5c_branch2c (BatchNormalizati (None, 7, 7, 2048) 8192 res5c_branch2c[0][0]

__________________________________________________________________________________________________

add_16 (Add) (None, 7, 7, 2048) 0 bn5c_branch2c[0][0]

activation_46[0][0]

__________________________________________________________________________________________________

activation_49 (Activation) (None, 7, 7, 2048) 0 add_16[0][0]

__________________________________________________________________________________________________

avg_pool (AveragePooling2D) (None, 1, 1, 2048) 0 activation_49[0][0]

__________________________________________________________________________________________________

flatten_1 (Flatten) (None, 2048) 0 avg_pool[0][0]

__________________________________________________________________________________________________

fc1000 (Dense) (None, 1000) 2049000 flatten_1[0][0]

==================================================================================================

Total params: 25,636,712

Trainable params: 25,583,592

Non-trainable params: 53,120

__________________________________________________________________________________________________

われわれ（植物研究者）の観点から使えそうなコト（in general)

Q.「ヒト（プロ）と同レベルならば人（プロ）がやればよい」のでは？

A.「どんな場所にも暗黙知・経験知を一瞬で持ち込める」ことに意味があると思います。

カルテを何千枚も見てきてパターン認識できる医者はいるbut

全発現データを何千枚見てきてパターン認識できてる人は多分いない

Q.「深層学習において識別（回帰）出来たら、その先はない」のでは？

A.「識別出来た理由が分からないことが多い ＆ 理由自体が求める答えである」ことが多いと思います。

■ Science of integrative info, like “Favors” or “Atmosphere”

Which ones would be OK for purchasing?

What criteria?

“Threshold”

in cucumbers ☚ 農学だと、こういうのが面白いな、と思っています。

われわれ（植物研究者）の観点から使えそうなコト（in detail)

〇一次的利用（農学だとかなり需要あり）自動障害果判別・病害判定・自動カウント（プロの目を現場に持ち込む)

〇二次的利用判別理由の生理学的解析：例えば発生運命の早期診断＆部位特異的トランスクリプトーム解析など

画像以外の「暗黙知」蓄積：ゲノムデータへのCNNの活用（進化学的指標の検出・塩基置換モデルの推定)

Giuffrida et al. 2018, Plant JSingh et al. 2018, Trend Plant Sci

実際にやってる人たちがいました＠SMBE2018

実際の解析例

岐阜県農業技術センター植栽「富有」11/13：調査・写真撮影 1,900果11/15：調査・写真撮影 400果11/20：調査・写真撮影 487果11/21：調査・写真撮影 387果

合計 3,174果

“へたすき”への

「暗黙知」：>60歳くらいの専門家のみ判別可能

これ、へたを取ってみないと分からない（へたを取ると商品価値ゼロ）

写真は「果頂部（へたが無い方）」から：選果場での基準に合わせて。➡ 見抜けるのか？

実際の解析例

Ans.「見抜ける」GUIツールでの解析例。>90%の精度が出力可能

Qus.「何を見抜いているのか？」信頼性の問題・研究に適用する可能性

➡ 診断理由の可視化が必要。

ところで、「診断理由」ってわかるの？

Black Box

■ 従来の考え方だと

「ネットワークの逆走は出来ない」

■ “診断理由” はブラックボックス

➡ 最近？出てきた

「ネットワークの逆伝播」

Fine heat-mapping of

contributing parts

>90%

disorder

FACE: Lapuschkin et al. (2018) arXiv

■ 「何を見て」国籍や性別を見分けているの？

：簡単に聞こえて、案外答えられない疑問

「診断理由」が分かることの強み。例えばの話。

「ネットワークの逆伝播」ツールの一例

GradCAM (Gradient-weighted Class Activation Mapping) (Selvaraju et al. 2016)

LRP (Layer-wise Relevance Propagation) (Bach et al. 2015)

実践編

では、実際に動かしてみましょう。まずは「画像診断」

必要なもの〇GPU搭載PC（CPUでも？）またはクラウド (この後、神沼先生より)〇学習/テストデータ（今回は画像）以下、選択可能。1. GUIツールDIGITSとNeural Network Consoleについて簡単に説明します2. CUIツール今回は「TensorFlow」をバックエンド（framework）に「keras」をラッパーライブラリ (higher API) として使います。「python」を言語としてkerasは書かれています。バックエンドとしては他にもcaffeやPyTorch, Chainer, NNablaなど多数あります。

「困ったらgithubまたはQiitaに聞け」

← Keras, VGG16で検索

直感的にできるGUIツール①DIGITS

■ 構築済みのネットワークを利用可能。

（AlexNet, GoogLeNet）

■ defaultで「かなりいい感じ」にやってくれる

⇔ この「いい感じ」自動調整が仇にも… (後述)

■ epochや画像の取り込みをいじりやすい

⇔ 反面、ネットワークのチューニングは難しい

■ セットアップがやや難。

■ Nvidiaのcaffeをバックエンドに

■ prototxtで重みづけモデルを出力化。

直感的にできるGUIツール①DIGITS

■ ローカルホストで扱う

■ 判別・物体認識がメイン

■ データセットを選択

■ 詳細オプションを設定

（defaultでもいい感じ）

やりたいこと選択

画像のデータセット登録

使うデータセットいろいろ

直感的にできるGUIツール①DIGITS

■ 結果の可視化

■ prototxtモデルを保存可能 (しかし、使えるはずのGrad-camが使えない）

■ epochごとにモデルは保存されている

直感的にできるGUIツール②

https://dl.sony.com/ja/

■ ネットワークをdrag-dropで設計可能。

⇔ 基本知識ゼロだときつい。

■ 結果の図 (acc/errの推移なども）を簡単出力

■ セットアップ簡単！！！

■ Sony製のNNablaをバックエンドに

： ただし、prototxt (caffe) でも出力化。

■ Nvidia製のGPU x 1のハードで十分演算可能

今回のコンセプトは、「Deep learningの仕組みは知らないけれど」「お試しで使ってみたい」or「とりあえず結果だけ知りたい」

Litchi Rambutan

■ 2クラスのcategorization

■ 教師データ（学習セット）はかなり少なめ（N<200)

Kerasで画像診断下準備: 必要ツールのインストール & データ/scriptダウンロード（終わってますよね？）

$ pip install tensorflow$ pip install keras$ pip install opencv-python$ pip install pandas$ pip install tqdm$ pip install sklearn (またはscikit-learn)$ pip install pillow$ pip install h5py$ pip install jupyter

画像データ

今回の画像データ学習後モデル（「きょうの料理」的な）

Jupyterで使うscripts

$ mkdir models

(後で必要になります）

$ jupyter notebook

➡ web browserにアクセス

（local hostが立ち上がっているはず）

Kerasで画像診断その1: 作業ディレクトリでjupyter notebookを立ち上げる。

Kerasで画像診断その2: 画像学習- ipynbを開いて、必要ツールをimport

今回みなさんがやるべきは「これを押すだけ」

「素人でもできる」がモットーなので。

あとは各自のデータに成型していってください。

Kerasで画像診断その3: 画像学習- データの読み込み・成型

ImageDataGeneratorで読み込む

./data/train/ 内のdirectoryを自動認識

今回は事情によりbatch sizeは極小。

Kerasで画像診断その4: 画像学習- 既存モデル＆既存画像で重みづけしたものを更新

VGG16の読み込み (kerasでカバー)

& imagenetの既学習

新データセットによる更新を許容

全結合層 (fully connection) 以降を新規に構築。

今回は2クラス分類

Imagenet: いわゆる「画像集」

このデータセットが既に学習されている。

Kerasで画像診断その5: 画像学習- main runおよび保存設定

Steps per epochとepochの設定。

今回はbatch size同様、事情により極小。

./models/ に学習モデルを保存。

HDF5形式で任意ですが、後のGrad-camなどと一致させる

いったん脱線その1：診断理由を検出するには？「ネットワーク逆伝播」の解説を

Grad-CAMでは

1. 全結合層がCNNとGlobal Average Pooling層に置き換わる必要性。

Convolution - Global average – Softmaxと経由させる

2. Convolutionの最終層の勾配情報で可視化

(i) Convolution最終層でGlobalAveragePoolingを行い、指定クラスの最終

層における重要度を決定。

(ii) 重要度に応じて各チャンネルをかけて加算していき、Reluを行ったもの

がGrad-CAM

1. 外す

2. カスタムしたものを足す

VGG16モデルの構造

3. Grad-camでは全結合直前のconvが大事

いったん脱線その2：みなさんの目の前の演算がなかなか終わらないワケ。

クラウドシステム（神沼先生がご紹介）

またはGPU搭載マシンで解析しましょう。

(20万円弱で十分に揃います）

閑話休題：Kerasで画像診断＆判断理由を検出その6: Grad-camの下準備

$ mv finetuned_VGG16.h5 ./models

または「finetuned_VGG16.h5」を「models」ディレクトリへ

Kerasで画像診断＆判断理由を検出その7: Grad-camの設定

サイズは全行程で一致させる

Kerasで画像診断＆判断理由を検出その8: 保存設定・画像診断・全結合層直前CNN層の取得

サイズは全行程で一致させる

学習モデルの指定

診断画像の選択

診断結果の表示

Grad-camにおける全結合層直前のCNN層の指定

＆ Grad-cam解析イメージの表示

Kerasで画像診断＆判断理由を検出その9: 診断結果の表示・診断理由の可視化

果肉 (flesh) や周囲の風景ではなく、

殻の表面を見て判断出来ていそうです。

➡ これで十分なのかな？次のスライドへ

100% ランブータンだと識別！！

Kerasで画像診断＆判断理由を検出その10: Wet研究者のニーズに応じて。情報学者とは方向性が異なる。

Grad-camのメリット■ Grad-CAMではクラス毎の影響度を可視化可能■ネットワークの制限や変更がいらない（複雑なネットワークでもOK）

表現力が高い（予測精度が高い）ことが至上命題ではない。（これは往々にしてオミクス解析の統計にも言える。理論 vs Wet研究）

ResNet50Acc=0.93Grad-CAM

AlexNetAcc=0.87

LRP

Grad-camのデメリット■ 細かい反応部分が分からない■ 既存のGuided Backpropと組み合わせ可能だが…未だ精度が低い

LRPのメリット■ pix単位の反応部位を可視化可能■ 実はInception modelくらいならBackprop可能らしい

LRPのデメリット■ pix毎のGuided Backpropのため、分岐が複雑なモデルは対応不可

Acknowledgements■ 深層学習

九州大学 大学院情報科学研究科

内田 誠一 博士・黒木 陵平

京都大学 フィールド研

伊勢 武史 博士・大西 信徳

■ 植物 (柿) サンプル

岐阜県農業技術研究センター

新川 猛 博士・鈴木 哲也・杉浦 真由

■ サンプリング

京都大学 大学院農学研究科 果樹園芸学研究室

前田 春香・牧本 里奈・増田 佳苗

ご連絡はこちらまで

takashia@kais.kyoto-u.ac.jp (1年くらいで使えなくなります)

clacla_takashi@yahoo.co.jp

伊勢武史 准教授京都大フィールド研

内田誠一 教授九州大学情報科学