Convolutional Pose Machines@conta_

緒方　貴紀 (twitter: @conta_)

CTO@ABEJA, Inc.

Computer Visionとか、Machine Learningを使った

プロダクト開発をやっています。

Self Introduction

Pose Estimation?

Related Work

Pictorial structures Hierarchical models

Sequential prediction Convolutional architectures

[A. Toshev and C. Szegedy, CVPR’2013]

[Tian et al., ICCV’2011][Mykhaylo et al., CVPR’2009]

[Ramakrishna et al., 2014]

Related Work

Pictorial structures Hierarchical models

Sequential prediction Convolutional architectures

[A. Toshev and C. Szegedy, CVPR’2013]

[Tian et al., ICCV’2011][Mykhaylo et al., CVPR’2009]

[Ramakrishna et al., 2014]

Pose Machines?

Confidence Maps[Ramakrishna et al., 2014]

Pose Machines

[Ramakrishna et al., 2014]

パッチから特徴量を抽出し、各Parts or NotのClassifierを用いて、Confidence Mapsを作りたい

Pose Machines

[Ramakrishna et al., 2014]

局所的な特徴を用いた推定はWeak

Partsによっては、局所的特徴だど推定できない。。。

Part Contextは非常に有効な特徴

Pose Machines

[Ramakrishna et al., 2014]

前段階での推定結果を用いて、各Partsの関係性を

事前情報無しにを活用出来ないか？

Pose Machines

Stage I Confidence

Head Neck L-Shoulder L-Elbow L-Wrist

g2g1 g3

Context Features

Context Features

Stage I Confidence Maps

Stage II Confidence Maps

Stage III Confidence Maps

Image Features

[Ramakrishna et al., 2014]

Pose Machines

Stage II Confidence

g2g1 g3

Context Features

Context Features

Stage I Confidence Maps

Stage II Confidence Maps

Stage III Confidence Maps

Image Features

Head Neck L-Shoulder L-Elbow L-Wrist

[Ramakrishna et al., 2014]

Pose Machines

Stage III ConfidenceHead Neck L-Shoulder L-Elbow L-Wrist

g2g1 g3

Context Features

Context Features

Stage I Confidence Maps

Stage II Confidence Maps

Stage III Confidence Maps

Image Features

Head Neck L-Shoulder L-Elbow L-Wrist

[Ramakrishna et al., 2014]

Pose Machines

Stage III ConfidenceHead Neck L-Shoulder L-Elbow L-Wrist

g2g1 g3

Context Features

Context Features

Stage I Confidence Maps

Stage II Confidence Maps

Stage III Confidence Maps

Image Features

Head Neck L-Shoulder L-Elbow L-Wrist

[Ramakrishna et al., 2014]

Pose Machines (Previous Work)

Pose Machines (Previous Work)

HoG Random Forests

Convolutional Pose Machines (CPM)

Deep Deep

Architecture of CPM

階層的なCNNによるPose Machinesの実現

Stage1

Stage1: Localな特徴量の学習

368x368のInputに対して160x160の範囲をカバー

各Parts + BackgroundのConfidence Maps (P+1)

＊MPII Human Pose Datasetだと P=14

Outputs

Stage 2

Stage2: Localな特徴量 + Part Contextによる学習

Stage T

Stage2と同じ構成のネットワークを積み上げていく

＊本研究ではStage6まで積み上げる

各StageのConfidence Mapsと教師データとのEuclidean Distance Loss

教師データ: 各PartsのGround truth locationからGaussian Peakを計算したもの

Loss Function

Stage2以降、全段階のConfidence Mapのおかげで

良い推定ができている

Spatial context from belief maps

3つのDatasetsで実験

- MPII Human Pose Dataset

- Leeds Sports Pose (LSP) Datase

- FLIC Dataset

Experiments

Results

Results

Stageは積み上げるとイイんやで

Results

Pose Machinesを上回る精度

Stageを重ねるごとに精度は向上

Results

State-of-the-art Performance (ﾄﾞﾔｧ

実装してみた

1. (色々頑張って実験した結果)いい感じの連続構成CNNによって、暗黙的な空間モデルの学習ができた

2. Graphical Modelによる推論無しに、階層構造のPredictionができた

Conclusion

We are hiring!

→ https://www.wantedly.com/companies/abeja

博士持ち大歓迎！