Click here to load reader

通用目标检测框架下的船只识别系统 - *2pt``复杂海情和气 … · ship identification | university of chinese academy of sciences 通用目标检测框架下的船只识别系统

  • View
    282

  • Download
    0

Embed Size (px)

Text of 通用目标检测框架下的船只识别系统 - *2pt``复杂海情和气 … · ship...

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Ship Identification with General Object Detection Frameworks

    2017 12

    11

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    2

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    not final!

    Titan X fine-tune

    4

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    1024 1024

    huochuan,714 565 1009 692huochuan,460 562 810 918huochuan,883 281 1024 398youlun,693 926 796 1009

    bounding box(x, y,w, h)

    6

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    1024 1024

    7

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Are U Kidding Me

    (a) (b)

    1:

    Bad ground truth labels hurt performance!

    8

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Recall = TPTP+FNPrecision = TPTP+FP

    F1 =2PR

    P + R

    0 0.2 0.4 0.6 0.8 10

    0.25

    0.5

    0.75

    1

    recall

    pre

    cis

    ion

    KITTI Pedestrian (moderate)

    DeepParts

    FasterRCNN

    FilteredICF

    pAUCEnsT

    Regionlets

    CompACTDeep

    3DOP

    SDP+RPN

    MSCNN

    2: PR

    9

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    R-CNN

    Race to the Top

    11

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Faster R-CNNFaster Region-based Convolutional Networks

    Multiway Classification

    BoxRefinement

    Box ClassifierFeature Extractor

    (vgg,incep+on,resnet,etc)

    BoxRegression

    ObjectnessClassification

    Proposal Generator

    CNN RPN Fast R-CNN

    image

    conv layers

    feature maps

    Region Proposal Network

    proposals

    classifier

    RoI pooling

    12

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    dark channel prior

    I(x) = J(x)t(x) + A(1 t(x))

    A

    14

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    dark channel prior

    I(x) = J(x)t(x) + A(1 t(x)) A

    14

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    dark channel prior

    I(x) = J(x)t(x) + A(1 t(x)) A

    14

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    SVM+

    15

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    /

    : = 4 : 1

    XXX(6XXX8) 18:38:20 3000 XXX(6XXX8) 18:38:35XXX 19:34:02

    XXX(4XXX8) 19:34:24 XXX(6XXX8) 20:38:52

    629924424473239 107572929618

    16

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    /

    : = 4 : 1

    XXX(6XXX8) 18:38:20 3000 XXX(6XXX8) 18:38:35XXX 19:34:02

    XXX(4XXX8) 19:34:24 XXX(6XXX8) 20:38:52

    629924424473239 107572929618

    16

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    /

    : = 4 : 1

    XXX(6XXX8) 18:38:20 3000 XXX(6XXX8) 18:38:35XXX 19:34:02

    XXX(4XXX8) 19:34:24 XXX(6XXX8) 20:38:52

    629924424473239 107572929618

    16

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    /

    : = 4 : 1

    XXX(6XXX8) 18:38:20 3000 XXX(6XXX8) 18:38:35XXX 19:34:02

    XXX(4XXX8) 19:34:24 XXX(6XXX8) 20:38:52

    629924424473239

    107572929618

    16

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    /

    : = 4 : 1

    XXX(6XXX8) 18:38:20 3000 XXX(6XXX8) 18:38:35XXX 19:34:02

    XXX(4XXX8) 19:34:24 XXX(6XXX8) 20:38:52

    629924424473239 107572929618

    16

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    /

    : = 4 : 1

    XXX(6XXX8) 18:38:20 3000 XXX(6XXX8) 18:38:35XXX 19:34:02

    XXX(4XXX8) 19:34:24 XXX(6XXX8) 20:38:52

    629924424473239 107572929618

    16

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    THE REMEDY (BUDA ET AL. 2017)

    The method that in most of the cases outperforms all otherswith respect to multiclass ROC AUC was oversampling.

    Oversampling should be applied to the level that totallyeliminates the imbalance.

    As opposed to some classical machine learning models,oversampling does not necessarily cause overfitting ofconvolutional neural networks.

    17

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Random saturation

    Random cropping Random contrast Random clipping & pasting Random dehazing efforts

    18

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Random saturation Random cropping

    Random contrast Random clipping & pasting Random dehazing efforts

    18

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Random saturation Random cropping Random contrast

    Random clipping & pasting Random dehazing efforts

    18

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Random saturation Random cropping Random contrast Random clipping & pasting

    Random dehazing efforts

    18

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Random saturation Random cropping Random contrast Random clipping & pasting Random dehazing efforts

    18

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    3: 19

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    RPN[Cai 2016]

    4: KITTI

    20

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    MSCNN (Multi-Scale CNN)Cai et al. 2016

    MSCNN

    * feature map

    21

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Sub-CNN (Subcategory-aware CNN)Xiang et al. 2016

    N M

    N = (N 1)M + N

    Conv feature maps

    Conv layers Subcategoryconv layer(shared weights)

    Heat maps

    RoIgeneratinglayer

    RoIs

    RoI poolinglayer

    Softmax

    Bboxregression

    Featureextrapolatinglayer

    Extrapolatedconv feature maps

    Subcategoryconv layer(shared weights)

    FC layer

    Pooled convfeature map

    Image pyramid For each RoI

    22

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Multi-crop Inference

    Ensembling: diversity is key

    23

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Takeaway Messages

    Ground truth matters

    Image dehazing works very well Trade-offs: speed / accuracy, precision / recall Addressing class imbalance: oversample Handling objects with scale variation

    feature up-sampling by deconvolution / feature extrapolation multi-scale feature fusion

    25

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Takeaway Messages

    Ground truth matters Image dehazing works very well

    Trade-offs: speed / accuracy, precision / recall Addressing class imbalance: oversample Handling objects with scale variation

    feature up-sampling by deconvolution / feature extrapolation multi-scale feature fusion

    25

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Takeaway Messages

    Ground truth matters Image dehazing works very well Trade-offs: speed / accuracy, precision / recall

    Addressing class imbalance: oversample Handling objects with scale variation

    feature up-sampling by deconvolution / feature extrapolation multi-scale feature fusion

    25

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Takeaway Messages

    Ground truth matters Image dehazing works very well Trade-offs: speed / accuracy, precision / recall Addressing class imbalance: oversample

    Handling objects with scale variation

    feature up-sampling by deconvolution / feature extrapolation multi-scale feature fusion

    25

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Takeaway Messages

    Ground truth matters Image dehazing works very well Trade-offs: speed / accuracy, precision / recall Addressing class imbalance: oversample Handling objects with scale variation

    feature up-sampling by deconvolution / feature extrapolation multi-scale feature fusion

    25

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Takeaway Messages

    Ground truth matters Image dehazing works very well Trade-offs: speed / accuracy, precision / recall Addressing class imbalance: oversample Handling objects with scale variation

    feature up-sampling by deconvolution / feature extrapolation

    multi-scale feature fusion

    25

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Takeaway Messages

    Ground truth matters Image dehazing works very well Trade-offs: speed / accuracy, precision / recall Addressing class imbalance: oversample Handling objects with scale variation

    feature up-sampling by deconvolution / feature extrapolation multi-scale feature fusion

    25

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Object Detection

    Ren, Shaoqing, et al. Faster R-CNN: Towards real-timeobject detection with region proposal networks. Advances inneural information processing systems. 2015.

    Huang, Jonathan, et al. Speed/accuracy trade-offs formodern convolutional object detectors. arXiv preprintarXiv:1611.10012 (2016).

    Huang, Jonathan. Object Detection at Google. IEEE SIDAS2017 Industry Keynote.

    . 2016.

    26

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    State-of-the-art

    Cai, Zhaowei, et al. A unified multi-scale deep convolutionalneural network for fast object detection. European Conferenceon Computer Vision. Springer International Publishing, 2016.

    Hong, Sanghoon, et al. PVANet: Lightweight Deep NeuralNetworks for Real-time Object Detection. arXiv preprintarXiv:1611.08588 (2016).

    Ren, Jimmy, et al. Accurate Single Stage Detector UsingRecurrent Rolling Convolution. arXiv preprintarXiv:1704.05776 (2017).

    Xiang, Yu, et al. Subcategory-aware convolutional neuralnetworks for object proposals and detection. IEEE WinterConference on Applications of Computer Vision. IEEE, 2017.

    27

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Dehazing and others

    He, Kaiming, Jian Sun, and Xiaoou Tang. Single image hazeremoval using dark channel prior. IEEE transactions onpattern analysis and machine intelligence 33.12 (2011).

    Berman, Dana, and Shai Avidan. Non-local image dehazing.Proceedings of the IEEE conference on computer vision andpattern recognition. 2016.

    Buda, Mateusz, Atsuto Maki, and Maciej A. Mazurowski. Asystematic study of the class imbalance problem inconvolutional neural networks. arXiv preprintarXiv:1710.05381 (2017).

    28

  • SHIP IDENTIFICATION | UNIVERSITY OF CHINESE ACADEMY OF SCIENCES

    Thank you!

    Code and models will be available at:

    https://github.com/sailordiary/ship-identification

    Questions?29

    https://github.com/sailordiary/ship-identification

    R-CNN

    Race to the Top