对地观测与识别年度进展 -...

对地观测与识别年度进展

夏桂松guisong.xia@whu.edu.cn

武汉大学

2018/4/24 1

Earth observation (EO) :

Understanding Earth’s surface via remote sensing technologies.

哪儿有什么、会怎么样

Earth Vision：Understanding Earth’s surface with images

多尺度、特殊边界条件的CV问题

Difficulties：less than 5% of EO Images were used

数据那么多，能看懂的没有几个

人工智能 + Earth Observation

2018/4/24 2

3

对地观测与识别年度进展之

Object Detection in Aerial Images

夏桂松

guisong.xia@whu.edu.cn

武汉大学

测绘遥感信息工程国家重点实验室

2018/4/24

Object Detection in Aerial Images (ODAI)

@JL-1 Satellite

@Digital Globe

@Google Earth

2018/4/24 4

Earth Observation

Earth observation (EO) is about understanding the planet Earth‘s

surface via remote sensing technologies.

2018/4/24 5

Challenges of ODAI

Large variations in the scale of objects

2018/4/24 6

Challenges of ODAI

Arbitrary orientation of instances

Densely packed & small instances

2018/4/24 7

What happened before 2016

Cheng-Han, A survey on object detection in optical remote sensing images. ISPRS J. Photo. & Remote Sensing, 20162018/4/24 8

2017: Data-driven & Deep Models for ODAI

Arbitrary orientation of instances :

• RICNN: [Li et. al., IEEE TGRS 2018]

• Faster-RCNN-OBB: [Xia et. al., CVPR’2018]

Small objects:

• Random Access Memories, [Zou et. al., IEEE TIP, 2018]

• Layout Proposal Network, [Hsieh et. al., ICCV’2017]

Benchmark dataset

• DOTA, [Xia et. al., CVPR’2018]

• DIUx xView,

2018/4/24 9

Arbitrary orientation of instances

• Rotation-Insensitive CNN [Li et. al., TGRS 2018]

2018/4/24 10

Arbitrary orientation of instances

• Rotation-Insensitive CNN [Li et. al., TGRS 2018]

multi-angle, multiscale and translation-invariant RPN Results

2018/4/24 11

Arbitrary orientation of instances

• Rotation covariant: Faster-RCNN-OBB: [Xia et. al., CVPR’2018]

RoiPooling

FCs

Output: quadrangle regressor

Softmax predict pP {( , ), 1,2,3,4}xi pyit t i

2018/4/24 12

Arbitrary orientation of instances

• Rotation covariant: Faster-RCNN-OBB: [Xia et. al., CVPR’2018]

2018/4/24 13

Sparsely distributed small objects

Random Access Memories [Zou et. al., IEEE TIP, 2018]

2018/4/24 14

Sparsely distributed small objects

Random Access Memories [Zou et. al., IEEE TIP, 2018]

2018/4/24 15

CNN + CFAR + Bayesian Estimation

Sparsely distributed small objects

Random Access Memories [Zou et. al., IEEE TIP, 2018]

2018/4/24 16

Layout Proposal Network: [Hsieh et. al., ICCV’2017]

Packed small objects

2018/4/24 17

Layout Proposal Network: [Hsieh et. al., ICCV’2017]

Packed small objects

Layout Proposal Networks

Using spatial kernel to encode contextual layout

information into detectors.

Idea: a predicted position with

more nearby cars can get higher

confidence and higher probability

to be localized as a car.

2018/4/24 18

Layout Proposal Network

2018/4/24 19

DOTA: A Large-scale Dataset for ODAI

About 200K instances (~0.5M now), 15 categories Annotated by oriented bounding boxes (OBBs) Large images reflecting scenarios in real applications

2018/4/24 20

[Xia et. al., CVPR’2018]

DOTA: A Large-scale Dataset for ODAI

mAP of the detection task with oriented bounding boxes

2018/4/24 21

DOTA: A Large-scale Dataset for ODAI

Xia, et. al. DOTA: A Large-scale Dataset for Object Detection in Aerial Images. CVPR’2018.

Link to DOTA Link to a contest on ICPR’2018

基金委空间信息网络重大研究计划-目标检测比赛（特等奖10万RMB）

2018/4/24 22

In 2018 : DIUx xView

DIUx xView 2018 Detection Challenge

OBJECTS IN CONTEXT IN OVERHEAD IMAGERY

2018/4/24 23

Lam et. al., xView:

Objects in Context in

Overhead Imagery,

arXiv:1802.07856,

2018,

xviewdataset.org

In 2018 : VisDrone2018

P. Zhu, L. Wen, X. Bian, H. Ling and Q. Hu, Vision Meets Drones: A Challenge.

http://www.aiskyeye.com

2018/4/24 24

References

• Cheng-Han, A survey on object detection in optical remote sensing images. ISPRS J.

Photo. & Remote Sensing, 2016.

• Li, et. al., Rotation-Insensitive and Context-Augmented Object Detection in Remote Sensing Images. IEEE TGRS, Vol. 56, No.4, pp.2337-2348, 2018.

• Xia, et. al. DOTA: A Large-scale Dataset for Object Detection in Aerial Images. CVPR’2018, https://captain-whu.github.io/DOTA

• Zou et. al., Random Access Memories: A New Paradigm for Target Detection in High Resolution Aerial Remote Sensing Images, IEEE TIP, Vol27, No.3, pp.1100-1111, 2018

• Hsieh et. al., Drone-based Object Counting by Spatially Regularized Regional Proposal

Network, ICCV’2017

• Lam et. al., xView: Objects in Context in Overhead Imagery, arXiv:1802.07856, 2018,http://xviewdataset.org

• P. Zhu, L. Wen, X. Bian, H. Ling and Q. Hu, Vision Meets Drones: A Challenge, arXiv2018, http://www.aiskyeye.com

2018/4/24 25

26

谢谢！

夏桂松

guisong.xia@whu.edu.cn

武汉大学

2018/4/24