38
Multi - sensor Data Fusion Techno Briefing Mr. Paveen Juntama Air Traffic Service engineering Research & Development Department (RD.AS.) Presented by

Multisensor Data Fusion : Techno Briefing

Embed Size (px)

DESCRIPTION

This presentation includes : - Introduction - Methodology - Data Fusion Techniques - ATC Applications - Current works

Citation preview

Page 1: Multisensor Data Fusion : Techno Briefing

Multi-sensor Data Fusion

Techno Briefing

Mr. Paveen Juntama

Air Traffic Service engineeringResearch & Development Department(RD.AS.)

Presented by

Page 2: Multisensor Data Fusion : Techno Briefing

Contents

Overview

Methodology

Fusion Techniques

ATC Applications

Current works in RD.AS. (วว.สว.)

2

Page 3: Multisensor Data Fusion : Techno Briefing

Overview

Overview

Methodology

Fusion Techniques

ATC Applications

Current works in RD.AS. (วว.สว.)

3

Page 4: Multisensor Data Fusion : Techno Briefing

Problem-solving techniques based on the idea of integrating many answers into a single; the best answer

Process of combining data or information from various sensors to provide a robust and complete description of an process of interest

Multilevel process dealing with automatic detection, association, correlation, estimation and combinationof data or information from single or multiple sources

Definitions :

OverviewMultisensor Data Fusion (MDF)

4

Page 5: Multisensor Data Fusion : Techno Briefing

Location and characterization of enemy units & weapons

Air to air / surface to air defense Battlefield intelligence Strategic warning etc.

Military applications :

OverviewMDF Applications

5

Central Monitoring systems (CMS) System Faults Detection Location & Identification Robotics & UAVs Medical etc.

Non military applications :

Page 6: Multisensor Data Fusion : Techno Briefing

Improves accuracy

Improves precision

Improves availability

Reduces uncertainty

Supports effective decision making

MDF provides advantages over a single sensor :

OverviewWhy MDF ?

6

Page 7: Multisensor Data Fusion : Techno Briefing

Methodology

Overview

Methodology

Fusion Techniques

ATC Applications

Current works in RD.AS. (วว.สว.)

7

Page 8: Multisensor Data Fusion : Techno Briefing

MethodologyFusion Architectures

8

Measurement Fusion (Sensor data Fusion)

Feature-level Fusion

Decision-level Fusion (High-level data Fusion)

Data Fusion requires combining expertise in 2 areas :

Sensors

Information integration

Data fusion is essentially an information integration problem.

Data fusion can be categorized into 3 main classes based on the level of data abstraction used for fusion :

Page 9: Multisensor Data Fusion : Techno Briefing

Direct fusion of data sensor

The sensors measuring the same physical phenomena are required.

Measurement Fusion (Sensor Data Fusion) :

MethodologyFusion Architectures

9

S1

Data LevelFusion

Association

S2

Sn

Feature

Extraction

Identity Declaration

Page 10: Multisensor Data Fusion : Techno Briefing

Involves the extraction of representative features from sensor data

Features is combined into a single concatenated feature vector that is an input to a fusion node

Feature-level Fusion :

MethodologyFusion Architectures

10

S1

Association

S2

Sn

Feature

Extraction

Feature LevelFusion

+Identity

Declaration

Page 11: Multisensor Data Fusion : Techno Briefing

Each sensor has made a preliminary determination of an entity’s location, attributes and identity before combining

Decision-level fusion algorithms are used such as weighted decision, Bayesian inference and Dempster-Shafer’s method

Decision-level Fusion :

MethodologyFusion Architectures

11

S1

S2

Sn

Identity Declaration

Feature

Extraction

Identity Declaration

Identity Declaration

Association

Declaration Level

Fusion

Page 12: Multisensor Data Fusion : Techno Briefing

Fusion Techniques

Overview

Methodology

Fusion Techniques

ATC Applications

Current works in RD.AS. (วว.สว.)

12

Page 13: Multisensor Data Fusion : Techno Briefing

Fusion Techniques

13

The available data fusion techniques can be classified into3 categories

Data Fusion

Data Association

Decision Fusion

State Estimation

Page 14: Multisensor Data Fusion : Techno Briefing

The process of assign and compute the weight that relates the observations or tracks from one set to the observation of tracks of

another set.

Fusion TechniquesData Fusion Techniques

14

Data Association Techniques

Algorithms commonly used

Nearest Neighbors(NN), Probabilistic Data Association(PDA), Joint PDA(JPDA), Multiple Hypothesis Test (MHT) etc.

Page 15: Multisensor Data Fusion : Techno Briefing

State estimation techniques aim to determine the state of the target under movement (typically the position) given the observation or

measurement.

Fusion TechniquesData Fusion Techniques

15

State Estimation (Tracking)

Algorithms commonly used

Maximum Likelihood (ML) & Maximum Posterior, Kalman Filter, Particle Filter, Covariance Consistency Methods etc.

Page 16: Multisensor Data Fusion : Techno Briefing

Decision Fusion techniques aim to make a high-level inference about the events and activities produced from the detected targets.

Fusion TechniquesData Fusion Techniques

16

Decision Fusion

Algorithms commonly used

Bayesian Methods & Dempster-Shafer Inference, AbductiveReasoning, Semantic Methods etc.

𝑥1(𝑛)

𝑥2(𝑛)

𝑥𝑛(𝑛)

𝑥(𝑛)|

|

|

Page 17: Multisensor Data Fusion : Techno Briefing

Fusion TechniquesData Fusion Techniques

17

Nearest Neighbors

Probabilistic Data Association

Joint PDA

Multiple Hypothesis Test

Maximum Likelihood

Kalman Filter*

Particle Filter

Covariance Consistency Methods

Bayesian Methods*

Dempster-Shafer Inference

Abductive Reasoning

Semantic Methods

*Bayesian approaches

Data Association State Estimation Decision Fusion

Page 18: Multisensor Data Fusion : Techno Briefing

Fusion TechniquesBayesian Approaches

18

Bayes’ theorem

where the posterior probability, P(Y|X), represents the belief in the hypothesis Y given the information X. This probability is obtained by

multiplying the a priori probability of the hypothesis P(Y) by the probability of having X given that Y is true, P(X|Y)

Page 19: Multisensor Data Fusion : Techno Briefing

Fusion TechniquesBayesian Approaches

19

A Recursive Bayesian Estimator : Kalman Filter

Address the general problem of trying to estimate the state of a discrete time process

Estimate a process using a recursive algorithm :– Prediction : estimate the process state at a certain time

– Correction : obtain feedback from noisy measurement

Page 20: Multisensor Data Fusion : Techno Briefing

Fusion TechniquesBayesian Approaches

20

The need of Kalman Filter ?

System

MeasuringDevice

SystemError Sources

Control

UnknownSystem State

Measurement Error Sources

System state cannot be measures directly

Estimation “optimally” from measurements is required

Correction

PredictionPrediction

++

MeasurementModel

ProcessModel

Updated+

-

Error

Kalman Filter

𝑥(𝑛)

𝑥(𝑛)

Page 21: Multisensor Data Fusion : Techno Briefing

Fusion TechniquesBayesian Approaches

21

Data Fusion with Kalman filter

MeasurementFusion

Track-to-trackFusion

Page 22: Multisensor Data Fusion : Techno Briefing

Fusion TechniquesBayesian Approaches

22

Example results of Kalman filtering

Page 23: Multisensor Data Fusion : Techno Briefing

ATC Applications

Overview

Methodology

Fusion Techniques

ATC Applications

Current works in RD.AS. (วว.สว.)

23

Page 24: Multisensor Data Fusion : Techno Briefing

ATC ApplicationsSurveillance Data Processing

24

VHF GS

SAT GS

ATC CENTRE

ADS GS

MLAT/WAMMODE SSSRPSR

SAT NAVINMARSATSAT COM

Surveillance sensor environment

Page 25: Multisensor Data Fusion : Techno Briefing

25

ATC ApplicationsSurveillance Data Processing

Page 26: Multisensor Data Fusion : Techno Briefing

26

ATC ApplicationsSurveillance Data Processing

Selection techniques

Radar 1

Radar 2

|

|

|

Radar N

Multiple plots

switching

Selected Plots

plots

plots

plots

Radar 1

Radar 2

|

|

|

Radar N

Mono radar tracking

Mono radar tracking

Mono radar tracking

Multiple tracks

switching

plots

plots

tracks

tracks

Multiple plots switching method

Multiple tracks switching method

Selected Tracks

plots tracks

Page 27: Multisensor Data Fusion : Techno Briefing

27

ATC ApplicationsSurveillance Data Processing

Average techniques

Multiple track average method

Multiple plot average method

Radar 1

Radar 2

|

|

|

Radar N

Mono radar tracking

Mono radar tracking

Mono radar tracking

Track-to-track

correlation

Track-to-track

fusion

Fused

Tracks

plots

plots

plots

tracks

tracks

tracks

Radar 1

Radar 2

|

|

|

Radar N

Plot-to-plot

correlation

Plot-to-plot fusion

Fused

Plots

plots

plots

plots

Page 28: Multisensor Data Fusion : Techno Briefing

The technique consists in using all plots coming from any radar to update a unique synthetic common track

The track update is performed in the fly as soon as sensor report are received so that the reduction of the meantime update in multi-radar configuration improves the accuracy of the track parameter estimation.

These techniques contain more complex algorithms (Association + State Estimation + Decision Fusion)

Variable update techniques :

28

ATC ApplicationsSurveillance Data Processing

N

N-1

N-2

Correlation

Track Management

Track UpdateTrack

Initiation

N-kOutput

Tracks created

Tracks initiated

Non

associated

plots

Association

PairingNon

associated

tracks

Tracks to update

Page 29: Multisensor Data Fusion : Techno Briefing

29

ATC ApplicationsSurveillance Data Processing

Comparison between techniques :

Selection & Average Techniques Variable Update Techniques

Low CPU load Medium to high CPU load

Low track accuracy Good track accuracy

Low track discrimination Good track discrimination

Manoeuvre detection in long time Manoeuvre detection in short time

Page 30: Multisensor Data Fusion : Techno Briefing

Current works in RD.AS.

Overview

Methodology

Fusion Techniques

ATC Applications

Current works in RD.AS. (วว.สว.)

30

Page 31: Multisensor Data Fusion : Techno Briefing

Current works in RD.AS.

31

System Architecture :

Multi Radar Tracking System (MRTS)

FusionSystem

SSR

SSR

SSR

ADS-B

Ground Station

Local Tracks

ADS-B Reports

System Tracks

Page 32: Multisensor Data Fusion : Techno Briefing

32

Study of System tracks & ADS-B reports

Characteristics System tracks ADS-B reports

Update rates 500 ms 0.3-3 ms

Update rates / target 5 s 1 s

Data source MRTS GNSS

Identification Mode 3/A Callsign, Mode S

PerformanceHigh Availability

Low AccuracyHigh Accuracy

Low Availability

Current works in RD.AS.

Page 33: Multisensor Data Fusion : Techno Briefing

Current works in RD.AS.

33

Study of System tracks & ADS-B reportsHorizontal Zoom

ADS-B reports lost in some periods (Low Availability) System tracks are less accurate in positioning compared to ADS-B

reports

Page 34: Multisensor Data Fusion : Techno Briefing

Current works in RD.AS.

34

Fusion system

Sensor data Feature vector IdentityDeclaration

System tracks (CAT62),

ADS-B reports (CAT21)

Metadata Fused Tracks

Track Management, Track Initiation and Filtering are responsible for the association, correlation and state estimation techniques.

While Track-to-track Fusion corresponds to a Decision-level Fusion scheme.

Page 35: Multisensor Data Fusion : Techno Briefing

Current works in RD.AS.

35

Results :

Remark :

There is only 1 ADS-B ground station in operation

Page 36: Multisensor Data Fusion : Techno Briefing

Current works in RD.AS.

36

Problems & Difficulties

The difficult synchronization due to different time sources between ADS-B GS & MRTS

Difficulty of track correlation due to target identification problems

Page 37: Multisensor Data Fusion : Techno Briefing

Current works in RD.AS.

37

Future works

Improve the synchronization mechanisms

Improve Fusion algorithms

Evaluate performance of the fusion system

Study possibilities for integrating data from new sensor types such as MLAT, WAM etc.

Study and characterize the system closing to the realistic environment as possible including a process model, a measurement model, Radar biases and ADS-B receiver biases.

Page 38: Multisensor Data Fusion : Techno Briefing

Conclusion

38

Data fusion can be performed at 3 levels :

– Sensor data

– Feature vectors

– High level inferences

Several techniques has been developed to process data fusion at each level.

Fusion techniques can be used with one or more techniques; data association, state estimation or decision fusion, each technique contains various algorithms.

The use of fusion techniques and methodology depends on the environment of the system which include sensor characteristics, integrated information etc.