View
1
Download
0
Category
Preview:
Citation preview
1
2© 2016 The MathWorks, Inc.
Using Model-Based Design to develop high quality
and reliable ADAS & Automated Driving Systems
김종헌차장
Senior Application Engineer
MathWorks Korea
3
Electronics & Active Safety: Helping to increase Road Safety
A reduction of 66% between 1991-2013.
Innovation of automotive safety & ADAS solutions made this possible.
76230
25938
?How much were road fatalities reduced over the last 20 years?
Evolution of Road Fatalities in EU (1991-2013)Source: Website European Commission
4
Challenges in ADAS & AD Development
Algorithm
– Complicate algorithm to analyze the environment
surrounding the car and make a decision in real-time
Perception
Localization
Situation analysis & Driving Behavior
Path Planning, Actuator Control
Process
– Vehicle prototype is difficlt for testing in early
engineering stage
– Limited flexibility to vary and explore different design
directions with real prototype
Test
– Vast amount of operating scenarios and data to test
(Millions of test km required)
5
Agenda
Challenges to automate the driving
Example workflow for ADAS algorithm development
– Managing & analyzing big data from vehicle fleet test
– Algorithms MATLAB/Simulink provide to support ADAS development
– Verifying and implementing ADAS algorithms using MBD
ROS Interface for Automated Driving system development
Q & A
6
MATLAB Gives Design Engineers the Environment for
Algorithm Design, Exploration, and Verification
Regression
Test
C Code
Algorithm
Interactive
Exploration
Data
7
Developing models for ADAS Algorithms
Regression
Test
C Code
Algorithm
Interactive
Exploration
Data
Regression
Test
C Code
Algorithm
Interactive
Exploration
Data
8
Example of Analysis and Re-simulation Tool
1. Gather data- Instrument Toolox- Vehicle Network Toolbox- Image Acquisition Toolbox
Model-Based Approach to Resource-Efficient
Object Fusion for an Autonomous Braking System
Jonny Andersson, Scania,
MathWorks Automotive Conference 2015
2. Re-simulate- MATLAB/Simulink- Parallel Computing Toolbox- MDCS
9
Example of Visualization and Algorithm development Tool
• Productivity
Data visualization
Multi-modal data analysis
• Allow Engineers Rapidly
Build and Share Tools
Easy-to-use plot functions
Build-and-deploy
Acquisition of Heterogeneous
Data (CAN & Image)
Video play
Vision/Radar data analysis
Sensor fusion algorithm development
Birds-Eye
View
10
Example of Analysis and Re-simulation Tool
1. Gather data- Instrument Toolox- Vehicle Network Toolbox- Image Acquisition Toolbox
2. Re-simulate- MATLAB/Simulink- Parallel Computing Toolbox- MDCS
3. Find situations- Various Data Analytics Tools
4. Classify and analyze- Various Data Analytics Tools- MATLAB Compiler- Report GeneratorModel-Based Approach to Resource-Efficient
Object Fusion for an Autonomous Braking System
Jonny Andersson, Scania,
MathWorks Automotive Conference 2015
11
MathWorks Tools Help Automate Ground Truth Labeling
Tools for
– Sensor input and visualization
– Video annotation
– Tracking and machine learning for
automation
Leverage parallel computing to
speed up training
Scale to big data architectures for
processing and searching
12
13
14
15
Developing Vision Using MathWorks Tools
Continental built their
tooling using MATLAB
16
Developing models for ADAS Algorithms
Regression
Test
C Code
Algorithm
Interactive
Exploration
Data
Regression
Test
C Code
Algorithm
Interactive
Exploration
Data
17
MathWorks Products for Vision Applications Development
Image Processing
Toolbox™
Contrast adjustment
Geometric transformations
Various filters
Segmentation
Object analysis
Computer Vision System Toolbox™
High-speed video I/O
Point Cloud processing
Tracking
Stereovision
Image Acquisition Toolbox™
Image capture from standard H/W
Analog, Camera Link, DCAM,
GigE Vision, USB camera, etc
Microsoft Kinect Support
Statistics and Machine Learning
Toolbox™
Multivariate statistics
Probability distribution
Machine learning
Experimental design
Statistical process control
18
Cascade DetectorComputer Vision System Toolbox - 4 Easy Steps to Create a Car Detector
Collect Car Image Data
(Training Images)
Label Objects of Interest
Train Object Detector
Test Detector Performance
19
Example of 3D Vision Algorithm DevelopmentComputer Vision System Toolbox
Stereo Calibration App.
Feature Detection/Extraction
Point Tracker
Feature Matching
Kd-tree based
Point Cloud
Object
Point Cloud
Visualization
Point Cloud
Matching
21
Example of Lidar Data Processing (Fitting and Removing Surface)Computer Vision System Toolbox
radar
vison
if(params.ptCloud.pcfitplane.enable)
if(params.ptCloud.pcfitplane.lowerHeight>0)
[~,~,outlierIndices] = pcfitplane(ptCloud,params.ptCloud.pcfitplane.lowerHeight,[0,0,1]);
ptCloud = select(ptCloud,outlierIndices);
end
if(params.ptCloud.pcfitplane.upperHeight>0)
[~,inlierIndices,~] = pcfitplane(ptCloud,params.ptCloud.pcfitplane.upperHeight,[0,0,1]);
ptCloud = select(ptCloud,inlierIndices);
end
end
22
Deep Learning for Object Recognition Neural Network Toolbox
Addressing Challenges in Deep Learning for Computer Vision
Challenge
Managing large sets of
labeled images
Resizing, Data augmentation
Background in neural
networks (deep learning)
Computation intensive task
(requires GPU)
Solution
imageSet or imageDataStore to
handle large sets of images
imresize, imcrop, imadjust,
imageInputLayer, etc.
Intuitive interfaces, well-documented
architectures and examples
Training supported on GPUs
No GPU expertise is required
Automate. Offload computations to a
cluster and test multiple architectures
CNN Object Detector
23
MathWorks Products for Radar Development
PA
LNA
DSP
Determination of waveforms
(range, resolution)
Phased Array System Toolbox
Signal Processing Toolbox
Instrument Control Toolbox
Algorithms for data analysis
Phased Array System Toolbox
DSP System Toolbox
Channel modeling (interference, noise)
Communications System Toolbox
Modeling of antenna arrays (no of elements, position)
Phased Array System Toolbox
Modeling of RF Impairments
(noise, non-linearity, frequency dependency)
SimRF
24
Phased Array System Toolbox
Waveform
GeneratorTransmitter
Transmit
Array
Signal
ProcessingReceiver
Receive
Array
Environment,
Targets, and
Interference
Waveforms
Pulse, LFM, FMCW, etc.
Tx Antenna Arrays
ULA, URA, etc.
Rx Antenna Arrays
ULA, URA, etc.
Beamforming, Matched
Filtering, Detection, CFAR,
STAP, etc.
Transmitter
Monostatic and Bistatic
Receiver
Monostatic and Bistatic
Environment effects,
impairments, interference
Algorithms
Simulation Tools
25
Phased Array System Toolbox Algorithms
Direction of ArrivalSpace-Time Adaptive
ProcessingDetection
28
• Calculate Ground Speed
• Object classification
• Filtering
• Offset Compensation
Data
Pre-processing
Zoning
Path
Estimation
Path
Estimation
How about Sensor fusion algorithm for FCW
Vision
Object
Radar
Object
Vision
LD
Vehicle
CAN
Sensor
Fusion
Kalman
Filter
Sensor Fusion
& Tracking
MIO: Most-Important Object
Threat
Assessment
Risk
Assessment
Maneuver
Analysis
Zoning
Find
MIOFCW
Kalman
Filter
29
• Calculate Ground Speed
• Object classification
• Filtering
• Offset Compensation
Data
Pre-processing
Zoning
Path
Estimation
Path
Estimation
Sensor fusion algorithm for FCW
Vision
Object
Radar
Object
Vision
LD
Vehicle
CAN
Sensor
Fusion
Kalman
Filter
Sensor Fusion
& Tracking
MIO: Most-Important Object
Threat
Assessment
Risk
Assessment
Maneuver
Analysis
Zoning
Find
MIOFCW
Kalman
Filter
30
Sensor Fusion
Made Easy by MATLAB CVST
Vision
Object
Radar
Object
Radar
VisionR1 R2 Rm
V1
V2
Vn
Radar
costMatrix
Assignments
V1 + R2
V2 + R1
Vn + Rm
Fusion
𝑓(𝑉1) + 𝑓(𝑅2)𝑓(𝑉2) + 𝑓(𝑅1)
𝑓(𝑉𝑛) + 𝑓(𝑅𝑚)
Fused Object List
Computer Vision System Toolbox™
[assignments, unassignedVisions, unassignedRadars] = ...assignDetectionsToTracks(costMatrix, param.costOfNonAssignment);
Pairs of visions and
associated radars
31
Kalman Filter
0x̂
0P
Initial state
& covariance
1ˆ
kx
1kP
Previous state
& covariance
Pk
APk1A
TQ
(1) Predict state based on physical model and previous state
(2) Predict error covariance matrix
Time Update (“Predict”)
K k Pk
H
T(HPk
H
TR)
1(1) Compute Kalman gain
(2) Update estimate state with measurement
Measurement Update (“Correct”)
?x k ?x kKk(zk H?x k
)
(3) Update the error covariance matrix
Pk (IK kH) Pk
kkk vHxz
Measurementkx̂
kP
Output of
updated state
1 kk
Current becomes previous
R : Sensor noise covariance matrix (measurement error)
K : Kalman gain
uw
][ TE wwQ
: Control variable matrix
: Process (state) noise
][T
kkk E eeP kkk xxe ˆ
: Process (state)
covariance matrix
(estimation error)
: Process noise
covariance matrix
v
H
: Measurement noise
k minimize P
A : State matrix relates the state at the
previous, k-1 to the state at the current, k
: Output matrix relates the state to the
measurement
kkkk wBuxAx
1ˆˆ
From sensor spec or experiment
32
Kalman Filter Made Easy by MATLAB CVST
0x̂
0P
Initial state
& covariance
1ˆ
kx
1kP
Previous state
& covariance Time Update (“Predict”)
Measurement Update (“Correct”) Current Measurement
Output of
updated state
1 kk
Current becomes previous
[z_pred,x_pred,P_pred] = predict(obj)
z_pred : prediction of measurementx_pred : prediction of stateP_pred : state estimation error covariance
at the next time step
[z_corr,x_corr,P_corr] = correct(obj,z)
z_corr : correction of measurementx_corr : correction of stateP_corr : state estimation error covariance
z
x_corrP_corr
Predicted state
x_pred
kalmanFilterSysObj = vision.KalmanFilter(A,H,'ProcessNoise',Q,'MeasurementNoise',R)
Computer Vision System Toolbox™
33
Regression
Test
Developing models for ADAS Algorithms
Interactive
Exploration
Data
AlgorithmRegression
TestAlgorithm
Interactive
Exploration
Data
C Code
34
Create components in MATLAB and reuse them in Simulink
35
Automate regression testing with Simulink Test
Specify tests and Interact with results
in the Test Manager
Generate a report to share results
36
Generate C code for your algorithm with MATLAB Coder
Software in the loop &
Processor in the loop
enabled by
Embedded Coder
MATLAB
C
37
Enable coverage of MATLAB and C code with
Simulink Verification and Validation
Collects coverage on MATLAB code
in “Normal” mode
Collects coverage on generated C
code in “Software in the loop” mode
38
39
Prototype on hardware with Simulink Real-Timeas seen in today’s Test drive your ADAS algorithms presentation
Algorithm Models
Vehicle and
Environment
Models
Forward
Collision
Warning
Autonomous
Emergency
Braking
40
41
User Stories
Ground Detection using Backward Camera
11
Dynamic Alignment of Radar Sensor
42
Agenda
Challenges to automate the driving
Example workflow for ADAS algorithm development
– Managing & analyzing big data from vehicle fleet test
– Algorithms MATLAB/Simulink provide to support ADAS development
– Verifying and implementing ADAS algorithms using MBD
ROS Interface for Automated Driving system development
Q & A
43
Ethernet
CPU2
Main CPU
Challenges of Doing It All by Yourself
Enc
IMU
LIDAR
Kinematics& Control
Remote
machine
Your
algorithm
Image pre-
processing
Camera
Localization &
Mapping
GUI
Visual SLAM
Motor
Controller
Map server
Local
Planner
Global
Planner
You may need…
• Reliable distributed architecture
• Algorithms for autonomy
• Path planning
• Collision avoidance
• Sensor processing
• Sensor data acquisition
• Simulation environments
WiFi
44
Automated Driving at BMW
45
Potential Frameworks @ BMW
In-house SW Commercial SW Open source
46
Potential Frameworks @ BMW
In-house SW Commercial SW Open sourceIn-house SW Commercial SW Open source
47
What is ROS (Robot Operating System)?
• An architecture for distributed inter-process
communication
• Packages for common algorithms and drivers
• Multilanguage interface (C++, Python, Lua, Java
and MATLAB)
Benefit for Automated Driving
• Sensor data acquisition
• Reliable distributed architecture
• Lots of “off the shelf” algorithms for autonomy Path planning / Collision avoidance /Sensor processing
• Simplified component compatibility through
standalone interfaces
• Integration with simulation environments
48
ROS Trends in Robotics Development
http://rosindustrial.org/ric-americas/
• #1 middleware for robotics
applications development
• Popular in research and
gaining great momentum in
industry
49
LIDAR RADAR GPS/IMUCamera
Autonomous Car as an Advanced Robotics System
Steering Actuator
Actuator ECUs
GAS/Brake Actuator
Motion Controllers
Planning
Localization Obstacle
avoidance
Global Map
Motion controlNODE
NODE
NODE
NODE
NODE
NODE
NODE
NODE NODE NODE
ROS: communication framework
and stack of libraries
50
With MATLAB/Simulink?
Custom C
C/C++ Code
Early Idea
MATLAB Code Simulink Model
Convert to ROS Node
by Hand
Generate
Code
ROS
Need to learn ROS and Linux
Should be familiar with C++
Difficult to get started
51
With RST
Custom C
Early Idea
MATLAB Code Simulink Model
ROS
Workflow with Robotics System Toolbox
Without RST
Custom C
C/C++ Code
Early Idea
MATLAB Code Simulink Model
Convert to ROS Node
by Hand
Code
Generation
with
Interface
Merge to Simulink
Operation
connecting
with ROS
Generate
Code
ROS
Need to learn ROS and Linux
Should be familiar with C++
Difficult to get started
52
What can be done with the Robotics System Toolbox?
ROSMATLAB Code
SM Models
Built-in
algorithms
Robot
ROS node
Simulation
environment
Networking
MATLAB on PC
53
Co-simulation with ROS
54
Co-simulation with ROS
55
What can be done with the Robotics System Toolbox?
ROSMATLAB Code
SM Models
Built-in
algorithms
Robot
ROS node
Simulation
environment
Networking
MATLAB on PC
56
Co-simulation with ROS
Robot URI
Ex) 192.168.204.150
Simulator URI
Ex)192.168.204.144
57
Implementation
58
What can be done with the Robotics System Toolbox?
ROSMATLAB Code
SM Models
Built-in
algorithms
Robot
ROS node
Simulation
environment
Networking
Code Generation
MATLAB on PC
Generate standalone ROS node from Simulink
59
Automated Driving at BMW
60
Automated Driving with ROS at BMW
** MICHAEL AEBERHARD, BMW, ROSCON 2015
61
The MATLAB environment helps engineers …
Gain insight by visualizing and analyzing data– Plotting helps you understand sensor behavior and “give life to the data”
– Creating Apps helps you simplify the analysis process
Speed up algorithm design to implementation iterations by
generating code– Generating C code helps the algorithm and implementation stay synchronized
– Automating regression testing helps algorithm and software engineers collaborate
Have a rapid interactive environment leveraging the ROS
communication framework and stack of libraries.
62
Q & A
Thank you for your attention!
Questions?
Recommended