ASCL of Jilin ASCL of Jilin UniversityUniversity吉林大学汽车仿真与控制国家重点实验室吉林大学汽车仿真与控制国家重点实验室

Confidential 11

Background & MotivationBackground & Motivation

Challenges Technological Challenges on Vehicle

Intelligence

Advancementg

ModelingModelingSimulationSimulation

Confidential

D i i i t l i t tBackground & MotivationBackground & Motivation

Driving environment now plays an important or even critical role on the safety and performance of intelligent driving Driverdriving.

DrivingDrivingweatherweather traffictraffic

ggEnvironmentEnvironment

El i C l

Confidential

Vehicle Electronic Control

roadroad

Motivation: Motivation: Challenges on Testing & ValidationChallenges on Testing & Validation

Setting up field testing for many features can be of long‐V2V Coordination

Collision Avoidance

g p g y gcycle and high‐cost, and even impossible.

Some safety critical features can be difficult, if not impossible to test in the field especially during the

Intelligent Vehicle

impossible, to test in the field, especially during the early development stages!

Confidential

V2V Coordination Pedestrian Collision

Technological AdvancementTechnological Advancement

Modeling on Battery and Electric Motors, on‐board communication

VehicleDynamics

driverdriver

weatherweather traffictrafficroadroadtiretire

radarradarwirelesswirelesscameracamera

Confidential

Modeling on Driving Environment andEnvironmental Sensing

Modeling & Simulation Plays Key Roles Modeling & Simulation Plays Key Roles

Advanced Development PlatformAdvanced Development Platform

Pure SimulationPure Simulation离线纯仿真平台离线纯仿真平台

RTRT--SIL/HIL SimulationSIL/HIL SimulationRTRT SIL/HIL SimulationSIL/HIL Simulation实时、软硬件在环仿真平台实时、软硬件在环仿真平台

DIL with 3D EnvironmentDIL with 3D Environment驾驶员在环三维场景仿真驾驶员在环三维场景仿真驾驶员在环三维场景仿真驾驶员在环三维场景仿真

InIn--Vehicle TestVehicle TestConfidential

InIn--Vehicle TestVehicle Test实验车试验实验车试验//标定标定//验证验证

Tools & Method on Vehicle Tools & Method on Vehicle IntelligenceIntelligence

Vehicle Performance

Virtual driving environment(road, traffic, weather, etc)

Virtual driving scenarios(ABS/TCS/ESP/ACC, self-parking)

l Simulated traffic

Vehicle Virtual Driving Vehicle Virtual Driving EnvironmentEnvironment

Environmental sensors(radar, camera, GPS/map, V2V, V2X)

Simulated traffic participants

(cars, pedestrians, barrels, etc)

Vehicle IntelligenceVehicle Intelligence

Confidential

Tools & Method on Vehicle Tools & Method on Vehicle IntelligenceIntelligence

Driver ModelDriver Model

Weather ModelWeather Model Traffic ModelsTraffic Models

Virtual Driving Virtual Driving EnvironmentEnvironment withwith

Environment & Sensor ModelsEnvironment & Sensor Models

Vehicle Dynamics ModelVehicle Dynamics Model Electronic Control

Confidential

Road & Field ModelRoad & Field Model

Vehicle Dynamics Modeling: Vehicle Dynamics Modeling: with Highwith High--Order, LargeOrder, Large--Nonlinearity Nonlinearity with Highwith High Order, LargeOrder, Large Nonlinearity Nonlinearity

and High Efficiencyand High Efficiency

Confidential

Vehicle Dynamics ModelVehicle Dynamics Model

RearSprung Mass

Unsprung Mass

Unsprung Mass

SystemSystem--based modeling based modeling approach with:approach with:hhighigh--order and largeorder and large--

Wheel Spin

p g gg ggnonlinearitynonlinearityhigh efficiency for realhigh efficiency for real--timetimehigh fidelityhigh fidelityp

SideslipFront

high fidelityhigh fidelity

Front

sw

~

Confidential

Vehicle Dynamics Model Vehicle Dynamics Model (under (under SimulinkSimulink))

Confidential

Vehicle Dynamics Model: Vehicle Dynamics Model: verificationverification

10

20

30

40

gle

[deg

]

CarsimPanosimDouble-Lane Change

-30

-20

-10

0

Ste

ering

Whe

el A

ng

Simulated speed at 80kp

0 1 2 3 4 5 6 7 8 9 10-40

Time [s]

Steering Angle

10

15 CarsimPanosim

0.2

0.3

0.4 CarsimPanosim

5

0

5

Yaw

Rat

e [d

eg/s

]

-0.1

0

0.1

tera

l Acc

eler

atio

n [g

]

-15

-10

-5

-0.4

-0.3

-0.2

Lat

ConfidentialLateral GYaw Rate

0 1 2 3 4 5 6 7 8 9 10-15

Time [s]0 1 2 3 4 5 6 7 8 9 10

0.4

Time [s]

Vehicle Dynamics Model: Vehicle Dynamics Model: verificationverification

Fishhook Simulation

Simulated speed at 80kph, with steering and path shown below

300 20

100

200

300

deg]

CarsimPanosim

-20

0

20CarsimPanosim

-100

0

100

erin

g W

heel

Ang

le [d

-60

-40

Y C

oord

inat

e [m

]-300

-200

-100

Ste

e

120

-100

-80

PathSteering Angle

0 1 2 3 4 5 6 7 8 9 10-300

Time [s]0 50 100 150

-120

X Coordinate [m]

Confidential

Vehicle Dynamics Model: Vehicle Dynamics Model: verificationverification

30 CarsimP i 0 6

0.8 CarsimPanosim

Fishhook Simulation

Yaw R

10

20

[deg

/s]

Panosim

0.2

0.4

0.6

erat

ion

[g]

Panosim

LateraRate

20

-10

0

Yaw

Rat

e [

-0.4

-0.2

0

Late

ral A

ccel

e

al G

0 1 2 3 4 5 6 7 8 9 10-30

-20

Time [s]

0 1 2 3 4 5 6 7 8 9 10

-0.8

-0.6

Time [s]

2.5 Carsim

80.4 Carsim

0.5

1

1.5

2

deg]

CarsimPanosim

79.6

79.8

80

80.2

ty [k

m/h

]

CarsimPanosim

Roll

1 5

-1

-0.5

0

Rol

l Ang

le [d

79

79.2

79.4Lo

ngitu

dina

l vel

ocit

l Ang

le

Speed

Confidential0 1 2 3 4 5 6 7 8 9 10

-2.5

-2

-1.5

Time [s]

0 1 2 3 4 5 6 7 8 9 10

78.4

78.6

78.8

Time [s]

Vehicle Dynamics Model: Vehicle Dynamics Model: verificationverification

Fishhook Simulation

2.5 CarSimP Si

4 CarSim

1

1.5

2

gle

[deg

]

PanoSim

1

2

3

km/h

]

PanoSim

Sides

Late

-0.5

0

0.5

Veh

icle

Slip

Ang

-1

0

1

Late

ral S

peed

[k

slip A

ngle

eral Spee

0 1 2 3 4 5 6 7 8 9 10-1.5

-1

Time [s]

0 1 2 3 4 5 6 7 8 9 10-3

-2

e edTime [s]

Confidential

Radar Modeling: Radar Modeling: with Combined Geometric and Physical with Combined Geometric and Physical with Combined Geometric and Physical with Combined Geometric and Physical

ApproachesApproaches

Confidential

Radar ModelingRadar Modeling Take into consideration of major physical factors, such as

electromagnetic wave propagation (echo signals), target and its RCS estimation and ambient noiseestimation, and ambient noise

Combined geometric and physical modeling approaches to achieve high fidelity while maintaining real-time computational efficiency

Target M d l

high fidelity while maintaining real time computational efficiency

Model

RCS Echo SignalRadar Model RCS

Estimation

gPropagation

Model

Ambient Model

Confidential

Radar ModelingRadar Modeling

Confidential

Radar ModelingRadar Modeling

Confidential

Camera ModelingCamera ModelingCamera ModelingCamera Modeling

Confidential

PanoSimPanoSim：：Camera ModelingCamera Modeling

Confidential

Basic Camera Modeling

PanoSimPanoSim：：Camera ModelingCamera Modeling

fr)2tan(2 fr

)2i (2 f)sin(fr

)2sin(2 fr

Fisheye & Wide Angle Camera Modeling and Validation

Fisheye Camera Modeling

Confidential

Camera Modeling and Validation

PanoSimPanoSim：：Camera Mounting & CalibrationCamera Mounting & Calibration

Take PicturesTake Pictures

Record Video

Confidential Video Playback

PanoSimPanoSim：：Camera Model ApplicationsCamera Model Applications

Traffic Sign Recognition Distance/Speed MeasurementTraffic Sign Recognition Distance/Speed Measurement

Confidential

Lane Marker DetectionObject Detection

Highly Correlated with the Results from Real ImageHighly Correlated with the Results from Real Image

Real Image Simulated Image

Lane detection using Hough transform

Road detection using watershed algorithm

Confidential

ddAn Integrated An Integrated Software ToolSoftware Tool

Confidential

PanoSimPanoSim：：Six Modules with Many FunctionsSix Modules with Many Functions

Main GUIs VehicleBuilder

FieldBuilder PanoCam

PanoPlot/PanoAnim MDL Generator

T ffiR d /Lid /CV hi l D i

Confidential

Traffic Weather/Light

Radar/Lidar/Camera Virtual Proving Ground

Vehicle Dynamics Wireless Comm

PanoSimPanoSim：：ApplicationsApplications

Vehicle Performance

Virtual Proving Ground（Road/Traffic/Weather）

Virtual Scenarios（ABS/ESP/ACC/LDW/UPA）

l ffEnvironmental Sensors（Radar/Lidar/Camera/Wireless/GPS）

Traffic（Cars/Pedestrian/Signal/Signs）

ADASADAS

Confidential

PanoSimPanoSim：：ApplicationsApplications

ConceptSystem

Performance

PrototypeSubsystemIntegration

ProductSubsystem

lid iProduct Validation

An Integrated Platform for Technology and Product Development

of Vehicle Intelligence

Confidential

of Vehicle Intelligence

PanoSimPanoSim：：ApplicationsApplications

Vehicle Performance

ADAS ACC/LDW/LKA/FCW Autonomous Parking

Brake/Steer/Suspension Handling & Stability

g

Environment Sensing Vision-based sensing & IP

Powertrain System Engine/Transmission Driveline & Driveability

g Radar Detection Sensor Fusion

I t lli t D i iy

Electronic Controls ABS/TCS

Intelligent Driving Planning & Decision Positioning/NavigationABS/TCS

ESPIntelligent Transportation V2V/V2I/V2X

A D i i

Confidential

Autonomous Driving

FieldBuilderFieldBuilder：：Building Testing FieldBuilding Testing Field

3D Virtual Proving Ground

Confidential

TrafficTraffic：：Traffic Modeling for ADASTraffic Modeling for ADAS

Good fidelity in traffic dynamics, while maintaining sufficient computational efficiency

Good flexibility in generating desired disturbances for ADAS feature development, while maintaining its nature of randomness

Confidential

PanoSimPanoSim：：Virtual Testing & VerificationVirtual Testing & Verification

C lli i Collision Avoidance

Adaptive Cruise Control

Confidential

Autonomous Parking

PanoSimPanoSim：：Seamless Seamless Connection to SimulinkConnection to Simulink

Auto Simulink Model Generation

Hi h Effi i t High Efficient Numerical Simulation

Confidential

RealReal--Time HIL/DIL Time HIL/DIL SimulationSimulation

Confidential

Simulation on Simulation on Vehicle Intelligence DevelopmentVehicle Intelligence Development

Extend the functionality of driver simulator with environmental sensingenvironmental sensing

Confidential

Camera Modeling: ApplicationsCamera Modeling: ApplicationsReal-time HIL/DIL simulation with camera model

Virtual Driving EnvironmentVirtual Driving Environment

Driver SimulatorCamera Model

3Cat Warmup Scalar for Costs

2Cat Warmup Scalar for Rate L imits

1Cat al yst Warmup Enabl ed

single(1)

single(0)

Action

Eng C at Warmup Enabled

Cat Warmup Scalar

Cat Warmup Enable

Scalar f or Rate Limits

Scalar f or Costs

Use Cat Warmup Fl ag from Eng

Max R at e

Min Rate

Max

Min

input

output

Rate and Range Limiter 2

Max R at e

Min Rate

Max

Min

input

output

Rate and Range Limiter 1

Merge

Merge3

Merge

Merge2

Merge

Merge1

u1if (u1)

else

If

Action

Cat Warmup Scalar

Cat Warmup Threshold

Cat Warmup Enable

Scalar f or Rate Limits

Scalar f or C osts

Contro l without Flag f rom Engine

7Ti me Step

6Min Rate Li mi t for Cost Scal ar

5Min Rate L imit for Rate Scalar

4Cat Warmup T hreshold

3Use Eng Cat Warmup Fl ag

2Cat Warmup Scalar

1Eng Cat Warmup Enabl ed

VeEOCR_r_SclForR ateLimRaw

VeEOCR_r_SclCatWarmupCost Raw

VeEOCR _r_SclC at WarmupCost

VeEOCR _r_SclForRateLim

3Cat Warmup Scalar for Costs

2Cat Warmup Scalar for Rate L imits

1Cat al yst Warmup Enabl ed

single(1)

single(0)

Action

Eng C at Warmup Enabled

Cat Warmup Scalar

Cat Warmup Enable

Scalar f or Rate Limits

Scalar f or Costs

Use Cat Warmup Fl ag from Eng

Max R at e

Min Rate

Max

Min

input

output

Rate and Range Limiter 2

Max R at e

Min Rate

Max

Min

input

output

Rate and Range Limiter 1

Merge

Merge3

Merge

Merge2

Merge

Merge1

u1if (u1)

else

If

Action

Cat Warmup Scalar

Cat Warmup Threshold

Cat Warmup Enable

Scalar f or Rate Limits

Scalar f or C osts

Contro l without Flag f rom Engine

7Ti me Step

6Min Rate Li mi t for Cost Scal ar

5Min Rate L imit for Rate Scalar

4Cat Warmup T hreshold

3Use Eng Cat Warmup Fl ag

2Cat Warmup Scalar

1Eng Cat Warmup Enabl ed

VeEOCR_r_SclForR ateLimRaw

VeEOCR_r_SclCatWarmupCost Raw

VeEOCR _r_SclC at WarmupCost

VeEOCR _r_SclForRateLim

Image P

ControllerRT Vehicle Model

Processor

ConfidentialHIL

行驶环境及交通模拟

传感数据

系统功能演示：车辆动力学实时仿真汽车ACC电子控制算法

虚拟传感器驾驶员动态模拟器

数据处理

实时车辆动力学模型

汽车ACC电子控制算法环境与环境传感模拟驾驶员在环动态模拟车载通信及网络模拟

控制算法

ACC控制命令

（硬件在环）

车载通信及网络模拟人机界面设计汽车电控与智能化系统集成

3Cat Warmup Scalar for Costs

2Cat Warmup Scalar for Rate L imits

1Cat al yst Warmup Enabl ed

singl e(1)

singl e(0)

Action

Eng Cat Warmup Enabled

Cat Warmup Scalar

C at Warmup Enable

Scalar f or Rate Limits

Scalar f or Costs

Use Cat Warmup Flag f rom Eng

Max Rate

Min Rate

Max

Min

input

output

Rate and Range Li mi ter 2

Max Rate

Min Rate

Max

Min

input

output

Rate and Range Li mi ter 1

Merge

Merge3

Merge

Merge2

Merge

Merge1

u1if (u1)

else

If

Action

Cat Warmup Scalar

Cat Warmup Threshold

Cat Warmup Enable

Scalar f or Rate Limits

Scalar f or Costs

Contro l wi thout Fl ag from Engi ne

7Time Step

6Mi n Rate L imit for Cost Scalar

5Min Rate Limit for Rate Scal ar

4Cat Warmup Threshold

3Use Eng Cat Warmup Fl ag

2Cat Warmup Scal ar

1Eng Cat Warmup Enabl ed

VeEOC R_r_SclForRateLimR aw

VeEOC R_r_SclC at WarmupCostRaw

VeEOC R_r_SclCatWarmupCost

VeEOC R_r_SclForRateLim

3Cat Warmup Scalar for Costs

2Cat Warmup Scalar for Rate L imits

1Cat al yst Warmup Enabl ed

singl e(1)

singl e(0)

Action

Eng Cat Warmup Enabled

Cat Warmup Scalar

C at Warmup Enable

Scalar f or Rate Limits

Scalar f or Costs

Use Cat Warmup Flag f rom Eng

Max Rate

Min Rate

Max

Min

input

output

Rate and Range Li mi ter 2

Max Rate

Min Rate

Max

Min

input

output

Rate and Range Li mi ter 1

Merge

Merge3

Merge

Merge2

Merge

Merge1

u1if (u1)

else

If

Action

Cat Warmup Scalar

Cat Warmup Threshold

Cat Warmup Enable

Scalar f or Rate Limits

Scalar f or Costs

Contro l wi thout Fl ag from Engi ne

7Time Step

6Mi n Rate L imit for Cost Scalar

5Min Rate Limit for Rate Scal ar

4Cat Warmup Threshold

3Use Eng Cat Warmup Fl ag

2Cat Warmup Scal ar

1Eng Cat Warmup Enabl ed

VeEOC R_r_SclForRateLimR aw

VeEOC R_r_SclC at WarmupCostRaw

VeEOC R_r_SclCatWarmupCost

VeEOC R_r_SclForRateLim

目标检测道路识别

人机界面（HMI）

ACC警示信息

Confidential

人机界面（HMI）

原型控制器

DriverDriverSimulatorSimulator

Engine HIL Engine HIL PlatformPlatform

Steering HIL Steering HIL

Environment Environment and Sensor and Sensor

ModelsModelsggPlatformPlatform

Driveline HIL Driveline HIL

Braking HIL Braking HIL PlatformPlatform

PlatformPlatform

Electric DriveElectric DriveElectric Drive Electric Drive HIL HIL PltformPltform Powertrain Powertrain

HIL PlatformHIL Platform

Confidential

Virtual Driving EnvironmentVirtual camera, radar, V2VVirtual camera, radar, V2V

Traffic and 3D environment

DIL SimulatorDIL simulation

HMI

P C llRT Vehicle Controls

Confidential

Prototype ControllerBy-wire actuators

In-vehicle communications

Conclusions:

Field testing can be very challenging on cost, time, d fl ibili d i i ibland flexibility, and sometimes even impossible.

Virtual driving environment can be in high fidelity g g ywith models of road, traffic and weather, and environmental sensors of radar and camera (and (V2V next).

It is proved to be very effective in the early design It is proved to be very effective in the early design, test and verification of vehicle intelligence.

Confidential