Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Japan Display Inc.
2018/03/02
車載向け高速カメラモニタリング システム(CMS)の開発
Takayuki Nakanishi,
Kazunori Yamaguchi,
Toshiharu Matsushima,
Fumitaka Goto,
Daichi Suzuki, and Satoshi Matsushima
Development of
Fast Response,
Low Latency
Real-time Camera and Display System
for Automotive Applications
2 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Outline
1. Introduction
• Background
• Primary experiment
2. Subject-Evaluation Results of the Latency
• Latency time test
3. Technologies of Our Prototype
4. Summary
3 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
1. Introduction
• Background
• Primary experiment
2. Subject-Evaluation Results of the Latency
• Latency time test
3. Technologies of Our Prototype
4. Summary
4 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Background
Opportunity:
• Improving fuel economy
• Reducing blind area
• Image recognition
• Designing flexibility
100 ms Delay
10 ms Delay
5.6 m
A few tens of centimeters
Moving Distance at 200 km/hr
200 ms Delay 11 m
5 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Primary Experiment - Setup
DSLR
60 fps (estimation)
latency: 280~320 ms (removed)
CMS120
120 fps
latency: <40 ms CMS240
240 fps
latency: <10 ms
Camcorder
60 fps (estimation)
latency: 40~80 ms
Tripod External display of DSLR
AMOLED, by Wi-Fi connection
CMS240
TN-LCD, HDMI
External display
for camcorder
IPS-LCD, HDMI Side
mirror
CMS120
in-house
IPS LCD,
stripe
RGB
Difficult to compare!
6 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Observation – In the cockpit
Optical mirror
CMS120
CMS240
Camcorder
• Ambient light at 100k Lux
• Comparing to side mirror, display brightness is too low
and barely see nothing
• Strong surface reflection occurs on KAMUY’s display
Problems
Noticeable delay using camcorder : 40-80ms delay
Noticeable motion blur : 240Hz case looks good
Strong surface reflection : need anti-reflection
Insufficient panel brightness : more luminance!
7 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Observation – Night view
Optical mirror
CMS120
CMS240
Problems
Poor contrast using digital system : AE control is important
Dark & poor image using optical mirror
Benefit
Good visibility at camcoder : Image processing effect
Camcorder
8 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
1. Introduction
• Background
• Primary experiment
2. Subject-Evaluation Results of the Latency
• Latency time test
3. Technologies of Our Prototype
4. Summary
9 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Latency Time Test
Participants
• 20 persons
Condition
• Delay from 0 to 117 ms
(0 to 7 frames)
• 6 type of Images, Random
10 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
17 33 50 67 83 100 117P
erc
en
tage
Latency Time (ms)
Accuracy rate of latency test : Video 1
Wrong
No Diff
Correct
Video 1: Waving Hands
• Accuracy rate significantly increases after latency time of 50 ms
• Fast and periodic moving object is easy to judge latency differences
Speed: 13.18 deg/sec
Periodic: Yes
Reference point: Yes
Can detect time difference correctly
Can’t detect time difference correctly
Say “same” means wrong answer
11 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Video 2: Person Passing By
• The accuracy rate is unstable, while “No diff” is around 30% until 117 ms
• Judgment for slowly moving object seems not easy
Speed: 7.14 deg/sec
Periodic: No
Reference point: Yes 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
17 33 50 67 83 100 117P
erc
en
tage
Latency Time (ms)
Accuracy rate of latency test : Video 2
Wrong
No Diff
Correct
12 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
17 33 50 67 83 100 117P
erc
en
tage
Latency Time (ms)
Accuracy rate of latency test : Video 3
Wrong
No Diff
Correct
Video 3: Train Moving Quickly
• The fastest moving object and no reference point in this movie
70 km/hr speed and train becomes blurred and looks all the same
• Obviously it’s a difficult scenario.
The percentage of “No Diff” is over 40% for all delay
Speed: 23.33 deg/sec
Periodic: No
Reference point: Insignificant
13 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
17 33 50 67 83 100 117P
erc
en
tage
Latency Time (ms)
Accuracy rate of latency test : Video 4
Wrong
No Diff
Correct
Video 4: Train Moving Slowly
• Two trains run in opposite direction at a speed around 30 km/hr
• Lots of reference point exists in this video,
e.g., the relative movement of two trains, or fixed electric pole
• The tester can judge the latency easily over 50 ms
Speed: 1.74 deg/sec
Periodic: Yes (relative movement of two trains)
Reference point: Yes
14 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Experimental Result – Latency time test -
Experiment Operation Type ND of Latency
Waving Hands Indirect Periodic
And w/ reference point
50 ms
Train Moving Slowly
Indirect Periodic
and w/ reference point
50 ms
Train Moving Quickly
Indirect Non periodic
And w/o reference point
> 100 ms
People Passing By
Indirect Non periodic
And w/o reference point
> 117 ms or unstable
• People can detect time difference from around 50 ms
• Without reference point, noticeable difference increase Over 100 ms
System Latency Requirement: < 50 ms
15 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Demands for CMS - from primary and latency time test -
Low Motion Blur
Low Latency
Maximum allowance: 50 ms
High Luminance and High Contrast Ratio
Need more brightness in daylight
Need more contrast at night driving
16 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
1. Introduction
• Background
• Primary experiment
2. Subject-Evaluation Results of the Latency
• Latency time test
3. Technologies of Our Prototype
4. Summary
17 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
High Frame Rate Driving
60 Hz
240 Hz
frame1
Image sensor
exposure
Panel driving
frame1
0
768 Line
Exposure time
frame2 frame3
frame2 frame3 time frame4 frame5
frame4 frame5
frame6
frame6
frame7
frame7
frame8
frame8
frame1
Image sensor
exposure
Panel driving
0
Exposure time
frame1
time frame2
768 Line
768 Line
768 Line
Low Motion Blur
Clear Image is Captured by Shorter Scanning Time
18 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Fast Response Time LC
BET(blur edge time) vs. frame rate
clear
blur
Low Motion Blur
Improve Blur by Using SLC-IPS
Short-range Lurch Control In-Plane Switching (SLC-IPS) T. Matsushima et al,
“New Fast Response Time In-Plane Switching Liquid Crystal Mode”,
SID 2015 digest papers, 43.2, pp 648-651, 2015
19 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Synchronized Capturing and Playback Low Latency
4.536 ms Latency from Imager to Panel
1frame
System processing
Image Sensor
VSYNC
Image sensor
exposure
Panel driving
Panel VSYNC
Image sensor
data out
1frame
Line
0
768
Line
0
768
Line
0
768
Line
0
768
Exposure time
4ms
Output delay
5.05us
System delay
179us
1H time
5.37us
2frame 3frame
2frame 3frame time
time
Output delay (1H)
5.05us
1line
2line
3line
System delay (1H)
179us
1line
2line
3line
1Horizontal time
5.37us
1line
2line
3line
4.356 ms
20 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Under 100,000 Lx
HSV color space for RGB display
New HSV color space for RGBW display
Saturation S
Hue H
Low power mode brightness
outdoor mode brightness
Brightness (V)
Excellent Visibility under Brighter Environment
“Outdoor Mode” of RGBW Technology
Original picture
RGBW Outdoor mode
Conventional RGB panel
High Luminance
21 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
R G B W
WM IP
LED Dr. ・ ・ ・
Backlight
RGBW Display
Backlight illumination Backlight illumination
Local Dimming with RGBW Technologies
1. New Image Processing IP for local dimming
2. Special Light Guide Design
for edge-lit backlight
Improve Contrast Ratio by Using Local Dimming
High Contrast Ratio
22 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Over View of Our Prototype CMS
Specification
Imager size 1/2.3 inch
Display size 6.5 inch diagonal
Pixel number 1,280 x 768
Resolution 231 ppi
Pixel arraignment RGBW
Luminance 2,000 cd/m2 @OutdoorMode
Contrast ratio 20,000:1 with local dimming
Color gamut 76.5%
Frame rate 240 Hz
Note IPS-NEO
Low Motion Blur
Low Latency
High Luminance and High Contrast
23 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
1. Introduction
• Background
• Primary experiment
2. Subject-Evaluation Results of the Latency
• Latency time test
3. Technologies of Our Prototype
4. Summary
24 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Comparison
R G B R G W
R G W R G B
R G B R G W
R G W R G B
Features Improvement Conventional RGB Display
Prototype
1
• Quick Response Time
Tr+Tf (25deg.) = 20 ms Tr+Tf = 8.3 ms
Tr+Tf (-20deg.) = 146 ms Tr+Tf = 51 ms
• High Frame Rate 60 Hz 240 Hz (4times faster)
2 • Low Latency • Direct Synchronization
from Imager to Display Over 100 ms 4.356 ms
3
• • High Luminance
1000 cd/m2 2000 cd/m2 (6.5inch-demo)
• Local Dimming • Dynamic Contrast
improvement 1000:1 20000 : 1
25 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Conclusion
The ultra-low latency CMS has been developed, by adopting
Low Motion Blur
SLC-IPS technology
High speed scan panel
Low Latency
Synchronization of imager and display panel
High Luminance and High Contrast Ratio
RGBW technology with outdoor mode
Local dimming technology
These technologies becomes very important for safe driving.
By adding functions to CMS, the safety far beyond conventional optical mirrors will be provided in the near future.
26 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
Thank you for your kind attention.
27 Copyright © 2017-18 Japan Display Inc. All Rights Reserved.
本資料は、『電子ディスプレイの人間工学シンポジウム2018』のために作成されたものであり、
当社の発行する株式その他の有価証券への勧誘を構成するものではありません。本資料に記載される業界又は市場動向に関する情報は、現時点で入手可能な情報に基づいて作成しているものであり、当社がその真実性、正確性、合理性及び網羅性について保証するものではありません。また、本資料に記載される当社グループの計画、見積もり、予測、予想その他の将来情報については、現時点における当社の判断又は考えにすぎず、実際の結果は、国内外の個人消費その他の経済情勢、為替動向、スマートフォンその他の電子機器の市場動向、主要取引先の経営方針、原材料価格の変動等により、本資料記載の内容と大きく異なることがあります。