all-weather vision for automotive safety: which spectral band? · camera spectral band day fog ,...
TRANSCRIPT
-
All-weather vision for automotive safety: which spectral band?
N. Pinchon, O. Cassignol, A. Nicolas, F. Bernardin, P. Leduc , J-P. Tarel, R. Brémond, E. Bercier, G. Julien
BerlinSeptember 11-12 2018
-
AWARE
2
AWARE: WHAT DOES IT MEAN ?
Clear weather,fog, rain, snow
Urban, country roads and highways (transportation)
Runways (aeronautic)
DAY AND NIGHT
All Weather
Enhanced vision
All Roads
-
3
AWARE: KEY FIGURES
Key partners
10
2
2
36
5.5M€
Applicative domainsTransportation & aeronautic
Demo modules forecastedM1 for premiun vehicleand aeronautic
M2 for all vehicleMonths of projectFrom june 2014 to june 2017
R&D Including 1.7M€ Financed by the french competitiveness,industry and servicesdirectorate (DGCIS) andIsère General Council
-
4
WE ARE AWARE
-
5
Are you able to drive or to operate safely
with such visibility conditions
AWARE ORIGIN
OF COURSE NOT !
HERE IS, THE "AWARE" PROJECT ORIGIN !
-
6
AWARE ORIGIN
French Road Safety Observatory (ONISR): "Les accidents corporels de la circulation 2014 –
Recueil de données brutes", May 2015
INJURY
CASUALTIESFATALITIES
NIGHT 32% 41%
WET ROAD 20% 20%
ADVERSE
WEATHER21% 23%
AUTOMOTIVE
-
7
AWARE: THE STAKE
Allow the safe assisted and
automated driving of vehicles,
day and night.
INCREASE ROAD SIGNS AND OBSTACLES PERCEPTION
IN ALL-WEATHER BY USING A LOW COST IMAGE SENSOR
ADAS FUNCTION
Pedestrian detection
Vehicle shape
recognition
Traffic signs recognition
Road marking
detection
-
wavelength
4 IMAGING TECHNOLGIES UNDER EXPERIMENT8
CAMERA SENSORSPECTRAL
BANDOPTICS
Visible
RGB
CMOS – SXGA
(1280 x 966)
3x8 bits, pitch
4.2 μm
0,4-0,65 µm
HFOV = 54°
VFOV = 40°
F-number = 2
Extended
NIR
CMOS – SXGA
(1280 x 1024)
10 bits, pitch
5,3 μm
0,4-1 µm
HFOV = 39°
VFOV = 31°
F-number = 2.9
Extended
SWIR
InGaAs – VGA
(640 x 512)
14 bits, pitch
15 μm
0,6-1,7 µm
HFOV = 39° VFOV =
31°
F-number = 1.8
LWIR
Microbolomet
re - VGA (640
x 482)
14 bits, pitch
17 μm
8-12 µm
HFOV = 44° VFOV =
33°
F-number = 1.2
VISIBLE RGB1
1
EXTENDED NIR2
2
EXTENDED SWIR3
3
LWIR4
4
-
OUTDOOR TEST
A75 – Col de la Fageole
Cerema test’s ground
FOG TUNNEL
Clermont Ferrand
Cerema facility
FIELD TEST : SET-UP9
-
OUTDOOR TEST FOG TUNNEL
FIELD TEST : RANGE CALIBRATION10
-
FIELD TEST : OUTDOOR SCENARIOS11
SCENARIO WEATHERVISIBILITY
DISTANCE
AMBIENT
LIGHT
TEMPE-
RATURE
7Day, heavy fog,
light snow75 m 5032 lux -1°C
11Day, heavy fog
and snow69 m 1096 lux -1°C
22
Day low light,
heavy fog and
snow
75 m 104 lux -0.8°C
26 Night, heavy fog 99 m 0 lux +1.5°C
33 scenarios
Under test
=> 4 most relevant
-
EX SCENARIO #1 : DAY, CLEAR WEATHER 12
-
EX SCENARIO #7 : DAY, HEAVY FOG, LIGHT SNOW13
-
EX SCENARIO #26 : NIGHT,HEAVY FOG, 14
Vehicle
Front
Vehicle
Back
-
In adverse conditions:LWIR detect 3x times longer
15
EXPERIMENTAL RESULTS:PEDESTRIAN DETECTION
CAMERAFOG DENSITY
For “visible” pedestrian
detection at 25m
Visible
RGBModerate (visibility range = 47 ± 10 m)
Extended
NIRHigh(visibility range = 28 ± 7 m)
Extended
SWIRHigh(visibility range = 25 ± 3 m)
LWIR Extreme(visibility range = 15 ± 4 m)
-
16
EXPERIMENTAL RESULTS: VEHICLE DETECTION & RECOGNITION
DETEC-
TION
RECO-
GNI-
TION
TASK VISIBLE LWIR
In adverse conditions:VIS/NIR/SWIR detect Headlight,LWIR recognize shape
-
17
EXPERIMENTAL RESULTS: TRAFFIC SIGNS RECOGNITION
#1
DAY, CLEAR
WEATHER
#7
DAY, HEAVY
FOG, LIGHT
SNOW
VISIBLE NIR SWIR
In adverse conditions:NIR provides best SNRLWIR can not recognize
-
EXPERIMENTAL RESULTS: ROAD MARKING DETECTION
18
CAMERA
SCENARIO
22Visi 75m
SCENARIO
26Visi 99m
Using Headlight
Visible
RGB53m 53m
Extended
NIR60m 53m
Extended
SWIR63m 53m
LWIR - -
In adverse conditions:VIS/ NIR /SWIR equivalent using additional lighting
-
CONCLUSIONS19
ADAS FUNCTION
CAMERA SPECTRAL BAND
DAY FOG, USING HEADLIGHTS
CAMERA SPECTRAL BAND
NIGHT FOG, USING HEADLIGHTSVIS NIR SWIR LWIR VIS NIR SWIR LWIR
Pedestrian, detection + ++ ++ ++++ + ++ ++ ++++
Vehicle lights detection + ++ +++ --
Vehicle shape recognition + ++ ++ ++ - - -- ++
Traffic signs recognition + ++ - -- + ++ - --
Road marking detection + ++ ++ - + ++ ++ -
In adverse conditions:
Visible RGB extended toNIR combined with LWIRprovide the best spectralbands to improve ADASperformances of detectionor recognition such asvehicle, pedestrian, roadmarking, and traffic signs.
-
20
Cliquez et modifiez le titreN. Pinchon1, O. Cassignol2, A. Nicolas2, F. Bernardin3, P. Leduc 4, J-P. Tarel5, R. Brémond5, E. Bercier6,
G. Julien7
1: VALEO, 34 rue Saint André 93012 Bobigny Cedex, France2: SAGEM, 21 Avenue du Gros chêne 95610 Éragny Sur Oise, France
3: CEREMA, 8-10 rue Bernard Palissy 63017 Clermont-Ferrand Cedex 2, France4: CEA / Léti - 17 rue des Martyrs - 38054 Grenoble cedex 9, France
5: IFSTTAR / LEPSIS, 14-20 Boulevard Newton, Cité Descartes, 77447 Marne la Vallée Cedex 2, France6: ULIS, 364 Route de Valence - ZI Les Iles Cordées BP 27 - 38113 Veurey-Voroize, France
7: NEXYAD, 95 Rue Pereire 78100 Saint-Germain-en-Laye, France