development effort of evaluation technology in …...44 development effort of evaluation technology...

　

42

Development Effort of Evaluation Technology in Automatic Parking System

Masao SENDO Nobuhito YOKOTA

Susumu TAKAHASHI

Shinichi SUGIURA


　 Nowadays, every automotive manufacturer is working toward practical use of autonomous driving in 2020 as a molestone, and the range of automation is expanding year by year. Accordingly, functions of autonomous driving are becoming more complicated, and in the development of sensing products, it is necessary to perform an evaluation assuming a huge market environment scene including defect factors so as to ensure market quality.　 This paper introduces an ideal evaluation process and development efforts of an evaluation technology to realize it for evaluation of the target recoginition performance in the automatic parking system we are currently working on.

In Japan, “Public-Private ITS Initiative /Roadmaps 2017” was announced in May, 2017. (Fig. 1) Adopting the automated driving level defined by SAE (Society of Automotive Engineers), Japan shows the stance to follow the international movement. Various automotive manufacturers already announced that they will launch an autonomous driving vehicle (level 3) in 2020. As for full autonomous driving (level 4), the activities by the private and public sectors as one toward the establishment of a system has been vitalized. When a driving assistance system began appearing on the market, the products which partially support “acceleration, deceleration, and steering” was the mainstream. However, in accordance with the change from “assistance” to “automatic”, higher safety is required from the market while the number of sensors such as image camera, millimeter wave radar and others increases and their detection range expands. In order to secure this high safety, collection of vast amounts of field driving data, and advanced analysis are needed.

Introduction 1.

Abstract

自動運転システムに係るロードマップ①：自家用自動運転車（1）

・準自動パイロット（SAEレベル2）

・SAEレベル2

・自動パイロット（SAEレベル3）

・完全自動運転（SAEレベル4）

【官民（S

IP1

含む）】

大規模社会実証に

向けた検討

【官民】

制度面等の

調査検討

【官民（S

IP1

含む）】

高速道路での

大規模実証実験

制度整備方針

（大綱）策定

交通事故死大幅削減、交通渋滞の緩和、物流交通の効率化、高齢者等の移動支援

世界一安全で円滑な道路交通社会

世界最先端のＩＴＳ

高速道での　自動運転

一般道での　自動運転

高度安全運転支援システム（仮称）

安全運転サポート車

【民間】研究開発・　　　実用化の推進

【関係各省】安全運転サポート車（サポカーS、サポカー）の普及啓発

【民間】研究開発・実用化の推進




【関係各省】制度の詳細検討、必要な制度改正等

準自動パイロット（L2）市場化

短期中期長期年度2017 2018 2019 2020 2021 2022 2023～25 2026～30

自動パイロット（L3）市場化※

高速道路完全自動運転（L4）市場化※

一般道路自動運転（L2）市場化

【民間】市場展開、　　　更なる高度化

【民間】市場展開

高度安全運転支援システム（仮称）市場化




【自家用車】

Fig. 1 Roadmap of Autonomous Driving System

This paper introduces establishment of efficient and effective evaluation process and development effort for evaluation technology with the case study.

43

DENSO TEN Technical Review Vol.1

VICT Enginering Group has launched the products in a wide range of field such as sensing, HMI (Human Machine Interface), and telematics. In sensing field, we proceed with the development of a target recognition technology with camera in addition to millimater wave radar as a sensor which functions as “eye” for realizing autonomous driving. These system consists of sensor itself and recognition algorithm. Naturally, as for developing better products, it is important to understand the required performance of the target system, and to develop it on the assumption how it is used in the market. At present, we are developing the sensing technology of automatic parking system, in which the performance responding to various parking scene is required. In ensuring performance for the system with the prefix “automatic”, major manufacturers already take action such as driving over millions km in the market field and establishing evaluation envirounment by large-scale simulation. These approaches are far larger than conventional product evaluation scale, and they are steadily expanding in future.(Fig. 2) According also to global trend, drastic evolution is required for the way of evaluation.

Fig. 2 Evaluation Range in Vehicle Automation Level

Autonomousdriving

Full Autonomousdriving

Full Autonomous

driving

System

range

Driver support type autonomous driving

Reference) Ministry of Land, Infrastructure and Transport：The study on auto-pilot system interim summary

（Autom

ation

）

（Evaluation range

）

100％

Large

Independent systemACC

Collision damage reduction systemlane keeping system

System sophisticatedFurther integration ,high accuracy

System integrationACC+

lane keeping Assist

We defined ideal evaluation as follows.① Basic performance evaluation (bench)② Market simulation evaluation (field)③ Market environment evaluation (market)

Fig. 3 shows the flow of evaluation for the target recognition performance by camera.①Evaluate a camera alone in limited environment (movement and brightness etc.) in basic performance evaluation. ②Perform evaluation which simulates market environment in evaluation field with mounting the camera on vehicle in market simulation evaluation. ③Evaluate total performance under actual environment (market) in market environment evaluation. In recent years, as for the problem of sensing development, serious problems are found at the final stage in V shape developmental process. When problem caused by basic design is found in the final evaluation stage, the range of design change expands and huge rework and development delay occurs. Then we accelerated the evaluation contents assuming market environment in the pre-stage evaluation (basic performance evaluation and market simulation evaluation), as shown in Fig. 4, and incorporated it into the basic performance evaluation, and clearly defined the index for the completion level in each stage in order to prevent a serious problem in the final stage. Also, we defined it as the ideal target to realize evaluation which covers various market environments in market environment evaluation.

Fig. 3 Product Evaluation Process

Bench　evaluation

Parking scene which includes various “trace”, “object”, and “environment” affects parking performance.Evaluate performance by simple evaluation condition and improve the quality.

A unit of camera Plural camerasActual vehicle

（Plural cameras）

Evaluation environment quantified error factor

③Market environment evaluation

Market adaptation (robustness)

①Basic performance evaluation②Market simulation

evaluation

Ideal evaluation process 3.

Autonomous driving technology and evaluation

2.

　

44


Fig. 5 3-axis Evaluation Equipment

Fig. 6 3-axis Evaluation Equipment Specification

・Synchronous control of Ｘ,Ｙ and θ axis・Camera mount, 360-degree rotation・Adjustment of camera angle （Pan / Tilt / Roll）

■Camera moving part

■Data output

■User interface　　　

・Setting of origin, moving trace, and moving velocity

・CAN output of moving position and velocity

・Trace drawing by dedicated software and operation

10m

variable

10m

θ axis

Plane view

Main function

Side view

Y axis fram

e

X axis frame

Y axis fram

e

② Market simulation evaluation : In this phase, we confirm the performance under actual environment by using a vehicle equipped with camera, and recognition target object after ensuring the safety. Therefore, safe and quantitative evaluation is needed under the environment assuming the usage of products in the market and driving pattern. DENSO TEN newly established a test field in 2017. (Fig. 7) A various type of parking frames which exist in the market are reproduced in this field. Also, evaluation vehicles are equipped with highly accurate GPS (Global Positioning System) and LiDAR (Laser Imaging Detection and Ranging), and evaluation can be performed by comparing the actual position of detecting object with detected result.

We explain target recognition performance evaluation on which we work right now to realize this evaluation process. As for the evaluation, safety is the first priority, and the method should be quantitative and efficient in order to judge the product function correctly. Therefore, we develop various evaluation technique, equipment, and tool to adjust each evaluation stage and we advance evaluation and analysis. ① Basic performance evaluation: We evaluate it by image by using a standard target whether a target recognition method itself operates as planned or not. In this phase, it is important to quantify the right recognition value and affecting error factor. If the position relation between sensor and recognition target object is clear, right extracting the problem is possible by comparing the recognition results. Then, we developed the equipment which reproduces the vehicle trace by moving a sensor dynamically. (Fig. 5) As shown in Fig. 6, the sensor is mounted on the intersection point of the frame which operates in X axis direction and Y axis direction. This equipment reproduces sensor motion mounted on vehicle by concurrent control of 3 axes with adding rotation function of θ direction. As the position of intersection is calculated in absolute coordinate, we can compare actual position relation with the recognition result by grasping position relation between the origin and the recognition target object. We can verify the design feasibility and the robustness with use of this equipment.

Fig. 4 Front-Loading of Evaluation

System design

Function design

Market environment evaluation

Market simulation evaluation

Basic performance evaluation

Algorithm design

Evaluation atdesign stage

Approach by evaluation technology 4.

45


For example , as shown in Fig. 8, Regarding the vehicle as the object, colar, shape, and size are described as property, and behaivour such as velocity, trace, and others are defined as method, which leads to systematization. For the object of each catalog, we can create various type of evaluation scene as shown in Fig. 9 by selecting and combining property, method, and paramater.

When we simply try different combinations with all objects in all catalog, a huge number of pattern are made. In the system which automatically judges the market environment, the quality must be secured in these evaluation scenes. However, performing all these patterns is unrealistic. Then, we challenged to narrow down the paramater based on the level of difficulty. For example, detecting parking frame is realized by detecting line segments which are edge extracted by the contrast (brightness difference) between line.

Fig. 8 Example of Object Systematizaton

Fig. 9 Evaluation Scene Created by Catalog

Vehicle catalog

Vehicle object

…

PropertyBody colorShapeSize

…

Method

Param

eter

VelocityTrace

…

Parking frame catalog

Parking frame object

Person/Roadside object/Weather　eｔｃ.

……

PropertyShape of lineSize of lineColor of lineColor of road surface

…

Parameterロ type/Ｕ type…Thickness：cm/length：cm

R:G:BR:G:B

…

…

Weather

Vehicle

Trace

Arrangement

Road

surface

Person

Roadside

object

Parking

frame

Fig. 7 Evaluation Field of Automatic Parking System

③ Market environment evaluation : In this phase, suitability for actual market environment is performed. It is a key how we encompass the scenes including error factor and evaluate under the market environment where a huge number of scenes exist. We introduce the effort to improve the completeness of market environment scene and efficiently evaluate the target object. Firstly, we have comprehensively extracted recognition target object which may exist in the market , and the object which affects to recognition by using brainstorming method based on past image data etc. Then, the extracted objects are grouped by type. This group is called “catalog” for convenience sake. Recognition target object which is categorized to each catalog is called object. We define colar, shape, and size as property, and define things having behaivour such as velocity, trace, and others as method. Some of property and method are quantitative and converted into numerical values.

Kobe Test Field 　Simulation Evaluation of Market Environment (Automatic Parking)

Combination of road surface and line color of parking frame

Asphalt road surface Concrete road surface

Handicap mark GutterSpecial road surface

Brick / Lawn / Iron plateStraight 100m Grid line Slope

Stone pavement

132m

100m

107m

坂路

駐車区画

駐車路面バリエーション

区画線種

特殊路面駐車枠

グリッド

直線路

←至

本社Ｄ棟

　

46


As for the development of automatic parking system, we perform efficient and effective evaluation by using evaluation technology. In future development of automatic parking system, the expansion of evaluation range is expected in accordance with system advancing and complicating. We will work on not only evaluation of actual environment but also using virtual technology and AI (Artificial Intelligence) so as to ensure the quality.

・MySQL is a trademark or registered trademark in U.S.A and other countries of MySQL, Inc. and Sun Microsystems, Inc. in U.S.A.

ReferenceThe prime minister’s official residence : Public-Private ITS Initiative / Roadmaps,pp.67 [2017]Ministry of Land, Infrastructure and Transport :The study on auto-pilot system Interim Summary [2013]

Conclusion 5. As shown in Fig. 10, we investigated the past image data and paint color of road surface, and made color chart. Then, we measured the brightness with the type of lighting and illuminance which was expected in the market environment. It is found that it is difficult to extract the edge by the combination of small brightness difference for each color measured. When the white line is used as the standard in Fig. 10, the brightness difference was smallest for light-blue color of road surface, and then detecting it was difficult. If extracting edge in these combination is possible, combination with low level of difficulty is not necessary to be performed, which enables to reduce the number of pattern as for the property and paramater of color. We proceed with efficient and effective evaluatuon by performing this kind of verification and focusing about other paramaters. Control and operation are also important because we expect that measurement data size will be huge even after focusing. As for the control and the operation, system for data search from various aspect and processing it with high speed is necceary in order to proceed with the data analysis and simulation. Then, we developed an original data base. By adopting PHP (Hypertext Preprocessor)+ MySQL as data base application with WEB-based GUI (Graphical User Interface), manager and user can operate it without burden. As for NAS (Network Attached Storage), we must prevent the data access speed from decreasing in accordance with increase of data. We introduced scall-out NAS to solve this problem which easily improves the system performance when added. (Fig. 11)

Fig. 11 Scale-out NAS

Inside Appearance

Fig. 10 Combination Narrowed Down by Color Evaluation

線/路面色想定輝度差

【Verification process】

Market environment

Easy

Difficult

Color of “line and road surface” Brightness of “color”

＝？×？Blue×Yellow

Green×Red　Light blue×Gray

White×Black

試験No.

輝度

D. Extract minimum combination

A：Create color chart of line and road surface

C: Analysis of brightness difference

蛍光灯太陽灯水銀灯ナトリウム灯LED灯

太陽灯

B:Brightness evaluation by chart D: combination difficulty for color of line/road surface

Combination

White×

Black

White×Red

White×

Green

White×Gray

White×

Light b lue

Picture Difficulty

A. Extract color of line and road surface in market

B. Calculation of brightness in market

C. Verify combination of brightness difference

47


Profiles of Writers

Susumu TAKAHASHI

VICT Engineering GroupSystem Evaluation Dept

Nobuhito YOKOTA


Masao SENDO


Shinichi SUGIURA


development effort of evaluation technology in …...44 development effort of evaluation technology...

Documents