achieving high test coverage for an asil system · • increased test coverage →increased...

Achieving high test

coverage for an

asil system

2019-05-07 MATTIAS SAVINAINEN - VOLVO CAR CORPORATION - VECTOR TESTING SYMPOSIUM 2019

1. Background• Why test automation

• Why virtual verification

2. Way of working• Tools used

• Test setup

3. How to scale up test coverage• vTESTstudio features

4. Virtual verification with vehicle models

5. High coverage virtual verification

6. Conclusions and results

2019-05-07

agenda

MATTIAS SAVINAINEN - VOLVO CAR CORPORATION - VECTOR TESTING SYMPOSIUM 2019

• Increased test coverage → Increased quality.

• Saves time on running tests.

• Reproducibility.

2019-05-07

The road to high coverage verification


Time should be spent on test

development, not running

tests.

Why test automation?

• Reduces number of needed test vehicles

• Can verify functions in early project phases.

• Reproducability

Why virtual verification?

2

7

11

1617

18

2014 2015 2016 2017 2018 2019

Active damping – number of software vehicle variants by year

Volvo XC90 launched.

Beginning of a new vehicle platform.

2019-05-07

software growth


2019-05-07

A change in test strategy


Manual testing, one variant

Manual testing, several variants

Automated testing, several variants

Automated testing, one variant

Virtual verification, vehicle models

2011 2013 2015 2016 2019




• Test setup





2019-05-07

agenda


2019-05-07

overview of software’s used


vTESTstudio is a powerful development

environment for creating automated ECU tests.https://www.vector.com/int/en/products/products-a-z/software/vteststudio/

“CANoe is the comprehensive software tool for

development, test and analysis of individual

ECUs and entire ECU networks.”https://www.vector.com/int/en/products/products-a-z/software/canoe/

“CarMaker can model real-world test scenarios

in the virtual world. It has real-time capable

models for vehicles, roads, drivers and traffic.”https://ipg-automotive.com/products-services/simulation-software/carmaker/

• Vector CANoe

• Vector vTESTstudio

• IPG CarMaker

2019-05-07

vt system


test setup


Signal exchange Signal exchange

CarMaker CANoe VT System

vTESTstudio ECU

Test object

Sensors




• Test setup





2019-05-07

agenda


Topics

• Function definitions

• Test case definitions

• Parameter lists

• Variant properties

2019-05-07

Scaling up testing with vteststudio


vTESTstudio is a powerful development

environment for creating automated ECU tests.https://www.vector.com/int/en/products/products-a-z/software/vteststudio/

2019-05-07

Vteststudio overview


2019-05-07

Example – testing different vehicle variants


”Check that a certain function

behaves correctly in X number

of variants.”

2019-05-07

Scaling up testing – function definitions


My first test case

MyFirstTestFunction

2019-05-07

Scaling up testing – test case definitions


2019-05-07

Scaling up testing – parameter lists


Parameters can either be

added as local values or

be created as parameter

list which are global and

can be used in other

tests also.

2019-05-07

Scaling up testing – parameter lists


Parameter lists can be edited directly in vTESTstudio or in Microsoft Excel.

vTESTstudio Excel

2019-05-07

Scaling up testing – test case execution


Test cases are executed from CANoe.

One test case will be created for each

call (variant).

2019-05-07

Scaling up testing – variant properties


Examples of useful variants

- Vehicle configuration

- Country / Market region

- ASIL level (ISO26262)

Variant properties are assigned to test cases in

vTESTstudio.

Test execution based on these properties are

done in CANoe.

”Today I need to test new software on a

Volvo V90 for EU market and run all

ASIL-A test cases”

• Which tests you run is configured in CANoe.

• The list of tests will update to only include those that matches the chosen properties.

2019-05-07

Scaling up testing – variant properties


The VT System is used to create electrical faults on sensors

- Short circuit

- Short to battery

- Short to ground

- High/low voltage

Two test case definitions are used to cover 72 fault injection tests on the ECU

2019-05-07

Test case example – electrical fault injections





• Test setup





2019-05-07

agenda


Topics

• Test case example – Active damping system

• CarMaker integration with CANoe

• CarMaker test cases scaled up using vTESTstudio

• Example of repeatable tests

Virtual verification with vehicle models


“CarMaker can model real-world test scenarios

in the virtual world. It has real-time capable

models for vehicles, roads, drivers and traffic.”https://ipg-automotive.com/products-services/simulation-software/carmaker/

Active damping - overview


• The ECU actively adjusts the dampers 500 times per second.

Customer benefits

• Increases comfort and handling.

• Enhances vehicle dynamics• Comfort mode

• Dynamic mode

• Eco mode

• Off-road mode

2019-05-07

Active damping – main features


ECU

Active

damping

algorithm

Sensors

Network

signals

Current[A]

outputDampers

Canoe – carmaker integration


Fast Data

eXchange protocol

(FDX)

CarMaker CANoe

Functional Mock-up Unit (FMU)

ECU

Active

damping

algorithm

Sensors

Network

signals

Current[A]

outputDampers

2019-05-07

Video example – Pitch


Will be played in full screen

2019-05-07

Video example - roll



test setup with carmaker


Closed loop test environment.

All data is being sent continuously at 500Hz.SensorsECU

Test object

test setup with carmaker


1. Vehicle motion data is sent

from CarMaker to CANoe.

2. Scripts in CANoe converts

vehicle motion data to analog

outputs from the VT System

3. The analog outputs are sent to

I/O ports on the ECU.

4. The VT System reads the

damper current output and sends

it to CANoe.

5. The damper current output is

sent to CarMaker.

6. The vehicle model calculates

new damper forces.

SensorsECU

Test object

Example – Evasive Maneuvre



Example – Evasive Maneuvre - canoe






• Test setup





2019-05-07

agenda


• Emergency braking is on going

• The vehicle will pitch forward

• The active damping software will adjust the dampers accordingly.

2019-05-07

Test case example – brake grip control


2019-05-07

When everything is connected


CANoe

vTESTstudio

VT System

CarMaker

High coverage

virtual verification

• Features used in vTESTstudio• Test case definitions

• Function definitions

• Variant properties

• Parameter lists

2019-05-07

When everything is connected - vteststudio


2019-05-07

When everything is connected - video






• Test setup





2019-05-07

agenda


2019-05-07

Conclusions - Test automation


2011 2015 2019

1 variant, manual tests 3 variants, automated tests

~1-2 weeks

1 variant, automated tests

~55 minutes ~60 minutes per variant

Active damping software, number of tests and time spent

• Verification in real vehicle• Planning

• Booking

• Preparation of vehicle

• Travel time

• Etc...

2019-05-07

• Virtual verification• Much faster to increase test coverage

• Time can be spent on implementing new test cases.

• Cost efficient.

Conclusions – virtual verification





2019-05-07

Conclusions - results





• Reproducibility





2019-05-07

Conclusions - results





• Reproducibility


Or the short summary:

It saves time and money.

2019-05-07

Final words – three things to remember


The importance of test automation.

2019-05-07




The benefits of virtual verification.

2019-05-07




The benefits of virtual verification.

The software quality increase of high coverage

testing.

achieving high test coverage for an asil system · • increased test coverage →increased...

Documents