Title Slide

Virtualized Embedded Systems for Testing and Development

Andreas Rasmusson, Göran Söderman Scania

Karl Meinke , Hojat Khosrowjerdi KTH-CSC

FFI Dnr 2013-05608

One Image

Agenda

• Driving factors, Needs

• Methods & Tools

• Case-studies

• Results & Experiences

2

Title and Content

3

Virtues explores methods for Quality Assurance oflarge, evolving, embedded software systems,with a focus on sw-development and sw-testing

How:1. Test the unmodified, true ECU-binaries2. Generate tests from the concise formal requirements

Needs:1. A test environment independent of source-code/architechture/OS2. Keep test code volume down

Title and Content

• 3 year project – 2014-2017

• FFI Dnr 2013-05608

• Learning-Based Requirements Testing on SIL and VHIL Platforms

• Software-in-the-loop testing Wincomp

• Virtualised hardware-in-the-loop testing QEMU

• Integrated with learning-based test tool LBTest

4

Scania/KTH project VIRTUES

Title and Content

TestTool

PerformanceTool

SecurityTool

EnergyTool

SILEmulator:WinComp

ECUComponent

SIL Tool Chain

ECU SoftwareComponent

Title and Content

6

VHIL Tool Chain

Title and Content

7

LBTest: Architecture

Incremental Learning Algorithm

Model Checker(NuSMV)

TestCaseGeneratorand Oracle

Model abstraction

Observed output

inon n = 1, 2, …

inon

Mn

verdict vn

Stochastic Equivalence

Checker

counterexample in

Multiple, parallellSUT processespossible

Communication wrapper

System Under Test

PLTL User Requirement

Title and Content

8

SIL: Stubbed x86-compiled app

Appl-SW

OS HL

OS LL(Stubbed)

x86

Wrapper

LBTest

ApplReq

Title and Content

9

VHIL: unmodified srec-file (guest-level)

Wrapper

LBTest

ApplReq

ISS (TCG)

srec

Appl-SW

OS HL

OS LL

Hw-modelCPU, MMU, InterruptCtrl,

Timer, MMIO, CAN …

gdb-server

Title and Content

GDB (x86)10

VHIL: unmodified srec-file (host-level)

Wrapper

LBTest

ApplReq

ISS (TCG)

srec

Appl-SW

OS HL

OS LL

Hw-modelCPU, MMU, InterruptCtrl,

Timer, MMIO, CAN …

gdb-server

Title and Content

We considered SIL requirements testing of 3 ECU applications

• remote engine start (ESTA),

• dual circuit steering (DCS),

• fuel level display (FLD).

11

SIL Case Studies

Learning-based Testing for Safety Critical Automotive ApplicationsH. Khosrowjerdi, K. Meinke, A. Rasmusson Proc. Int. Symp. on Model Based Safety Assessment, 2017

Title and Content

Benchmark LBTest against an in-house test suite using mutation testing

Requirements Capture Challenges

“the electric motor had been on while the second sensor had a flow or noflow, since the last engine restart”

(emotor = on & (sensor2 = flow | sensor2 = noflow) Since ignition = restart)

12

Dual Circuit Steering (DCS)

Learning-based testing of automotive ECUsSophia BäckströmMasters project, KTH

Title and Content

• Tested 7 hours 24 minutes

• 60 state model,

• 800 transitions

• 97% convergence

• 5 requirements errors found – but none were safety critical

13

SIL Test Results for DCS

Learning-based testing of automotive ECUsSophia BäckströmMasters project, KTH

Title and Content

We considered VHIL requirements testing of 3 ECU requirements

• bit-flip Checksummed Memory Area,

• stuck-at in Checksummed Memory Area,

• (TBD) stuck-at in MMIO

14

VHIL Case Studies

Virtualized-Fault Injection with Learning-based Requirements TestingH. Khosrowjerdi, K. Meinke, A. RasmussonSubmitted to ICST2018

Title and Content

15

Bit-flip in Checksummed Memory Area

Inject errors via scripted debugger

guest-level, with symbols

Bit-flip

Title and Content

Conclusions

• Successful case studies on real-products

• Both SIL and VHIL testing can be handled by one tool

• Formal requirements improved Scania’s product understanding

• Instruction Set Simulation is useful for ecu-testing.

• Test the realization. It contains more behavior than the source-code.

Last Page - Blue

Andreas RasmussonSenior Engineer | Systems Development | Scania CV ABPhone: +46 (0)8 553 894 64 Mobile: +46 (0)700 831 914Address, 151 87 Södertälje, Swedenandreas.rasmusson@scania.com

www.scania.com | www.facebook.com/scaniagroup

Last Page/Chapter Divider

18