978-1-4799-5001-0/14/$31.00 ©2014 IEEE 8A6-1

DESIGN FOR ARINC 653 CONFORMANCE: ARCHITECTING INDEPENDENT VALIDATION OF A SAFETY-CRITICAL RTOS

Ahmet Alptekin, Yunus Yilmazer, Ugur Usug, Feyzullah Koca, The Scientific and Technological Research Council of Turkey (TUBITAK), Kocaeli, Turkey

Koray Incki, Ozyegin University, Istanbul, Turkey

Abstract The ARINC 653 specification not only provides

a standard application programming interface for an RTOS, but also specifies how to validate an ARINC 653 based RTOS. ARINC 653 Part 3 Conformity Test Specification specifies test procedures for validation of ARINC 653 Part 1 (Required Services Specification).

Existing ARINC 653 verification suites and packs do not provide platform-independency, maintainability gained by an open source framework, a reliable communication protocol, and automated testing principles at the same time.

This paper introduces a brand new validation suite, GVT-A653 which is platform-independent and ensures conformance to ARINC 653 specification. The suite is based on TETware (trademark of OpenGroup) and builds upon Continuous Integration (CI) principles. It also brings flexibility by providing various protocols including Avionics Full-Duplex Switched Ethernet (AFDX) Network that provides deterministic communication required in avionics applications.

Introduction ARINC 653 standard defines an interface

between Real Time Operating System (RTOS) and application programs. This standard guarantees to conform Integrated Modular Avionics (IMA) partitioning that provides the execution of one or more avionics applications both spatially and temporally independent from each other.

IMA provides many benefits in terms of reducing the unnecessary weight due to the excessive cabling and facilitated reuse of software components in diverse platforms. However, employing IMA in avionics has altered the way avionics systems are

architected. Since the introduction of IMA architecture in 1990s, RTOSs have been essential for avionics system architectures. Relying on a modular RTOS, IMA architecture extremely simplifies development of avionics system software. Avionics system software providers develop their products for a certain RTOS and can reuse it on all compatible systems.

ARINC 653 standard provides agreeing with IMA partitioning that provides spatial and temporal independency of different avionics applications running on the same module [1]. The ARINC 653 specification defines the fundamental set of functionalities that an RTOS must provide. ARINC 653 Application Programming Interface (API) has been at the forefront of safety-criticality and portability issues in avionics domain. Most commercially and militarily available conventional avionics systems utilize ARINC 653 Application Executive (APEX) API as the main API in safety-critical system development.

Software developed against standard API specification might be easily ported from one RTOS to another. The ARINC 653 specification not only provides for a standard application programming interface for an RTOS, but also defines test procedures to demonstrate the compliance of the API behavior of RTOS.

In this paper, a brand new validation suite is proposed for ARINC 653 compliant RTOSs, named GVT-A653. The suite is developed fully adhering to ARINC 653 Part 3 (Conformity Test Specification) [2], which defines test procedures to verify the conformity of RTOS implementations for ARINC 653 Part 1 [3]. GVT-A653 is applied on an in-house developed ARINC 653 compliant RTOS. Even though there are few commercially available validation tools for ARINC 653 conformance testing, these solutions are not sufficient in terms of

8A6-2

independency, infrastructure maintenance, automated testing, and reliable communication.

The main architecture of GVT-A653 is based on TETware [4], which provides an easy-to-use multi-platform uniform test framework into which local, remote, distributed, and real-time test suites can be incorporated. Thus it is unique in employing an open source test framework for architecting an independent scheme. GVT-A653 builds upon continuous integration (CI) [5] principles so as to automatically execute test cases in a predetermined timely fashion. The suite provides different types of communication methods between host and target systems. There are three communication methods that are currently implemented in the suite: Serial Communication (RS-232), AFDX Network [6], and User Datagram Protocol (UDP).

This paper is organized as follows: the architecture of GVT-A653 is explained in detail how it is designed using TETware and several communication methods. Next, it is mentioned how automated testing is adopted to GVT-A653 by using CI principles. Then, some numerical implementation details about the test cases and some experimental results belonging to these test cases are presented. Later, existing solutions of ARINC 653 conformance testing are expressed. Finally, we explain about future works and conclude this paper.

Overall Test Suite Architecture The proposed conformance test suite, GVT-

A653 consists of two parts. First part of the test suite, Host Computer System (HCS) is an application which controls the execution of the tests and runs on a host development environment (Linux or Windows). The HCS communicates with the target system using one of the communication protocols. The second part, Target Computer System (TCS) is a test environment that contains a set of partitions. These partitions execute the tests, evaluate the test results and return a PASS/FAIL token back to the HCS. On both HCS and TCS sides, the communication system is built upon the communication subsystems of the TETware Test Manager that controls the execution of the test cases and provides opportunity to develop new communication types between the host and the target system. As shown in Figure 1, the architecture of

TETware Test Manager consists of two parts: host system and real-time system.

Figure 1. Simple TETware-RT Testing Model

In the host system, there is an executable file named tcc that controls the execution of tests and gathers all the test results available after execution. The list of test cases is given in a scenario file and all test results are written to a journal file. tcc uses a Test Case Manager (TCM) executable file and TETware API that is linked into the TCM. This API library can be modified to support different communication protocols. The communication subsystem might be implemented using UDP sockets, serial line or AFDX ports. Similarly, the target system has a communication library (TETware-RT API Library) that is linked into the executable of each test to communicate with the host system.

GVT-A653 with AFDX Network AFDX Network employs dedicated bandwidth

while providing deterministic quality of service (QoS). Using AFDX Network in GVT-A653 provides a reliable infrastructure as suggested by IMA for avionics applications.

The overall architecture of GVT-A653 with AFDX Network is illustrated in Figure 2. The TCM has to link with the specific subsystem library, AFDX communication API, which simulates the AFDX communication on HCS. This API is responsible for creating the virtual ports and communicating with the TCS. There are two virtual AFDX ports in the TETware API. The first port is named as “Host Sender QP”. It sends a message that includes the

8A6-3

running test case name. The second port is named as “Host Receiver QP” which receives the test results from TCS.

The TCS test environment contains three special ARINC 653 partitions which are responsible for

executing the specified tests and returning the test results to the HCS by working in a coordinated fashion. These partitions are System Partition, Master Partition, and Auxiliary Partition which are respectively detailed.

Figure 2. GVT-A653 Architecture with AFDX Network

System Partition is responsible of the communication between the HCS and the TCS. It also loads next test case upon completion of the previous one. It contains the RT components of the TETware to establish communication. This communication system is implemented using two ARINC 653 queuing ports which are connected to the AFDX ports of the HCS. These ports are named as “Target Sender QP” and “Target Receiver QP” respectively. In addition, there are four ARINC 653 sampling ports in this partition. “Test Name SP” delivers the test case name to the Master Partition. “Sync SP” provides a synchronized communication between the System Partition and the Master Partition. Before the Master Partition sends the test case’s result, it sends a specific message, thus, System Partition can realize that the test is finished.

After the test is completed, “Result SP” receives the result (PASS or FAIL) from the Master Partition. Then, “Reset SP” sends a reset message to Master Partition for soft resetting and executing a new test case.

Master Partition plays the master role in the TCS by executing each test case. There are four ARINC 653 sampling ports used to communicate with the System Partition. At the beginning of the each test, the Master Partition starts in NORMAL mode and waits the System Partition to get the test procedure to be executed. As soon as “Sys Test Name SP” receives the test case name from the System Partition, it starts to execute the test procedure. As the Master Partition executes the test scenarios, it sends a message to synchronize with System Partition

8A6-4

and after sending the test case result. After receiving the reset message which indicates an acknowledgement of delivered test result from the System Partition, it goes to WARM_START mode for executing the next test if exists. In case of the test procedures are associated with partition level recovery actions or inter partition communications, the Master Partition needs additional two sampling communication ports in order to transfer some essential information to the Auxiliary Partition. These ports are named respectively “Aux Test Name SP” and “Aux Synchronize SP”.

Auxiliary Partition is an assistant partition that works generally on slave mode to help the Master Partition. The Auxiliary Partition is only required when the test procedures need another partition to cover the usage of some ARINC 653 services. The Auxiliary Partition includes two ARINC 653 sampling ports for getting test case name and providing the synchronization between the Master Partition and the Auxiliary Partition. In the Auxiliary Partition, the “Aux Test Name SP” is used to know which test case is executed. After a test case finishes, the Auxiliary Partition goes to WARM_START mode for servicing a new test case using the synchronization port.

GVT-A653 with Serial Line or UDP In order to use GVT-A653 validation suite with

different communication types (UDP and serial line protocols), we reconstructed the communication subsystem. When serial or UDP socket communication is selected, the TCM is linked to a corresponding communication library instead of the AFDX API on the HCS. The AFDX queuing ports in the TCS is taken out from the System Partition. In this formation the System Partition uses another TETware-RT API library that is produced for the serial or UDP socket communication.

Continuous Integration Automated testing is crucial in this project

because of the huge amount of functions need to be tested individually. Also each API function test consists of multiple test cases.

Testing a function consists of cleaning, building and running steps. Also, it is needed to make sure

that a compiled ARINC 653 system is ready according to the latest updated codes. During the execution of each test, the status of the system need to be checked since some tests may crash the system or put the system in an inconsistent state. Therefore, the system may need to be restarted before passing to next tests. After running all tests, the results of each function are collected in an output file. If these steps are not done automatically, it consumes a huge amount of time and may lead to incorrect testing. Existing automated testing tools such as Jenkins [7] has been checked but not worked as flexible as we needed. Therefore, we also developed our own continuous integration tool as a module for this project.

Each function to be tested is placed inside a scenario file before the test begins. The general architecture of our automated testing tool is displayed in Figure 3. The main steps are:

• Cleaning and building the latest codes from the source control system (in this project Rational ClearCase) for ARINC 653 compatible RTOS.

• Running the operating system on the target platform.

• Getting the next APEX function from the scenario file.

• Cleaning, building, and running all test cases of the selected function.

• Waiting for the test result. If no result returns from the system within the timeout period, we record the system situation in a dump file and restart the system for the next function to be tested. If test returns with a result, we continue testing with the next one. It should be mentioned that not all of the test cases for the selected function has to be successful. The tool records passed failed or unresolved cases in a result file to be analyzed later.

• Shutting down and reporting. If all tests are completed in the scenario file, we shut-down the system and e-mail the results to the recipients.

8A6-5

Figure 3. Continuous Integration for GVT-A653

Conformance Testing We have verified the compliance of our RTOS

with the ARINC 653 Part 1 using the GVT-A653 validation suite. For each APEX service in ARINC 653 Part1 document; test requirement, functional test procedure, robustness test procedure and service macro sections are defined in ARINC 653 Part 3 document. As a part of GVT-A653, defined sections for all services are implemented and successfully executed on our RTOS.

This test suite contains all test cases specified in ARINC 653 Part 3 document. Furthermore, additional test cases are included for Service Access Point (SAP) port services and Multiple Module Schedule services described in ARINC 653 Part 2 (Extended Services) document [8]. There are 245 test cases for 56 services of the ARINC 653 Part 1 and 33 test cases for SAP and Multiple Module Schedule services in the ARINC 653 Part 2 within the scope of GVT-A653. The test cases for the implemented subset of ARINC 653 Part 2 services are designed from scratch. Figure 4 shows the number of services and test cases implemented for GVT-A653.

Figure 4. Detailed Number of Services and Test Cases

Some errors are detected in ARINC 653 Part 3 document during the implementation phase of the test cases. We have found 74 errors that can be analyzed in two groups; the first group includes logical errors and the second group includes errors due to lack of algorithmic statements in the test cases. The errors caused by:

• Logical inconsistencies include scheduling and synchronization contradictions and redundant expressions. For instance, on the page 231 of ARINC 653 Part 3 document; the test case T-API-INTRA-0260:0010 contains two SEND_BUFFER call in the P1 process. First SEND_BUFFER call sends “test” message with timeout and second SEND_BUFFER call sends “wrong” message without timeout to Master process through the ID of buffer B1. The second SEND_BUFFER service call in P1 process is unnecessary since it is not either received or checked by master process.

• Lack of algorithmic statements are intra or inter partition communication calls which have no return value checks and absence of information on priority or period of some processes. For example in the test case T-API-INTRA-0260:0065 for RECEIVE_BUFFER service, “Process P1 is created” information is given however, the priority and period of process P1 is not given in the environment scenario. Process P1 must have a higher priority than Master process, because it must preempt Master

8A6-6

process for the correct operation flow of the test case. To make test case to work as expected, its environment scenario is modified with the expression “process P1 is created with priority = REGULAR_P1_PROCESS_PRIORITY and aperiodic”.

For the errors detected, 74 errata reports are prepared and sent to the ARINC 653 APEX Software subcommittee. 67 of these errata reports are accepted and will be published in the next version of the document by the subcommittee.

GVT-A653 makes itself invaluable during development of an ARINC 653 conformant RTOS for a software team. The tool has greatly reduced verification and validation time of the RTOS.

Related Work There are two available conformance testing

tools that have been presented to validate ARINC 653 based RTOSs. One of them has been developed by UniTESK Lab [9] as a test suite, which ensures conformance to ARINC 653 specification. The test suite includes test specifications of ARINC 653 Part 3. By using requirements-based testing methodology which is expressed in the DO-178B standard “Software Consideration in Airborne Systems and Equipment Certification” document [10], they have revealed errors in the ARINC 653 Part 3. They have added new test cases that are uncovered in ARINC 653 Part 3 including combinations of the requirements and system test cases for basic subsystems of ARINC 653 Part 1. A commercial ARINC 653 Verification Tool (AVT) [11] is another conformance test suite for the ARINC 653 Part 1. It allows an independent verification of the presence of required services and their conformance with the standard. Basically it consists of two main parts which communicate each other to handle the tests. The AVT Host controls the execution of the tests and runs on a host development environment. The AVT Host communicates with the target using interfaces which are set of abstract communication primitives. For each different target and host, this layer has to be implemented using the available implementation dependent communication services. The AVT Target consists of set of partitions that are responsible for executing the specified tests. Tests are evaluated by

the AVT Target and the test results are returned as a “PASS/FAIL” token back to the AVT Host. Alternatively, AVT Host can be embedded within AVT Target. In this case, the test results can be reported to an implementation dependent device. There is also an ARINC 653 Part 1 conformance test pack which is designed for LithOS, to demonstrate compliance of the API behavior and determine whether it meets with the ARINC 653 standard [12]. Although, this test pack includes behavioral tests, definitional test, and interface tests, it is not a complete platform-independent suite.

The main contribution and advantages of GVT-A653 tool which are not available on other tools are utilizing CI principles, offering platform-independency, building upon an open-source framework, and supporting a reliable communication protocol at the same time.

Future Work In 2012, ARINC 653 Part 4 (Subset Services)

was developed, which defines a subset of the API specified by ARINC 653 Part 1 [13]. Although, ARINC 653 Part 1 services involve ARINC 653 Part 4 services, some services in Part 1 have become un-used or modified in ARINC 653 Part 4 specification. Therefore, corresponding functional and robustness test procedures in ARINC 653 Part 3 specification do not cover the requirements of the ARINC 653 Part 4 completely. Hence, some test cases have to be updated or written from scratch for the ARINC 653 Part 4 Specification. The next phase of the GVT-A653 validation suite will contain the test specification to comply with ARINC 653 Part 4.

The APEX Subcommittee continues the discussion of multi-core processors usage on avionics platform. It is considered to add new APIs to satisfy the requirements of ARINC 653 Multi-Core Module. We are planning to support the development of ARINC 653 Multi-Core Module by generating new test procedures that comply with the Multi-Core APIs.

Conclusion The ARINC 653 specification defines the major

set of functionalities that an RTOS must support to be used in IMA systems. Therefore, independent

8A6-7

validation of an RTOS becomes an important issue to be conformant with the specification.

The existing literature shows that there are some validation tools as test suites or packs in order to check conformity of an RTOS. The tool we developed is novel and notable with several characteristics by:

• Employing an open source test framework in order to adapt useful updates in the framework.

• Supporting continuous integration for partial verifying by an automated build and test to detect integration errors as quickly as possible.

• Enabling platform-independency for RTOS developers to validate the conformance of any RTOS based on ARINC 653 Part 1 specification.

• Providing several communication methods including deterministic AFDX Network.

References [1] J. Rushby, June 1999, Partitioning in Avionics Architectures: Requirements, Mechanisms and Assurance. Technical Report NASA CR-1999-209347, SRI International, California, USA.

[2] Airlines Electronic Engineering Committee (AEEC), Avionics Application Software Standard Interface - ARINC Specification 653 - part 3 (Conformity Test Specification), 2006, ARINC Inc.

[3] Airlines Electronic Engineering Committee (AEEC), Avionics Application Software Standard Interface - ARINC Specification 653 - Part 1 (Supplement 2 - Required Services), 2006, ARINC Inc.

[4] Test Environment Toolkit TETware-Realtime User Guide, December 2002, the Open Group.

[5] Eun H. Kim, Jong C. Na, Seok M. Ryoo, 2009, Test Automation Framework for Implementing Continuous Integration, International Conference on Information Technology: New Generations, pp. 784–789.

[6] Airlines Electronic Engineering Committee AEEC), Avionics Application Software Standard Interface - ARINC Specification 664 - P7-1 (Avionics Full-Duplex Switched Ethernet Network), September 2009, ARINC, Inc.

[7] Jenkins, http://jenkins-ci.org/, accessed 15 August 2014.

[8] Airlines Electronic Engineering Committee AEEC), Avionics Application Software Standard Interface - ARINC Specification 653 - Part 2 (Extended Services), January 2007, ARINC Inc.

[9] Institute for System Programming of the Russian Academy of Sciences (ISPRAS), Software Testing for Embedded Avionics Systems,

[10] RTCA/DO-l78B, Software Considerations in Airborne Systems and Equipment Certification, December 1, 1992.

[11] Skysoft Portugal S.A., April 2007. ARINC 653-1 API/OS Verification Tool User Manual, version 1.2 edition, Portugal.

[12] Masmano, M., Valiente, Y., Balbastre, P., Ripoll, I., Crespo, A., Metge, J.J., 2010, LithOS: A ARINC-653 Guest Operating for XtratuM, Nairobi (Kenya), Proc. of the 12th Real-Time Linux Workshop.

[13] Airlines Electronic Engineering Committee AEEC), Avionics Application Software Standard Interface - ARINC Specification 653 - Part 4 (Subset Services), June 2012, ARINC Inc.

Acknowledgements This research has been mainly supported by

Informatics and Information Security Research Center of The Scientific and Technological Research Council of Turkey (TUBITAK-BILGEM). Also, we wish to thank Dr. Ferhat Ucan for pre-reviewing the paper and Ercan Gungor for TETware integration.

Email Addresses Ahmet Alptekin, ahmet.alptekin@tubitak.gov.tr

Yunus Yilmazer, yunus.yilmazer@tubitak.gov.tr

Ugur Usug, ugur.usug@tubitak.gov.tr

8A6-8

Feyzullah Koca, feyzullah.koca@tubitak.gov.tr

Koray Incki, koray.incki@gmail.com

33rd Digital Avionics Systems Conference October 5-9, 2014