FP7 - ReMeDi – 610902 1

EU FP7-ICT-2013.10.2

ICT for Cognitive Systems and Robotics - 610902

Work Package 3: Perception and Context – Aware Robot Control

Deliverable D3.2: First skeleton of the robot control

architecture

Release date: 03-12-2014

Status: restricted

Document name: ReMeDi_WP3_D3.2

WPx: Work Package Nr. Dxxx: Deliverable Number (compare DoW)

FP7 - ReMeDi – 610902 2

EXECUTIVE SUMMARY

This deliverable reports on the architecture of a multi-level robot control system

resulting from T3.1 comprising architecture documentation and integrated components.

The architecture is built on the basis of available components (Xenomai, Orocos, ROS).

The current integrated hardware and software framework includes modules for path

following, obstacle avoidance, and navigation and is complemented by results of T3.4

focusing on the design of a graphical interface for the DiagUI. Besides, a number of

software modules and testbeds have been developed and documented, which are of a

technical nature but they are important from the integration perspective.

FP7 - ReMeDi – 610902 3

Deliverable Identification Sheet

IST Project No. FP7 – ICT for Cognitive Systems and Robotics - 610902

Acronym ReMeDi

Full title Remote Medical Diagnostician

Project URL http://www.remedi-project.eu

EU Project

Officer

Christoph Klein

Deliverable D3.2 First skeleton of the robot control architecture

Work package WP3 Perception and Context – Aware Robot Control

Date of delivery Contractual M10 Actual 03-12-2014

Status Version 1.0 Final

Nature Prototype

Dissemination

Level

Restricted

Authors

(Partner)

Krzysztof Arent WRUT, Mateusz Cholewiński WRUT, Wojciech

Domski WRUT, Ida Góral WRUT, Janusz Jakubiak WRUT, Mariusz

Janiak WRUT, Łukasz Juszkiewicz WRUT, Bogdan Kreczmer

WRUT, Krzysztof Tchoń WRUT, Bartłomiej Stańczyk (ACCREA),

Adam Kurnicki (ACCREA), Luc Van Gool (ETHZ), Andrea Fossati

(ETHZ), Carlo Alberto Avizzano (SSSA), Emanuele Ruffaldi

(SSSA), Angelika Peer (TUM)

Responsible

Author

Krzysztof Arent Email Krzysztof.arent@pwr.edu.pl

Partner WRUT Phone +48 71 320 2726

Keywords control architecture, hardware – software framework, mobile platform:

control & navigation

Version Log

Issue Date Rev

No.

Author Change

02-12-2014 1.0 Krzysztof

Arent

incorporation of the reviewers' comments

FP7 - ReMeDi – 610902 4

TABLE OF CONTENTS

Executive Summary .......................................................................................................... 2

1. Introduction .................................................................................................................. 6

2. Control System Architecture of the ReMeDi System .................................................. 8

2.1. Logical structure .................................................................................................... 8

2.2. Design of G-DiagUI ............................................................................................ 17

3. Hardware-Software Framework for Integration of the ReMeDi System ................... 19

4. Selected Topics ........................................................................................................... 23

4.1. Xenomai, Orocos, ROS: basic software environment ......................................... 23

4.2. Xenomai s626 driver ........................................................................................... 23

4.3. OROCOS s626 component .................................................................................. 24

4.4. Simple manual controller of a motor ................................................................... 24

4.5. Conversion of a Simulink scheme into OROCOS component ............................ 25

4.6. Xenomai & FreeRTOS: motor control in distributed architecture ...................... 26

4.7. Modular sonar system .......................................................................................... 27

5. ReMeDi System Component: Mobile Base Carol ...................................................... 29

6. Conclusion .................................................................................................................. 30

References ...................................................................................................................... 31

LIST OF FIGURES

Figure 2.1-1: General system logical architecture (Kurnicki, et al. 2014) ....................... 8 Figure 2.1-2: The SysML bdd [package] ReMeDi system [diagram] ............................. 9

Figure 2.1-3: The SysML bdd [package] ACC [ACC diagram] ................................... 10 Figure 2.1-4: The SysML bdd [package] ETHZ [ETHZ diagram] ............................... 10

Figure 2.1-5: The SysML bdd [package] SSSA [SSSA diagram] ................................. 10 Figure 2.1-6: The SysML bdd [package] TUM [TUM diagram] .................................. 11

Figure 2.1-7: The SysML bdd [package] WRUT [diagram] ......................................... 12 Figure 2.1-8: The SysML ibd [block] Robot [diagram] ................................................. 13 Figure 2.1-9: The SysML ibd [block] MobileBase [diagram]........................................ 14 Figure 2.1-10: The SysML ibd [block] Navigation [diagram] ....................................... 15 Figure 2.1-11: The SysML ibd [block] DiagUI [diagram] ............................................. 15

Figure 2.1-12: The SysML ibd [block] AR [diagram] ................................................... 16 Figure 2.2-1: State flow for G-DiagUI ........................................................................... 17 Figure 2.2-2: Potential screenshot during palpation ....................................................... 18 Figure 2.2-1: Hardware - Software Framework for Integration of the ReMeDi System:

distributed architecture ................................................................................................... 20

Figure 2.2-2: Hardware - Software Framework for Integration of the ReMeDi System:

centralised architecture ................................................................................................... 20

Figure 4.4-1: Manual controller...................................................................................... 24 Figure 4.4-2: Logical scheme of manual controller ....................................................... 24 Figure 4.5-1: From Simulink scheme into OROCOS component .................................. 26 Figure 4.6-1: Hardware................................................................................................... 27

FP7 - ReMeDi – 610902 5

Figure 4.6-2: System architecture ................................................................................... 27 Figure 4.7-1: A prototype of a receiver / transmitter sensor ......................................... 28 Figure 4.7-2: A structure of a sonar system.................................................................... 28 Figure 4.7-1: Carol – autonomous navigation ................................................................ 29

LIST OF TABLES

Table 2.2-1: Delivered components for initial design of G-DiagUIError! Bookmark

not defined. Table 4.1-1: Delivered components for the use case “Xenomai, Orocos, ROS: basic

software environment” ................................................................................................... 23 Tabela 4.2-1: Delivered components for the use case “Xenomai s626 driver” .............. 24 Table 4.3-1 Delivered components for the use case “OROCOS s626 component” ....... 24 Table 4.4-1 Delivered components for the use case “Simple manual controller of a

motor’’ ............................................................................................................................ 25 Table 4.5-1 Delivered components for the use case “Conversion a Simulink scheme into

OROCOS component’’ .................................................................................................. 26 Table 4.6-1 Delivered components for the use case “Xenomai & FreeRTOS: motor

control in distributed architecture” ................................................................................. 27 Table 4.7-1: Delivered components for the use case “Modular sonar system” .............. 28

Table 4.7-1 Delivered components for the system component “Carol” ......................... 29

LIST OF ABBREVIATIONS

Abbreviation Description

PR Public Report

WP Work Package

Partner Abb. Description

TUM Technische Universität München

ACC ACCREA Bartlomiej Marcin Stanczyk

LUM Medical University of Lublin

PLUS ICT&S Center Salzburg

SSSA SSSA - Scuola Superiore Sant’Anna

ETHZ Eidgenössische Technische Hochschule Zürich

WRUT Wroclaw University of Technology

FP7 - ReMeDi – 610902 6

1. INTRODUCTION

System specification done in (Kurnicki, et al. 2014) for the ReMeDi robot shows that it

is a very complex system where the main components are: a manipulator with

a palpation effector, a haptic interface and a mobile platform. Control design and

implementation for such a system is a multi-stage procedure which is not well specified

and relies on individual properties of the system itself and its components. This

document focuses its attention on the selected stages of the mentioned procedure:

overall control system architecture for the ReMeDi robot, control of a ReMeDi mobile

platform and a graphical interface for a diagnostician interface (DiagUI).

It is well known that although robot control software architectures have attracted

considerable attention for more than the last decade there are no commonly accepted

solutions and procedures leading to a good design of an architecture. In practice each

robotic system has its own, unique control software architecture and the same is

expected for the ReMeDi robot.

However, due to importance of an architecture for the whole ReMeDi system we

decided to not develop it from scratch but to apply the guidelines elaborated and used in

the European FP7 project BRICS, (Kraetzschmar, et al. 2010), especially in the field of

component-based software technologies and communication middleware. The

fundamental concept of component-oriented programming (COP) is the a software

component that is a unit of composition with contractually specified interfaces and

explicit context dependencies only. A software component can be deployed

independently and is subject to composition by third parties (Szyperski 2002). There are

several robotics framework utilizing component-oriented programming Orocos

(Bruyninckx, Soetens i Koninckx 2003), OpenRTM (Anado, Suehiro i Kotoku 2008),

ROS (Quigley i inni 2009), OPRoS (Korean Institute for Advanced Intelligent Systems

2014), GenoM (Mallet, et al. 2010), SmartSoft (Schlegel i Worz 1999), and Proteus

(Groupe de Recherche en Robotique 2014). Extensive review of robotics middlewares

has been presented in (Shakhimardanov, Hochgeschwender, et al. 2010) and (Kramer i

Scheutz 2007). Following the BRICS guidelines, the OROCOS and ROS frameworks

has been proposed as a ReMeDi system middlewares. The OROCOS is general purpose

modular framework for robot and machine control which can be build ontop real-time

system like Linux with Xenomai extension. The framework provides basically a set of

libraries, among which the Real Time Toolkit (RTT) library delivers infrastructure and

functionality to build component based systems. In (Willaert, et al. 2012) the OROCOS

framework has been used to build a highly dynamic force-feedback bilateral controller.

The Robot Operating System (ROS) is one of the most widely used middleware for

robotic applications. It defines a software components called nodes that can

communicate each other through topics using publish/subscribe message passing

mechanism or services that are an equivalent of RPC. Also, ROS provides a robotics

community a large amount of ready to use robotic components and tools. Recently ROS

has been applied in several medical robotics projects: Raven II (Hannaford, et al. 2013),

Toyota HSR (Hashimoto, et al. 2013), Cody (King, Killpack and Kemp 2010) and Care-

O-bot 3 (Graf, Parlitz and Hagele 2009). A safety framework for component-based

medical robots has been presented in (Jung and Kazanzides 2013)

It should be noted, that along with the logical architecture we undertake issues related to

its implementation, such as optimal base software.

FP7 - ReMeDi – 610902 7

The mobile platform is a basic component of the ReMeDi system. The requirements,

associated with the platform, concerning control, are collected in D4.2 (Kurnicki, et al.

2014). They comprise map-based navigation and obstacle avoidance. Implementation of

such control features can be based on ready to use software components provided the

base software for implementation of the system architecture is suitably chosen. On the

other hand, the initially appointed base software can be comprehensively and reliably

validated by means of a mobile platform.

The graphical interface for the DiagUI is inherent part of the control software

architecture. It must be explicitly represented in the architecture and finally

implemented. To this end a suitable graphical interface design has to be done, usually in

an iterative process.

All the issues highlighted above are under consideration in the following parts of this

document. The document is organized as follows. In Section 0 the logical structure of

the control system architecture is presented together with the design of the graphical

interface for the DiagUI. In Section 0 the hardware-software framework is proposed for

implementation and integration of the whole system on the basis of the control system

software architecture. In Section 4 several use cases are presented to highlight selected

important aspects of the proposed basic software frameworks. In Section 0 one system

component - a mobile platform capable of navigating with basic software frameworks is

presented and discussed. In Section 6 conclusions are provided.

FP7 - ReMeDi – 610902 8

2. CONTROL SYSTEM ARCHITECTURE OF THE REMEDI SYSTEM

This section presents the current state of development of a logical ReMeDi system

architecture and initial design of one of its components which is a graphical interface for

diagnostic interface DiagUI (G-DiagUI).

2.1. Logical structure

The logical structure for the ReMeDi system has been initially proposed in (Kurnicki, et

al. 2014). It has a form a high-level diagram, presented in Figure 2.1-1, which

comprises the main system components identified through user studies carried out in

(Moser, et al. 2014).

Patient Site Diagnostician Site

3D Vision

System

Haptic interface

module

Audio System

USG Probe

Dummy

Cam

eras

USG Device

Ste

thosc

ope

Navigation

module

Mobile

Platform

Control &

Safety

System

Platform

Drive &

Locking Unit

Laser

sensors

Bumper

Proximity

sensors

Arm Control

& Safety

System

Arm Drive

Unit

Central Motion Control

and Safety System

Head/Neck

Control &

Safety

System

Head/Neck

Drive Unit

Force/Torque

Sensor

Contact/

Proximity

Sensors Power

System Unit

Power System

User Interface

Blood

preasure,

temperature

sensor

Telepresence

module

Interaction &

Visualisation

Head/Gaze

Tracker

USG Control

Panel

Keyboard,

3D Mouse

Pulpation

Interface

Foot Switches

Camera

Telepresence

module

LoudS

pea

ker

s

Dead man

Switch

Emergency

button

Assistant Interface

Dead man

Switch

Scr

een

Docking

Station

Mic

rophones

Perception & Interaction

Patient Monitoring

Environmental Condition

Recognition

Vision &Audio Acquisition

Face Analyses

Interaction module

Posture Estimation

Figure 2.1-1: General system logical architecture (Kurnicki, et al. 2014)

From the perspective of integration of the whole system the diagram needs further

clarification. Description of inputs and outputs has to be formalised. Particular

components have to be decomposed into subsystems with properly defined

interconnections, hierarchy, inputs and outputs. The functionality of each component

must be precisely specified. Useful guidelines for doing this can be found

in (Kraetzschmar, et al. 2010).

According to the component-based architecture development methodology, the ReMeDi

system has to be decomposed into a set of components representing particular system

functionality. Components may represent atomic and complex structures, depending on

the current state of a system development and the currently available level of detail of

knowledge about the system. Each component must have well defined interfaces and

may be parameterized.

The ReMeDi system components are systematically collected in a carefully structured

formal component specification table. This table provides a common format that allows

precise definition of components, in particular, their inputs and outputs. It has a form of

a spreadsheet which is shared by all the partners (http://accrea.myredmine.com, Task:

control system architecture: the formal component specification table). Thanks to this,

components can be added, updated and synchronized to each other in the course of the

project. The field names of the formal component specification table are collected in

Table 2.1-1. It should be clearly stated that the necessary condition for a successful

software system integration is a completely filled formal component specification table

with all the control system architecture components specified.

FP7 - ReMeDi – 610902 9

Table 2.1-1: The field names of the formal component specification table

Component

name

Doctor

(D) or

Patient (P)

Partner

Behavioral pattern

(sink, source,

filter or other)

Standard or

real-time

Activity (aperiodic,

periodic, event driven)

Short

description

Inputs

Name Description Message structure Standard or

real-time Remarks

Type Name Description Units

Outputs

Name Description

Message structure Standard or

real-time

Activity

(periodic,

event

driven)

Remarks Type Name Description Units

Parameters Other

requirements,

resources,

remarks

Type Name Description Units

The ReMeDi control system architecture is described in a form of a set of SysML

diagrams (SysML), presented in Figure 2.1-1 to Figure 2.1-12. The diagrams follow

directly from the formal component specification table and the general system logical

architecture in Figure 2.1-1. They are organised as follows. The SysML block definition

diagram (bdd) of the ReMeDi system is presented in Figure 2.1-2. The components

have been collected into packages managed by a responsible partner. The package

content is presented in Figure 2.1-3 to Figure 2.1-7. The ReMeDi system structure is

illustrated with SysML internal block diagrams (ibd) in Figure 2.1-8 to Figure 2.1-12.

Figure 2.1-2: The SysML bdd [package] ReMeDi system [diagram]

FP7 - ReMeDi – 610902 10

Figure 2.1-3: The SysML bdd [package] ACC [ACC diagram]

Figure 2.1-4: The SysML bdd [package] ETHZ [ETHZ diagram]

Figure 2.1-5: The SysML bdd [package] SSSA [SSSA diagram]

FP7 - ReMeDi – 610902 11

Figure 2.1-6: The SysML bdd [package] TUM [TUM diagram]

FP7 - ReMeDi – 610902 12

Fig

ure

2.1

-7:

Th

e S

ysM

L b

dd

[p

ack

age]

WR

UT

[d

iagra

m]

FP7 - ReMeDi – 610902 13

Fig

ure

2.1

-8:

Th

e S

ysM

L i

bd

[b

lock

] R

ob

ot

[dia

gra

m]

FP7 - ReMeDi – 610902 14

Fig

ure

2.1

-9:

Th

e S

ysM

L i

bd

[b

lock

] M

ob

ileB

ase

[d

iagra

m]

FP7 - ReMeDi – 610902 15

Figure 2.1-10: The SysML ibd [block] Navigation [diagram]

Figure 2.1-11: The SysML ibd [block] DiagUI [diagram]

FP7 - ReMeDi – 610902 16

Fig

ure

2.1

-12:

Th

e S

ysM

L i

bd

[b

lock

] A

R [

dia

gra

m]

FP7 - ReMeDi – 610902 17

2.2. Design of G-DiagUI

The diagnostician’s interface DiagUI basically consists of all the components at the

Diagnostican Site in Figure 2.1-1. In some contexts the DiagUI can be also extended by

the Assistant Interface at the Patient Site. From the perspective of a doctor the most

exposed and used software component of the DiagUI is a graphical interface.

An initial project of the G-DiagUI is presented in (Arent, et al. 2014). The starting point

was (Moser, et al. 2014). As a result of the careful analysis of use cases several

examination techniques have been identified:

basic techniques: interview, observation, palpation, auscultation of the chest,

electrocardiography, ultrasonography of aorta, heart and abdomen,

teleconferencing, recording and comparison of the examination results;

auxiliary techniques: chest X-ray, computed tomography, pulse oxiometry,

blood pressure measurement, laboratory blood tests (in some use cases the basic

technique can become an auxiliary technique).

In order that a doctor could apply the techniques mentioned above, G-DiagUI should

possess suitable functionalities, such as (Arent, et al. 2014): patient face view, patient

body view, probe view, communication with patient, patient emotional state, cameras

control, visualisation of the probe distance, visualisation of force stethoscope sound,

communication with an assistant, visualisation of ECG results, ECG monitoring, access

to patient data, recording an a few more. It must be emphasized that during particular

examinations only selected functionalities have to be available. To describe this well,

basic work modes were identified. They and their mutual relations are presented by

means of a state flow diagram shown in Figure 2.2-1.

Figure 2.2-1: State flow for G-DiagUI

For each work mode an appropriate analysis is carried out in (Arent, et al. 2014) using

use cases diagrams, which comprise a doctor, an assistant and appropriate

FP7 - ReMeDi – 610902 18

functionalities. The resulted diagrams form a basis for the design of particular windows

in the G-DiagUI. The a priori approved underlying design principle consists in keeping

the G-DiagUI in a maximally simple form. It is illustrated by Figure 2.2-2 where a

collection of graphical windows associated with the palpation working mode is

presented.

Figure 2.2-2: Potential screenshot during palpation

FP7 - ReMeDi – 610902 19

3. HARDWARE-SOFTWARE FRAMEWORK FOR INTEGRATION OF THE

REMEDI SYSTEM

The control system architecture presented in Section 0 requires a suitable hardware-

software framework for deployment and integration of system components and their

arrangements. It follows from (Kurnicki, et al. 2014) that a ReMeDi robot is a very

complex system consisting of many components, running concurrently on different

platforms. Hence, the hardware- software framework for a ReMeDi robot should be

very flexible, easily reconfigurable, open to changes and modifications, with the

architecture rather distributed than centralised.

The general concept of the hardware-software framework distributed architecture has

been presented in Figure 2.2-1. From a hardware side, this framework consists of

several computation devices like PCs or Advance RISC Machines(ARM)-based

computers and a number of custom embedded controllers. The number of computation

devices and also custom controllers is not limited. The absolute minimum is one

computation device. In this case we obtain a centralized architecture, presented

in Figure 2.2-2.. We envision that the ReMeDi system will take a distributed form with

diverse computation devices. The form of these devices, in particular their peripherals,

will evolve in the course of the project.

FP7 - ReMeDi – 610902 20

Figure 2.2-1: Hardware-Software Framework for Integration of the ReMeDi

System: distributed architecture

Figure 2.2-2: Hardware- Software Framework for Integration of the ReMeDi

System: centralised architecture

FP7 - ReMeDi – 610902 21

The intention is that computers should perform high-level control tasks that require high

computational power and large resources. But it is not obligatory and low-level control

algorithms can be hosted as well. Computers can be equipped with any number of

internal or external devices that extend their functionality and communication abilities,

e.g. cameras, range finders, multifunctional I/O cards and WiFi network cards.

Computers are supervised by real-time Linux with Xenomai extension. Xenomai

provides hard real-time support to user-space applications, is well integrated with the

GNU/Linux environment and implements many programing interfaces, including native

and POSIX. The computer software stack is based on two robotic frameworks: ROS and

OROCOS. Not time critical components should be implemented as ROS nodes, while

time critical as OROCOS components. This distinction is due to the fact that OROCOS

is fully integrated with Xenomai. OROCOS components are associates with real-time

threads and utilize Xenomai infrastructure. ROS hasn’t been designed as real-time

framework in mind, however it provides a rich set of tools, services and ready-to-use

application building blocks. OROCOS and ROS frameworks are well integrated, each

framework provides common interfaces and a transport layer for communication

between components located within one machine, and also spread over several

machines. Custom embedded controllers are microcontroller-based small devices, which

can be easily adopted to any specific requirements regarding functionality, resources

and dimensions. Such devices are dedicated to performing low level control tasks like:

motor control, signal conditioning and sensor fusion. Usually this kind of tasks requires

specialized resources, real-time response and high stability. For this reason custom-

embedded controllers, are managed by FreeRTOS, a tiny footprint, hard real-time

operating system dedicated for small embedded devices. FreeRTOS has very portable

source code structure, is predominantly written in C, typically its kernel binary image is

in the range of 4K to 9K bytes, and it supports 34 different architectures including ARM

Cortex-M. Applications hosted by custom controllers are implemented mainly as RTOS

real-time tasks. Critical application parts can be implemented as bare metal procedures

running directly on hardware. The number of applications running on each device is

limited only by its resources. A communication between components running on a

different machine is possible with a ROS publish-subscribe message passing

mechanism and also with a OROCOS CORBA framework. This transport layers are

based on a standard Ethernet IPv4 protocol and can be used for not time critical

communication. A time critical communication between a computational devices as

well as custom embedded controllers is possible with a RTnet framework, a hard real-

time network protocol stack for Xenomai and recently also for FreeRTOS. The RTnet

operates on a standard Ethernet hardware, implements common Ethernet protocols in a

deterministic way, and provides a standard POSIX socket API. In future, it is planned to

implement a publish-subscribe protocol.

The most important features of ROS, OROCOS, Xenomai, FreeRTOS, RTnet are

collected in

FP7 - ReMeDi – 610902 22

Table 2.2-1.

FP7 - ReMeDi – 610902 23

Table 2.2-1: Selected features of software frameworks

software

framework

features

ROS Robot Operating System

Open source license

Supported languages: C++, Python

Agent-based programming model

Message passing

– Topics – publish/subscribe model

– Service – remote operation

Name and Parameter Services

Application building blocks: coordinate system transform services, visualization

tools, debugging tools, robust navigation stack, arm path planning, object

recognition, ...

www.ros.org

OROCOS Open Robot Control Software

Open source license

Supported languages: C++, LUA and native scripting

Component-Oriented Programming mode

Framework components

– Orocos Toolchain

– Kinematics & Dynamics Library

– Bayesian Filtering Library

Real-time control and communication

Integrated with Xenomai and ROS

www.orocos.org Xenomai Real-time development framework cooperating with the Linux kernel

Open source license

Hard real-time support to user-space applications

Integrated with the GNU/Linux environment

Supported interfaces (skins): Native, POSIX, pSOS+, uITRON, VRTX, Vx-Works

and RTDM,

Supported architectures: x86(SMP), x86 64(SMP), ARM(SMP), Blackfin, Nios II,

PowerPC(SMP)

A real-time data acquisition framework Analogy

www.xenomai.org

FreeRTOS Real-time operating system from Real Time Engineers Ltd

Open source license

Tiny footprint – typically a RTOS kernel binary image will be in the range of 4K to

9K bytes

Supports 34 architectures, including ARM Cortex-M

Very portable source code structure, predominantly written in C

Queues, binary semaphores, counting semaphores, recursive semaphores and

mutexes for communication and synchronisation between tasks, or between real time

tasks and interrupts

FreeRTOS+ ecosystem: CLI, IO, FAT, UDP/IP, TCP/IP, TLS/SSL, Nabto,

Trace, Certified kernel (SafeRTOS)

www.freertos.org

RTnet Hard real-time network protocol stack for Xenomai and RTAI

Open source license

Operates on standard Ethernet hardware

Supports several popular NIC chip sets, including Gigabit Ethernet

Implement UDP/IP, TCP/IP (basic features), ICMP and ARP in a deterministic way

POSIX socket API for real-time user space processes and kernel modules

FreeRTOS port is available (RTmac, RTcfg, TDMA, Socket API, UDP/IP, ARP)

www.rtnet.org

FP7 - ReMeDi – 610902 24

4. SELECTED TOPICS

With reference to the hardware- software framework presented in Section 0 several use

cases have been developed. They highlight and present selected, but very important

stages in the process of deployment of architectural components in the proposed

software-hardware framework.

4.1. Xenomai, Orocos, ROS: basic software environment

The document (Domski, Installation and integration of Xenomai, OROCOS, ROS and

RTnet on an industrial PC: step by step procedure 2014) shows, step by step, how to

install software components characterized in

FP7 - ReMeDi – 610902 25

Table 2.2-1 and how to ensure that these components can work together properly so that

the whole system is well functioning. The exact list of software components is the

following: Ubuntu 12.04 LTE, Unity 3D, Qt, ROS (Fuerte), Xenomai, RTnet,

OROCOS. The hardware basis consists of: Supermicro Mainboard X9SPV-M4-3Q,

Intel CORE i7-3612QE, Kingston KVR13LSE9/8, INTEL 530 SSD MLC 120GB. A

suitable disk image has been prepared and is available.

Table 4.1-1: Delivered components for the use case “Xenomai, Orocos, ROS: basic

software environment”

component location

report (Domski, Installation and integration of Xenomai, OROCOS, ROS and RTnet

on an industrial PC: step by step procedure 2014)

source code no

virtual disc

image

http://robrex.ict.pwr.wroc.pl/~wdomski/VirtualBoxUbuntuImage/Ubuntu-ready.vdi

4.2. Xenomai s626 driver

As mentioned in Section 0 Xenomai delivers functionality of creating Linux drivers

which can meet hard real-time requirements. A special role is played by Analogy

framework that is designed for handling the communication with various data

acquisition cards in a Linux kernel space. For the purposes of the ReMeDi project the

Analogy driver for multifunction analog/digital I/O Sensoray model 626 board has been

developed. It refers to the centralized architecture, presented in Figure 2.2-2. Currently,

the driver is available at the Xenomai web site (this is the first contribution of the

ReMeDi project to the open source community). This driver is capable of handling all

the available ports of the Sensoray 626 card:

– 48 digital inputs/outputs,

– 16 analog inputs,

– 4 analog outputs,

– 6 encoders.

FP7 - ReMeDi – 610902 26

Table 4.2-1: Delivered components for the use case “Xenomai s626 driver”

component location

report (Domski, Xenomai Linux kernel driver for Sensoray 626 multi I/O board

and associated S626_task OROCOS component 2014)

source code http://git.xenomai.org/xenomai-

jro.git/commit/?h=sensoray&id=1f7d776b94eaaa8529b38d413ece6c3fabbf251f

http://robrex.ict.pwr.wroc.pl/svn/remedi/wd/s626_kernel/tags/v.1.0/

(virtual)

disc image

no

4.3. OROCOS s626 component

This component is comprehensively presented in (Domski, Xenomai Linux kernel

driver for Sensoray 626 multi I/O board and associated S626_task OROCOS component

2014). It is a software bridge between the Xenomai s626 driver and user accessible

space. In other words, this component is a representation of a multifunction

analog/digital I/O Sensoray model 626 board in OROCOS. Importantly, s626_task

component delivers a mechanism, which allows to interact with Xenomai s626 driver

while not violating the hard real-time constraints

Table 4.3-1 Delivered components for the use case “OROCOS s626 component”

component location

report (Domski, Xenomai Linux kernel driver for Sensoray 626 multi I/O board and

associated S626_task OROCOS component 2014)

source code http://robrex.ict.pwr.wroc.pl/svn/remedi/wd/s626_task/tags/v.1.0/

(virtual)

disc image

no

4.4. Simple manual controller of a motor

This use case is comprehensively presented and discussed in (Cholewiński and Domski

2014).

Figure 4.4-1: Manual controller

Figure 4.4-2: Logical scheme of manual

controller

FP7 - ReMeDi – 610902 27

It demonstrates how to use s626 components to implement a simple manual controller

of a motor and highlights all aspects of the s626 component.

The involved hardware is the following:

PC with Sensoray 626 PCI ,

simple PCB board (the hardware is presented in Figure 4.4-1 while its logical

structure in Figure 4.4-2 ) equipped with 2 monostable switches (Button #1 &

Button #2), 1 bistable switch (Button #3), two-axis joystick ,

servo controller with analog input for motor control (Maxon Escon 70/10),

motor (Maxon 305014) with incremental encoder (the same as in Figure 4.6-1).

Scenario of the demonstrator is as follows. After switching the bistable switch,

movement of a joystick causes proportional motions of a motor. Encoder readings are

displayed on a screen. Turning on monostable switches is reported on a monitor.

Comments in the source code are instructive. The following aspects are highlighted:

representation of a physical object in OROCOS,

acquisition of measurements from particular input ports of multifunction card

SENSORAY m626 via s626 component,

setting values on individual output ports of multifunction card SENSORAY

m626 via s626 component.

Table 4.4-1 Delivered components for the use case “Simple manual controller of a

motor’’

component location

report (Cholewiński and Domski 2014)

source code http://robrex.ict.pwr.wroc.pl/svn/remedi/mch/SimpleManualController/tags/v.1.0

(virtual)

disc image

no

4.5. Conversion of a Simulink scheme into OROCOS component

In the ReMeDi project Matlab & Simulink are basic tools for design and deployment of

control algorithms among majority of the technical partners. Having this fact in mind,

the procedure of converting a Simulink diagram into OROCOS real-time component

has been carefully studied and described in (Domski, From Simulink block diagram to

real-time OROCOS component 2014) with the help of a few simple case studies. The

idea is depicted in Figure 4.5-1.

FP7 - ReMeDi – 610902 28

Figure 4.5-1: From Simulink scheme into OROCOS component

It is worth of noting that although the Orocos Simulink Toolbox is currently available,

knowledge of the full procedure guarantees higher flexibility in design and

implementation of real-time Orocos components. The procedure consists of three basic

steps:

Simulink (make a diagram in Simulink; configure the simulation parameters;

generate code; analyse the documentation from Simulink Report Generator).

OROCOS (generate OROCOS component template; copy the source files from

Simulink into the src directory of the OROCOS component; declare in

<name>_component.cpp the header files from Simulink; define input and

output of a component in the constructor; include the method <name>_initialise

from Simulink into the method startHook of the component ; bind the inputs and

the outputs of the Simulink model and the component by including _step

method from Simulink into updateHook metdhod from the component; Compile

component.

Deployer (create a script to load and connect components and to set the time

period for periodic components; run the script in Deployer).

Table 4.5-1 Delivered components for the use case “Conversion a Simulink scheme

into OROCOS component’’

component location

report (Domski, From Simulink block diagram to real-time OROCOS component

2014)

source code http://robrex.ict.pwr.wroc.pl/svn/remedi/wd/Simulink_OROCOS/tags/v.1.0

(virtual)

disc image

no

4.6. Xenomai & FreeRTOS: motor control in distributed architecture

This use case (Janiak 2014) refers to the distributed hardware- software architecture

depicted in Figure 2.2-1. It is a simple manual axis motion controller that is shown in

Figure 4.6-1.

FP7 - ReMeDi – 610902 29

Figure 4.6-1: Hardware

Figure 4.6-2: System architecture

The system architecture is presented in Figure 4.6-2. It consists of two devices, a regular

PC running Linux with Xenomai extension, and an Axis Controller running a

FreeRTOS. Devices communicate each other using a RTnet. On the PC side there is one

regular Linux process panel.tcl and two real time tasks: rtsnd, rtrec. The panel is a

very simple user interface that allows for setting a control value and enabling/disabling

a selected controller. The rtsnd real-time task receives control commands from the user

interface through a rtpipe and forwards them to the axis controller using RTnet. The

axis controller sets its analog outputs according to the received control commands, and

sends measurements from an analog and a quadrature input through RTnet.

Measurements are received by the rtrec task and displayed in a console.

Table 4.6-1 Delivered components for the use case “Xenomai & FreeRTOS: motor

control in distributed architecture”

component location

report (Janiak 2014)

source code http://accrea.myredmine.com/attachments/download/98/AxisCtrExample.tar.gz

(virtual)

disc image

no

4.7. Modular sonar system

As indicated in Figure 2.2-1, a sonar system is a potential candidate for integration with

the ReMeDi robotic system on the basis of a distributed architecture. Basically, the

more receiver/transmitter sensors are present on the walls of a mobile platform, the

better its performance of obstacle avoidance. Therefore scalability, specific to a

distributed architecture, is desired and useful in this context.

FP7 - ReMeDi – 610902 30

Figure 4.7-1: A prototype of a

receiver / transmitter sensor

Figure 4.7-2: A structure of a sonar system

In (Kreczmer 2014) the principles of a new sonar system for the ReMeDi mobile

platform are outlined. This system is currently under development. The distinguishing

feature is that the new system will be capable of not only measure distances from

obstacles but also identify selected shapes of obstacles. The basic components: range

finder modules and sonar system controller are shown in Figure 4.7-2. Note, that it is

envisioned that the system will communicate with the rest of the robotic system via

RTnet. At the moment, a prototype of receiver/transmitter sensor is tested. This sensor

is shown in Figure 4.7-1. It is the essential part of the range finder module.

Table 4.7-1: Delivered components for the use case “Modular sonar system”

component location

report (Kreczmer 2014)

source code no

(virtual)

disc image

no

FP7 - ReMeDi – 610902 31

5. REMEDI SYSTEM COMPONENT: MOBILE BASE CAROL

This section presents a ReMeDi system component of which the first prototype has been

implemented. Two aspects are important here. The prototype in the current form is a

fulfilment of a declaration made with respect to WP3 in the Description of Work in the

Annex I of the ReMeDi Grant Agreement. Secondly, implementation of this prototype

relies on the whole hardware- software framework presented in Section 0 (restricted to

the centralised architecture), techniques and the software outlined in Section 4, design

guidelines formulated in Section 0 and demonstrates functioning of the whole software

technology.

Carol, manufactured by ACCREA, is a physical model of a ReMeDi mobile platform.

The robot is shown in Figure 4.7-1. It was used to implement and to initially test

autonomous navigation functionality that is based on Xenomai – OROCOS – ROS

software framework. The details are discussed in (Jakubiak and Juszkiewicz 2014), here

we present only selected information.

Figure 4.7-1: Carol – autonomous navigation

The basic hardware components of the control system are a mini-ITX industrial

computer (the same type as in Section 4.1) and a control box. The software framework

was installed according to the procedure described in (Domski, Installation and

integration of Xenomai, OROCOS, ROS and RTnet on an industrial PC: step by step

procedure 2014). The control system architecture is implemented both in the form of

OROCOS components (they operate in real - time) as in ROS nodes. The basic

OROCOS components are: S626 (see Section 4.3), wheels driver (obtained on the basis

of automatically generated code from Simulink, see Section 4.5), state machine, inverse

kinematics and odometry. The last two mentioned components are embedded in ROS

topics. Map-based navigation comes from the ROS navigation stack.

A number of tests carried out with the navigation system on Carol (that is illustrated in

Figure 4.7-1) have shown that the implemented software system is functioning very

well and the behaviour of the robot is satisfactory.

Table 4.7-1 Delivered components for the system component “Carol”

component location

report (Jakubiak and Juszkiewicz 2014)

FP7 - ReMeDi – 610902 32

source code http://robrex.ict.pwr.wroc.pl/svn/remedi/mobile_base/platform

http://robrex.ict.pwr.wroc.pl/svn/remedi/mobile_base/carol_nav

http://robrex.ict.pwr.wroc.pl/svn/remedi/mobile_base/reset_handler/tags/D3.2

http://robrex.ict.pwr.wroc.pl/svn/remedi/mobile_base/odometry/tags/D3.2

http://robrex.ict.pwr.wroc.pl/svn/remedi/mobile_base/inverse_kinematics/tags/D3.2

http://robrex.ict.pwr.wroc.pl/svn/remedi/mobile_base/wheels_driver/tags/D3.2

disc image http://robrex.ict.pwr.wroc.pl/~ljuszkie/obrazCarol2.tgz

6. CONCLUSION

A first skeleton of the ReMeDi robot control architecture has been proposed. The

general logical system architecture initiated in [D4.2, 2014] has beebn further

developed, wherein the description of the individual components are formalized. The

main attention focused on two aspects: efficiency of a qualitative analysis and feasibility

of integration of the ReMeDi system. The proposed procedure of system architecture

development, in particular component description, guarantees providing all the

necessary information for designing and updating a system architecture at present and in

the course of the project.

The inherent issue to the logical system architecture is its deployment. In this context

two hardware- software architectures were presented: distributed and centralised. It is

envisioned that the ReMeDi system will be a composition of both of them. As a base

software the following systems are proposed: Linux Xenomai, OROCOS, ROS. These

software systems are commonly used and well verified in many projects. It is important

in a view of complexity of the ReMeDi control software system. Notable is that they are

well integrated, flexible and open to other software.

The proposed software framework for deployment and integration is also motivated by

the results of use case studies. They facilitate installation and configuration of the base

software system as well as show how to deploy easily and efficiently control strategies

developed in Simulink – willingly and frequently used by majority of technical partners

in the ReMeDi project. Summing up, it has been depicted a bridge, in some sense,

linking the proposed base software for ReMeDi project and the software environments

for controllers design used by majority of partners.

From the perspective of implementation of a control software architecture two results

deserve special attention and are potentially very useful for further development of the

ReMeDi software system. Autonomous robot navigation has been implemented based

on Linux Xenomai, OROCOS and ROS on a mobile robot platform Carol provided by

ACCREA, which is a physical model of the ReMeDi mobile base. Low-level control

strategies were developed and automatically ported from Simulink. It should be stressed

that it is a first large and complete hardware- software component from the perspective

of the ReMeDi system. In addition to this, having in mind the current logical control

architecture, an initial design of a Graphical DiagUI has been developed.

FP7 - ReMeDi – 610902 33

REFERENCES

Arent, Krzysztof, Mateusz Cholewiński, Janusz Jakubiak, and Bogdan Kreczmer. “Initial project of a

graphical user interface for DiagUI.” SPR 11/2014, Wrocław University of Technology, 2014.

Azamat Shakhimardanov, Jan Paulus, Nico Hochgeschwender, Michael Reckhaus i Gerhard K

Kraetzschmar. Best Practice Assessment of Software Technologies for Robotics. BRICS, FP7,

http://www.best-of-robotics.org/pages/publications/BRICS_Deliverable_D2.1.pdf, 2010.

Bogdan Kreczmer. „Modular sonar system.” SPR 10/2014, Wrocław University of Technology, 2014.

Cholewiński, Mateusz, and Wojciech Domski. “Development of OROCOS component for

implementation a simple manual controller of a motor based on S626 task component.” SPR

7/2014, Wrocław University of Technology, 2014.

Christian Schlegel i R. Worz. „The software framework SMARTSOFT for implementing sensorimotor

systems.” Intelligent Robots and Systems, 1999. IROS '99. Proceedings. 1999 IEEE/RSJ

International Conference on. 1999. 1611-1616.

Clemens Szyperski. Component Software: Beyond Object-Oriented Programming. Boston, MA:

Addison-Wesley Longman Publishing Co., Inc., 2002.

Domski, Wojciech. “From Simulink block diagram to real-time OROCOS component.” SPR 15/2014,

Wrocław University of Technology, 2014.

Domski, Wojciech. “Installation and integration of Xenomai, OROCOS, ROS and RTnet on an industrial

PC: step by step procedure.” SPR 5/2014, Wrocław University of Technology, 2014.

Domski, Wojciech. “Xenomai Linux kernel driver for Sensoray 626 multi I/O board and associated

S626_task OROCOS component.” SPR 6/2014, Wrocław University of Technology, 2014.

Graf, Birgit, Christopher Parlitz, and Martin Hagele. “Robotic Home Assistant Care-O-bot 3 Product

Vision and Innovation Platform.” In Human-Computer Interaction. Novel Interaction Methods

and Techniques, Lecture Notes in Computer Science, by Julie A. Jacko, 312-320. Berlin

Heidelberg: Springer, 2009.

Groupe de Recherche en Robotique. Platform for RObotic modeling and Transformations. 2014.

http://www.anr-proteus.fr.

Hannaford, B., et al. “Raven-II: An Open Platform for Surgical Robotics Research.” Biomedical

Engineering, IEEE Transactions on, April 2013: 954-959.

Hashimoto, K., F. Saito, T. Yamamoto, and K. Ikeda. “A field study of the human support robot in the

home environment.” Advanced Robotics and its Social Impacts (ARSO), 2013 IEEE Workshop

on. 2013. 143-150.

Herman Bruyninckx, Peter Soetens i Bob Koninckx. „The Real-Time Motion Control Core of the

Orocos.” IEEE International Conference on Robotics and Automation. 2003. 2766--2771.

Jakubiak, Janusz, and Łukasz Juszkiewicz. “Deployment of low level control and navigation in a mobile

platform Carol based on Xenomai-ROCOS-ROS software.” SPR 12/2014, Wrocław University

of Technology, 2014.

James Kramer i Matthias Scheutz. „Development environments for autonomous mobile robots: A

survey.” Autonomous Robots, 2007: 101-132.

Janiak, Mariusz. “Axis motion controller - example of Xenomai, RTnet, FreeRTOS usage.” SPR 14/2014,

Wrocław University of Technology, 2014.

Jung, Min Yan, and Peter Kazanzides. “Run-time Safety Framework for Component-based Medical.” 4-th

Workshop on Medical Cyber Pisical Systems. Philadelphia, PA, USA, 2013.

King, Chih-Hung, Marc D. Killpack, and Charles C. Kemp. Effects of Force Feedback and Arm

Compliance on Teleoperation for a Hygiene Task. Vol. 6191, in Haptics: Generating and

Perceiving Tangible Sensations, Lecture Notes in Computer Science, by AstridM.L. Kappers,

JanB.F. van Erp, WouterM. Bergmann Tiest, & FransC.T. van der Helm, 248-255. Berlin

Heidelberg: Springer, 2010.

Korean Institute for Advanced Intelligent Systems. OPRoS. 2014. http://opros.or.kr.

FP7 - ReMeDi – 610902 34

Kraetzschmar, Gerhard K, Azamat Shakhimardanov, Jan Paulus, Nico Hochgeschwender, and Michael

Reckhaus. Specifications of Architectures, Modules, Modularity, and Interfaces for the BROCRE

Software Platform and Robot Control Architecture Workbench. D2.2, BRICS, FP7,

http://www.best-of-robotics.org/pages/publications/BRICS_Deliverable_D2.2.pdf, 2010.

Kurnicki, Adam, et al. “System Definition, Technical Specifications and Safety Analysis.” ReMeDi, FP7,

2014.

Mallet, Anthony, Cédric Pasteur, Matthieu Herrb, Séverin Lemaignan, and Felix Francois Ingrand.

“GenoM3: Building middleware-independent robotic components.” Robotics and Automation

(ICRA), 2010 IEEE International Conference on. 2010. 4627-4632.

Morgan Quigley i inni. „ROS: an open-source Robot Operating System.” ICRA Workshop on Open

Source Software. 2009.

Moser, Christiane, et al. "User requirements and design guidelines." D1.1, FP7, ReMeDi, 2014.

Noriaki Anado, Takashi Suehiro i Tetsuo Kotoku. „A Software Platform for Component Based RT-

System Development: OpenRTM-Aist.” Simulation, Modeling, and Programming for

Autonomous Robots, Lecture Notes in Computer Science. Berlin: Springer, 2008. 87--98.

Shakhimardanov, Azamat, Nico Hochgeschwender, Kraetzschmark, and Gerhard K. “Component Models

in Robotics Software.” Proceedings of the 10th Performance Metrics for Intelligent Systems

Workshop, PerMIS '10. New York, NY, USA: ACM, 2010. 82--87.

SysML. SysML Open Source Specification Project. http://sysml.org/, brak daty.

Willaert, Bert, Brecht Corteville, Dominiek Reynaerts, and Hendrik Van Brussel. “A Mechatronic

Approach Towards Bilateral Teleoperation for Keyhole Surgery.” International conference on New

Actuators. 2012. 144--147.