1

Towards a Functional System Architecture forAutomated Vehicles

Simon Ulbrich, Andreas Reschka, Jens Rieken, Susanne Ernst, Gerrit Bagschik, Frank Dierkes, Marcus Nolte,and Markus Maurer

Abstract—This paper presents a functional system architecturefor an automated vehicle. It provides an overall, generic structurethat is independent of a specific implementation of a particularvehicle project. Yet, it has been inspired and cross-checked witha real world automated driving implementation in the Stadt-pilot project at the Technische Universitat Braunschweig. Thearchitecture entails aspects like environment and self perception,planning and control, localization, map provision, Vehicle-To-Xcommunication, and interaction with human operators.

Index Terms—Functional System Architecture, Automated Vehi-cles, Autonomous Driving

I. INTRODUCTION

WHEN software is developed for an automated vehicle,it is a bottom-up process for many teams. If existing

building blocks are just hacked together in some way, this willlead to complex system designs. Yet, having a well structuredfunctional system architecture is key. It has central impact onthe system design and technical software development for anautomated vehicle — often for several years.

This paper presents an overall functional system architecturefor an automated vehicle. Implementation independent mod-ules are grouped such that there are clean interfaces amongthese modules.

This functional system architecture differs from others bystrictly using hierarchy and functional separation. It underwentseveral iterations. Earlier versions of this architecture havebeen published by [1]–[9]. It has strongly been influenced bythe functional system architecture developed by Dickmanns’group [10]–[13]. Matthaei & Maurer [8], and Matthaei [9, p.37 ff.]. The following architecture discussions will be basedon the last architecture revision in English [8].

Some refinements have already been published in German[9, p. 37 ff.]. A goal of this article is to make our researchinsights accessible to the international scientific community.Apart from that, several other enhancements have been madeto incorporate more recent research such as the definition ofinterfaces, e.g., the definition of a scene and a situation in

S. Ulbrich was with the Institute of Control Engineering at TU Braun-schweig, Germany and is now with Audi AG.

A. Reschka, J. Rieken, S. Ernst, G. Bagschik, F. Dierkes, and M. Nolte arewith the Institute of Control Engineering at TU Braunschweig, Germany. A.Reschka is currently visiting student researcher at Stanford University.

M. Maurer is professor and head of the Institute of Control Engineering atTU Braunschweig, Germany.

Manuscript received March 10th, 2017.

Ulbrich et al. [14], or functional safety considerations resultingfrom the application of the architecture in the aFAS project[15].

This paper is organized as follows. First different approachesfor structuring driving tasks and their processing levels areintroduced and compared with other approaches in the litera-ture. Then the functional system architecture from this paperis presented by outlining its main columns and clarifyingits interfaces and comprised activities. For each aspect themodifications to the state of the art are presented and providedwith a root cause for these. In the end, open issues arehighlighted and a conclusion is drawn.

II. BACKGROUND

The ISO 26262 standard proposes a functional system archi-tecture as a part of the system design and defines “modu-larity”, “adequate level of granularity”, and “simplicity” asrequirements to the architectural design [16, part 4, p. 13].As a property of a modular system design the ISO 26262proposes “hierarchical design”, “precisely defined interfaces”,“maintainability”, “testability”, and “avoidance of unnecessarycomplexity” (ibid.).

To provide a structure for human behavior, Rasmussen [17]distinguishes “skill-based behavior”, “rule-based behavior”,and “knowledge-based behavior” as three levels of perfor-mance of skilled human operators. On the lowest, skill-basedlevel, reactive, sensory-motor activities take place withoutconscious control. On a rule-based level, decisions are takenbased on a previously stored set of rules. If a situation is notfamiliar and there is no stored rule for it, knowledge-basedbehavior may be applied. Here a new strategy for goal archivalis developed from existing knowledge.

Donges distinguishes “navigation”, “guidance”, and “stabi-lization” as three hierarchical levels of driving tasks in hispublication from 1982 cited in [18]. Similar to Riemersma[19] and Michon [20, p. 489] citing his inaugural lecture from1971, one elementary (operational) layer is used for coursekeeping and speed control, a second (tactical) layer is for anybehavior planning, and a third one is for strategic planning.

Hale et al. [21, p. 1383] suggest that the three levels of drivingtasks and the three levels of Rasmussen are rather orthogonalto each other. While Donges’ driving tasks address what taskis to be solved, Rasmussen addresses how it is solved. Table Iillustrates the relationship between both. Most of the time

arX

iv:1

703.

0855

7v2

[cs

.SY

] 3

0 M

ar 2

017

2

TABLE IEXAMPLES FOR DRIVING TASKS AND THEIR PROCESSING LEVELS BASED

ON HALE et al. [21, P. 1383] DEPICTED AS IN MUIGG [22, P. 8]

Processing levelSkill-based Rule-based Knowledge-based

Dri

ving

task

Navigation Dailycommute

Choicebetween

familiar routes

Navigating inforeign town

GuidanceNegotiating

familiarjunctions

Passing othercar

Controlling askid on icy

roads

StabilizationRoad

followingaround corners

Driving anunfamiliar car

Learner onfirst lesson

stabilization tasks are handled on a skill-based level. However,if for example a car is unknown to the driver, he or she may nothave these subconscious skills to address the task. Yet, thereare learned rules that may be used. If even these rules need tobe formed from knowledge, because it is the first time a freshlearner drives a car, it would be knowledge-based behavior.On a tactical level, passing another car typically involvessituation assessment and a certain amount of stored rules andexperience, e.g., how much of a gap is necessary to overtake.Humans address this typically by rule-based behavior. Last ofall, navigation may display the widest variety of processinglevels in everyday driving. Blindly commuting the same streetevery day without looking for, e.g., changed traffic signs,may almost be skill-based behavior. It becomes rule-based,if it involves active tactical decisions between several routeoptions, and turns into knowledge-based behavior, if a drivernavigates in an unknown city for the first time.

Transferring the concept of driving tasks and processing levelsfrom human drivers to a technical system provides a startingpoint for a technical architecture. In fact, this provides ahierarchical abstraction of driving tasks as in Maurer [13] andMatthaei et al. [23]. Another distinction of tasks for automateddriving may be derived from different processing steps ofperceiving and acting.

Extending work by Zapp [24], who described a functionalcontrol-cycle for automated vehicles, Hock et al. [10] showedan inverted “U” shaped signal flow from sensors to actua-tors with a hierarchical separation of processing levels fordriving tasks. Further specification and an exhaustive systemdescription can be found in Dickmanns et al. [11]. Forinstance in [25, p. 239] and more clearly in [12, p. 441],Dickmanns presents the concept of separating “recognition”from “behavior (execution)” as well as the idea of aggregatingfeatures into abstract symbolic representations. Maurer [13]and Dickmanns [12, p. 185] highlight multiple feedback loopsat different hierarchical levels constituting the signal flowin our architecture nowadays. In [25], [26, p. 595], [12,p. 387], Dickmanns also illustrates the usage of a dynamicknowledge base and background knowledge that is now namedas context1 modeling in this architecture. Additionally, [13, p.40 ff.], [27, p.73 ff.], [28, p. 64], [12, p. 442] identify the

1Context is here understood as the part of discourse that surrounds andrepresents an element.

central role of system capabilities and translate them into ahierarchical structure of abstraction. [12, p. 442] separatesbetween “scene understanding”, “planning”, and “gaze andlocomotion control”. As in [13], the situation assessment israther part of the perception. In [11], situation assessmentstretches into both worlds: the planning column as well asthe perception column. In this article, the goal- and value-independent scene/context modeling is considered to be part ofthe perception column. The goal- and value-specific situationextraction and situation assessment is considered to be partof the planning and control column. The idea of “situationaspects” as a result of a “situation assessment” is presented in[28, p. 51 ff.] for automated driving.

Matthaei [9, p. 25 ff.] provides a more comprehensive literaturereview on the various forms of functional system architecturesthat have been used by different teams in automated drivingas well as in robotics. For a broad literature review the readeris referred to his dissertation. Yet, we would like to relateour article to some recent publications on functional systemarchitectures for automated vehicles.

Tas et al. [29] compare the functional system architectures ofseveral automated vehicles. They summarize the advantagesand disadvantages of a distributed, modular architecture suchas ours. They highlight the importance of fault detection,diagnosis, and self monitoring for system robustness in au-tomated driving. Further, they unify the visual representationof three other vehicle architectures ([30]–[32]) into a com-mon visualization scheme. Here, [29]–[31] use a hierarchicalstructuring of driving tasks as in Donges [18] for “missionplanning”, “behavior and motion planning”, and “vehiclecontrol and actuation”. Perception, localization, and “vehiclestate estimation” (cf. our self monitoring) are not hierarchicallystructured in [29], [31], or [32]. A “scene understanding”or “environment model” seems to be understood similarly toour context/scene modeling as a central point for informationaggregation. [29] suggests to consider Vehicle-To-X (V2X)communication as an “array of redundant sensors”. If such anapproach is chosen, it is of particular importance to keep inmind that V2X information can be uncertain, incorrect, andeven intentionally misleading. Further V2X communicationmight provide information on different levels of abstraction.Hence, we treat information from V2X differently than infor-mation from onboard sensors (cf. section III-C).

Behere & Torngren [33] identify core components in a func-tional system architecture and group them under “perception”,“decision and control”, and “vehicle platform manipulation”.Their components resemble mostly our components. Yet, notall of our components are part of their architecture. Addi-tional components they identified are “energy management”for “battery management” and “regenerative braking” and“reactive control” for reflex responses to unexpected stimuli asin automated emergency braking. For us, energy managementis considered as part of the vehicle. Reactive control is indeedconsidered. It is part of the stabilization module and its low-latency data link explained in section III-B. Like [29], Behere& Torngren [33] consider V2X communication similar to

3

Fig. 1. Functional system architecture of an automated vehicle. Blocks represent modules for activities; arrows show information flows (TS & TL state est.= traffic sign & traffic light (recognition and) state estimation, feature extrac. = feature extraction, dyn. env. = dynamic environment, aug. = augmentation, la.cros. = lane crossing handling, la. cha. = lane change handling, exec. mon.= execution monitoring, HMI = human machine interface, V2X = Vehicle-To-X)

a sensor/actor. Finally, Behere & Torngren [33] identifiedthe otherwise often neglected aspect of “diagnosis and faultmanagement”. We agree to its importance. In our architectureit is part of self perception (cf. section III-A) and executionmonitoring (cf. section III-B).

The functional system architecture presented here has beeninspired by and applied to the Stadtpilot project for automateddriving in urban environments and the aFAS project for anunmanned protective vehicle for highway hard shoulder roadworks. Hence, it is not just a top-down concept from a sketchboard but has actually been proven to work in real worldautomated driving. It underwent several iterations. The foun-dations have been laid in [1]–[3], [5], the concept for contextmodeling has been developed in [4], [6], [8], [14], environmentperception has been refined in [7]–[9], self representation hasbeen addressed by [15], [34], [35], and localization and mapprovision has been discussed by [8], [9], [36]. The remainderof this article will show the status quo of our functionalsystem architecture and present the enhancements comparedto previous publications.

III. FUNCTIONAL SYSTEM ARCHITECTURE

Figure 1 illustrates a revised architecture based on [1]–[9],[11]–[13].

The vertical abstraction layers of the functional system ar-chitecture are aligned to the levels of driving tasks fromDonges [18], Riemersma [19], and Michon [20, p. 498]. Oneelementary (operational) stabilization layer is used for coursekeeping and speed control, a second (tactical) guidance layeris for any behavior planning and a third one is for strategicplanning (navigation). Albus [37, p. 281] suggested the use ofsuch a hierarchical structure not only for behavior planningand control but also for perception. Nothdurft [38] transferredthe concept of Oberlander et al. [39], to differentiate contextinformation in particular for digital maps2 by “topological,”“semantic,” and “metric” properties to the field of automateddriving. In Figure 1, the terms road-level relate to the roadnetwork topology, lane-level to the semantic relationshipsamong lanes and feature-level to the metric properties usedfor a localization within a lane.

Certain modifications have been made by the team at theInstitute of Control Engineering at TU Braunschweig after the

2Based on [40] and [14], we understand a map as a diagrammatic repre-sentation of an area’s scenery.

4

publication of previous architecture versions in Matthaei &Maurer [8] and Matthaei [9, p. 37 ff.]. The following sectionsdescribe the current state of the functional system architectureand discuss the recent modifications. It is explained along theinverted-“U”-shaped main signal flow through the componentsin the architecture.

A. Environment and Self Perception

1) Interfaces: The environment and self perception columnhas interfaces to localization and map provision, behaviorplanning and control, and with the communication column.Within the column there is an interface towards sensors.

Perception has an interface towards the automated vehicle’ssensor systems. They have been clustered into environmentsensors covering external aspects around the vehicle (exte-roceptive) and vehicle sensors to obtain information aboutthe vehicle itself and its internal state (proprioceptive). En-vironment sensors are sensors like cameras, lidar, and radarsensors but also conventional sensors like a thermometer ora rain sensor. Vehicle sensors provide information about themovement or pitch of the ego vehicle, but also informationabout the charging/filling level of the battery/the fuel tank,for example. In a hardware architecture, sensor data featureextraction and even model-based filtering may be allocated toa sensor itself. Yet, in a functional system architecture, theinterface between the sensor block and the subsequent featureextraction is raw sensor data.

Although the perception column is primarily based on sensordata from within the column, it may use map informationtogether with a pose within that map as input on differenthierarchical levels of abstraction. On a macroscale level, thereare topological road network maps used to augment perceivedinformation with a-priori map information. On a mesoscalelevel, lane level map information may be used to augmentcontext modeling even beyond the limited field of view fromon-board sensor systems. On a microscale level within alane, feature information may be used to provide additionallandmarks or to stabilize lane tracking. Likewise an inputmight be Vehicle-To-X information obtained from other trafficparticipants or infrastructure.

The algorithms in the planning and control column are theprimary data user of the perception column. On a navigationlevel, a road network together with a traffic flow may be usedto calculate an optimal route. At the tactical level (guidance),a scene3 as defined in Ulbrich et al. [14] is provided. On anoperational level (stabilization), the perception may providesimple features and state variables as a low latency shortcutto low level control as in Maurer [13, p. 42].

3A scene describes a snapshot of the environment including the scenery anddynamic elements, as well as all actors’ and observers’ self-representations,and the relationships among those entities. Only a scene representation in asimulated world can be all-encompassing (objective scene, ground truth). Inthe real world it is incomplete, incorrect, uncertain, and from one or severalobservers’ points of view (subjective scene) (cf. Ulbrich et al. [14]).

Perceived information is provided on different levels of ab-straction (road-level, lane-level, feature-level) for map updatesor mapping. Sensor data (gyroscopes, wheel tick sensors, ...)from the perception column may directly be used for localiza-tion and map provision.

Last of all, perception data may directly or indirectly be usedfor broadcasting information via Vehicle-To-X communicationor visualization. The authors assume that there will alwaysbe a goal and value specific context selection. Thus, rathera for others as relevant classified situation subset will becommunicated or visualized. Yet, also with this intermediatestep, communication will be at least based on informationfrom perception.

2) Comprised Activities: Figure 2 provides details on theenvironment and self perception. The dashed line symbol-izes the separation between the perspective to the outside(environment perception) and the often neglected perspectiveto the inside (self perception) as in Maurer [13, p. 58 ff.],Bergmiller [41, p. 145 ff.], and Reschka et al. [34]. Similar asin Matthaei [9, p. 51], a green color codes that only relativelycertain internal information has been used. The blue colorindicates that only internal sensors and/or environment sensorinformation has been used. The violet color indicates thatadditionally map data with all possible errors in map-relativelocalization and incorrect, possibly outdated map informationhas been used. The yellow color indicates perceived data usedfor map updates and Vehicle-To-X information.

Sensor data is used for feature extraction and subsequentmodel-based filtering. Feature extraction and model-basedfiltering is performed regarding several aspects. This includeslane detection and tracking, dynamic element tracking4, oc-cupancy grid modeling plus subsequent feature extraction anddata filtering, traffic sign and traffic light recognition and stateestimation, as well as self monitoring of the automated vehicle.Input to this block are raw or processed sensor data andpossibly feature-level map data. Models are used to identifyentities, associate measurements to entity hypotheses and trackentities over time. In lane tracking, dynamic element tracking,and traffic light and traffic sign recognition a temporal valida-tion or tracking is typically performed after an extraction ofrelevant features. In occupancy grid mapping, widely used forthe stationary environment, a similar temporal filtering resultsfrom a probabilistic filtering performed in different cells ofthe occupancy grid itself. Entities and properties of these aregenerated by a subsequent feature extraction from that grid.5

Any of the sensors are mounted to the automated vehicle.Thus, their sensor data will be ego-relative. To transformsensor data into a stationary coordinate system, it is necessaryto estimate ego motion. This is part of the data filtering in selfperception. We further suggest to integrate self monitoring intoself perception. The threshold between a self monitoring and

4“Dynamic objects” form the set of “dynamic elements” by extending themwith non-object-model-compliant elements (cf. Ulbrich et al. [14]).

5The feature extraction and model-based filtering is not discussed in furtherdetail here. Some details are provided in Rieken et al. [7] and will be discussedin a future publication specifically on this topic.

5

Fig. 2. Environment and self perception based on Rieken et al. [7] and Matthaei & Maurer [8]. Green = only subject to vehicle sensor errors; blue = subject toenvironment sensor errors; violet = also subject to map and localization related errors; yellow = perceived environment features and Vehicle-To-X information(dyn. env. = dynamic environment, TS & TL state est. = traffic sign & traffic light (recognition and) state estimation, extrac. = extraction, pas. = passive, IR =infrared sensors, USS = ultrasonic sensors, thermo = thermometer, PMD = photonic mixing device, rain = rain sensors, odomet. = odometry sensors, gyros. =gyroscope, consum. = consumption sensors, fill lev. = fill level sensors, steer. ag. = steering angle sensors, rotation = rotation sensors, acceler. = accelerationsensors, environ. = environment, V2X = Vehicle-To-X)

self representation on a context modeling level seems vagueat first. The authors suggest to use the same differentiationas for other entities. The self monitoring provides informationabout entities of the ego vehicle and their attributes like healthstates or errors. The self representation provides semantic linksbetween those entities to derive a full context not only aboutthe environment but also about the automated vehicle itself.

The information from the feature extraction and model-basedfiltering is used for context/scene modeling (cf. Ulbrich et al.[14]). This subsumes several aspects of information modeling,aggregation, and association. Scenery modeling combines laneinformation with a scenery model. This scenery model mayuse a-priori map data and a position in this map from thelocalization and map provision column in Figure 1. Dynamicenvironment modeling may interact with the scenery modelto incorporate model-based information. Dynamic elements,for example, are more likely to move along lanes or paths.6

Dynamic elements and the scenery are associated with each

6For safety applications and to model non-rule compliant behavior, it isessential that this is only an information augmentation. The initial trackingresults still need to be maintained to avoid crashing into non-rule compliantdynamic elements.

other to obtain an environment model. This is combinedwith the self representation of the ego vehicle to yield acontext/scene model. This scene representation is transmittedto modules in the planning and control column. Matthaei [9,p. 52] differentiates a “local” scenery and scene modelingfrom an “extended” one. The first is solely based on perceivedinformation and incorporates no map-related information. Itsoutput can be used for updating a map with perceived infor-mation. The distinction avoids loops in the information flowand self-confirming hypotheses of confirming map data withmap-supported perception data.

The perception column is completed by modeling a road-levelenvironment. This subsumes a possible road topology andtraffic flow identification to estimate which lanes constituteroads and whether these roads are congested or blocked.7 Sofar, this module has not been implemented in the Stadtpilotor aFAS project. The road network is simply piped throughas it is from an a-priori map from the localization and mapprovision column towards subsequent modules.

7A lane level traffic flow identification may still be considered as part ofthe context modeling.

6

3) Enhancements to the State of the Art: The modificationsare shown towards Matthaei & Maurer [8] as the last broadlyaccessible publication of our functional system architecturein English. The sensors’ block is identical; feature extractionand model-based filtering has only been marginally modifiedregarding the self perception. Here, Matthaei only mentionedthe aspect of “motion estimation” and a rather vague “datafiltering” (Matthaei & Maurer, 2015, p. 162; Matthaei, 2015,p. 51). Yet, as in Maurer [13, p. 58 ff.], Bergmiller [41, p.145 ff.], and Reschka et al. [34] this is only part of the selfperception. It may further include friction coefficient estima-tion, vehicle component wear-and-tear estimation, componentdiagnosis, energy level estimation, etc.

The aspect of traffic sign and traffic lights has marginallybeen modified. Matthaei & Maurer [8, p. 162] called it trafficsign and traffic light “detection”, Matthaei [9, p. 51] called ittraffic sign and traffic light “state estimation”. Of course it isnecessary to detect, recognize the position, type and in caseof traffic lights the state of an element. Other than trackingstationary lane markings/lanes, Matthaei assumes no need foran ego motion compensation for traffic signs and lights. Trafficsign and traffic light estimation has not been implemented inthe Stadtpilot project. If this is purely frame-based, it mayindeed not need an ego motion estimation. If it stabilizes trafficsign/light hypothesis over time, it will need an ego motion.Thus it has been linked by a dotted line.

Context modeling has been restructured. Matthaei’s differenti-ation between “local” scenery/scene modeling and “extended”scenery/scene modeling8 have been both subsumed under onlyone scenery modeling and scene modeling with correspond-ing submodules. A dynamic environment modeling has beenintroduced as an analogon to scenery modeling for staticenvironment aspects. This may include steps of validatingdifferent tracks of dynamic elements against each other. Forinstance if the contours of different elements overlap it mightbe a sign of actually tracking the same object twice rather thanin fact observing a collision. Further, Matthaei & Maurer [8,p. 162] and Matthaei [9, p. 51] called the step of associatingsemantic information about the automated vehicle “vehiclestate modeling”. Aligned with Bergmiller [41, p. 145 ff.], theauthors prefer self representation as a name for this block.Last of all, the name of the overall module seems odd at first.While it is named context modeling, its output is only a scenefrom a scene modeling as in Matthaei & Maurer [8]. With thedefinitions from Ulbrich et al. [14] it is indeed correct to havea scene as an output. Yet, the process itself entails aspectsof context modeling, too. Thus the name of the module isextended to context/scene modeling.

Similar to Matthaei [9, p. 51], road topology identification andmodeling as well as traffic flow identification are summarizedin a block above context modeling. The block has beenrenamed from “road topology and traffic flow modeling” to amore general road-level environment modeling. Linguistically,this makes room to identifying and modeling one day aspects

8This entails information from a-priori map data about static and movableelements.

like ferryboats affecting the mission planning due to limitedoperating hours as a part of this block. Moreover, an arrow be-tween road-level environment modeling and the context/scenemodel has been added to represent such an information flowof high-level road information towards information in a scene.

B. Planning and Control

1) Interfaces: The planning and control column has interfacesto the perception and communication column and towards theactuators within the column. Inputs from perception are:

• A road network together with a traffic flow informationfor navigation.

• The scene described as in Ulbrich et al. [14] for tacticalplanning (guidance).

• Features with state variables as a low latency shortcut tocontrol as in Maurer [13, p. 42].

Outputs exist within the column towards actuators. Theseentail gas, gears, brake commands, and steering. Yet, it mayalso include actuation of other vehicle components like thehorn, indicators, or headlights. It may even include openinga door lock or the trunk for freight delivery or loading, oractivating the wipers for removing dirt from the windscreen.

Interfaces towards the communication column will be detailedin its according section.

2) Comprised Activities: Figure 3 illustrates details on theplanning and control in a functional system architecture. Thecolor coding for information flows is the same as in sectionIII-A. Modules for planning and control use the previouslymentioned scene as a central interface on a tactical level. Themodules have been divided into three levels according to thehierarchy of driving tasks in Donges [18].

On a strategic level (navigation), the road network, informationabout traffic flows or blockages and an externally providedmission are used for navigation purposes. A mission planningas in Dickmanns [12, p. 405] or Gregor et al. [42, p. 81 ff.] en-tails planning certain events like a cargo or passenger pickup.They result in waypoints between which a route needs to beplanned. A route planning yields a —with respect to someoptimization criteria— best route but also route alternatives.9

The calculation of route alternatives may be triggered byevents or upon request of the tactical level (guidance). Thenavigation may consider skill restrictions of an underlyingguidance layer. If, for instance, the battery of an electricvehicle is too low to take a shorter but more energy consumingroute through a mountain area. Route alternatives to reach themission goals are recalculated to reflect ego position changes.

The guidance modules use this information to render a missionexecutable. They use the current scene to select relevantaspects and to augment it with additional information to deriveone or several situation representations for the automated

9The aspect of route alternatives has so far not been implemented in theStadtpilot or aFAS project.

7

Fig. 3. Submodules for planning and control. Blue = subject to environment sensor errors; violet = also subject to map and localization related errors; yellow= Vehicle-To-X information for perception (V2X = Vehicle-To-X, dyn. = dynamic, traj. = trajectory, alt. = alternative, HMI = human-machine-interface)

vehicle. Such a situation is used for situation assessmentand behavior planning regarding several situation aspects.Among those are regular driving within a lane, lane changes,lane crossings (e.g., at intersections), free space navigationfor parking, etc. (cf. “driving maneuvers [...] for automatedvehicles”, [35, p. 122 ff.]). Situation assessment for these sit-uation aspects entails application specific situation assessmentexpert algorithms and also skill and ability monitoring for thatparticular situation aspects. Behavior planning entails not onlymaneuver selection but also planning about how a maneuvershould be executed. This how does not include detailed veloc-ity profile planning but rather a sequence of tactical behaviordecisions like longitudinal and/or lateral adjustments to a gapor stopping points in an intersection, indicator activationsor maybe even honking one’s horn. The guidance blockis completed with execution monitoring of all components,which ensures reliability (continuity of correct service) andavailability (readiness for correct service) [43]. This executionmonitoring has ultimate control over deactivating the systemor its modules.

Output of the tactical guidance layer is a set of target posesfor maneuvers. A target pose commands the stabilization layerwhat to plan for. This may entail a target position, orientation,velocity (and further derivatives), constraints for trajectoryplanning like a drivable area, a reference corridor, sampling

ranges or target deviation costs, and a symbolic maneuver typeinformation.

The maneuver information may be utilized by an underlyingstabilization level to switch between algorithms as in Maurer[13, p. 74]. The target pose may be linked to a vehicle with acertain id to perform longitudinal vehicle following. It may beset to the center of a neighboring lane for lane changing or itis set towards a gap in traffic for longitudinal adjustments toprepare lane changing. For parking, this pose may contain agoal position and orientation in a parking lot. Even at complexintersections, this interface seems sufficient to cover, e.g., tostop at a stop sign, proceed to a line of sight and finally turningthrough a lane with oncoming traffic.

Depending on the actual implementation only one or several10

target poses may be handed over to the stabilization level. Incase of the latter, target pose selection is implicitly done bythe knowledge of selection rules in the stabilization level.

The stabilization subsumes trajectory planning and low levelcontrol, and execution monitoring as three major aspects. Tra-jectory planning calculates trajectory candidates for all thesetarget poses. Low level control translates those trajectories

10So far, only one target pose has been implemented in the Stadtpilot oraFAS project.

8

into actuator control variables. Execution monitoring detectsdeviations between what is planned and executed.

Trajectory planning as in Werling [44], or path planning with asubsequent velocity profile planning as in Kammel et al. [45],Hundelshausen et al. [46], Wille [5], Broggi et al. [47] can begeneralized into a three step procedure of trajectory alternativegeneration, trajectory alternative assessment and selection, andtransforming the results into a representation to be used forlow level control.

A path or trajectory planning may entail a subsampling offurther target poses around the provided target poses as inWerling [44, p. 42]. Based on a cost function, the besttrajectory, according to a cost criteria, is selected.11 Dependingon the implementation of the trajectory planning, it is neces-sary to transform the trajectory from a geo-stationary, localcoordinate system of a scene or situation towards an ego-vehicle bound coordinate system in which the actuators andlow level controllers operate. If trajectory planning is executedin a Frenet frame, this transform is performed as a last step.

A future point on this trajectory is used as input to the lowlevel controllers to command a steering angle, brake pressure,or acceleration rate to the actuators of the automated vehicle.To reduce latency, it may be necessary to obtain direct featureupdates from the previously mentioned model-based filteringalgorithms directly on the stabilization level. These featureupdates may be incorporated into the low level controllers oreven the trajectory planning.

Once more, the stabilization level entails execution monitoringto ensure the correct functioning of these algorithms andpossibly to inform the guidance module about issues onthe stabilization level. Examples of this driving task relevantinformation are if no collision-free trajectory can be calculatedor if the execution of commanded behavior is not possibledue to physical limitations in the vehicle’s dynamics. Thisfeedback is either used for execution monitoring in the tacticallevel or even to adopt the tactical behavior planning (guidance)or strategic mission planning (navigation). For instance, ifchanging lanes to a highway exit lane jam-packed with trafficrequires high relative velocity adjustments and thus highdiscomfort in trajectory planning, it may even affect the routeplanning by avoiding such a maneuver and simply takingan alternate route by choosing a next exit further down thehighway. Likewise, even low level control may provide suchfeedback by reporting control deviations. If a high slip angleindicates issues in vehicle stability, it may even affect tacticalbehavior planning by changing to a lane with better friction.

3) Enhancements to the State of the Art: On the strategiclevel of navigation, the route planning has been renamed to amore general mission planning. When the scope of automateddriving becomes wider, mission planning may not only containroute planning but even mission elements [48, p. 43] likecargo pickup, or refueling. Matthaei & Maurer [8] mention

11Selecting the best point could once more be considered as tactical decisionmaking. Hence, one could argue the necessity of a trajectory selection arbiterblock within the guidance module. For simplicity, it is excluded in Figure 3.

a “selection of a next navigation point” as a submodule of thenavigation block. Only transferring the next navigation pointto a guidance module imposes a severe limitation becauseseveral route alternatives may exist. This can be illustratedin the earlier mentioned example of an automated vehicleperforming a lane change onto an off-ramp jam-packed withtraffic. If there is a high risk to exceed the skills of the vehicle,it may be better to avoid such a risky lane change and accepta marginal detour rather than to enforce exiting where it wasplanned.

This is not only a thought experiment but rather a real worldissue and addressed by the lane advice in Ulbrich & Maurer[49]. For that reason, the authors deviate from Matthaei &Maurer [8] by assuming not only one but several routes asan output of the route/mission planning and dropping the“selection of a next navigation point” altogether. Only if thealternatives are known, an informed tactical decision aboutfollowing or deviating from what was planned at the naviga-tion level is possible. Likewise to incorporate such knowledgeabout limited skills from a tactical level (either from the selfrepresentation as part of the scene) or the situation assessmentand behavior planning itself into the mission planning, anupward facing arrow from guidance to navigation is added.

Deep changes have been made to tactical planning comparedto Matthaei & Maurer [8]. As illustrated in Ulbrich et al. [14] agoals- and value specific context selection and augmentation isadded as an intermediate step between a goals- and value inde-pendent scene and a goals- and value related situation. Theremay be one or several situation data structures for differentaspects of behavior planning. They can be used as an inputor even be augmented by modules for situation assessment.12

For instance, the results of a gap quality assessment mightbe fed back into a situation. That information could be usedin an adaptive cruise control target pose selection module totemporarily reduce a time gap towards a front vehicle to avoidrestricting gap adjustments to a gap slightly in front.

Behavior planning is used as an additional block to reflectnot only a maneuver selection but likewise the earlier intro-duced planning about how a maneuver should be executed.The earlier introduced execution monitoring is added as anadditional block to the planning and control column. No clearopinion has yet been formed if it is necessary to includeexecution monitoring as a separate block or if every blockis supposed to have a sub-aspect of execution monitoring.Yet, as mentioned earlier, it is indeed important to includethe upward information flow from stabilization to guidance.13

It was missing in [8] and has now been added.

The stabilization block has been detailed compared to Matthaei& Maurer [8]. A feature updating block has been added to

12Other than Matthaei & Maurer [8] the authors prefer the less ambiguousterm situation assessment instead of situation analysis. Yet, a situation is ratherthe input of a situation assessment than its output. Only some situation aspectsmay be needed for other modules in situation assessment and thus fed backinto the situation data structure.

13This extension is based on discussions with Professor Chris Gerdes,Stanford University in 2014.

9

reflect the updating process of, e.g., vehicle distances and ve-locities for low latency stabilization (cf. [13, p. 42]). Trajectorytarget poses from the guidance level may be associated todynamic elements. Their dynamic state variables may be up-dated based on more recent information directly from model-based filtering while bypassing the latency induced by themore comprehensive context modeling, situation assessment,and behavior planning. This leads to faster reactions in timecritical scenarios.

The set of actuators has been extended by adding indicators,the horn, door locks, wipers, lights, etc. Matthaei & Maurer [8,p. 164] highlight that some actuators are used for the purposeof tactical communication (cf. “implicit communication” inUlbrich et al. [50]). These actuators (or rather: devices) havenot been part of the functional system architecture so far,neither as part of the communication column nor of theactuator block. Due to their similar nature as activating abrake light, they are all grouped under the actuator module.A module from the tactical guidance level may actuate thosedevices through the operational stabilization level.

At last, the “planning and control” column has been renamedfrom the linguistically ambiguous term “mission accomplish-ment”.

C. Communication

1) Interfaces: The interfaces of the communication are illus-trated in Figure 3. At the strategic level for navigation tasks,a mission may directly be commanded from an operator via ahuman-machine interface or even remotely via Vehicle-To-Xcommunication. The mission may entail a route destination aswell as goal criteria like a route with most comfort in auto-mated driving, shortest travel distance, or the most economicroute alternative. As a feedback, the system may communicatea planned route, resulting from the commanded mission. Yet,the system may even provide route alternatives to an operatorto enhance mission selection. The authors agree with Matthaei[9, p. 56] that for a SAE-level-5 system (cf. [51]) of anautomated vehicle, the only necessary input is on a strategiclevel (navigation). Yet, for the sake of informing an operatoror in case of not-level-5 systems, additional communicationinterfaces are necessary.

At a tactical level for guidance tasks, a situation is used as aninterface for visualization and Vehicle-To-X communication.While the situation for Vehicle-To-X communication may bedifferent from the situation for behavior planning of the egovehicle, it is still a situation because not every aspect that ispart of the scene will be relevant for the (assumed) goals andvalues of any of the information recipients in Vehicle-To-Xcommunication, or legal to be transmitted (cf. “enhancements”section). Likewise, a situation for visualization will probablybe simplified and temporarily smoothed to reduce distraction.Yet, it is still a situation because it shows what is relevantregarding the goals and values of an operator or interestedpassenger. It may entail information about planned maneu-vers as part of the situation aspects derived from planning

and control. Predictive warnings to inform a passenger mayeither be considered as part of the situation or as a separateinformation interface from the guidance module towards thecommunication column.

In the opposite direction (towards perception and map pro-visioning), the communication column provides Vehicle-To-Xinformation to be incorporated into the scene and possibly like-wise on a feature or road level. Likewise, a desired maneuvermay be commanded from an operator to the guidance module[9, p. 57]. This could be to command an operator-initiated lanechange but also to command an emergency stopping maneuveror a driver takeover request.

At the operational level (stabilization), short term warningsmay be issued or desired setpoints commanded [9, p. 57].Short term warnings could be the activation of an electronicstability control system in case of a higher than intendedslipping angle on a low friction road. A desired setpoint couldbe the timegap towards a leading vehicle for an adaptive cruisecontrol driver assistance system. For a future level-5 systemthese interfaces may not be necessary anymore, because bydefinition the system needs to handle all these aspects withoutdriver intervention. Yet, as long as there is a transition be-tween humans used to drive a vehicle by themselves and fullautomation these interfaces may still exist as a legacy for along time.

2) Comprised Activities: An automated vehicle may have acommunication interface for communicating with an operatoror passenger (human-machine interface, HMI), as well asfor technical communication with other traffic participants orthe infrastructure via a Vehicle-To-X (V2X) communicationinterface.

The human-machine interface entails both directions of com-munication: On the one hand, to obtain input from an operatoror passenger and on the other hand to provide information.A special case are automated vehicles being monitored bya central tele-operation unit. Here the aspect of a human-machine interface and the usage of communication networksare combined. Matthaei [9, p. 56] envisions the idea of nav-igation or guidance inputs for traffic management or clearingcorridors for emergency vehicles. For the latter, the reliabilityand guaranteed coverage of current communication networksis an issue. Yet, at least the technically less demanding cen-trally controlled deactivation of an automated driving functionwithin a certain amount of hours could be useful to ensure theabsence of hazardous states caused by a bug, after such a bughas been discovered in the fleet of automated vehicles.

The aspect of Vehicle-To-X communication entails commu-nication with other traffic participants or infrastructure. De-pending on what other vehicles are able to provide the rangeof applications is wide. Current research initiatives like Ko-HAF14 address aspects like obtaining map updates from fleets,collaborative perception, and coordinating cooperative drivingmaneuvers among traffic participants. Algorithms to imple-ment such behavior are spread among the modules in the other

14http://www.ko-haf.de/, visited on Nov. 29th, 2016.

10

three columns of the functional system architecture. Yet, theactual communication interface for 802.11p wireless local areanetwork communication, cellular network communication, orother communication channels is part of this column.

3) Enhancements to the State of the Art: Certain modificationshave been made to the communication column since it waspublished in Matthaei & Maurer [8].

Regarding interfaces, changes have been made to some con-tents of existing arrows. The interface between navigation andcommunication in Matthaei [9, p. 57] is extended by not onlyexchanging a “route” but rather a “mission” as input to thenavigation and by adding the aspect of route alternatives forthe opposite information flow.

While Matthaei15 assumed collaboration happens over theinterface left of the perception column, the authors suggestto use the existing communication interface in the communi-cation column. To the authors, there is no need for a separateinterface in the functional architecture, because aspects fromthe perception column can be exchanged with one interfaceat the very left. To allow an information flow from thecommunication column to the perception and localization andmap provision columns, additional links have been added.

For transmitting Vehicle-To-X information, it is assumed thata full scene will probably never be sent but only a relevantextract of the aspects assumed to be relevant for the informa-tion recipients and their archival of their anticipated goals andvalues (situation for Vehicle-To-X communication). If littleinformation exists about the goals and values of the informa-tion recipients, only obviously irrelevant aspects (e.g., privacy,what was seen inside of buildings by accidentally lookingthrough windows) may be excluded and thus the relevantextract may almost converge against the full information froma scene.16 If legislation and communication channel width willever allow to broadcast a full scene, the aspect of informationselection could be dropped and the link between the perceptionand communication blocks becomes bidirectional.

The localization and map provision column can exchangeV2X information with the communication interface. Thus, theblocks in localization and map provision can receive and sendupdates of map data on all layers of the architecture.

D. Localization and Map Provision

1) Interfaces: The localization and map provision column hasinterfaces with the perception column to exchange:

• road-level map features and map updates,• lane-level map features and map updates,• feature-level map features and map updates, and

15Internal report “Cooperation, Collaboration, and Communication” fromMarch, 2015.

16In the distinction between a scene and situation in Ulbrich et al. [14]the focus was rather on goals and values of a vehicle. Here the distinctionhas similarly been extended towards goals and values to be considered forcommunication as they are stipulated by authorities, e.g., privacy.

• vehicle and environment sensor data.

Further, it has an interface with localization sensors. Accordingto [8], the localization sensors like those in a global navigationsatellite system (GNSS) are not part of the environmentsensors at the bottom but are rather noted on the left dueto providing information on higher abstraction levels.

Within the column, information is exchanged between thedifferent hierarchical levels. The upward information flowrepresents the use of, e.g., low level map features to extracthigher level lane information. Likewise, there is an informationflow downwards: Information about the existence of a roadmight be used to establish semantic relationships and supportlane hypotheses in a lane level map.

2) Comprised Activities: The automated vehicle needs tolocalize itself relative to its maps to make use of informationin these maps. The aspect of map provision entails providingmap information to other modules as well as the process ofmapping and map updating in order to have such informationto share. All these aspects are depicted in Figure 4.

Localization and map provision is executed on different hi-erarchical levels. Nothdurft [38] transferred the concept ofOberlander et al. [39] to distinguish map information bytopological, semantic, and metric properties to the field ofautomated driving. Based on Du et al. [52], Matthaei & Maurer[8] differentiated between macroscale (road-level), mesoscale(lane-level), and microscale (within lane) map information andlocalization in those maps.

On each level, localization sensors provide data input to obtainan absolute, global pose from localization algorithms. Thisinformation is often combined in Bayesian filtering approacheswith inertial movement data (cf. blue data flow in Figure 4)to provide a position even between position fixes from, e.g.,a satellite-based localization sensor. Current approaches aredifferentiated by their depth of data fusion (loosely, tightly,ultra-tightly-coupled) and summarized by Skog & Handel [53]cited by Matthaei [9, p. 43].

An absolute global pose is used together with perceivedenvironment features to obtain a map-relative pose estimation.This map-relative pose is used to retrieve map information andto provide it to modules in the perception column in order toaugment perceived information by map information.

Depending on the implementation, a second data flow fromthe perception column towards the map provision column mayexist. This is to use features and a concurrently obtained map-relative pose to update maps with perceived information. Thisconcurrent map-relative localization and map-updating processmay be repeated on the earlier introduced hierarchical levels.Information may be exchanged between the levels to keepmaps consistent.

Different technologies exist to serve the different (vertical)levels in the functional system architecture with different needsfor accuracy. On a macroscale level (roads), global navigationsatellite system solutions found in today’s vehicle entertain-

11

Fig. 4. Localization and map provision based on Matthaei [9, p. 45]. Blue = subject to environment sensor errors; violet = also subject to map and localizationrelated errors; yellow = features for map updates (estim. = estimation)

ment systems are largely sufficient. For mesoscale localizationon a correct lane as well as for microscale localization withina lane, a higher accuracy is needed. Signal distortions inionospheric layers can be compensated by utilizing differentcarrier frequencies and correction data from ground stationsmay be used to increase accuracy. Yet, accuracy as well asreliability are insufficient to serve as a single, non-redundantsource for localization in automated driving. This becomesparticularly obvious in urban environments or complex multi-level highway interchanges.

All information from the localization and map-provision col-umn is subject to errors in the localization as well as errors inthe maps itself. At the time of writing there is no guaranteeon information integrity and timeliness of data. Thus, incorrectlocalization or map data may possibly propagate to subsequentmodules and compromise decisions and behavior. To ensureawareness of this, every module that uses map data is coloredin violet.

3) Enhancements to the State of the Art: The localizationand map provision has been restructured. The “external data”and “absolute global localization” columns in Matthaei &Maurer [8] have been summarized into one “localization andmap provision” column. “External data” was renamed to mapprovision to ensure that modules are activities, as in the

Unified Modeling Language (UML) standard. Hence, “externaldata” is —similar to, e.g., a “scene”— a data container andthus an arrow rather than a module. We think that the levelof abstraction for “localization” and “external data” seemedless aggregated than for example “perception” or “planningand control” which form other columns. Further, the titles ofthe map provision blocks have been changed towards whatthey actually provide: Maps. The “world modeling” used byMatthaei & Maurer [8] leaves room for confusing it withthe term’s connotation in the community to where it reflectsactivities which are here summarized under context modeling.Matthaei & Maurer [8] did not provide details within themap provision block. The refinements in Matthaei [9, p. 45]within those blocks have now been incorporated to make themaccessible to a non-German-speaking audience.

IV. OPEN ISSUES

Despite long and intense discussions, there are still severalopen issues in the functional system architecture. Three aspectswill be highlighted here:

First of all, the name of the context modeling seems counter-intuitive due to the fact that it only outputs a scene. Indeed, ascene is part of the context and according to its wide definition(cf. section II), a full context may never be represented. Yet,certain context information may be used for better scene

12

modeling. Thus, the term “context modeling” for the overallblock seems more appropriate.

Secondly, in architecture discussions with other researchgroups in the Uni-DAS society17 the idea was voiced forfeedback from stabilization modules towards model-basedfiltering modules. That is, to adopt models if, e.g., the egovehicle is not following a planned trajectory when it is drifting.

Thirdly, no clear answer has yet been provided where a driveror passenger18 monitoring camera should be located in thearchitecture. One could argue that it is irrelevant if an operatorprovides a maneuver input by a button or the camera and thatit thus shall be part of the human machine interface. Likewise,it may be considered as a sensor and part of the perceptioncolumn. A driver or passenger monitoring is so far not part ofthe Stadtpilot project.

Furthermore, an open point is the clear differentiation between“occupancy grid mapping” in the perception column and“feature map provision” in the localization and map provisioncolumn. Occupancy grid mapping is necessary in perceptionfor local dynamic maps, free space extraction, or dynamicclassification. If static elements are aggregated in a globalfeature map, it is part of the map provision column. Hence,the age of features to be typically still maintained in the gridor map is a distinguishing factor, but there is still room for abetter distinction between both.

In current discussions about the potentials and demands ofautomated vehicles, a server-based shared map is a key tothe availability for automated vehicles. It is not explicitlymodeled in the architecture, since we think it is part of theV2X connectivity. A more sophisticated integration into thearchitecture of the ego vehicle seems not helpful, as it wouldchange the focus from the aspired architecture for a singleautomated vehicle towards an overall architecture for a wholetraffic system. That would require several additional aspectslike trusted authorities for information validation or trafficmanagement authorities, which are out of scope of this article.

Moreover, there are still discussions on the point whethernavigation or guidance has ultimate decision power if aplanned route is followed or a route alternative is selected.If a traffic jam is detected, it is clearly a navigation task toadapt the route. Vice versa, if enforcing to take a highway exitwould result in a collision, it is the tactical guidance layer thatdecides to not take the exit and to request a replanned route toreflect the reality of having missed that particular exit. Thereis a gray area in between where following the route is stillwithin the specifications of what the automated vehicle cando, but where in the given situation it is just now, tactically abetter choice to rather pick a route alternative with a minimaldetour to avoid risk or maintain comfort goals. As in section

17Uni-DAS workshop on functional system architectures in October 2015in Darmstadt, Germany. www.uni-das.de

18Might be necessary in a SAE level 5 system to help minors or elderlypassengers for instance in case of a medical emergency situation or to ensurethat they remain seated while driving.

III-B, we see these decisions to be under the decision-makingauthority of the guidance level, but not without controversy.

Another issue is where predictions are to be found in thearchitecture. To the authors, a prediction is rather a tool to beused in several modules. For instance, model-based filteringwill use prediction models. Likewise, a situation predictionmight be necessary in the guidance module or a movementprediction in the stabilization module. One could ask if thereis a prediction even in the context model to provide notonly the current but even future scenes. A possible way toillustrate predictions in the architecture could be to extend thetwo-dimensional architecture by a third dimension in whichprediction is an additional layer. This comes to the price ofvisual distinctiveness and presentability. Another way could beto introduce multiple views on the architecture for particularaspects.

Furthermore, the allocation of self representation to a partic-ular block in the architecture is not as clear as it seems. Forsure, it is mainly a bottom up process to aggregate informationfrom vehicle sensors. Yet, execution monitoring might detectthat a vehicle’s deviation from its intended trajectory is highand thus the maneuver capabilities of that vehicle are limited.In other terms, there is goal and value specific informationfor self modeling in the planning and control column. Hence,certain aspects of self modeling could be spread over severalhierarchical levels and columns in the architecture and thuslimit the conceptual rigorousness that structure diagrams ofthe architecture suggest. Once more, a third dimension with aseparate layer for self representation could alleviate this issue.In this layer not only the self representation, but also all formsof self monitoring and execution monitoring could be placed.The result could be aggregated in the scene/context modeland used for decision making and control in the planning andcontrol column.

Possibly not fully covered is the aspect of cooperation andcompetition between multiple agents. So far, implicitly co-operative behavior [54] and explicit Vehicle-To-Infrastructurecommunication [55] has been implemented in the Stadtpilotproject. Yet, it seems likely that future research on cooperationand competition may not be fully covered in the architecture.We assume an additional view on the architecture might berequired to cover these aspects with all its various facets.

Last of all, the role of Vehicle-To-X communication is stillsubject to discussions. While the current communication col-umn is eligible to broadcast information from the planningand control or perception column, an opposite communicationflow for Vehicle-To-X data input is harder to incorporate.Currently, this induces a right-to-left information flow thatcontradicts the main signal flow direction otherwise goingfrom left-to-right. A workaround would be once more toopen a third dimension or additional view for Vehicle-To-X communication as it has interfaces with many blocks. Apossible implementation specific addition to the architecturecould be a data flow from the decision modules to other trafficparticipants or the infrastructure via Vehicle-To-X and viceversa. E.g., the selected route, the selected maneuver as part

REFERENCES 13

of the situation for Vehicle-To-X communication, or a plannedtrajectory on the stabilization level.

V. CONCLUSIONS

This article presented a refined functional system architecturefor an automated vehicle. The concept of hierarchy andfunctional separation has been introduced and applied. Theinterfaces between the modules have been detailed and themodifications to the state of the art have been presented. To theauthors, this functional system architecture is still an organicstructure that will be modified and refined to address the openissues.

ACKNOWLEDGMENT

The authors would like to thank the Stadtpilot team at TUBraunschweig for their valuable inputs and reviews of thisarticle. We thank Volkswagen Group Research and the Auto-Pilot team for their financial and institutional support of theunderlying research of this article. Further we thank the Fed-eral Ministery for Economic Affairs and Energy for fundingour research as part of the aFAS project, the German ResearchFoundation for supporting us in the Controlling ConcurrentChange project, and the Daimler and Benz Foundation forfunding the Value Based Decision Making project. We thankProf. Mykel Kochenderfer for hosting Andreas Reschka atStanford Intelligent Systems Lab while this article was written.

REFERENCES

[1] Wille, J. M., Saust, F. & Maurer, M., “Comprehensivetreated sections in a trajectory planner for realizingautonomous driving in Braunschweig’s urban traffic,”International IEEE Conference on Intelligent Trans-portation Systems (ITSC), pp. 647–652, 2010.

[2] Saust, F., Bley, O., Kutzner, R., Wille, J. M., Friedrich,B. & Maurer, M., “Exploitability of vehicle relatedsensor data in cooperative systems,” in InternationalIEEE Conference on Intelligent Transportation Systems(ITSC), 2010, pp. 1724–1729.

[3] Reschka, A., Bohmer, J. R., Gacnik, J., Koster, F.,Wille, J. M. & Maurer, M., “Development of Soft-ware for Open Autonomous Automotive Systems inthe Stadtpilot-Project,” in International Workshop onIntelligent Transportation (WIT), 2011, pp. 81–86.

[4] Nothdurft, T., Hecker, P., Frankiewicz, T., Gacnik, J.& Koster, F., “Reliable Information Aggregation andExchange for Autonomous Vehicles,” in IEEE VehicularTechnology Conference (VTC), 2011, pp. 1–5.

[5] Wille, J. M., “Manoverubergreifende autonomeFahrzeugfuhrung in innerstadtischen Szenarienam Beispiel des Stadtpilotprojekts (English Title:Maneuver-Comprehensive Autonomous VehicleGuidance in Urban Scenarios in the Example of theStadtpilot Project),” Dissertation, TU Braunschweig,2012.

[6] Ulbrich, S., Nothdurft, T., Maurer, M. & Hecker, P.,“Graph-based context representation, environment mod-eling and information aggregation for automated driv-ing,” in IEEE Intelligent Vehicles Symposium (IV), 2014,pp. 541–547.

[7] Rieken, J., Matthaei, R. & Maurer, M., “TowardPerception-Driven Urban Environment Modeling forAutomated Road Vehicles,” in International IEEE Con-ference on Intelligent Transportation Systems (ITSC),2015, pp. 731–738.

[8] Matthaei, R. & Maurer, M., “Autonomous driving -a top-down-approach,” At - Automatisierungstechnik,vol. 63, no. 3, pp. 155–167, 2015.

[9] Matthaei, R., “Wahrnehmungsgestutzte Lokalisierung infahrstreifengenauen Karten fur Fahrerassistenzsystemeund automatisches Fahren in urbaner Umgebung (En-glish Translation: Perception-Supported Localization inLane-Level Maps for Advanced Driver Assistance Sys-tems and Automated Driving in Urban Environments),”Dissertation, TU Braunschweig, 2015.

[10] Hock, C. & Dickmanns, E. D., “Intelligent Navigationfor Autonomous Robots Using Dynamic Vision,” inCongress of the International Society for Photogram-metry and Remote Sensing (ISPRS), 1992, pp. 900–915.

[11] Dickmanns, E., Behringer, R., Dickmanns, D., Hilde-brandt, T., Maurer, M., Thomanek, F. & Schiehlen,J., “The seeing passenger car ’VaMoRs-P’,” in IEEEIntelligent Vehicles Symposium (IV), 1994, pp. 68–73.

[12] Dickmanns, E. D., Dynamic Vision for Perception andControl of Motion. London, United Kingdom: SpringerInternational Publishing, 2007.

[13] Maurer, M., “Flexible Automatisierung von Straßen-fahrzeugen mit Rechnersehen (English Translation:Flexible Automation of Road Vehicles with MachineVision),” Dissertation, Universitat der Bundeswehr inMunchen, 2000.

[14] Ulbrich, S., Menzel, T., Reschka, A., Schuldt, F. &Maurer, M., “Defining and Substantiating the TermsScene, Situation, and Scenario for Automated Driving,”in International IEEE Conference on Intelligent Trans-portation Systems (ITSC), 2015, pp. 982–988.

[15] Stolte, T., Reschka, A., Bagschik, G. & Maurer, M.,“Towards Automated Driving: Unmanned ProtectiveVehicle for Highway Hard Shoulder Road Works,” inIEEE International Conference on Intelligent Trans-portation Systems (ITSC), 2015, pp. 672–677.

[16] ISO 26262:2011 Road vehicles - Functional safety.Geneva, Switzerland: International Organization forStandardization (ISO), 2011.

[17] Rasmussen, J., “Skills, rules, and knowledge; signals,signs, and symbols, and other distinctions in humanperformance models,” IEEE Transactions on Systems,Man, and Cybernetics, vol. SMC-13, no. 3, pp. 257–266, 1983.

[18] Donges, E., “A Conceptual Framework for ActiveSafety in Road Traffic,” Vehicle System Dynamics: In-ternational Journal of Vehicle Mechanics and Mobility,vol. 32, no. 2-3, pp. 113–128, 1999.

REFERENCES 14

[19] Riemersma, J. B. J., “Perception in traffic,” UrbanEcology, vol. 4, no. 2, pp. 139–149, 1979.

[20] Michon, J. A., “A Critical View of Driver BehaviorModels: What Do We Know, What Should We Do?”In Human Behavior and Traffic Safety, Evans, L. &Schwing, R. C., Eds., New York, USA: Springer In-ternational Publishing, 1985, pp. 485–524.

[21] Hale, A. R., Stoop, J. & Hommels, J., “Human errormodels as predictors of accident scenarios for designersin road transport systems,” Ergonomics, vol. 33, no. 10-11, pp. 1377–1387, 1990.

[22] Muigg, A., “Implizites Workloadmanagement - Konzepteiner zeitlich-situativen Informationsfilterung im Auto-mobil (English Translation: Implicit Workload Manage-ment - Concept of a Temporal-Situative Information Fil-tering in an Automobile),” Dissertation, TU Munchen,2009.

[23] Matthaei, R., Reschka, A., Rieken, J., Dierkes, F.,Ulbrich, S., Winkle, T. & Maurer, M., “AutonomousDriving,” in Handbook of Driver Assistance Systems -Basic Information, Components and Systems for ActiveSafety and Comfort, Winner, H., Hakuli, S., Lotz, F. &Singer, C., Eds., Springer, 2016, pp. 1519–1556.

[24] Zapp, A., “Automatische Straßenfahrzeugfuhrung durchRechnersehen (English Translation: Automated RoadVehicle Guidance by Computer Vision),” Dissertation,Universitat der Bundeswehr in Munchen, 1988.

[25] Dickmanns, E. D. & Graefe, V., “Dynamic monocu-lar machine vision,” Machine Vision and Applications,vol. 1, no. 4, pp. 223–240, 1988.

[26] Dickmanns, E. & Muller, N., “Scene recognition andnavigation capabilities for lane changes and turns invision-based vehicle guidance,” Control EngineeringPractice, vol. 4, no. 5, pp. 589–599, 1996.

[27] Siedersberger, K.-H., “Komponenten zur automatis-chen Fahrzeugfuhrung in sehenden (semi-)autonomenFahrzeugen (English Translation: Components forAutomated Vehicle Guidance in Perceiving (Semi)-Autonomous Vehicles),” Dissertation, Universitat derBundeswehr in Munchen, 2003.

[28] Pellkofer, M., “Verhaltensentscheidung fur autonomeFahrzeuge mit Blickrichtungssteuerung (English Trans-lation: Behavior Decision Making for Autonomous Ve-hicles with Gaze Control),” Dissertation, Universitat derBundeswehr in Munchen, 2003.

[29] Tas, O. S., Kuhnt, F., Zollner, J. M. & Stiller, C.,“Functional system architectures towards fully auto-mated driving,” in IEEE Intelligent Vehicles Symposium(IV), 2016, pp. 304–309.

[30] Wei, J., Snider, J. M., Gu, T., Dolan, J. M. & Litkouhi,B., “A behavioral planning framework for autonomousdriving,” in IEEE Intelligent Vehicles Symposium (IV),2014, pp. 458–464.

[31] Jo, K., Kim, J., Kim, D., Jang, C. & Sunwoo, M.,“Development of Autonomous Car - Part II: A CaseStudy on the Implementation of an Autonomous Driv-ing System Based on Distributed Architecture,” IEEE

Transactions on Industrial Electronics, vol. 62, no. 8,pp. 5119–5132, 2015.

[32] Kunz, F., Nuss, D., Wiest, J., Deusch, H., Reuter,S., Gritschneder, F., Scheel, A., Stubler, M., Bach,M., Hatzelmann, P., Wild, C. & Dietmayer, K., “Au-tonomous driving at Ulm University: A modular, robust,and sensor-independent fusion approach,” in IEEE In-telligent Vehicles Symposium (IV), 2015, pp. 666–673.

[33] Behere, S. & Torngren, M., “A functional architecturefor autonomous driving,” in Proceedings of the FirstInternational Workshop on Automotive Software Archi-tecture, 2015, pp. 3–10.

[34] Reschka, A., Bagschik, G., Ulbrich, S., Nolte, M.& Maurer, M., “Ability and skill graphs for systemmodeling, online monitoring, and decision support forvehicle guidance systems,” in IEEE Intelligent VehiclesSymposium (IV), 2015, pp. 933–939.

[35] Reschka, A., “Fertigkeiten- und Fahigkeitengraphenals Grundlage des sicheren Betriebs von automa-tisierten Fahrzeugen im offentlichen Straßenverkehr instadtischer Umgebung (English Translation: Skill andAbility Graphs as Basis for Safe Operation of Auto-mated Vehicles in Urban Environments),” Dissertation,forthcoming, TU Braunschweig, 2017, forthcoming.

[36] Matthaei, R., Bagschik, G. & Maurer, M., “Map-relative localization in lane-level maps for ADAS andautonomous driving,” in IEEE Intelligent Vehicles Sym-posium Proceedings, 2014, pp. 49–55.

[37] Albus, J. S., “Mechanisms of planning and problemsolving in the brain,” Mathematical Biosciences, vol. 45,no. 3, pp. 247–293, 1979.

[38] Nothdurft, T., “Ein Kontextmodell fur sicherheitsrele-vante Anwendungen in der autonomen Fahrzeugfuhrung(English Title: A Context Model for Safety-RelevantApplications in Automated Vehicle Guidance),” Disser-tation, TU Braunschweig, 2014.

[39] Oberlander, J., Uhl, K., Zollner, J. M. & Dillmann,R., “A region-based SLAM algorithm capturing metric,topological, and semantic properties,” in InternationalIEEE Conference on Robotics and Automation (ICRA),2008, pp. 1886–1891.

[40] Oxford Dictionaries, Map. URL: https : / / en .oxforddictionaries . com / definition / map (visited on02/10/2017).

[41] Bergmiller, P., Towards Functional Safety in Drive-by-Wire Vehicles. Berlin, Germany: Springer InternationalPublishing, 2015.

[42] Gregor, R., Lutzeler, M., Pellkofer, M., Siedersberger,K.-H. & Dickmanns, E., “EMS-Vision: a perceptualsystem for autonomous vehicles,” IEEE Transactionson Intelligent Transportation Systems, vol. 3, no. 1,pp. 48–59, 2002.

[43] ISO/IEC/IEEE 24765: Systems and software engineer-ing – Vocabulary. International Organization for Stan-dardization (ISO), 2010.

[44] Werling, M., “Ein neues Konzept fur die Trajek-toriengenerierung und -stabilisierung in zeitkritischenVerkehrsszenarien (English Translation: A Novel Con-

15

cept for Trajectory Generation and Stabilization inTime-Critical Traffic Scenarios),” Dissertation, Karls-ruher Institut fur Technologie, 2010.

[45] Kammel, S., Ziegler, J., Pitzer, B., Werling, M., Gindele,T., Jagzent, D., Schroder, J., Thuy, M., Goebl, M.,Hundelshausen, F. von, Pink, O., Frese, C. & Stiller, C.,“Team AnnieWAY’s autonomous system for the 2007DARPA Urban Challenge,” Journal of Field Robotics,vol. 25, no. 9, pp. 615–639, 2008.

[46] Hundelshausen, F. von, Himmelsbach, M., Hecker, F.,Mueller, A. & Wuensche, H.-J., “Driving with tentacles:Integral structures for sensing and motion,” Journal ofField Robotics, vol. 25, no. 9, pp. 640–673, 2008.

[47] Broggi, A., Buzzoni, M., Debattisti, S., Grisleri, P.,Laghi, M. C., Medici, P. & Versari, P., “Extensive Testsof Autonomous Driving Technologies,” IEEE Trans-actions on Intelligent Transportation Systems, vol. 14,no. 3, pp. 1403–1415, 2013.

[48] Gregor, R., “Fahigkeiten zur Missionsdurchfuhrung undLandmarkennavigation (English Translation: Abilitiesfor Mission Execution and Landmark Navigation),”Dissertation, Universitat der Bundeswehr in Munchen,2002.

[49] Ulbrich, S. & Maurer, M., “Situation Assessment inTactical Lane Change Behavior Planning for AutomatedVehicles,” in International IEEE Conference on Intelli-gent Transportation Systems (ITSC), 2015, pp. 975–981.

[50] Ulbrich, S., Großjohann, S., Appelt, C., Homeier, K.,Rieken, J. & Maurer, M., “Structuring Cooperative Be-havior Planning Implementations for Automated Driv-ing,” in International IEEE Conference on IntelligentTransportation Systems (ITSC), 2015, pp. 2159–2165.

[51] Society of Automotive Engineers (SAE), Taxonomy andDefinitions for Terms Related to On-Road Motor VehicleAutomated Driving Systems J3016 Sep. 2016 (revised),Society of Automotive Engineers (SAE), 2016.

[52] Du, J., Masters, J. & Barth, J., “Lane-level positioningfor in-vehicle navigation and automated vehicle loca-tion (AVL) systems,” in International IEEE Conferenceon Intelligent Transportation Systems (ITSC), 2004,pp. 35–40.

[53] Skog, I. & Handel, P., “State-of-the-Art In-Car Naviga-tion: An Overview,” in Handbook of Intelligent Vehi-cles, Eskandarian, A., Ed., London, United Kingdom:Springer International Publishing, 2012, pp. 435–462.

[54] Ulbrich, S. & Maurer, M., “Probabilistic online POMDPdecision making for lane changes in fully automateddriving,” in International IEEE Conference on Intelli-gent Transportation Systems (ITSC), 2013, pp. 2063–2067.

[55] Saust, F., Wille, J. M. & Maurer, M., “Energy-Optimized Driving with an Autonomous Vehicle inUrban Environments,” in IEEE Vehicular TechnologyConference (VTC Spring), 2012, pp. 1–5.

Simon Ulbrich is currently working at Audi AG onpiloted driving systems and is pursuing his PhD atTU Braunschweig at the Institute of Control Engi-neering. Before that, he finished a Diploma Degreein Electrical Engineering/Industrial Engineering atTU Braunschweig and a Master of Science Degreein Industrial and Systems Engineering at the GeorgiaInstitute of Technology. His main research interestsare tactical behavior planning and context modelingfor automated driving.

Andreas Reschka is currently pursuing a PhD atthe Institute of Control Engineering at TU Braun-schweig and visiting the Stanford Intelligent Sys-tems Laboratory. He holds a Bachelor of Sciencein Network Computing from Technische UniversitatBergakademie Freiberg and a Master of Science inInformation Management and Information Technol-ogy from Universitat Hildesheim. His main researchtopics are self-awareness, safe behavior, functionalsafety, and development processes for autonomousvehicles.

Jens Rieken works as a research assistant at the In-stitute of Control Engineering at TU Braunschweigsince 2012 and is currently pursuing his PhD. Heholds a Master of Science Degree in Electrical En-gineering from TU Braunschweig. His main researchtopics are environment perception and scene under-standing for urban scenarios, as well as designingalgorithms for point cloud processing.

Susanne Ernst is with the Institute of Control Engineering at TU Braun-schweig since 2015 and is currently pursuing her PhD. She holds a Masterof Science Degree in Mechanical Engineering from TU Braunschweig. Hermain research topics are behavior recognition of other traffic participants anddecision making for automated vehicles.

Gerrit Bagschik works as a research assistant atthe Institute of Control Engineering at TU Braun-schweig since 2013 and is currently pursuing hisPhD. He holds a Master of Science Degree inComputer and Communications Engineering fromTU Braunschweig. His main research topics arefunctional safety and scenario based hazard analysisfor automated vehicles.

16

Frank Dierkes works as a research assistant atthe Institute of Control Engineering at TU Braun-schweig since 2013 and is currently pursuing hisPhD. He holds a diploma degree in ComputerEngineering from RWTH Aachen University. Hismain research topic is behavior generation underuncertainty for automated vehicles.

Marcus Nolte works as a research assistant at the In-stitute of Control Engineering at TU Braunschweigsince 2014 and is currently pursuing his PhD. Hereceived his Master of Science in Electrical Engi-neering from TU Braunschweig. His main researchtopics are self-awareness and motion planning forover-actuated automated vehicles.

Markus Maurer has held the chair for Vehicle Elec-tronics at TU Braunschweig since 2008. His mainresearch interests include autonomous road vehicles,driver assistance systems and automotive systemsengineering. From 2000 to 2007 he was active inthe development of driver assistance systems at AudiAG.