proceedings of the first international workshop on ... · micro-reboot support for multi-server...

Proceedings of The First International Workshop on Proceedings of The First International Workshop on DDependable ependable UUbiquitous biquitous NNodes odes (IWDUN 2007)(IWDUN 2007)in conjunction within conjunction withthe International Symposium the International Symposium on Ubiquitous Computing Systems(UCS2007)on Ubiquitous Computing Systems(UCS2007)

Tokyo Denki University Kanda CampusTokyo Denki University Kanda Campus, , Tokyo, Japan Tokyo, Japan November. 25th, 2007November. 25th, 2007

Purpose and Scope • Recent years have seen a rapid proliferation of micro ubiquitous nodes. As well as off-the-shelf

cellular phones, mobile audio players, and portable game devices, wireless sensor nodes are of great interest for researchers especially for developing ubiquitous computing systems. Among those proposed by the academia, particle nodes and Motes have been commercialized so that we can deploy them for practical applications like environmental monitoring, remote health care, and context captures. Despite the range of development of sensor nodes, systems support that ensures their dependability has not been investigated. We believe that the use of such micro ubiquitous nodes in practical applications necessitates the systems support that enables users to leverage from those applications without being burdened by malfunction of the nodes, malicious users attacking them, or inconsistent behaviors of them. The primary motivation of this workshop is hence to finding research issues inherent in achieving dependable micro ubiquitous nodes, looking at existing research on them, and systems on them. The intention of this workshop is to bring together researchers and practitioners from a wide variety of disciplines with the goal to discuss, identify, share experiences and formalize key issues surrounding the challenge of building dependable micro ubiquitous nodes. The immediate goal will be to investigate the key issues from a variety of angles influenced by experience and background of the participants. The ultimate goal will be to formalize the idea of dependability for micro ubiquitous nodes and to define research agenda to stimulate further research. Submission to the workshop should combine at least one of the following aspects:

• Application Scenarios Deserving Dependability: What kind of application scenarios will be benefited from what kind of dependability? What are the hurdles in deploying real world application scenarios with micro ubiquitous nodes, such as sensor nodes? What sorts of dependability can be provided by applications themselves?

• Middleware Support for Dependability: What kind of middleware systems can be provided to achieve dependability? What kind of real world application scenarios require middleware support for dependability? What kind of dependability can be provided by middleware systems?

• Operating Systems Support for Dependability: What kinds of functionalities should be provided by operating systems on micro ubiquitous nodes like sensor nodes? Which functionalities have been covered by the existing embedded product/research operating systems? What directions should the operating systems research attack to?

• Surveys on Dependability for Embedded Systems: What kinds of sensor nodes have been proposed? How many of them have focused on or mentioned dependability? What kind of dependability is achieved by them?

• Hideyuki Tokuda, Keio University(Japan)• Kazunori Takashio, Keio University(Japan)• Jin Nakazawa, Keio University(Japan)• Masayuki Iwai, Keio University(Japan)• Michael Beigl, TU Braunschweig (Germany)

Workshop Organizers

iwdun2007

1

Program

15:00 - 15:10 Opening remarksHide Tokuda

15:10 - 16:10 Paper SessionKernel Data Structure-based Runtime Monitoring Lei Sun (student), Hiroo Ishikawa, Tatsuo NakajimaDepartment of Computer Science, Waseda University, Tokyo, Japan

Micro-Reboot Support for Multi-Server Operating Systems Hiroo Ishikawa and Tatsuo Nakajima Department of Computer Science, Waseda University, Tokyo, Japan

An Operating System Support for Dependability of Micro UbiquitousNodes Hideyuki Tokuda, Junichi Yura, Jin Nakazawa, Masayuki Iwai and Kazunori TakashioGraduate School of Media and Governance, Keio University

16:10 - 16:30 Break

16:30 - 17:00 Invited Talk I*Michael Beigl, TU BraunschweigTitle: Using Redundancy for Dependability in Ubiquitous Computing System

17:00 - 17:30 Invited Talk II*Christian Decker, TeCOTitle:Feedback Control of Ubiquitous Computing Systems

17:30-17:50 discussion

iwdun2007

2

Invited Talk I

• 16:30 - 17:00

• Michael Beigl, TU Braunschweig

Title: Using Redundancy for Dependability in Ubiquitous Computing System

• Abstract:

In Ubiquitous Computing Systems, small, tiny and low-priced sensor equipped hardware is often used to build up appliances, smart objects and smart environments. Such small devices are not calibrated, so overall these devices suffer from systematic and statistical error of the sensors. The thesis of this talk is, that high redundancy even with low-reliable sensor systems leads to high reliability, if scalability issues are solved. This talk will present methods to work around that problem, and to even reach high dependability using low-reliable sensor sources. The talk introduces the use of the proposed method in a concrete setting in the context of the EU funded RELATE project. Some examples and improvements of this method over a naive approach will be shown.

iwdun2007

3

Using Redundancy for Dependability in Ubiquitous Computing SystemeThe First International Workshop on Dependable Ubiquiotus Nodes (IWDUN)

Michael BeiglTU BraunschweigInstitut für Betriebssysteme und Rechnerverbundwww.duslab.de

Michael Beigl Use Redundancy for Dependability 2

Motivation: Ubicomp Sensor Systems

Target application scenariosHigh node densityEnvironmental habitat monitoringUbiquitous computingHuman interactionsAugmented objects

The Problem (and a part of the solution)Many simple, small, low cost nodesSingle measurement is not so precise => mass effect of low cost hardware, consumer electronicNoise and errors occur

iwdun2007

4


Some Examples of Ubicomp Sensor Systems

RELATE systemPeer to Peer localisationBased on Ultrasound (and Infrared, andRSSI, and magnetic, and…)Low cost hardwareMobileStand-aloneDistributed operationRelative positions and angles


Measurement Errors

Two types of errors:

Systematic▫ Are repeatable▫ Depend on the environment,

temperature, hardware decalibration

Statistical▫ Thermal noise▫ Induction

Addressing ErrorsStatistical▫ Repeated measurements average out the error

Systematic errors▫ Manual / Automatic calibration (example: temperature)▫ Knowing the type of error beforehand

iwdun2007

5


Error Analysis

Systematic Error are repeatable, huge and critical

Angle of arrival measurements taken by a single node in different spatial configurations. The error bars indicate the tenth and ninetieth percentile levels of the observed errors at a given angle. The angle-of-arrival error offsets for a given system state tend to be tightly constrained, and thus systematic


Measurement Error Model

Model for errors:r~= measured valuer = real valuen = noise (statistical error)t = system state (a vector containing position, angle, calibration, temperature, air pressure….)f(t,r) systematic error

Error of angle measurementsin RELATE

iwdun2007

6


Idea: Average out using redundancy

Idea: measure the same quantity (e.g. position) with many nodes redundantly and average the results

the statistical error can be “averaged out”The systematic error among nodes is different, therefore can be averaged out if its distribution over the nodes is bias-free

To achieve this:The systematic f(t,f) must be averaged over t (system state)The systematic error must mainly depend on one component in the system state which can be varied during runtime. Here:the systematic error mainly depends on the receive angle


Motivation of averaging statistical errors

Removal of systematic error using data collected from three nodes to estimate the range between two others:• Systematic error: accuracy can be improved by averaging as the systematic error of different nodes is also different among them• statistical error: is random => averaging helps always

iwdun2007

7


Multiple, Simultaneous Measurements

Steps through the process1) A transmits: B to F can receive

and determine their relative position

2) B transmits: A, C to F can receive and determine their relative position

3) C to F can put their two measurements together (vector addition) and determine position A to B

4) We have 4 different measurements of the vector A to B (held by C to F)

5) The 4 results can be averaged to reduce the systematic and statistical error


The Problem: Scalability

The more nodes are averaging out the error thebetter the resultBut: The more nodes are averaging, the moretraffic on the networkIdea: use of cooperativetransmissionwithnon-destructiveoverlay of signalsMulti-SDJSOnly ONE Packetinstead of N

iwdun2007

8


Non-destructive overlay of signals

TransceiverNo complex (De-) Modulation. In our case: no detection of summation signalsBuild using simple analog components onlyNo Synchronisation of phase or carrier frequencyAddressing the destructive interference problem with overlaying noise

CommunicationN to 1 communication of non-coherent sourcesOverlay = Summation in the channelSummation may be detectedTransceiver uses signal/energydetection


Parameter Estimation in SDJS

Received Data is estimated slot-by-slotInformation is then computed/addedEstimation process:

Start: count the number of received jam signals

1. ML-Point estimation:Give an estimationFor k (MLE) 2. MAP-Confidence interval:

Give an interval, [kmin,kmax] that contains the actual k with a given confidence (e.g. 90%)

In both cases: look-up table that can be prepared (no computation on nodes necessary)

iwdun2007

9


Simulation of Location Errors

RELATE US nodes modelError Models (from 500000 RELATE measurements):

Angle of arrival: ▫ Systematic: Gaussian µ=0, sigma=14°▫ Statistical: Gaussian µ=0, sigma=1.4°

Angle of emission:▫ Uniform over -45°..45°

Ranges:▫ Systematic error: Right-side half-Gaussian with µ=0,

sigma=3.5cm▫ Statistical error: Gaussian µ=0; sigma=4.9mm


Simulation of Location Errors

SDJS ModelData fusion resolution 2cm, 2° (SDJS scheme resolution)Errors through SDJS are negligible, as SDJS can be parameterized appropriately (statistics in the network protocol)

iwdun2007

10


Simulation: Statistical Error

50 nodes in a single cell, 5m by 5m room1000 different setups with random locations of the 48 “helper” nodes


Simulation: Direct Measurement and with removing statistical Error

1000 Results for direct and averaged measurements (50 nodes): the averaged measurements are closer to the real value

Gradual improvements when the number of nodes increases after removing statistical error

iwdun2007

11


Benchmark Results

Accuracy improvements


Results of enhancement processReal World settings

80,8%17mm90mm30

80,7%16mm85mm25

85,4%6mm48mm20

85,7%7mm48mm15

Enhance-mentiSERNaïve Approach

Settings with max. distance error in %

iwdun2007

12


Systematic error occur in sensor systemsEspecially in ultrasound systemDue to decalibration of the transducersDue to production variation

Redudancy can solve this problem Scalability can be used by special RF-schemes: Broadcast nature of sound and RFPeer-to-peer scenariosWorks fully distributed

Conclusion


Using Redundancy for Dependability in Ubiquitous Computing Systeme

THANK YOU FOR YOUR ATTENTION!

QUESTIONS?

iwdun2007

13

Invited Talk II

• 17:00 - 17:30•

Christian Decker, TeCO

• Title:Feedback Control of Ubiquitous Computing Systems

• Abstract:Ubiquitous computing systems run a variety of jobs and tasks. They encounter highly dynamic situations while providing services with quality demands. Cooperation and collaboration among the computing nodes is required to enable efficient and dependable data processing. Thistalk will introduce a control theory approach in operating systems to form a new framework for cooperation and collaboration. Selected examples will show the applicability and benefits of this approach.

iwdun2007

14

Kernel Data Structure-based Runtime Monitoring

Lei Sun, Hiroo Ishikawa, Tatsuo NakajimaDepartment of Computer Science,Waseda University, Tokyo, Japan

{sunlei, ishikawa, tatsuo}@dcl.info.waseda.ac.jp

Abstract

In this paper, a kernel data structure-based runtime moni-tor is presented for commodity systems in microkernel ar-chitecture. Inside the monitor, the protection specificationof the kernel data structure has been introduced to checksystem runtime consistency. The specification that consistsof a set of consistency constraints and corresponding re-pair actions provides the normal behavior regulations forkernel objects. During its runtime deployment, once in-consistency has been detected, the monitor will use relatedactions to recover the system.The evaluation experimentresults indicate our prototype system could correctly de-tect the presence of security attacks and system anomalybehavior. Moreover the monitor system is fully software-based without introducing any specific hardware. And themonitor also requires no modifications to the ABI betweenthe application program and guest OS, so that legacy ap-plications could be reused.

1 IntroductionNowadays with the rapid development, the consumer prod-ucts exhibit more and more complicated behavior. Lack-ing of the system runtime support, some anomaly insidesuch complex systems are often hard to reproduce and di-agnose. Moreover embedded systems suffer the securityattacks as well. For instance, modern consumer productsare usually equipped with network connectivity so that theycould download data and even execute it. As for virus pro-grams, they might inject malicious code into the target hostthrough the Internet and then compromise the system. Forlacking of enough technical knowledge, the users usuallycould not solve such security problems themselves. Thererises the problem that how to guarantee the system reliabil-ity at runtime.

Our research contributes the monitoring method whichextracts system runtime information from certain criticalkernel data structures to help enhance system level relia-bility. To our vision, modern embedded system designs arestill limited by system resource. Traditional logging-based

system trace and diagnosis solutions usually require largepersistent memory to store the logging data. While for onlydealing with a small amount of critical kernel data struc-tures, the data structure-based method becomes an alterna-tive candidate to the embedded system design for its light-weight. Moreover, the kernel data structure could fully rep-resent the actual system runtime states. It means that oncethe kernel data structure inconsistency occurs, whether dueto internal bugs or malicious attacks, the system anomalycould be detected immediately. Inside our prototype sys-tem, as a convenience we reuse the data structure specifica-tion language presented by Demsky and Rinard, in whichthe system developers could define a simple high-level se-curity specification for kernel data structure. Inside therunning system, the kernel data structure will be checkedperiodically against it to conclude whether the system be-havior follows the specification. Once the violation hasbeen detected, the related repair action will be taken.

The remainder of the paper is structured as following:Section 2 describes related work. Section 3 contains anoverview of the monitor system architecture and some de-sign issues during its development. Section 4 is about eval-uation experiment results of using our prototype monitor-ing system to detect the malicious attacks and section 5concludes the paper.

2 Related WorkThe monitoring system presented in this paper was in-spired by the work of following research areas: L4microkernel[1], system monitors[2] and specification-based data structure repair[3]. In the following, we willgive a brief introduction.

Recently L4 microkernel has drawn great attention dueto its highly modular architecture, efficient IPC mechanismand compact kernel. In contrast to traditional monolithickernel OS implementations, the microkernel approach onlyinvolves minimal trusted code which provides little sys-tem services. The kernel implements only address spaces,inter-process communication and basic scheduling. Allservers run in user mode and are treated exactly like any

iwdun2007

15

other applications. Since each server has its own addressspace, all these objects could be protected from one an-other. So the microkernel architecture appears to be a rea-sonable candidate to run the monitoring code and performsecurity tasks.

In the recent literature there has been some researchon monitoring systems, for instance, co-processor-basedmonitors[2]. While our system features in the software-based method designed for deployment in a single embed-ded system. Compared with the solutions which requireextra hardware support, our solution could greatly decreasethe deploying cost during the production manufacturing.

Specification languages have traditionally been used tohelp developers explore conceptual design properties in theabsence of any specific implementation, such as UML[4].And currently most safety-critical systems have adoptedthe similar technical solutions to help correctly representthe information that a program manipulates, hence guar-antee the system data structures consistent. In our moni-tor system, we have reused the data structure specificationlanguage[3] with necessary adaptation especially for ker-nel context.

3 System ArchitectureFor our runtime detection and repair approach, the specifi-cation language is used to establish a connection betweenthe low-level data layout in the runtime system and thehigh-level conceptual properties. Its main idea is to de-fine a high-level specification for kernel data that providesa simple but accurate representation of how kernel objectsin runtime memory relate to one another, as well as a setof constraints that must hold on those data objects for theintegrity of the kernel to remain intact.

The monitor system is developed as several loadablekernel modules and an independent server on microkernel.It has a two-layer structure: the lower layer (sensor mod-ules) is deployed as several loadable kernel modules whichdirectly deal with the kernel data structure; while the up-per layer (monitor manager) is implemented as an inde-pendent server from the guest kernel, which is in chargeof the integrity verification and dynamic configuration ofsensor modules. They are communicated via microkernelIPC messages.

Its deployment could be divided into two phrases, oneis offline phrase and the other is runtime. In the offlinephrase, the system developers write the specifications forthe system and compile it together with the source codeinto individual sensor modules, so that at the runtime, itwill cooperate with each other to perform the detection andrepair. While for the runtime phrase, it repeatedly picks upa constraint and evaluates it in the context of the current

data structures to check consistency violations in runtimesystem. If any inconsistency has been detected, the relatedaction will be invoked, otherwise the detection will con-tinue.

3.1 Sensor ModulesSensor modules encapsulate the low-level data structureof the kernel and also expose certain necessary interfaceswhich could be used to build specification cases for sys-tem level data structure. They are responsible for verifyingand repairing the underlying kernel data structure. Insidethe monitoring prototype system, sensor modules are orga-nized according to their categories, as process scheduling,file system, memory management, network and system sig-nal.Kernel Data Structure Access. The implementation ofthe sensor modules depends heavily on the LKM(loadablekernel module) mechanism which is a common method toextend the kernel of an existing operating system. Usingthe loadable kernel modules, we could access the kerneldata structure from the module, so that most of the relatedkernel source code could remain unchanged. The sensormodules are compiled into several relocation object filesand then dynamically loaded into the kernel address spaceduring bootstrap. Once the sensor modules are loaded intothe guest OS, they are running within the kernel space sothat they have full access to the related protected kerneldata structures. While only the LKM mechanism is notsufficient to meet all the requirement to fulfill the func-tion of accessing kernel data structure inside the monitor-ing system. For example, there is still no existing kernelAPI for accessing runqueue kernel data structure, even forthe latest kernel. While it is important for detecting kernellevel malicious attacks such as process hiding, some re-lated kernel source code has to been modified to expose thedata structure to the module. For example, as for the pro-cess scheduling module, in Linux kernel 2.6.10 the mainsource code of the kernel lies at kernel/sched.c its unmod-ified version is 4678 lines, after patching is 4872 lines, themodified lines could be 3.982%. And the newly introducedsource code are mainly functions could be invoked by thekernel module to access the unexposed kernel data struc-tures such as runqueue. As the development of the otherkernel module is in progress, since only this kind of modi-fication in source code level occurs on exposing the kerneldata structure to the kernel modules, so the related modifi-cation could be expected within such small ratio.Specification-based Detection. Inside the sensor mod-ules, the specification descriptive language is used to de-fine the specification used in runtime system. Each speci-fication contains model definition rules and a set of consis-tency constraints. The model definition rules identify the

iwdun2007

16

different kinds of objects and relations in the abstract viewand define a translation from the abstract model to the con-crete data structure. The consistency constrains the con-sistency properties of the data structure; these constraintsare expressed at the level of abstract objects and relationsin the model, they also constrains the relationship betweenthe model and the concrete data structure. The procedureof the specification deployment could be divided into threesteps:

1. Understand the kernel data structures. As for the pro-cess scheduling sensor modules, we are focus on thestudy of process scheduling kernel data structures.For example, the primary data structure for processmanagement inside kernel is the task struct structure.All processes are represented by a task struct instancewithin the kernel. Sometimes, maybe we only inter-ested in its specific data field, such as children is thelist head of its child process, pid is the unique processidentifier inside the system etc. So before define thedata sets and write constraints, you have to get famil-iar with related data structures and the exact meaningof interested data fields.

2. Define global data sets, relations and methods. Thebasic abstract data model is based on the low-levelkernel data structures, while it stands for the exactmeaning for the security cases. The most complextasks for the programmer is to define an effective andefficient compact abstract data model. The followingscript presents two definitions of the data sets and itsget() methods. The All Task represents all processesinside the system, while the Runqueue Task stands forthe processes inside the runqueue, which is ready torun.

set All_Task(task_t*);task_t* set All_Task::get(int i){

struct task_struct* p;for (p = init_task; i>0; i--)

p = next_task(p);return p;

}set Runqueue_Task(list_head*);list_head* set Runqueue_Task::get(){

struct prio_array* array = rq->active;int idx=sched_find_first_bit(array->bitmap);struct list_head *q = array->queue + idx;return q;

}

3. Write constraints. The following is a simple samplecode providing the security constraints on the childprocess number of a process. It indicates that eachprocess which belongs to the All Task data sets, itschild process number should be less than 100, other-wise the warning message will be sent to the systemadministrator. While maybe the situation is differentcase by case, the script just demonstrates its simplic-ity.

[for t in All_Tasks], true=> t.childnum<100: notify_admin("Warning child num >=100");

3.2 Monitor ManagerErrors in the monitor system may cause system crash, sohow to guarantee the consistency of the monitor itself isa very critical problem. Moreover the sensor modules areimplemented as loadable kernel modules, so how to guar-antee its integrity is also very important.

Monitor manager manages the user-programmed speci-fications, as well as their installation/ removal and dynamicconfiguration. It also verifies the integrity and consistencyof sensor modules at runtime. Once errors occurred in-side the monitor system, the monitor manager will trackits global id of the sensor module. The monitor managerwill unload the module and continue to execute other con-straints. If the unloaded module results in a fatal error, suchas kernel data structure inconsistency, the monitor managerwill suspend the system and notify system administrators.

4 EvaluationTo evaluate our system, we have set up the experiment thatwe performed on a system running based on the latest L4microkernel implementation L4Ka::Pistachio and the guestOS kernel version is 2.6.10. It is a Dell Dimension 4700machine with 512MB RAM, single 2.4 GHz Pentium(R)processor. Using our system, we have implemented (inC) several specifications designed to protect the Linux 2.6kernel data structure related with process scheduling. Wethen tested our specifications against implementations ofthe real attacks, such as the process hide attack. As for im-plementation of the CPU overhead evaluation, the L4 pro-cess data structure has been extended with two extra datafields to store the return value of rdtsc instruction and theprevious process id. Every 10,000 context switch, the sys-tem will dump out the value of two data structures, as theraw data used in the following discussion. Table 1 providesthe detail about the processes inside the prototype system,for the convenience of the statistic, pids have been mappedinto sequential integers.

Table 1: Process descriptionsPid Description

1 L4 system timer irq handler2 L4 system kernel debug handler3 L4 system timer handler5 Guest OS timer handler7 L4 system monitor8 Guest OS syscall handler9-11 Application workload processes

Figure 1 shows the context switch within the L4 micro-kernel system based on the sampling data of 10,000 times

iwdun2007

17

context switch and monitoring interval is set to 500 mil-liseconds. X-axis represents the CPU cycles counter insidethe system, it is marked from the rdtsc instruction sup-ported in Intel IA32 platform. Y-axis represents the pid.The sampling data is dumped after the system processesand application processes transit into a stable state.

0

2

4

6

8

10

12

5e+09 6e+09 7e+09 8e+09 9e+09 1e+10

Figure 1: Context switch inside the system

From Figure 1, we could see that when the applicationworkload process, whose pid is 9, has been blocked by thekernel monitoring operation just before the 5e+09th CPUcycle point, its execution has been delayed. While as forthe CPU consumption of the monitor server, whose pidis 7, is 0.2177%, still keeps low. As the monitor serverworks closely with the guest OS system call handler, wealso have designed various application test workload pro-grams and we found the CPU overhead keeps to almost thesame value.

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

0 200 400 600 800 1000

Figure 2: Monitor system CPU overhead

In Figure 2, the X-axis represents the monitoring inter-val ranging from 50 milliseconds to 1000 milliseconds andY-axis represents the percentage of the CPU resource usedfor monitor server program. It indicates the CPU consump-

Table 2: Monitor system overheadMonitor Interval CPU Overhead(ave)

1000 0.1137594%500 0.2133799%250 0.4557544%125 0.9380917%50 1.800082%

tion usage changes almost linearly along with the monitor-ing interval within the L4 microkernel system. To give afair statistic result, we define the measurement as long aswe could: the sample date will be dumped every 10,000times context switch. The data is also summarized to theTable 2, from which it could be concluded that the CPUoverhead will be less than 1.8% even when the monitoringinterval is set to 50 milliseconds.

5 Discussion and ConclusionAs of this time based on the implemented process schedul-ing sensor module, several related case studies have beenimplemented, such as hide process detection. With thedevelopment of the other sensor modules of kernel sub-system, further overall system overhead measurementshould be performed. While now only CPU overhead hasbeen taken into consideration. Based on the current re-sults of the scheduling overhead experiments, we found themonitor system easy to use and customize. Integrating themonitor into the existing system could help to protect thekernel from malicious attack.

References[1] Jochen Liedtke. “On µ-Kernel Construction.” In Pro-

ceedings of the 15th ACM Symposium on OperatingSystem Principles (SOSP), Dec 1995.

[2] Nick L. Petroni, Jr., Timothy Fraser, Jesus Molina,William A. Arbaugh. “Copilot-a Coprocessor-basedKernel Runtime Integrity Monitor.” In Proceedings ofthe 13th USENIX Security Symposium, Aug 2004.

[3] Brian Demsky, Martin C. Rinard. “Automatic De-tection and Repair of Errors in Data Structures.” InProceedings of 18th Annual ACM SIGPLAN Con-ference on Object-Oriented Programming, Systems,Languages, and Applications (OOPSLA), Oct 2003.

[4] The Unified Modeling Language (UML)http://www.uml.org, 2005.

iwdun2007

18

Micro-Reboot Support for Multi-Server Operating Systems

Hiroo Ishikawa Tatsuo NakajimaDepartment of Computer Science, Waseda University

[email protected]

Abstract

Self-healing is an important property for computingnodes that are required to operate for a long time with-out dedicated administrators such as sensor nodes. Thecurrent operating systems for such computing nodesprovide insufficient self-healing support; they lack fail-ure containment and/or failure notification for theirsubsystems and user tasks. The former prevents awhole system from crashing, and the latter is necessaryfor recovering the failure. We adopt microkernel archi-tecture for the former requirement and propose failurenotification support for the latter.

1 Introduction

Sensor nodes are required to operate for a longtime without any dedicated administrators. Some ofthem are connected to the Internet for delivering lo-cal weather monitoring etc. [1]. Some other aggregateand relay the data from other wireless sensor nodes [2].Even some of recent or future consumer electronics areequipped with sensors and wireless connectivity. Thosesensor nodes are assumed to run 24/7 and required tohandle failures by themselves. The current operatingsystems, however, make few efforts for availability; awhole system is simply terminated if it suffers from afailure. All the services provided by the node are alsoterminated. Failure containment and notification arethe facilities to improve the availability of the sensornodes.

Failure containment is realized by protection do-mains, which logically separate memory regions bymeans of hardware (i.e MMU) or software tools (e.g.compilers) so that the operating system kernel candetect invalid accesses over domain boundary due toerrors. However, many software systems on a sen-sor node are constructed as monolithic programs. Itshould be divided to multiple protection domains interms of its reliability. Moreover, we believe that soft-

ware for future sensor nodes should be made of mul-tiple cooperative processes for security reason. Someresearch projects show a secure construction of an In-ternet server where it is divided into multiple processesdepending on access privileges to data [3][4]. As forsensor nodes connected to the Internet, not only theapplications, but also the operating system should beconstructed in that way.

Failure notification is another important feature ofa reliable operating system. In the situation where ev-ery application is constructed of multiple cooperativeprocesses, a failure on a process may break the consis-tency of the application. The failure notification givesan opportunity for the other processes to deal with thefailure. The current operating systems tend not to no-tify a failure of a process to the other processes. Failurenotification is necessary for a reliable operating systemsuch as a multi-server operating system.

We propose a multi-server operating system thatsupports micro-reboot (uRB) as the first class abstrac-tion. uRB is a technique originally proposed for Inter-net servers to improve service availability by restart-ing a part of a system which suffers from a failure [5].Its core idea is to improve availability by reducing thetime for recovery by rebooting a part of a system ratherthan increasing the available time. Unlike the originalidea of uRB, there are several challenges to apply uRBat the operating system level. While a web applicationserver is clearly separated to application logic and data,a process is not separated to logic and computationalstate because it is a more general abstraction than aweb application server. Consequently, the operatingsystem kernel is required to preserve process state overits restart for the system consistency. First, the kernelshould provide the mechanism to notify the restart ofa process to the other running processes. Second, thekernel should provide the notion of persistent memoryor continuations.

This paper is structured as follows. The next sectiondiscusses the key principles of ArcOS. Then, the imple-mentation of ArcOS is described in Section 3. Section 4

iwdun2007

19

shows some related work. Then we conclude the paperin Section 5.

2 OS Support for uRB

The core of our uRB support is to find the smallestcollection of processes, called a restart group, to recovera failure. We use restart notifications to form a restartgroup. Each process that has received the notificationinvokes its event handling procedure to join the currentrestart group. The processes in a restart group arerestarted at the same time by the kernel. This restartmay trigger another restart notification (Figure 2.2).

The kernel maintains dependency trees and processstate to realize the uRB procedure described above. Adependency tree represents the relationships betweenprocesses. A restart notification is delivered along withthe dependency trees. The process state machine ex-presses the state transitions corresponding to uRB. Thekernel controls the delivery of a notification by meansof the state machine.

1st generation 2nd generation 3rd generation

Figure 1. A dependency tree and the prop-agation of restart notifications. The arrowsrepresent the notifications of restart. Thegray circles represent the processes restart-ing, which join a restart group. The white cir-cles represent the processes not restarting.The dotted circle represents the restart groupin the first generation. A notification is prop-agated if a process restarts.

2.1 Dependency Trees

A dependency tree represents the relationships be-tween processes in order to propagate the failure andrestart of a process. We assume that any two processesthat interact with each other have a client-server rela-tionship. A server process is always a parent node ofthe client node in a dependency tree.

The kernel expands a dependency tree when a clientprocess establishes a connection to a server process.

In contrast, a node is removed from a tree when theprocess is terminated or explicitly disconnect from itsserver process.

A dependency tree is accessible from any process init. It maintains the state of the processes in it so thata process that has received a notification can make adecision of recovery depending on the decisions of thedependent processes.

2.2 Propagation of Restart Notification

A process can keep its consistency with the processto be restarted by receiving a notification of the restart.For instance, the process receiving the notification mayimmediately close the connection to the crashed pro-cess, or it may keep the connection after the processrestarted and send data again for the process so thatit recovers its pre-crash state.

In case one of the processes decides to restart in re-sponse to the notification, another notification of therestart of the process is generated and delivered to thesub-tree. Figure 2 illustrates the delivery of a noti-fication. A delivery of notification wakes up the mainthread of a process. The notification handler is invokedby the main thread upon a restart notification. The no-tification handler is allowed only to make a decision forjoining or leaving the current restart group so that thesystem can form a restart group immediately after afailure detection. Depending on the decision, anothernotification is delivered to the next process.

1. notification

2. handling

3. notification

4. handling

Figure 2. Notification delivery and the orderof its procedure.

3 ArcOS

ArcOS is a multi-server operating system based onthe L4 microkernel [6]. Operating system services suchas device drivers, file systems, etc. run in user modeas other user tasks do. Thus, each OS service is iso-lated from each other and from the kernel. The ker-nel analyzes the dependencies between process to findthe subset of the whole system to be micro-rebooted.

iwdun2007

20

The current implementation depends on the commu-nication abstractions provided by the L4 microkernel.Since IPC and shared memory are the ways processesinteract with each other, the privileged task, called Arcroot task, watches them to maintain dependency trees.

Arc root task (ART) is in charge of memory map-ping control in addition to task lifecycle control suchas thread creation/deletion and interrupt delivery touser tasks. If two tasks attempt to share a memory re-gion, ART maps certain number of pages to both tasks’address spaces. At this moment, client-server role ofthe tasks is determined: a task that has initiated pagesharing becomes a client, and a task that has acceptedpage sharing becomes a server. Page mapping rela-tionships are stored in the memory mapping databasemaintained by ART.

Upon a failure of a task, ART looks through thememory mapping database and finds server tasks ofthe task. Then, ART propagates a failure notificationto form a restart group as described in Section 2.2.

4 Related Work

ChorusOS recovers an actor (a process under Cho-rusOS) from a failure by restarting it [7]. To preservethe state of actors, it provides persistent memory tothem. To recover the latest state of an actor, it makescheckpoints of it. Chorus also defines a restart groupto manage a set of actors that are connected by de-pendencies. As a result, restart groups form a restarthierarchy. A restart action is propagated in a way thata parent of the restart group tree takes in charge ofits children. If the parent fails to obtain normal re-sponse from its restarted children, the grand parentrestarts the parent. Chorus provides some additionalprimitives for propagation, such as the types of restartactions.

Nooks gives a separate address space to each Linuxdriver so that it is isolated from the kernel and restartindependently [8]. The shadow driver is provided totake over a device driver during its reboot in order toprevent applications that have used the device driverfrom crashing because of its short absence.

CuriOS is a research operating system that supportprocess restarting [9]. Its operating system servicesand user programs are written in C++ so that theycan receive system errors such processor exceptions ormemory violation through C++ exception mechanism.A server is allowed to store a part of its state corre-sponding to a client to the client’s address space sothat the server can recover the state after a restart.

5 Concluding Remarks

Micro-rebooting enables a system to heal itself. Fail-ure notification supported by an operating system iden-tifies the group of processes that would be affected bythe failure and is necessary for micro-rebooting of asystem. This paper has shown the dependency treeand the notification delivery as technologies to realizethe failure notification inside a system. We are cur-rently implementing these features on a multi-serveroperating system.

References

[1] Hiroshi Esaki and Hideki Sunahara. Live E!Project; Sensing the Earth with Internet WeatherStations. In Proceedings of the 2007 Interna-tional Symposium on Applications and the Internet(SAINT’07), January 2007.

[2] Omprakash Gnawali, Ben Greenstein, Ki-YoungJang, August Joki, Jeongyeup Paek, Marco Vieira,Deborah Estrin, Ramesh Govindan, and EddieKohler. The Tenet Architecture for Tiered SensorNetworks. In Proceedings of the 4th ACM Confer-ence on Embedded Networked Sensor Systems (Sen-Sys’06), October 2006.

[3] Niels Provos, Markus Friedl and Peter Honeyman.Preventing Privilege Escalation. In Proceedings ofthe 12th USENIX Security Symposium. August 2003.

[4] Maxwell Krohn. Building Secure High-PerformanceWeb Services with OKWS. In Proceedings ofthe 2004 USENIX Annual Technical Conference(USENIX’04), June 2004.

[5] George Candea, Shinichi Kawamoto, Yuichi Fujiki,Greg Friedman, and Armando Fox. Microreboot – ATechnique for Cheap Recovery. In Proceedings of the6th Symposium on Operating Systems Design andImplementation (OSDI’04), December 2004.

[6] L4 eXperimental Kernel Reference Manual, Ver-sion X.2, System Architecture Group, Departmentof Computer Science, Universitat Karlsruhe, Novem-ber 2006.

[7] Vadim Abrossimov, Frederic Hermann, JeanChristoph Hugly, Frederic Ruget, Eric Pouyoul, andMichel Tombroff. Fast Error Recovery in CHO-RUS/OS: The Hot-Restart Technology. TechnicalReport, Chorus Systems, Inc. 1996.

iwdun2007

21

[8] Michael Swift, Muthkaruppan Annamalai, RianBershad, and Henry Levy. Recovering DeviceDrivers. In Proceedings of the 6th Symposiumon Operating Systems Design and Implementation(OSDI’04), December 2004.

[9] Fransis M. David and Roy H. Campbell. Buildinga Self-Healing Operating System. In Proceedings ofthe 3rd IEEE International Symposium on Depend-able, Autonomic and Secure Computing (DASC’07),September 2007.

iwdun2007

22

An Operating System Support for Dependability ofMicro Ubiquitous Nodes

Hideyuki Tokuda, Junichi Yura, Jin Nakazawa, Masayuki Iwai, and Kazunori TakashioKeio University, JST CREST

5322 Endo Fujisawa Kanagawa, 252-8520 JAPAN

I. Introduction

Rapid progress of hardware technology has been result-ing in the creation of micro-range embedded devices, suchas wireless sensor nodes. These devices play importantroles in ubiquitous computing, since they provide sens-ing, networking, and actuation capabilities. We thus callthem as micro ubiquitous nodes. They are, however, exe-cuted with proprietary operating system without sophisti-cated power management, real-time execution support, orother resource management, even though their resourcesare quite limited. They contain a limited capacity of bat-tery, a limited amount of storage, a limited speed of CPU,and so on. Such limitation causes fatal errors during theirusage. For example, sensor nodes stop transmitting sensordata due to power failure. Micro ubiquitous nodes are thusrequired to have a certain level of dependability, which canbe represented as a set of reliability, safety, and availability.

This paper addresses our challenges towards realizationof dependability in micro ubiquitous nodes. We are de-veloping a framework atop Linux to achieve overarchingresource management mechanism. Our initial target ismainly on wireless sensor nodes like Intel’s Imote2[1], whichare executed with XScale CPU. Our aim is developing theframework first for such a resource-rich micro ubiquitousnode, and then port it to resource-limited ones includingPIC or AVR-based nodes. In this paper, we first describesthe challenges inherent in supporting dependability in mi-cro ubiquitous nodes. We then introduce the testbed plat-form we have developed to implement and evaluate thesystem support. We finally summarize the initial researchoutcome that we developed aiming at supporting a guar-anteed data recording in a mobile sensor database.

II. Challenges

The key of our challenge is to guarantee the worst-caseavailable time (WCAT) of a micro ubiquitous node, whoseexamples are shown below.• A Wireless sensor node is a node transmitting or

receiving sensor data acquired from the sensors on itto or from other nodes.

• A Digital audio recorder, player is a node that canrecord audio via a microphone, and play back audioclips including the recorded one.

• An IP phone is a node that a user can talk to some-body via the Internet transmitting and receiving audiostreams.

We see the followings as common issues in these nodes.

A. Power Management

All these nodes execute with a battery, whose capacity islimited. On the other hand, their users may require theirexecution for a duration of time. For example, wirelesssensor nodes used in an environmental monitoring appli-cation may be required to execute for one week. A digi-tal audio recorder may need to record voice during a two-hour speech. These examples show the need for a mecha-nism that guarantee execution of micro ubiquitous nodesfor WCAT based on the applications’ requirement.

We investigate the mechanism based on Dynamic Volt-age and Frequency Scaling (DVFS) of CPU, and a fine-grained control of power modes of peripheral devices.

B. Storage Management

All these nodes require certain amount of storage. Forexample, the digital audio recorder needs to accommodatetwo-hours of voice. Other devices also require storage spacefor hardware and operating system logging. Logging is re-quired for testing of the devices, since it increases repro-ducibility of bugs. It also involves time constraint, such asa vendor of an IP phone require detailed log in last thirtyminutes. These examples show the need for a mechanismto guarantee availability of storage space for WCAT.

The storage space in micro ubiquitous nodes is quite lim-ited, such that a MICA2 Mote sensor node contains as lit-tle as 256KB flash memory. We thus investigate a storagemanagement mechanism based on fidelity control of user-level applications.

C. Network Management

Two of the above devices, wireless sensor nodes and IPphones, depend on a wireless network connectivity, hencerequiring certain quality of radio. A wireless sensor nodetransmitts sensor data through a network, and an IP phonedoes so for voice communication. The following two causequality decrease. First is network handover, which wouldfrequently happen in an IP phone due to its mobility. Ifa handover causes a long period of disconnection from thenetwork, the voice communication would be terminated.The second is lowered radio strength that the nodes facewhen they disengage from a corresponding node or a net-work access point. A wireless sensor node, when it executeswith a bad quality of radio, wastes power for sensor datatransmission, since the data may be lost.

The required quality of radio must be issuable by thenodes to make applications on them dependable. We thus

iwdun2007

23

Fig. 1. Picture of NUTS

investigate a framework that enables applications to issuethe quality, be notified when the quality underruns.

III. NUTS: A Testbed Platform

We are developing a testbed platform, called a Next-generation Ubiquitous Tolerant Sensor (NUTS) node, tohave (1) a generic micro ubiquitous node with a series ofperipheral devices including storage, network, and sensorsthat can be applied to the example devices above, and(2) enough hardware support for power management. Awireless sensor node, for example, does not equip powermanagement IC that enables an OS to estimate remain-ing battery life. NUTS provides such hardware support asa basis of mechanisms towards the challenges. Figure 1shows a picture of NUTS, and Table I is its specification.It consists of the three component boards.

TABLE I

Specification of NUTS

Gumstix Linux BoardModel Verdex XM4-BTCPU Marvell PXA270/400MHz

Network BluetoothSize 80 × 20 × 5 (mm)

Main BoardPeripherals 3.6” LCD, GPS, LED (RGB)

I/O USB, Audio, Serial, I2CNetwork IEEE802.11 (opt.)Power DC5V, Li-Ion Battery (1000mAh)Size 100 × 50 × 11 (mm)

Sensor BoardMPU Crossbow MPR2600

Sensors Light, Temp., Accel., RFIDPeripherals Rotary Encoder, LED (RGY)Network ZigBee

Size 70 × 50 × 8 (mm)

Gumstix Linux Board

The brain of NUTS is an off-the-shelf uLinux board,called Gumstix (http://www.gumstix.com). It equips anXScale PXA270 processor, 32MB flash memory, 128MBSDRAM, and Bluetooth network interface. It connects tothe main board and the sensor board described below.

Main Board

The guts of NUTS is a cumtom-made circuit board thatequips a power management unit (PMU) and peripheraldevices including an AC97 codec, a USB connector, an

0

6400

12800

0 6 12 18 24 0

6

Sto

rage U

sage (

blo

cks)

Thre

shold

of P

roxim

ity F

ilter

Elapsed Time (hours)

Storage UsageThreshold

0

6400

12800

0 6 12 18 24 0

6

Sto

rage U

sage (

blo

cks)

Thre

shold

of P

roxim

ity F

ilter

Elapsed Time (hours)

Storage UsageThreshold

Proximity-based Filter (guarantee: 2) Proximity-based Filter (guarantee: 6)

Fig. 2. Data recording in proximity-based filter

LCD, and a GPS receiver. Energy consumption of eachperipheral device can be controlled by the PMU by switch-ing it on and off. The PMU is then controlled by softwarethrough I2C bus.

Sensor Board

The skin of NUTS is a custom-made circuit board pig-gybacking a MICAz Mote. A range of sensors, including arotary encoder, an RFID reader, an accelerometer, and athermometer, provide a powerful sensing capability. Wire-less sensor nodes, which is included in our application de-vices, can leverage this capability.

IV. Results of Guaranteed Data Recording

We have developed a mobile sensor database that facil-itates storing data guaranteing worst case execution timeand worst-case data quality. It filters sensor data by dis-carding redundantly received data. The key feature isto adjust the threshold of a filter dynamically dependingon the remaining storage size. The adjustment is donewithin the worst-case data quality specified by a user withmax/min values of the threshold.

Figure 2 shows changes in storage usage and threshold byadapting a proximity-based filter (a filter based on similar-lity of contiguous data). In this evaluation, a user specifies24 hours as a worst case execution time. The left figureshows a case that the worst-case data quality is 2, and theright one shows the case that the worst-case data qualityis 6. In the former case, the filter can not reduce receiveddata adequately, and that the system stops recording inabout 13 hours. In the latter, the threshold of the filter ischanging between 0 and 6. As a result, the system contin-ues to store data in 24 hours.

V. Summary

This paper introduced our challenges towards depend-ability of micro ubiquitous nodes. This project is nowfunded by Japan Science and Technology Agency (JST)for 6 years starting from the year 2006. We plan to de-velop a framework, in the Linux kernel, which provides ap-plications to request time constraint against resources likepower, storage, and network. The last half of the projectwill be given for work with vendors to commercialize theresearch result.

References

[1] Intel Corporation, “Intel mote 2 overview”.

iwdun2007

24

http://www.ht.sfc.keio.ac.jp/iwdun2007/http://www.ht.sfc.keio.ac.jp/iwdun2007/

Cover Design @ Masayuki IWAICover Design @ Masayuki IWAI

proceedings of the first international workshop on ... · micro-reboot support for multi-server...

Documents