Simulation in Large Scale Sensor Networks

Barney MaccabeUniversity of New Mexico

Santa Fe, New Mexico 17 Oct 2006

Scalable Systems LabScalable Systems LabThe University of New Mexico

Scalable Systems LabScalable Systems Lab

The University of New Mexico Collaborators

•UNM – sensor network systems software• James Horey, PhD Student

• Jean-Charles Tournier, Postdoc

• Eric Nelson, MS student

• LANL – sensor network system simulation• Sean Brennan, MTS (& UNM PhD student)

• Angela Mielke, project lead

• SNL – multi-scale system simulation for MPPs• Rolf Riesen

• Keith Underwood

“I have finally reached the point in my career where I can live on the results of my graduate students”

Scalable Systems LabScalable Systems Lab

The University of New Mexico

Multiscale Simulation for Systems Software

•Use simulation to study systems software approaches• fault tolerance: simulate a large system and inject

faults• impact and viability of systems software at scale:

how does it work

•More “emulation” than “simulation,” for now•Traditional use of simulation in sensor

networks• application testing• ad-hoc routing protocols

Scalable Systems LabScalable Systems Lab

The University of New Mexico Basic simulation tools

•ns-2 – network simulation• packet level simulation, great for studying changes

to TCP/IP

• plug-ins for wireless protocols

• ad-hoc wireless routing protocols

• Simple Scalar processor simulation• instruction set simulation

• great for studying branch prediction and out-of-order execution

•Mambo

Scalable Systems LabScalable Systems Lab

The University of New Mexico More

•Tiny OS simulation (TOSSIM)• http://www.tinyos.net/• develop applications, small number of nodes

•EmStar• http://cvs.cens.ucla.edu/emstar/• sensor networks involving Linux-based platforms

• SWANS• http://jist.ece.cornell.edu/• like ns2, better scalability, Java apps

•and a whole host of others...• http://en.wikipedia.org/wiki/Sensor_network

Scalable Systems LabScalable Systems Lab

The University of New Mexico

Sensor Network Environment

Mote

Mote

Mote

Mote

Stargate

Mote

Mote

Mote

Mote

Stargate

Mote

Mote

Mote

MoteMote

MoteMote

StargateMote

• Motes – sensing

• 1,000’s

• limited compute, communication and, power

• Stargates – compute

• 100’s

• more compute, memory, power, and communication

• Computation in the network

Scalable Systems LabScalable Systems Lab

The University of New Mexico Software Stack

Spreadsheet Programming

Active Management

Kensho

Kaizen

Micro Services

Hardware

Application

Language

Mapping policy

Mapping mechanism

Quality of service

Hardware abstraction

Com

posa

bilit

y

• Programming

• writing the functions that will run on the nodes of a sensor network

• Tasking

• distributing functions to the nodes of a sensor network

• Groups

• Static and dynamic

• Collective operations

Scalable Systems LabScalable Systems Lab

The University of New Mexico Basic Simulation

• Goal: evaluate software stack

• Kensho API library

• Each sensor simulated by a process (linked with the Kensho library)

• Wireless communication provided by a broadcast daemon

• Sensor reads come from environment simulator

Environment

Kensho

Node

Read Sensor

Broadcast

Send Message

Kensho

NodeKensho

Node Kensho

Node

Scalable Systems LabScalable Systems Lab

The University of New Mexico

Fire Detection:Group Operations

Fig. 5. The resolution of the collective data decreased as the sizeof the groups increased, while the total number of sensors remainedconstant.

Each node within a region runs a simple monitoringfunction that detects the presence of fire. Upon detectinga fire, the sensor publishes a warning message indicatingthe current temperature to the group leader. By countingthe number of fire warnings and averaging the tempera-ture values, the group leader then determines how muchof the region is engulfed in fire.

We observed that the resolution of our collective datadecreased as the size of the groups increased. This isshown in Figure 5. Although the data resolution wasadversely affected, decreasing the number of groupsalso decreases the communication overhead since moredata is being compressed by the collective functions.This ability to quickly change the resolution of the databy changing group membership allows Kensho usersto systematically manipulate important communication-resolution tradeoffs.

B. Mobile Object Tracking

We also developed a mobile-object simulator and anassociated tracking application in Kensho. As observedin Figure 6, an object starts at position (5, 5) and movesin a clockwise, circular fashion at a velocity of 1.35units per time interval. Sensor nodes are placed in 1 unitincrements in a grid layout. When the object is withina certain radius of a magnetometer sensor, the sensorregisters a positive value. This value decays inverselywith the square of the distance. Sensor nodes within thesensor radius employ a centroid algorithm similar to theone presented in [6] to estimate the position of the object.Currently we do not attempt to predict the direction ortrajectory of the object.

On average, the tracking is accurate to approximately

half the distance between sensor nodes. This is partlydue to the fact our simulation does not simulate back-ground noise; each sensor node is able to accuratelyestimate its distance to the object. More exhaustive andrigorous studies of tracking algorithms can be foundelsewhere[11][12].

For this application, we employ dynamic groups.Besides a simple change in the type of group used,the launch script remains similar to the one used forfire detection. As nodes register a positive magnetometervalue, they join the dynamic group and begin processingthe computation associated with that group. As the objectmoves away from a node, the node leaves that group andstops processing. The node computation for this appli-cation is identical to the one used in our fire detectionapplication with the exception of the sensor type. Thecollective computation collects the data published bythe nodes and performs the centroid algorithm as shownin Figure 7. Employing dynamic groups in this fashionis similar to the approach taken by EnviroTrack[11],although our system is not specifically designed formobile object tracking applications.

C. ObservationsThere are several interesting observations which result

from the previous two example applications. First, bothapplications employ similar tasking schemes. We alsoobserve that tasking has a major impact on the resolutionof the observed data, communication overhead, and theuseful lifetime of the sensor network application. Theseobservations indicate that tasking serves an importantrole in the development of an application and new toolsto help users manage these tradeoffs are necessary.

We also observe that both applications use identi-cal node-level functions. In both cases, nodes collectenvironmental data (temperature and magnetometer re-spectively) and relay the data to the collective func-tion. By treating node functions and collective functionsseparately, Kensho allows applications to share nodefunctions easily and encourages code re-use. This func-tional separation between node and collective functionsis shared in other projects such as Tenet[13].

V. CONCLUSION

Research in programming abstractions attempts toalleviate the difficulty of writing complex, distributedalgorithms, but does not fully address the additionalobstacles associated with tasking. These additional ob-stacles impede the development and deployment of newsensor network applications. Our library, Kensho, ex-plicitly addresses the role of tasking in a distributedsensor network. Users are able to task the sensor network

4

smaller groups: higher

resolution, more

communication

Scalable Systems LabScalable Systems Lab

The University of New Mexico

Motion Tracking: Dynamic Groups

Fig. 6. The object starts at position (5, 5) and moves at a velocity of 1.35 units per time interval. The circles indicate the sensors that aretriggered by the presence of the object. The horizontal line in the right figure represents the normalized path of the object, while the pointsindicate how far from the actual object the algorithm predicted the object to be.

Fig. 7. Each node in the group collects environmental data using a collect_data call to the Kensho Node API. It then publishes that data tobe consumed by the collective computation. This collective computation collects the data from the nodes using a collect_data call.

using both static and dynamic group methods. These twomethods effectively abstract both spatial and temporaldivisions in responsibility in the sensor network.

In this paper, we presented the design and initialimplementation of a lightweight communications andtasking library. Our current memory overheads indicatethat a port to an existing sensor platform is viable.We also developed two applications, fire detection andmobile-object tracking, to demonstrate the usefulness ofour library. Both applications are tasked in a straightfor-ward way thanks to the simple group-based semantics ofKensho. Initial results indicate that Kensho furthers thegoals of practical sensor network deployments.

A. Acknowledgments

This paper is supported in part by the U.S. Departmentof Energy/NNSA and Los Alamos National Laboratoryunder contract number DE-AC52-06NA25396 and ap-proved for public release under LA-UR-06-6041.

REFERENCES

[1] A. Mainwaring, J. Polastre, R. Szewczyk, D. Culler, and J. An-derson, “Wireless Sensor Networks for Habitat Monitoring,” inWSNA, 2002.

[2] P. Juang, H. Oki, Y. Wang, M. Martonosi, L.-S. Peh, andD. Rubenstein, “Energy-Efficient Computing for Wildlife Track-ing: Design Tradeoffs and Early Experience with ZebraNet,” inASPLOS, 2002.

[3] R. Musaloiu, A. Terzis, K. Szlavecz, A. Szalay, J. Cogan, andJ. Gray, “Life under your feet: A wireless soil ecology sensornetwork,” in EmNets, 2006.

[4] P.-C. Shen, C. Srisathapornphat, and C. Jaikaeo, “Sensor Informa-tion Networking Architecture and Applications,” IEEE PersonelCommunication Magazine, vol. 8, no. 4, pp. 52–59, August 2001.

[5] K. Whitehouse, C. Sharp, E. Brewer, and D. Culler, “Hood: aneighborhood abstraction for sensor networks,” in MobiSys, 2004.

[6] M. Welsh and G. Mainland, “Programming sensor networks usingabstract regions.” in NSDI, 2004.

[7] D. Chu, K. Lin, A. Linares, G. Nguyen, and J. Hellerstein, “Sdlib:A sensor network data and communications library for rapid androbust application development,” in IPSN/SPOTS, 2006.

[8] J. Hill, R. Szewczyk, A. Woo, S. Hollar, D. E. Culler, and K. S. J.Pister, “System Architecture Directions for Networked Sensors,”in ASPLOS, 2000.

[9] C.-C. Han, R. Kumar, R. Shea, E. Kohler, and M. Srivastava,“Sos: A dynamic operating system for sensor nodes,” in MobiSys,2005.

[10] H. Abrach, S. Bhatti, J. Carlson, H. Dai, J. Rose, A. Sheth,B. Shucker, and R. Han, “Mantis: System support for multimodalnetworks of in-situ sensors,” in WSNA, 2003.

[11] T. Abdelzaher, B. Blum, Q. Cao, Y. Chen, D. Evans, J. George,S. George, L. Gu, T. He, S. Krishnamurthy, L. Luo, S. Son,J. Stankovic, R. Stoleru, and A. Wood, “Envirotrack: Towardsan environmental computing paradigm for distributed sensornetworks,” in ICDCS, 2004.

[12] C. M. Aprea, D. M. Alde, J. T. Rutledge, S. S. Giandoni, A. M.Mielke, and J. S. Dreicer, “Algorithms for seismic tracking ofvehicles using distributed sensor networks,” submitted to Pureand Applied Acoustics, 2005.

[13] R. Govindan, E. Kohler, D. Estrin, F. Bian, K. Chintalapudi,O. Gnawali, R. Gummadi, S. Rangwala, and T. Stathopoulous,“Tenet: An architecture for tiered embedded networks,” Centerfor Embedded Networked Sensing, Tech. Rep., 2005.

5

Fig. 6. The object starts at position (5, 5) and moves at a velocity of 1.35 units per time interval. The circles indicate the sensors that aretriggered by the presence of the object. The horizontal line in the right figure represents the normalized path of the object, while the pointsindicate how far from the actual object the algorithm predicted the object to be.

Fig. 7. Each node in the group collects environmental data using a collect_data call to the Kensho Node API. It then publishes that data tobe consumed by the collective computation. This collective computation collects the data from the nodes using a collect_data call.

using both static and dynamic group methods. These twomethods effectively abstract both spatial and temporaldivisions in responsibility in the sensor network.

In this paper, we presented the design and initialimplementation of a lightweight communications andtasking library. Our current memory overheads indicatethat a port to an existing sensor platform is viable.We also developed two applications, fire detection andmobile-object tracking, to demonstrate the usefulness ofour library. Both applications are tasked in a straightfor-ward way thanks to the simple group-based semantics ofKensho. Initial results indicate that Kensho furthers thegoals of practical sensor network deployments.

A. Acknowledgments

This paper is supported in part by the U.S. Departmentof Energy/NNSA and Los Alamos National Laboratoryunder contract number DE-AC52-06NA25396 and ap-proved for public release under LA-UR-06-6041.

REFERENCES

[1] A. Mainwaring, J. Polastre, R. Szewczyk, D. Culler, and J. An-derson, “Wireless Sensor Networks for Habitat Monitoring,” inWSNA, 2002.

[2] P. Juang, H. Oki, Y. Wang, M. Martonosi, L.-S. Peh, andD. Rubenstein, “Energy-Efficient Computing for Wildlife Track-ing: Design Tradeoffs and Early Experience with ZebraNet,” inASPLOS, 2002.

[3] R. Musaloiu, A. Terzis, K. Szlavecz, A. Szalay, J. Cogan, andJ. Gray, “Life under your feet: A wireless soil ecology sensornetwork,” in EmNets, 2006.

[4] P.-C. Shen, C. Srisathapornphat, and C. Jaikaeo, “Sensor Informa-tion Networking Architecture and Applications,” IEEE PersonelCommunication Magazine, vol. 8, no. 4, pp. 52–59, August 2001.

[5] K. Whitehouse, C. Sharp, E. Brewer, and D. Culler, “Hood: aneighborhood abstraction for sensor networks,” in MobiSys, 2004.

[6] M. Welsh and G. Mainland, “Programming sensor networks usingabstract regions.” in NSDI, 2004.

[7] D. Chu, K. Lin, A. Linares, G. Nguyen, and J. Hellerstein, “Sdlib:A sensor network data and communications library for rapid androbust application development,” in IPSN/SPOTS, 2006.

[8] J. Hill, R. Szewczyk, A. Woo, S. Hollar, D. E. Culler, and K. S. J.Pister, “System Architecture Directions for Networked Sensors,”in ASPLOS, 2000.

[9] C.-C. Han, R. Kumar, R. Shea, E. Kohler, and M. Srivastava,“Sos: A dynamic operating system for sensor nodes,” in MobiSys,2005.

[10] H. Abrach, S. Bhatti, J. Carlson, H. Dai, J. Rose, A. Sheth,B. Shucker, and R. Han, “Mantis: System support for multimodalnetworks of in-situ sensors,” in WSNA, 2003.

[11] T. Abdelzaher, B. Blum, Q. Cao, Y. Chen, D. Evans, J. George,S. George, L. Gu, T. He, S. Krishnamurthy, L. Luo, S. Son,J. Stankovic, R. Stoleru, and A. Wood, “Envirotrack: Towardsan environmental computing paradigm for distributed sensornetworks,” in ICDCS, 2004.

[12] C. M. Aprea, D. M. Alde, J. T. Rutledge, S. S. Giandoni, A. M.Mielke, and J. S. Dreicer, “Algorithms for seismic tracking ofvehicles using distributed sensor networks,” submitted to Pureand Applied Acoustics, 2005.

[13] R. Govindan, E. Kohler, D. Estrin, F. Bian, K. Chintalapudi,O. Gnawali, R. Gummadi, S. Rangwala, and T. Stathopoulous,“Tenet: An architecture for tiered embedded networks,” Centerfor Embedded Networked Sensing, Tech. Rep., 2005.

5

Tracking Accuracy

Fig. 6. The object starts at position (5, 5) and moves at a velocity of 1.35 units per time interval. The circles indicate the sensors that aretriggered by the presence of the object. The horizontal line in the right figure represents the normalized path of the object, while the pointsindicate how far from the actual object the algorithm predicted the object to be.

Fig. 7. Each node in the group collects environmental data using a collect_data call to the Kensho Node API. It then publishes that data tobe consumed by the collective computation. This collective computation collects the data from the nodes using a collect_data call.

using both static and dynamic group methods. These twomethods effectively abstract both spatial and temporaldivisions in responsibility in the sensor network.

In this paper, we presented the design and initialimplementation of a lightweight communications andtasking library. Our current memory overheads indicatethat a port to an existing sensor platform is viable.We also developed two applications, fire detection andmobile-object tracking, to demonstrate the usefulness ofour library. Both applications are tasked in a straightfor-ward way thanks to the simple group-based semantics ofKensho. Initial results indicate that Kensho furthers thegoals of practical sensor network deployments.

A. Acknowledgments

This paper is supported in part by the U.S. Departmentof Energy/NNSA and Los Alamos National Laboratoryunder contract number DE-AC52-06NA25396 and ap-proved for public release under LA-UR-06-6041.

REFERENCES

[1] A. Mainwaring, J. Polastre, R. Szewczyk, D. Culler, and J. An-derson, “Wireless Sensor Networks for Habitat Monitoring,” inWSNA, 2002.

[2] P. Juang, H. Oki, Y. Wang, M. Martonosi, L.-S. Peh, andD. Rubenstein, “Energy-Efficient Computing for Wildlife Track-ing: Design Tradeoffs and Early Experience with ZebraNet,” inASPLOS, 2002.

[3] R. Musaloiu, A. Terzis, K. Szlavecz, A. Szalay, J. Cogan, andJ. Gray, “Life under your feet: A wireless soil ecology sensornetwork,” in EmNets, 2006.

[4] P.-C. Shen, C. Srisathapornphat, and C. Jaikaeo, “Sensor Informa-tion Networking Architecture and Applications,” IEEE PersonelCommunication Magazine, vol. 8, no. 4, pp. 52–59, August 2001.

[5] K. Whitehouse, C. Sharp, E. Brewer, and D. Culler, “Hood: aneighborhood abstraction for sensor networks,” in MobiSys, 2004.

[6] M. Welsh and G. Mainland, “Programming sensor networks usingabstract regions.” in NSDI, 2004.

[7] D. Chu, K. Lin, A. Linares, G. Nguyen, and J. Hellerstein, “Sdlib:A sensor network data and communications library for rapid androbust application development,” in IPSN/SPOTS, 2006.

[8] J. Hill, R. Szewczyk, A. Woo, S. Hollar, D. E. Culler, and K. S. J.Pister, “System Architecture Directions for Networked Sensors,”in ASPLOS, 2000.

[9] C.-C. Han, R. Kumar, R. Shea, E. Kohler, and M. Srivastava,“Sos: A dynamic operating system for sensor nodes,” in MobiSys,2005.

[10] H. Abrach, S. Bhatti, J. Carlson, H. Dai, J. Rose, A. Sheth,B. Shucker, and R. Han, “Mantis: System support for multimodalnetworks of in-situ sensors,” in WSNA, 2003.

[11] T. Abdelzaher, B. Blum, Q. Cao, Y. Chen, D. Evans, J. George,S. George, L. Gu, T. He, S. Krishnamurthy, L. Luo, S. Son,J. Stankovic, R. Stoleru, and A. Wood, “Envirotrack: Towardsan environmental computing paradigm for distributed sensornetworks,” in ICDCS, 2004.

[12] C. M. Aprea, D. M. Alde, J. T. Rutledge, S. S. Giandoni, A. M.Mielke, and J. S. Dreicer, “Algorithms for seismic tracking ofvehicles using distributed sensor networks,” submitted to Pureand Applied Acoustics, 2005.

[13] R. Govindan, E. Kohler, D. Estrin, F. Bian, K. Chintalapudi,O. Gnawali, R. Gummadi, S. Rangwala, and T. Stathopoulous,“Tenet: An architecture for tiered embedded networks,” Centerfor Embedded Networked Sensing, Tech. Rep., 2005.

5

Group is defined by magnetometer activation

Scalable Systems LabScalable Systems Lab

The University of New Mexico N-Sim: Scalable Simulation

•Goal: run large scale sensor network applications without modification

•Be able to control communication and node faults

•Order of magnitude speedup

• Initial focus on typical software stack: TinyOS and Linux

•Long term will be used to study the Kensho/Kaizen stack

Scalable Systems LabScalable Systems Lab

The University of New Mexico Virtualization using Xen

Hardware

HostOS 1

Dom 0

Xen Virtualization Layer

HostOS 2

HostOS 3

HostOS 4

App App App App

• Xen provides base level virtualization

• sharing CPU between multiple OSes

• Dom 0 provides access to other resources (e.g., network)

• Type 1

• para-virtualization

Virtual Machines

Scalable Systems LabScalable Systems Lab

The University of New Mexico

N-sim: Scalable simulation for sensors

• Simulate each sensor or stargate using a separate virtual machine

• Direct execution of applications

• On modified operating systems

• Simulation daemon runs in Dom 0

• network and sensing

Hardware

Dom 0

Xen Virtualization Layer

App App Mote App

VirtualStargates

VirtualMotes

Mote App

Linux Linux TinyOS

TinyOS

Xeno Linux

Xeno Linux

TinyOS

wedge

TinyOS

wedge

Daemon

Scalable Systems LabScalable Systems Lab

The University of New Mexico Scaling: Cluster of nodes

• Use multiple cluster nodes

• Simulation Daemons coordinate and exchange communications

• MPI for communicaiton and coordination

Daemon

Daemon

DaemonDaemon

Daemon

Scalable Systems LabScalable Systems Lab

The University of New Mexico

Full system simulation for MPP design

Library

Process

Network Simulator

Library

Process

Library

Process

Scalable Systems LabScalable Systems Lab

The University of New Mexico Multiscale issues

•Want full, cycle accurate, simulation

• from detailed processor simulation

• device drivers

• to large scale application

• including all software layers

• impact of changing implementation of services

•Also want simple, proof of concept simulation

• fast, clean and simple

Scalable Systems LabScalable Systems Lab

The University of New Mexico Thanks

•LANL & NNSA

•SSL UNM

• Prof Patrick Bridges

• Postdocs

• Patricia Crowley (now at Gonzaga)

• Patrick Widener