Proceedings of the ASME 31st International Conference on Ocean, Offshoreand Arctic Engineering

OMAE 2012June 10-15, 2012, Rio de Janeiro, Brazil

OMAE2012-83812

DRAFT: MODELING AND INSPECTION APPLICATIONS OF A COASTALDISTRIBUTED AUTONOMOUS SENSOR NETWORK

Nicholas M. Patrikalakis∗Joshua Leighton

Georgios PapadopoulosDepartment of Mechanical EngineeringMassachusetts Institute of Technology

Cambridge, MA 02139Email: {nmp,jleight,gpapado,gbarb}@mit.edu

Gabriel WeymouthHanna Kurniawati

Pablo Valdivia y AlvaradoTawfiq Taher

Rubaina KhanCenter for Environmental Sensing and Modeling

Singapore-MIT Alliance for Research and TechnologySingapore

Email: {weymouth,hannakur,pablov,tawfiq,rubaina}@smart.mit.edu

ABSTRACT

Real time in-situ measurements are essential for monitor-ing and understanding physical and biochemical changes withinocean environments. Phenomena of interest usually display spa-tial and temporal dynamics that span different scales. As a result,a combination of different vehicles, sensors, and advanced con-trol algorithms are required in oceanographic monitoring sys-tems. In this study our group presents the design of a distributedheterogeneous autonomous sensor network that combines under-water, surface, and aerial robotic vehicles along with advancedsensor payloads, planning algorithms and learning principles tosuccessfully operate across the scales and constraints found incoastal environments. Examples where the robotic sensor net-work is used to localize algal blooms and collect modeling datain the coastal regions of the island nation of Singapore and toconstruct 3D models of marine structures for inspection and har-bor navigation are presented. The system was successfully testedin seawater environments around Singapore where the water cur-rent is around 1-2m/s.

∗Address all correspondence to this author.

INTRODUCTION

This study presents the autonomous sensor network beingdeveloped at the Center for Environmental Sensing and Model-ing (CENSAM) in Singapore and its applications to coastal envi-ronments. Ocean observing and prediction systems present chal-lenges for vehicles, sensors, motion planners, data assimilationand predictive models. In addition, coastal environments haveparticular challenges such as low depths and commercial vehi-cle traffic which increase the likelihood of collisions. Our groupis developing a distributed heterogenous autonomous sensor net-work that combines underwater, surface, and aerial robotic ve-hicles along with advanced sensor payloads, planing algorithmsand learning principles to successfully operate across the scalesand constraints found in coastal environments. Its effectivenessrelies on the seamless operation of all vehicles, their safe interac-tion with each other and the environement, the capability of col-lecting pertinent data, and the capabity to analyze the collecteddata and use information within to optimze the sampling process.The two applications explored herein portray examples of basictools for these objectives. The study first presents a descriptionof the configuration and components of the autonomous sensornetwork developed by our group and subsequently two applica-tions are described in detail: a procedure for algal bloom mon-itoring and 3D surface reconstruction. Preliminary results are

1 Copyright c© 2012 by ASME

summarized within each section and conlcuding remarks and fu-ture work are outlined.

AUTONOMOUS SENSOR NETWORKOcean observing and prediction systems have inspired both

theoretical and experimental work. The Autonomous OceanSampling Network (AOSN) project [1, 2] outlined the signifi-cant challenges for the practical implementation of such systems.Other studies since have explored both hardware (vehicles andsensors) [3, 4] and software (motion planners, data assimilation,predictive models) [5–7] for use in robotic sensor networks to ob-serve and forecast conditions in marine environments. This studypresents an update on the autonomous sensor network being de-veloped at CENSAM [8] and its applications to coastal environ-ments. Fig. 1 shows a diagram of the typical spatial distributionof the active network nodes within CENSAM’s network, whichinclude research vessels, autonomous surface vehicles (ASVs),autonomous underwater vehicles (AUVs), and unmanned aerialvehicles (UAVs). Large distances (∼ 103 meters) are covered bythe air vehicles (quadrotors) whose mission is to rapidly scan ar-eas of interest and identify potential target zones. In addition tonavigation related hardware, quadrotor sensor payloads includedigital and infrared cameras. If a quadrotor identifies a featureof interest its location is geo-tagged and relayed to the rest of thenetwork, subsequently a research vessel deploys autonomous un-derwater vehicles (Iver2 submarines from Oceanserver) and au-tonomous surface vehicles (SCOUT ocean kayaks equipped foroceanographic and undersea testing) within the vicinity of theareas of interest (∼ 102 meters). The AUVs and ASVs performfiner scanning passes inside the target zones to provide denserdata for further analysis. In addition to inertial navigation andcommunication equipment, the surface and underwater vehiclesare equipped with oceanographic sensors to measure pH, temper-ature, conductivity, chlorophyll, rhodamine, turbidity, and dis-solved oxygen. Spectrophotometers (AC-9 from Wetlabs) arealso used to measure the absorption and attenuation coefficientsof the elements within the sampled zones.

ALGAL BLOOMSThe first application discussed for the robotic sensor net-

work is the location and scientific survey of algal blooms in thecoastal regions of Singapore. Harmful algal blooms are an in-creasing problem in the coastal waterways surrounding Singa-pore due to urban development as well as longer term climatechanges. A toxic algal bloom in December 2009 resulted in200,000 fish killed in the Pasir Ris area alone, and a secondbloom by a different algal species occurred a month later in thesame location. An understanding of the underlying ecological,chemical, tidal, and hydrodynamic factors is needed to developreliable models of these events. However, modeling a system

FIGURE 1. Stratified heterogeneous sensor network. Aerial, surfaceand underwater vehicles along with research vessels are used to spandifferent spatial and temporal scales and optimally scan and monitorcoastal regions.

with so many features and such a broad range of time and lengthscales is extremely difficult, and an autonomous network of sens-ing vehicles is an ideal approach to capture the data essential todevelop such models. Using flexible kernel regression methods,simingly disparate data sources can be combined in a meaningfulway with heavier weighting given to more accurate data sources(such as GPS position data of the ASVs). Once compiled, thiskind of information can be used to establish correlations betweenmeasured parameters across vehicles and to map out distribu-tions of key parameters such as chlorophyll. As described inthe previous section, quadrotors, equipped with digital and ther-mal cameras, scan large distances to identify areas of interest,in this case potential bloom zones. A research vessel deploysAUVs and ASVs within the vicinity of the areas of interest. TheAUVs and ASVs perform finer scanning passes inside the targetzones to provide denser data for further analysis. The vehiclesare shown in Fig 2 along with one identified algal bloom in theJohor Straight between Singapore and Malaysia. Data from thequadrotor cameras and the in-situ measurements taken along thepaths of the AUVs and ASVs must then be assembled into a co-herent picture of a bloom.

The geo-tagged and time-stamped data are processed toidentify correlations between the measured variables to help de-velop models of the bloom activity based on more easily pre-dicted or measured variables such as temperature, local waterdepth, and tidal dynamics. An example of such data is presentedin Fig 3 for all the sensor variables over all the data collected inone season. Because the sensors are not uniform across the ve-hicles and sensors occasionally malfunction, these correlationsmust be made between pairwise observations using Pearson’smethod.

While this kind of correlation information is important formodel building, the data from the ASVs and AUVs must be tiedback in to data from the quadrotor cameras and samples takenfrom research vessels. These sources of data are crucial for lo-

2 Copyright c© 2012 by ASME

(a) (b)

(c) (d)

FIGURE 2. The quadrotor (a), AUV (b) and ASV (c) measurementvehicles and a bloom captured by the quadrotor’s aerial camera in theJohor Straight (d).

FIGURE 3. Correlation matrix of measured data assembled frommultiple test sites along the northern coast of Singapore. Correlationsbetween all the sensor variables have been shown to highlight major cor-relations, including the total height of the water column, the water tem-perature, the pH, conductivity, chlorophyll, rhodamine, and dissolvedoxygen. The shape and color of the ellipse indicates if the correlationis weak (white circle) strongly positive (blue front leaning) or stronglynegative (red back leaning), from [12].

x

y

-2

-1

0

1

-1 0 1 2

Depth0

10

20

30

40

50

(a)

x

y

-2

-1

0

1

2

-2 -1 0 1 2

Depth0

10

20

30

40

50

(b)

0 500 1000 1500

1015

2025

3035

4045

Sample #

Depth

(c)

FIGURE 4. Depth data from a single AUV run. Figure (a) showsthe path of the AUV and the data collected. Figure (b) shows the 2Dreconstruction using regularized kernel regression with Figure (a) dataoverlay ( [11]). Figure (c) shows the raw data (black) and kernel regres-sion (blue) which shows that signal is regularized without loosing thepeaks.

3 Copyright c© 2012 by ASME

cating the ASVs ans AUVs relative to the bloom activity which isidentified visually (as in Fig 2(d)) or through positive identifica-tion of a harmful bloom species in a water sample. This requiresthat the sensor data from individual paths be extrapolated overthe whole region as in Fig 4 where data from an AUV samplingpath is extrapolated using regularized kernel regression, specifi-cally the νSVR (Support Vector Regression) of [10] and [9]. Asshown, the method expands the useful domain of the data fromthe path to the full region, while regularizing the signal withoutloosing the peaks.

SURFACE RECONSTRUCTIONAnother application of the robotic sensor network is to con-

struct 3D models of marine structures. Marine structures arestructures that are partially submerged. For safety, marine struc-tures need to be inspected regularly for wear and tear. Manualinspection is tedious and even a small oversight can have severeconsequences for the structure and the people inside or around it.A robotic system that can construct 3D models of marine struc-tures would enable remote inspection and allow human field in-spectors to focus on higher risk areas. This inspection strategyreduces the likelihood of oversights and improves the safety ofmarine environments. In this section we present the hardwareand software infrastructure to construct 3D models of marinestructures. Our current system uses a single ASV with multi-ple sensors. The hardware setup can be easily implemented onmultiple ASVs to speed up inspection of large marine structureswhenever necessary. Various multi-robot mapping algorithms,such as [13], can then be used to merge the 3D scan data fromdifferent ASVs.

Hardware setupDue to the water currents and wakes that may move the ASV

adversely, we would like to use a scanning sensor that can fin-ish each scanning cycle quickly and has a relatively wide fieldof view. The high scanning frequency allows each scanning cy-cle to be completed before the ASV drifts significantly far fromthe position where the scanning cycle was started. This reducesthe need to adjust different points within a single scan accordingto the ASV movement, and hence simplifies 3D model recon-struction. The wide field of view provides significant overlapsbetween subsequent scans of the environment, despite the unin-tended movement of the ASV due to water currents and wakes.Significant overlaps between subsequent scans allows mergingsubsequent scans into one coordinate system without knowingthe scanner pose when the scans were taken, which is an impor-tant capability for 3D model reconstruction when the accuracyof the robot’s positioning sensors is low. Note that the abovesensor requirements do not need to be satisfied by both the sen-sor that scans above the waterline and the underwater part of the

structures. When one of these sensors satisfies the above require-ments, we can use the information for merging the scan data fromone of the sensors to help the merging process of the scan datagenerated by the other sensor. In this work, the above require-ments are satisfied by the sensor that scans the above the water-line part of the structures. To scan the above the waterline weuse a Velodyne HDL-64E S2, a 3D LiDAR (Light Detection andRanging) that finishes each scanning cycle in 0.1 seconds. Ineach scanning cycle, the LiDAR captures the entire 3600 hori-zontal and 26.80 vertical field of view with 0.090 × 0.40 reso-lution. Unfortunately the Velodyne LiDAR cannot be mountedin its standard configuration on the ASV. When the LiDAR ismounted in its standard configuration, it sits low on the water rel-ative to the structures we would like to scan. Since the LiDAR’svertical field of view spans from −24.80 to +20 and its rangelimit is 50m, in its standard configuration, the LiDAR can onlyscan parts of the marine structures from the water surface up toaround 2 meters above the water surface. This is insufficient forour purpose. To overcome this difficulty, we mount the LiDARin an inverted configuration, thereby generating a vertical field ofview that spans from −20 to +24.80 and enables the LiDAR toscan parts of the marine structures from the water surface up toaround 20 meters above the water surface.

However, mounting the LiDAR in an inverted configurationmakes the ASV less stable. Since the LiDAR is quite heavy(around 13 kilograms) and its center of gravity lies near itsbottom, which would sit high up in an inverted configuration,mounting the LiDAR in an inverted configuration significantlyraises the center of gravity of the entire system. As a result,the ASV becomes less stable, especially in roll, and vulnerableto capsizing when operating in rough water environments. Tomount the LiDAR in an inverted configuration and maintain sta-bility, we designed a platform to mount the LiDAR in an invertedconfiguration on the ASV with two primary considerations: ve-hicle stability and sensor visibility. Fig 5(b) shows the mountingplatform. The entire mounting structure is made with lightweightaluminium extrusions. The four posts bearing the weight of thesensor are tied together with triangulating pieces in the hull ofthe ASV to create a rigid platform to mount the LiDAR that isrobust to motion in all directions. Furthermore, the four posts arenarrow so that they have a minimal effect on the data collectedby the LiDAR. Experiments with the LiDAR in this mount showthat these posts cast insignificant shadows on the LiDAR returns.In order to ensure the stability of the ASV, we keep the cen-troid of the craft low by mounting the LiDAR as low as possiblewithout encroaching on the sensor’s field of view. Additionally,to protect the ASV from rolling, its most vulnerable direction,we attach port and starboard stabilizers. These stabilizers con-sist of buoyant pontoons mounted on an aluminum square extru-sion assembly that is fixed directly to the LiDAR’s mount. Thepontoons are streamlined to minimize the added drag. To scanthe underwater part of the marine structure of interest, we use

4 Copyright c© 2012 by ASME

(a) (b) (c)

FIGURE 5. (a) The ASV setup for scanning the above and underwater parts of a marine structure. Note that additional pontoons are attached to theASV to improve its stability. (b) The Velodyne LiDAR and the mounting platform for placing the LiDAR in an inverted configuration on the ASV. (c)The BlueView MB2250 micro-bathymetry sonar mounted in a sideways configuration.

FIGURE 6. Processing steps for 3D model reconstruction of partiallysubmerged marine structures.

a 3D Microbathymetry sonar from BlueView (model MB2250-45). The MB2250-45 sonar uses 256 beams with one degreebeam width in the elevation. Since we are interested in mappingmarine structures, instead of mounting the sonar in the defaultforward-looking configuration, we mount it sideways on the ve-hicle (Fig 5(c)). The overall setup of is shown in Fig 5(a).

Software for 3D model reconstructionThe main difficulty in 3D model reconstruction is that GPS

signals are often blocked by the structures themselves during op-eration. Furthermore, we need to merge the 3D scan data of theabove the waterline part of the structures with 2D scan data of theunderwater part of the structures. To overcome the lack of GPSinformation, we use a scan matching technique to construct the3D model for above the waterline part of the structures. We thenuse the transformations, computed by the scan matching algo-rithm for the above the waterline part, to construct the 3D modelof the underwater part of the structures. We combine the 3Dmodel of above the waterline and underwater parts to construct acomplete 3D model of the partially submerged marine structure.

(a)

(b)

(c)

FIGURE 7. Mesh-based map for the above the waterline part of thejetty.

A diagram of this process is shown in Fig 6.

Experimental ResultsWe have tested our system successfully to construct a jetty

in Pulau Hantu, Singapore. The water currents during operationwere around 1-2m/s. The reconstructed 3D model can be seen inFig 7 and Fig 8. Higher resolution mesh-based maps are shownin Fig 7, while lower resolution point cloud maps are shown inFig 8. The maps can be used for inspection (high resolution) ornavigation in cluttered environements (lower resolution).

5 Copyright c© 2012 by ASME

(a)

(b)

FIGURE 8. Point clouds map for the above and underwater part ofthe jetty.

CONCLUSIONS AND FUTURE WORKThe vehicles and operational configuration of the coastal

distributed autonomous sensor network being developed at CEN-SAM were presented in this study. Two applications showcasingimportant tools for ocean surveys and modeling were described:algal bloom monitoring and 3D surface reconstruction. Algalblooms represent a major concern in coastal environments asthey have significant humanitarian and economical impacts. Ourgroup uses the exploration and monitoring capabilities of a sen-sor network along with statistical analysis to enable increasinglyaccurate models of bloom activity. Aerial, surface, and under-water vehicles were combined to sample and detect biochemicalpreconditions for algal blooms in the coasts of Singapore. Awealth of biochemical data (ph, chlorophyll, temperature amongothers) is being analyzed using learning models to understandand predict algal bloom occurrences. Inspection and operationin cluttered environments requires 3D maps to facilitate the op-eration of autonomous vehicles. 3D feature reconstruction of amedium sized marine structure (jetty) was successfully tested us-ing surface vehicles equipped with LIDAR and sonar. Both lowand high resolution maps can be generated with the presented ap-proach which can be used for either navigation or inspection de-pending on mission goals. Our group will continue testing thesetools in larger scale missions involving more vehicles as well asdeveloping further tools for oceanographic studies.

REFERENCES[1] T. Curtin, J. G. Bellingham, J. Catipovic, and D. Webb,

“Autonomous Ocean Sampling Networks”, Oceanography,6(3), 86-94, 1993.

[2] T. Curtin and J. G. Bellingham, “Autonomous Ocean Sam-pling Networks”, Journal of Oceanic Engineering, 26(4),2001.

[3] D.C. Webb, P.J. Simonetti, and C.P. Jones, “SLOCUM:an underwater glider propelled by environmental energy”,Volume 26, Issue 4, IEEE Journal of Oceanic Engineering,2001.

[4] S.M. Smith, P.E. An, K. Holappa, J. Whitney, A. Burns, K.Nelson, E. Heatzig, O. Kempfe, D. Kronen, T. Pantelakis,E. Henderson, G. Font, R. Dunn, and S.E. Dunn, “The Mor-pheus ultramodular autonomous underwater vehicle”, Vol-ume 26, Issue 4, IEEE Journal of Oceanic Engineering,2001.

[5] J.G. Bellingham, and J. S., Willcox, “Optimizing AUVOceanographic Surveys,” Proceedings AUV ’96, 391-398(1996).

[6] J.S. Willcox, J.G. Bellingham, Y. Zhang, and A.B. Bag-geroer, “Oceanographic Surveys with Autonomous Under-water Vehicles: Performance Metrics and Survey Design”,IEEE Journal of Oceanic Engineering, 26(4), 711-725,2001.

[7] T. Schneider and H. Schmidt, “Unified Command and Con-trol for Heterogeneous Marine Sensing Networks”, Journalof Field Robotics, 27(6), 876-889, 2010.

[8] N. M. Patrikalakis, F. S. Hover, B. H. Ooi, H. Zheng, K. T.Yeo, W. Cho, T. Bandyopadhyay, A. C. H. Tan, H. Kurni-awati, T. Taher, and R. R. Khan, “Infrastructure for MobileSensor Network in the Singapore Coastal Zone”, Proceed-ings of the 20th International Society of Offshore and PolarEngineering Conference (ISOPE), Beijing, China.

[9] Bernhard Scholkopf and Alexander J. Smola. Learningwith Kernels: Support Vector Machines, Regularization,Optimization and Beyond, MIT Press, 2001.

[10] Vladimir Vapnik. The Nature of Statistical Learning The-ory. Springer-Verlag, 1995.

[11] Hadley Wickham. ggplot2: elegant graphics for data anal-ysis. Springer New York, 2009.

[12] Duncan Murdoch and E. D. Chow. ellipse: Functionsfor drawing ellipses and ellipse-like confidence regions, Rpackage version 0.3-5, 2007.

[13] S. Thrun, W. Burgard and D. Fox. A real-time algorithmfor mobile robot mapping with applications to multi-robotand 3D mapping Proc. of IEEE International Conferenceon Robotics and Automation 2000

ACKNOWLEDGMENTThe research described in this project was funded in whole

or in part by the Singapore National Research Foundation (NRF)through the Singapore-MIT Alliance for Research and Technol-ogy (SMART) Center for Environmental Sensing and Modeling(CENSAM).

6 Copyright c© 2012 by ASME