http://adb.sagepub.com/Adaptive Behavior

http://adb.sagepub.com/content/21/5/404The online version of this article can be found at:

 DOI: 10.1177/1059712313491615 2013 21: 404 originally published online 29 July 2013Adaptive Behavior

Ritzmann and Roger D QuinnKathryn A Daltorio, Brian R Tietz, John A Bender, Victoria A Webster, Nicholas S Szczecinski, Michael S Branicky, Roy E

Blaberus discoidalisA model of exploration and goal-searching in the cockroach,   

Published by:

http://www.sagepublications.com

On behalf of: 

  International Society of Adaptive Behavior

can be found at:Adaptive BehaviorAdditional services and information for    

  http://adb.sagepub.com/cgi/alertsEmail Alerts:

 

http://adb.sagepub.com/subscriptionsSubscriptions:  

http://www.sagepub.com/journalsReprints.navReprints:  

http://www.sagepub.com/journalsPermissions.navPermissions:  

http://adb.sagepub.com/content/21/5/404.refs.htmlCitations:  

What is This? 

- Jul 29, 2013OnlineFirst Version of Record  

- Sep 10, 2013Version of Record >>

at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from at OhioLink on October 20, 2013adb.sagepub.comDownloaded from

Original Paper

Adaptive Behavior21(5) 404–420� The Author(s) 2013Reprints and permissions:sagepub.co.uk/journalsPermissions.navDOI: 10.1177/1059712313491615adb.sagepub.com

A model of exploration andgoal-searching in the cockroach,Blaberus discoidalis

Kathryn A Daltorio1, Brian R Tietz1, John A Bender2, Victoria AWebster1, Nicholas S Szczecinski1, Michael S Branicky3, Roy E Ritzmann2

and Roger D Quinn1

AbstractCockroach shelter-seeking strategy may initially look like an undirected random search, but we show that they areattracted to darkened shelters. They arrive at a shelter in about half the time control cockroaches take to reach thesame location with no shelter present. We were able to identify six statistically significant trends from the behavior of134 cockroaches in 1-min naıve walking trials with four different shelter configurations. By combining these trends into amodel, we built a stochastic algorithm that significantly biases a simulated agent toward a target location. We call thismodel RAMBLER (Randomized Algorithm Mimicking Biased Lone Exploration in Roaches). RAMBLER could be adaptedfor a mobile robot equipped with an onboard camera and antenna-like contact sensors.

KeywordsAnimal behavior model, cockroach wall-following, bounded biased random walk, biologically inspired robotics

1 Introduction

Insect navigation behaviors have inspired advances invisual landmarking (Collet and Land, 1975; Yu, Leeand Kim, 2012), motion detection (Borst and Egelhaaf,1989), path selection (Dorigo & Gambardella, 1997),sensory integration (Wehner, 2003), high speed wall-following (Cowan, Lee, & Full, 2006; Chapman andWebb, 2006), and visual processing (Huber, Franz, &Bulthoff, 1999). However, one review article claims thatless has been done with their basic searching, despite itsvalue in conditions where other strategies fail (Franzand Mallot, 2000).

Finding a goal in an unknown environment is animportant task for exploration. Consider controlling amobile robot. Gradient descent controllers arrive atsome targets (local minima) quickly, while missing oth-ers, for example a goal behind a concave barrier.Systematic sweep approaches may be more complete,but less efficient and require more spatial memory.Incorporating randomness removes the memory andplanning requirement, but can add even more ineffi-ciency. Convergence has been demonstrated for severalways of adding stochastic terms to nonholonomicsource seeking controllers (Liu and Krstic, 2010;Stankovi and Stipanovic, 2009). When sensors are

limited, a simple strategy of pivoting a randomamount, stepping forward to measure a gradient, andthen moving either forward or backward based on thesign of the measured gradient direction has been pro-posed (Azuma, Sakar, & Pappas, 2010).

Still, these algorithms must be tuned for each appli-cation to balance the inherent trade-off between greedygoal tracking (direct but incomplete) and stochasticexploration (finds even hidden goals eventually). Arobotic controller may be tested in a several exampleenvironments, but animals have been testing theirstrategies in complicated real environments for life-times. Understanding the way animals address theexploration-exploitation trade-off not only shows usmore about the animal, but also provides a tested

1Mechanical Engineering Department, Case Western Reserve University,

Cleveland, Ohio, USA2Biology Department, Case Western Reserve University, Cleveland,

Ohio, USA3Electrical Engineering and Computer Science Department, Case

Western Reserve University, Cleveland, Ohio, USA

Corresponding author:

Kathryn Daltorio, Bio-Robotics Laboratory, 421 Glennan Building, 10900

Euclid Avenue, Cleveland, OH 44106, USA.

Email: kathryn.daltorio@case.edu

feasible tactic for naıve exploration. For example, twodesert arthropods use widening spirals when path inte-gration errors (or researchers’ interference) place themaway from their homes (Hoffmann, 1983; Muller andWehner, 1994). Such structure is not the norm, how-ever. More often, animal movements have been insight-fully analyzed with two-state random strategies, aforward motion state and a reorientation state: Run-and-tumble in E. coli (Nicolau, Armitage, & Maini,2009), relocation and local search in foraging animals(Benichou, Coppey, Moreau, & Voituriez, 2006), inter-mittent Levy patterns for long distances (Bartumeus,2009), and various other types of random walks(Borger, Dalziel, & Fryxell, 2008). Fish motion hasbeen characterized as a Persistent Turning Walker,allowing for the adjustment of angular velocity insteadof heading (Gautrais et al., 2009).

Cockroaches are especially interesting because theyare common in experimental biology. Already, cock-roaches have provided insight into tactile wall-following(Camhi and Johnson, 1998; Chapman and Webb, 2006;Cowan et al., 2006), turning (Harley, English, &Ritzmann, 2009), climbing and tunneling (Lewinger etal., 2005). It is even possible to build cockroach-scalerobots. For example, Asadpour, Tache, Caprari,Karlen, and Siegwart (2006) developed a matchbox-sized robot that uses hierarchical potential-field basedbehaviors and can influence cockroaches’ resting sitesby example (Halloy et al., 2007). These and otherexperimental successes raise new questions about howthe insects’ brains make decisions about when to doeach of these tasks. Turning, for example, is associatedwith a complex multi sensory area of the brain (Guoand Ritzman, 2013; Harley et al., 2009) whose functionis not yet understood, but can be probed in walkinginsects (Zill, 2010). Understanding the intact behaviormay help us hypothesize about how turning decisionsare made in the animal as well as provide a baseline forlater experiments with altered insects.

Previously, walking cockroach trajectories were fitto an isotropic diffusion random-walk model (Jeansonet al., 2003) and the result was implemented on a wall-following robot (Jost et al., 2004). The same group con-firmed that cockroaches prefer resting under dark shel-ters (Canonge, Sempo, Jeanson, Detrain, &Deneubourg, 2009), but no shelter-seeking bias wasreported. That is, their path to the shelter location wasnot suggested to have been influenced by the observa-tion of a shelter.

In the experiments presented here, we found not onlyrandomized behavior but also goal-seeking behavior.The path length to shelter is a little over half the pathlength to the shelter location in an empty arena (noshelter; Daltorio et al., 2012). To further understandthis adaptive behavior, we analyzed parameters that arebiased by the shelter, and conditions in which theyapply. Following these trends gets a simulated agent to

a goal with significantly less walking: 62% of the timethat would otherwise be required. This algorithm,RAMBLER (Randomized Algorithm MimickingBiased Lone Exploration in Roaches) would bestraightforward to implement robotically because itrequires little memory or computation. The purpose ofthis work is to characterize cockroach behavior in anunfamiliar environment. To be implementable on arobot, the model should be based on step-by-step deci-sions. To be extensible to other arenas in future, themodel should characterize the likelihood of all possiblerelevant choices. To be relevant to biology, the modelshould be sufficient to capture the metrics of the overallbehavior in the arena.

2 Insect experimental set-up

We are interested in exploration: how a cockroachapproaches an unknown environment. Although thereis interesting literature on how cockroaches can reactquickly to signs of danger, the escape response mayrepresent only a small fraction of cockroach foragingmovements. These escape behaviors are special well-defined reactions to immediate perceived threats thatcan be elicited even if the brain is damaged, althoughcan be modulated by sensory context (Schaefer andRitzmann, 2001) or terminated by entering a shadedshelter (Okada and Toh, 1998). We want to know whatdecisions the cockroach makes when it has as muchtime as it typically uses to decide with all its faculties.We have shown in our previous work that that prod-ding the animal in our arena results a transient periodof significantly higher walking speed (Bender et al.,2011), so we believe that the unprodded behavior is sig-nificantly different from escape response behavior.

Cockroaches are known to prefer shaded shelters(Okada and Toh, 1998; Canonge et al., 2009). Theexploration problem is how greedily to pursue promis-ing features (such as shaded shelters) vs. how much totake advantage of the nearest features (such as walls) ina situation where the best choice is unknown. We can-not know the individual cockroaches’ precise goals (incomputer science terms: the expected utility function ofeach choice). However, if we know that certain featuresare preferred, we can assume that they have a higherutility and we can analyze how that higher expectationaffects the decisions about which way to go. The arenawalls are needed contain the animals, but we can varythe presence of a shelter to see how the promise of ashaded shelter biases the search of the arena.

To understand how cockroaches approach explora-tion, we built a 91-cm2 arena with a small startingchamber (Figure 1). For each trial, an adult femalecockroach (Blaberus discoidalis) from a laboratory col-ony was placed in the start chamber. After 20 s, a gatebetween the start chamber and the arena was opened.

Daltorio et al. 405

If the animal did not leave the start chamber after 4min, the trial was aborted and a new cockroach waschosen. The median time in the start chamber was 11 s.Time in the chamber did not correlate with the direct-ness of the path to the shelter. The animals were notprodded in any way during the trial—neither to encour-age them to leave the start chamber nor to keep themmoving during the trial.

The animals were recorded exploring the arena withan overhead camera. In control trials, the arena wasempty; otherwise, a 153 15 cm red, translucent shelterwas placed in one of three locations (Figure 1) toattract the cockroaches. The arena was lit with incan-descent bulbs to approximately 1500 lux. The lumi-nance underneath the shelter was 300 lux. Each animalwas used only once; each trial was for 60 s. The arenawas cleaned between trials with detergent and water toremove evidence of previous trials. Sheets of blackcloth shrouded the arena to remove perspective cuesand environmental bias. Each animal’s behavior wasrecorded at 20 Hz with a digital video camera using theMotmot image acquisition package (Straw andDickinson, 2009). The position of the center of thecockroach’s body and its orientation in each frame ofthe videos were extracted using the averCaltechMultiple Fly Tracker (version 0.1.5.6; http://ctrax.berlios.de/) and the associated FixErrors toolbox forMATLAB (MathWorks, Inc., Natick, MA, USA;Branson, Robie, Bender, Perona, & Dickinson, 2009).

Because there were still occasional inversions of thetracked angle, we also corrected any instantaneous180� flips. All further analysis was done in MATLAB.

3 Analyzing cockroach data

To determine the behavior of the animals with respectto the shelter, we tested animals in an empty arena (withno shelter), but analyzed the data as if there were a shel-ter. First, we see that they prefer to rest under shelters.The cockroaches spend 10–20 s under these sheltersover the course of a 60-s trial. In empty trials, the cock-roaches spend 2–7 s under the shelters locations overthe course of the trial, depending on the shelter locationas shown in Figure 2.

To determine if the animals were indeed following adirected search and not an isotropic diffusion model, asin Jeanson et al. (2003). For each of the three shelterlocations, we measured the distance walked before theanimal first arrived at a shelter location (Figure 3).Regardless of the location of the shelter, the insectstook significantly shorter paths to the shelter than tothe same area in the shelter-less trials. In fact, the ani-mals are arriving at the shelter locations in about halfthe time and with a little over half the total path lengthwhen there is a shelter present. Figure 4 considers sev-eral other metrics in detail for the SC shelter location.Whether considering time to shelter location, distancewalked (path length) to shelter location, coverage of

Figure 1. Cockroaches were permitted to explore an enclosed arena, as seen through the camera mounted overhead for tracking.In each trial, the shelter was placed in one of three different locations, shown here is the SC location, and the EC and NE locationsare outlined. Note that these short descriptors (SC, NE, EC) are meant to evoke the compass directions as a mnemonic whenviewed with the start chamber on top. However, the arena walls were not intentionally aligned with a magnetic compass. In fact, theNE and EC shelter locations were placed on alternating sides of the arena, and the tracks were flipped in post processing to mitigateany affects of asymmetry. In this video frame, a cockroach is walking away from the entrance chamber and toward the shelter. Thearena is 91 cm2 with 10-cm high walls that are greased to discourage climbing. On the right, a sample cockroach arrives at shelter inthe SC location after a series of turns, which we fit to a series of straight-line segments connected by pivots.

406 Adaptive Behavior 21(5)

the wall or arena before the shelter location is reached,there is a significant difference when there is a shelterpresent. However, walking speed, distance walked perminute and coverage per minute are not significantly

different. The other shelter locations have similarresults. This suggests that the shelter-seeking behavioris a subtle modulation of the behavior in the emptyarena.

We wanted to know how the animal got to the shelterfaster, but every animal path was different. We couldnot account for the variation in turning and speed fromtimestep to timestep. We needed a way to parse the pathfor analysis. Our previous work (Bender et al., 2011)indicated that the animal uses different gaits (sloweramble instead of a faster trot) when close enough to thewall for antennal contact. This context-dependent beha-vior is the reason we have divided the behavior intostates based on wall contact. Blaberus discoidalis has anaverage antenna length of about half the body length.These antennae are almost always positioned be the firstpart of the cockroach to touch the wall. After encoun-tering a wall with the antenna, the cockroach followsthe wall on the side of the first antenna to contact. After‘‘Wall Following’’, there are three observable actions,which will correspond to states in our controller: anglingaway from the wall in the arena (Wall Depart), reversingdirection on the wall (Wall Turnaround), and arrivingat corners (Corner Exploration).

When away from the wall, the cockroaches execute aseries of seemingly random turns of varying curvature.First, we tried correlating the angular velocity to envi-ronmental parameters. However, wall parameters andprevious state variables generally dominated shelter-related parameters such as relative shelter angle anddistance to shelter. The resulting fit became elaborate

Figure 3. The path length from the starting point to three shelter locations is measured by tracking the visual center of thecockroaches’ bodies. There are 33 trials with the shelter in the center of the mid-back wall (referred to as SC), 17 trials with theshelter in corners closest to the top (NE), 40 trials with the shelter along the side-wall (EC) and 44 trials with no shelter, as a control.Each graph above compares the distance walked to a red shelter (colored histogram bars) with the distance walked to that samelocation in an empty arena with no shelter (gray histogram bars) for control. In each case, the path to the shelter location is shorterwhen there is a shelter present. p-values for the difference between shelter and no shelter distributions are calculated using kstest2(Matlab’s 2 sampled Kolmogorov–Smirnov test): SC p-value=0.0000011, EC p-value=0.013, NE p-value = 0.02. In 19% of control trialsand 6% of shelter trials, the cockroach had not arrived at the shelter location at the end of the trial so the total path length was used.These data shows that the behavior of the cockroach before arriving at the shelter is directed toward the shelter in some way.

Figure 2. The total time out of a 60-s trial spend under ashelter location. If the animals encounter a shelter, they tend tostay under it longer than they stay under the shelter location intrials in which there was no shelter. This is especially true forSC and EC locations, which have significant t-test and p-values.When there is no shelter in the arena, the NE corners still makegood resting places because they are in a corner and close tothe entrance.

Daltorio et al. 407

but did not lend itself to an insightful or elegant imple-mentation. So instead we fit these curves into a combi-nation of zero-turn-radius pivots and straight lines,shown in Figure 1. Pivots and straight lines are conve-nient for many types of robots. How the continuousanimal motion data is segmented into turns is sensitiveto sampling rates (Tourtellot, Collins, & Bell, 1991) andthe choice of segmentation thresholds in our case.However, we would argue that fitting the tracks tothese types of turns allows us to compare tortuosity (byhow many pivots and their magnitude) and bias (turndirection) for different conditions.

Our segmentation process is as follows: Each time acockroach left the wall for more than 0.5 s, the pathbetween wall-to-wall was broken down into a series ofline segments. First, a single line segment was consid-ered with endpoints at the wall departure point and atthe wall arrival point. The squared distances from theline segment to each tracked point were weighted byspeed and averaged. If that value was greater than1 mm2 or if any tracked point were further than 10 mmfrom the line, an additional segment was required. Eachtracked point in the segment was considered as a pivotpoint. If the criteria were still not met, additional pivotswere added by randomly selecting 100 combinations ofpivot points and then performing bounded minimiza-tions on those combinations, picking the best combina-tion for that number of pivots and re-evaluating the fitcriteria. Because the animals were more likely to depart

the wall or walk a little faster just after they exited theentrance chamber, the first 200 mm of the insect’s walkaway from the starting point were not included.

4 Extracting trends

In some trials, the animal would nearly approach theshelter before veering aside to find the shelter againlater. We hypothesized that a state based model wouldbe a good way to characterize the data if we found theright transition parameters. We were able to measureeach of these transition parameters from the cockroachtracks to build our model. In these data, we looked fortrends to capture the shelter seeking behavior that arenot also present in the shelter-less arena trials. Weexpected that some trends might be evident only whenthe shelter was at certain positions relative to the bodyof the cockroach, so we often divide the data based onperceived relative angle of the shelter. For example, theshelter is perceived at 0� if the animal is headingdirectly toward the shelter. If the divisions (or bins)divide the data too finely, there won’t be enough datain each bin to be confident of the results. If the divi-sions are too wide, trends that the animal only exhibitsunder narrow set of conditions will be missed. As such,we adjusted the bin width to narrow confidence inter-vals. Confidence intervals are bootstrap confidenceintervals with 1000 resamples of the mean. When confi-dence intervals for empty arena trials do not overlap

Figure 4. Insects’ performance measures for getting from the start chamber to the SC shelter location. The average animal walksover 2 m to get to the shelter. For control, without a shelter, the mean path length to the SC shelter location (virtual shelter) is over5 m. Over the course of the 1-min trial, all animals walk about 10 m. Coverage is measured with a grid assuming that all pointswithin 50 mm of the head are considered reached. While the overall wall and area coverages were similar (60% in 1 min), the timeand distance to the SC location was shortened by the presence of a darkened shelter there. Note that the mean speeds, measuredbefore arrival at the shelter, are the same. Each dot is a single trial. Error bars are 90% bootstrap confidence intervals on mean.Since only parameters related to how long it took to get to the shelter are different, the shelter seeking behavior is likely a subtleadaption to the exploration behavior in the shelter-less arena.

408 Adaptive Behavior 21(5)

confidence intervals for shelter trials, we consider theresult significant. Note that a lowercase p will indicateexplicit significance testing and an uppercase P refersto ratio or probability in later figures.

4.1 Speeds—apparently unaffected by shelter

The walking speed varies with the wall proximity, asshown in Figure 5. Our previous paper (Bender et al.,2011) suggests that this is because when the animal isout of antennal contact range of the wall they assume adifferent gait. We chose to group the data by whether itwas within 40 mm from the wall because this representsa large jump in the data. The antennae are too small totrack in the videos, but since the antennae are about 25mm long and touch the wall at an angle, this divisiondivides the data approximately by antennal contactstate. The presence of a shelter in the arena does notsignificantly affect these speeds.

4.2 Wall departures—more frequent if shelterbehind

In the arena, insects eventually encounter one of thewalls that bounds the arena. A typical cockroach turnsin the direction opposite the first antennal contact(Harley et al., 2009). They follow along the wall,

keeping the first-wall-contacting antenna in contact.Eventually, they depart the wall, going back into thearena and losing antennal contact. We characterizedthe decision to depart the wall by counting the numberof instances in which the animal crossed the wall-following threshold (40 mm). To make sure we werenot double-counting any animals that may have wav-ered at the edge of the threshold, we only counteddepartures in which the animal had been under thethreshold for at least 0.1 s prior to the departure. Thenumber of departures divided by the distance walked(the sum of the displacements between the trackedpoints) is the departure rate. To get a confidence inte-gral, we considered each point along the wall in anarray. If that point is a departure (the last point beforethe distance was greater than 40 mm) we assigned thevalue of the array to be 1, otherwise it was assigned 0.The average of this array, weighted by the speed, is thesame as the departure rate. We used the bootstrap com-mand to resample this array and take weighted averagesof the subsamples 1000 times. The 5th and 95th percen-tile of those subsamples averages define the 90% boot-strap confidence intervals, which are shaded around thelines in Figure 6.

For both empty arena trials and shelter trials, thereare about 1.9 departures per meter walked along thewall. However, when the wall departure rate is broken

Figure 5. Animal speed (measured by distance between body centers tracked at 20 Hz) increases with distance to nearest wall.Black is shelter-less arena control, blue is with a darkened shelter in the arena, but they are not significantly different. Average wallspeed in the empty arena=75 mm/s and with shelter=88 m/s. Average speed away from walls in empty arena=190 mm/s, withshelter=196 mm/s. Note that where the blue and gray error bars overlap the result is a darker blue. Error bars are 90% bootstrapconfidence on the mean. The diagram below is a scaled representation of the arena with the area near the wall in gray. n=44cockroaches in empty arenas, n=90 cockroaches in arenas with shelter. Each animal was recorded for 60 s, or 1200 frames, son=52800 frames in empty arenas and n=108,000 frames in shelter arenas.

Daltorio et al. 409

down by the angle of the shelter with respect to thewall, there are significant differences (Figure 6).Because the body angles of the insect varied slightly asthey walked, we considered the wall angle and sub-tracted from that the relative direction from the bodycenter to the center of the shelter location. We foundthat if the animal is along a wall that leads it awayfrom the shelter (135–180� angle with respect to theshelter), it is much more likely to depart the wall.Otherwise, the insect is slightly more likely to stayalong the wall. This trend was true for all the differentshelter locations, although the exact rates varied. Thus,the cockroach departs the wall more when its distanceto the goal is increasing.

To leave the wall, cockroaches turn their bodiesaway from the wall at and begin walking in the arena ina direction that is no longer parallel to the wall. We willrefer to this angle as the departure angle. To measurethis, we found the angle between the insect’s new pathin the arena and the wall the insect was following. Weapproximated the insect’s path as a line between twokey points. The first point is the departure point: whenthe animal crosses the wall threshold 40 mm away fromthe wall. The second point is when the animal reached150% of the threshold (60 mm) away from the wall.The body angle at the departure point would be a simi-lar measure, however it turns out to be noisier to trackand more threshold-sensitive. The average departureangle was 30� in trials without the shelter and 35� in

trials with a shelter (Figure 7). Note that this is quanti-tatively similar to the results for young cockroaches ina circular arena (Jeanson et al., 2003).

4.3 Wall turnarounds—infrequent when facingshelter

Occasionally, an insect walking along the wall will stopand make a 180� turn. Then the animal follows the wallback the way it came, staying in the range of theantenna throughout the maneuver. We will refer to thisaction as a ‘‘turnaround.’’ To identify these turn-arounds in the tracked data we looked for times whenthe animal was not moving parallel to the wall andchecked to see if the wall walking before and after werein opposite directions. (Note that turnarounds in cor-ners are considered separately.) The turnaround ratewas then calculated as the number of turnarounds permeter walked. In the empty arenas, the overall turn-around rate was 0.54–0.74 (90% bootstrapped confi-dence interval) turnarounds per meter walked. Forshelter trials, it was overall 25% less. We believe thiscan be explained by differences that occur when facinga shelter. In trials with shelter, there was a significantdifference in turnaround rate when the cockroach facedthe shelter within 633� vs. when it was not facing theshelter. Furthermore, when the cockroach was not fac-ing the shelter the turnaround rate was not significantlydifferent from the turnaround rate with no shelter. This

Figure 6. Wall departure rates are higher when cockroach is walking away from shelter. This is a trend visible in all shelterlocations (SC, NE, EC, which are labeled in Figure 1.) In gray all the shelter-less trials are compared to the all the trials with shelterin any of the three locations in blue. The shaded boxes are the bootstrap 90% confidence intervals. Diagrams below the graphindicate the angle of the shelter location (as a shaded wedge) relative to the cockroach as it walked along the wall for each bin ofdata. There were 576 departures of the 60,568 frames in which a cockroach was along the wall when there was a shelter in thearena. Without a shelter, there were 273 departures of 25,964 frames in which a cockroach was along the wall.

410 Adaptive Behavior 21(5)

suggests that the presence of a shelter in front of theanimal reduces the tendency to turn around by aboutan order of magnitude as shown in Figure 8. However,because of the infrequency of turnarounds, it is unclearwhether the same trend exists in the control data whenfacing the virtual shelter.

4.4 Corner behavior—generally unaffected byshelter

The corners were considered separately from the walls.The animals often stopped in the corners, exploring the

joints between the arena walls with their antennae, andsometimes trying to climb up the walls. We defined acorner region as within 10 cm of one of the four cor-ners. Most of the time, the animals left the cornerregion on the unexplored wall. In 29% of the cornervisits in the empty trials, the animal exits the corneralong the same wall it came in on (Figure 9). In theshelter trials, for most conditions, the ratio of turn-arounds was not significantly different from the emptyarena trials. The one exception is the NE corner of thetrials with the shelter in the NE corner. The shelter isincentive for the animal to linger longer in the corner,

Figure 8. The rates of turnarounds per meter indicate that there are fewer orientation reversals while along the wall if facing theshelter. Error bars represent 90% bootstrap confidence. In trials with shelters, there were 48 turnarounds out of 60,568 framesalong the wall. In trials with shelters, there were 48 turnarounds out of 25,964 frames.

Figure 7. The distribution of departure angles follows a distribution similar to results in Jeanson et al. (2003). Relative frequencyrefers to number of samples per bin, normalized by the total number of samples (208 for the empty control case, 423 for the withshelter case). This distribution can be compared to a truncated exponential with rate 1/mean, i.e., the probability of ending the pivotis 0.03 per degree.

Daltorio et al. 411

and perhaps reorient or lose track of the direction ithad previously followed. Thus, it is not surprising thatin the NE shelter data the turnaround rate is higher,closer to the expected 50% rate if the animal had noknowledge of its previous wall following direction.

4.5 Arena pivot incidence rate—less if facing shelter

The tracks of individual cockroaches can be approxi-mated by periods of straight walking interspersed withpivots (Figure 1). To determine whether a shelteraffected the tendency to stop walking straight andpivot, we counted the number of pivots per distancewalked to get a pivot rate. By subdividing these data,

we can relate the pivot rate to the angle of the bodyrelative to the shelter (Figure 10). We did not want theboundary to unnecessarily affect the pivot rates so weonly considered line segments that ended in pivots andexcluded any line segments that ended against the wallor in the shelter. The pivot rates for the empty arenaand arena with shelter are similar except when the per-ceived angle of the shelter is small. That is, when thecockroach is essentially facing the shelter, the tendencyto turn is less. While the quantitative data presented inFigure 10 are dependent on the parameters that definethe fit, the fact that there are fewer turns when facingthe shelter does indicate that the animals are walkingstraighter when facing the shelter.

Figure 10. Number of pivots divided by distance walked, not counting pivots that occur under the shelter. Confidences are 90%bootstrap. Short lines are other intervals of 0.2 radians or 11� to demonstrate that the exceptionally low rate when facing shelter isunique. Using a constant pivot rate implies an exponential distribution of segment lengths with average segment length of one overthe pivot rate (in our case 174 mm for 5.7 pivots/m and 84 mm for 11.9 pivots/m). The diagram to the right shows the mean straightlengths and sectors of ± 11� relative to the body angle. If the direction to the shelter is within the sector, the pivot rate will belower and thus the segments will be longer on average.

Figure 9. Cockroach behavior when encountering four wall corners and also starting chamber entrance. Colors are differentshelter location configurations, groupings are specific corners encountered in the arena. The means by shelter configuration areplotted as horizontal lines. Only the average corner rate, excluding the NE corner rate for the NE corner shelter location is used.That subset is excluded because the cockroach is under a shelter. The widths are 90% bootstrap confidence intervals. The n=134cockroaches entered corners n=631 times.

412 Adaptive Behavior 21(5)

A different way to look at how frequently to turnwould be to model the cockroach’s decision as a choiceat some distant point to go directly to the goal withoutturning away. We examined how likely a cockroach fac-ing the shelter was to continue all the way to the shelter.We looked at each straight that did not start under theshelter. We compared the angle to the center of the shel-ter from the start point with the angle of the straight. Ifthe difference was within 17� (or 0.3 radians), it wasconsidered ‘‘facing.’’ This angle was chosen because itwas enough to include enough instances of the controlanimals facing the spot in the empty arena where theshelter would be. Some of the facing animals continuedall the way to the shelter location boundary with onlyslight pivoting or without pivoting (note that these datawere not included in Figure 10 since they end in theboundary). The probability of this occurring, groupedby shelter location distance, is plotted in Figure 10.From the pivot rates in Figure 11, we can predict thereto be an exponential survival curve from the constantpivot rate. The control data seems consistent with theprediction from Figure 10, but when there is a shelter inthe arena, the probabilities are higher by about 10%.

4.6 Arena pivot magnitude—no shelter effect found

To build a model, we also needed to quantify theamount of turning at each pivot. The pivot angle is the

difference between headings of successive straight seg-ments. Pivots were often in the same direction as theprevious pivot, i.e., a persistent or correlated randomwalk. Therefore, in Figure 12, the sign of the magni-tude is multiplied by the sign of the previous pivot (sopivots on the right half plane are pivots in the samedirection as previous.) The distribution shows thatmost pivots are less than 45�. Very shallow pivots areless detectable using this fit method since a theoreticalpivot of magnitude close to zero would not be consid-ered a pivot. No significant difference in pivot magni-tude was seen in the shelter and empty arena trials.

4.7 Pivot direction—if facing shelter, straightensmore

There is little difference in the plots in Figure 12 for theempty arenas and arenas with shelter. We wondered ifthe turning behavior was dependent on shelter percep-tion. While there was no clear trend in the pivot meanswith respect to shelter angle, 57% of the pivots are inthe direction of the shelter as opposed to 53% for thecontrol. This small but significant difference did notsufficiently capture the behavior in simulation withouttaking into account the direction of the previous pivot.(Note: we checked that the further history of previouspivots does not seem to affect the ratio of pivot direc-tions, i.e., Markov chain order 1.) Further, when

Figure 11. One might wonder, do the animals ever decide to go to the shelter at the beginning of a straight line? If so, we wouldexpect the percentage of the animals that walk directly to the shelter from a given location to be higher than the expectedexponential distribution from the pivot rate. This is the probability of the animal going directly to the shelter or going to the shelterwith only slight pivots if the animal is facing the shelter location within 0.3 radians (17�) at the start of the straight. When there is noshelter, the experimental data seems to line up with the pivot rate prediction. If there is a shelter, we could justify walking directly tothe shelter an extra 10% above the pivot rate prediction.

Daltorio et al. 413

grouped by egocentric shelter angle, the distinction issignificant only when the shelter is in front of the ani-mal within about 33�, as in Figure 13. The percentageleft pivot following right pivots or vise versa is a func-tion of the closeness of the fit of the animal turningdata. However, the sensitivity to shelter angle suggeststhat cockroaches tend to change turning direction morewhen turning away from the shelter, a trend not depen-dent on the fit of pivots and straights.

Note that the pivoting might also be biased by othergeometric parameters. Preliminary data suggest thatincluding a few wall factors in the algorithm mayimprove overall simulation agreement with experimen-tal data by about 5%.

5 Result: an algorithm we can verify withsimulation

To understand how much of the behavior these identi-fied trends captured, we combined the behavioral mea-surements in Figures 6–13 into a finite state machinemodel. To differentiate our model from unbiased mod-els (Creed and Miller, 1989; Jeanson et al., 2003) oraggregating models (Asadpour et al., 2006) or wall-following models (Chapman and Webb, 2006; Cowanet al., 2006), we describe it as Randomized AlgorithmMimicking Biased Lone Exploration in Roaches(RAMBLER). We used two sets of parameters: one forthe shelter-less arenas (control) and one with averagedparameters of the three shelter configurations. This cor-responds to all the blue (shelter) and gray (empty) datain Figures 6–13.

RAMBLER requires only wall-following ability,detection of the direction (left or right) of the goalwithin angle ranges, and minimal state-to-state memory(to remember whether the previous pivot was left orright). RAMBLER’s output is velocity and angular

velocity of the agent. Thus, in MATLAB, we simulateda non-holonomic vehicle following RAMBLER.

The agent’s initial orientation and position are firstsampled from the cockroach after they exit of the start-ing chamber for each shelter configuration type (SC,NE, EC). Then the simulated agent is in the ‘‘straight’’state. The agent goes straight until either encounteringa wall or pivoting with probability equal to pivot inci-dence rates in Figure 10 (multiplied by the speed and.05-s timestep to obtain rates per timestep). Pivot direc-tions are determined probabilistically according toFigure 13. Pivot magnitudes are sampled from Figure12; if the animal was facing a shelter when entering astraight, 10% of the time no pivots were performeduntil the shelter is reached (Figure 11).

Along the wall, the simulation prevents the agentfrom crossing the wall boundary and gradually adjuststhe heading to be parallel to the wall. Instantaneousturnarounds happen at rate in Figure 9. Departure ratesin Figure 6 determine when to end wall-following andbegin Depart State. After pivoting the required angle,sampled from Figure 7, Depart State ended when theagent was away from the wall by 150% of wall thresh-old (40 mm) or when 20 timesteps passed. An overviewwith rounded parameters can be found in Figure 14.Stopping to pivot slows the simulated agent, so walkingspeeds were scaled up by 1.7 such that the net speedagreed with the overall speeds of the cockroaches. Anexample simulation trial is shown in Figure 15.

Figure 16 represents 500 simulated trials for eachshelter configuration. Even though the measurementsof model parameters were averaged over all three shel-ter location types, the results have good agreement foreach type. This shows that the trends identified inFigures 6–13, do result in an algorithm that guides theanimal toward the shelter. The metrics in Figure 4(such as walking speed, and covered fraction of the

Figure 12. Distribution of pivot angles from tracked cockroach data fitted with the iterative method described above. Here thepivot angles have been multiplied by the sign of the previous pivot angle so positive values are pivots in the same direction as theprevious pivot. 70.6% of pivots are in the same direction as the previous pivot.

414 Adaptive Behavior 21(5)

wall) also align well with model predictions. The modelpredicts 2 m longer path length and 10% more cover-age of the arena after a shelter is found. This makessense since our model does not capture pausing at theshelter.

Further to evaluate the relative effectiveness of eachtrend, we turned off each of the six trends individually,leaving the other five in place. The results, in Table I,show that turning in arena (Figures 10, 11 and 13) isresponsible for 5/6 of the performance gains and theon-the-wall behavior in Figure 6 is responsible for theremaining 1/6 of the performance increase for this arenashape. The trends in Figure 7 and in Figure 7 could beeliminated, but we leave them in because they may beimportant in future testing in other arenas. Note thatthe trends in Figures 10 and 11 encode similar informa-tion (how much to pivot) but they might have differentrobotic implementations. Perhaps cockroaches aremore likely to make decisions step by step as in Figure10 but a robot designer might want to use some steeringtoward the goal if the straight-line segments are prone

to drift errors. This combination approach seems tobetter approximate the behavior in the arena (as shownin Figure 16).

From the angle values summarized in Table 1, wecan also see that the most important visual feedbackinformation is whether the shelter is in front of the ani-mal, and whether the shelter is in front left or frontright.

6 Conclusions and future work

We have analyzed over 2 h of cockroach exploratorytracks that initially appeared random and undirected.From that data, we identified and quantified six trends(Table 1). Each trend is simple and fairly straightfor-ward to implement on a robot, and requires no map-ping of the environment. A simulation of these trendscaptures the behavior of cockroaches in our arena well.To our knowledge, this is the first paper to build anexploration algorithm by quantifying how cockroachesuse walls to arrive at new visual goals. This new

Figure 13. The probability of turning toward the shelter is a function of the direction of the previous turn. About 70% of thepivots are in the same direction as the previous pivot. So the probability of turning toward the shelter is about 70% if the animal isalready turning toward the shelter (left) and about 30% if the animal is turning away (right). However, if the animal is facing theshelter within 33� and turning away from the shelter, then it is significantly more likely to switch pivot directions (as in the centerdiagram). These are the only conditions in which the trials with shelter are significantly different than the empty arenas. Shelter angleis the angle between the straight heading before the pivot and the angle to the shelter. There were n=552 pivots fit to trials withshelters and n=254 pivots fit to trials without shelters.

Daltorio et al. 415

evidence suggests that cockroach navigation is neithergoal-optimal nor dependent on systematic explorationstrategies. However, with a few sensory inputs (low-res-olution vision and tactile), they are achieving both arri-val at the goal (the shelter) and exploration of most(~60%) of the arena and wall. Advanced planning algo-rithms may be able to optimize solutions, but our workon the cockroach suggests that optimal may not alwaysbe required. The RAMBLER algorithm could be usedon a low-cost cleaning robot that wanders an area butoccasionally needs to find a recharging station. Wall-following in animals has been linked to exploration,especially in blind species (Patton, Windsor, &Coombs, 2010), so this algorithm could help a searchand rescue robot with limited senses balance using well-defined paths like walls and going off-path to investi-gate potential targets. A group of such robots could bereleased and alert a remote operator when targets are

Figure 14. Proposed state-based diagram of Randomized Algorithm Mimicking Biased Lone Exploration in Roaches (RAMBLER).There are states for along the wall, in the corner, and in the arena. ‘‘Follow Wall’’ continues until a turnaround or departure israndomly selected at shelter-orientation dependent rates or until a corner is encountered. ‘‘Straight’’ continues until a wall isencountered or a pivot is randomly selected.

Figure 15. An example track of a simulated agent controlledwith Randomized Algorithm Mimicking Biased Lone Explorationin Roaches (RAMBLER).

416 Adaptive Behavior 21(5)

found. Distributed systems or swarms of robots mayget better coverage with stochastic behavior. TheRAMBLER algorithm could be an alternative motionfor the types of multi-robot gate-controlled behaviorsuggested by Bobadilla, Martinez, Gobst, Gossman,and LaValle (2012).

The animal’s apparently random choices may beaffected by many other factors such as noisy sensoryinput (like the antenna brushing the floor, or a reflec-tion from the wall, or detecting an odor in the air) or ahidden mental state (such as fatigue, or an associativememory.) We tried to regulate many of these factors

using many trials with many different individuals.There is room for improvement in the fit of Figure 16,so perhaps there are other factors that we have yet tofind in the data that would match the animal behavioreven better. From this benchmark, we would like todetermine the contribution of remembered relative wallpositions, the frustration of not being able to get to thegoal, and the choice of multiple goals by adding morefeatures to the arena such as barriers, competing goalsor adverse stimuli. We could also compare this explora-tion behavior with homing behavior toward a remem-bered location, or with behavior with aversive stimuli

Figure 16. The head of a simulated agent travels a similar distance to the head of a cockroach, indicating that this model predictsmuch of the cockroach shelter seeking behavior. Initial conditions sampled from the first time cockroach got 400 mm into arena.Percentages in the legend are the ratio between path length with shelter to path length without shelter for simulation andcockroaches. One averaged set of Randomized Algorithm Mimicking Biased Lone Exploration in Roaches (RAMBLER) transitionparameters seems to capture much of the behavior for three different shelter types.

Table 1. Trend contribution summary.

Fig Shelter trend descriptiona Angleb�%c

6 150% more frequent wall departures when facing away > 135� 5%7 Depart wall more acutely by 5� – 0%8 Turnaround on wall less (10%) when facing shelter < 33� 0%10 Pivot less (50%) when facing shelter < 11� 4%11 Occasionally (10%) walk straight to shelter < 17� 11%13 Turn more (P=50% instead of 30%) to counter previous turn if shelter is detected on opposite side < 33� 10%

Altogether, the above trends result in path length to shelter 71% that of path length to virtual shelter, and time to shelter 65% of time to virtual

shelter; 1000 trials for each shelter location, sampled from initial condition of cockroach trials. These percentages were found by sampling the initial

conditions at the instant the animal exited the starting chamber. While SC and EC shelter locations were not systematically affected by initial

condition, we found that sampling the cockroach data deeper into the arena (farther from the start chamber) increased the path length to the NE

location for both empty and shelter trials. Sampling after a path length of 400 mm agreed better with our experimental data (show in Figure 16),

which suggests that the animal might not notice that corner immediately upon exit from the start chamber. (The overall ratios are 65% path length,

56% time.) aA summary of the difference associated with the shelter trials in the figure data. bAngle of body orientation relative to the shelter that

the description applies to (0� corresponds to when the shelter is centered in the animals’ field of view. 180� indicates facing away from shelter).

Note that in the case of Figure 6, we changed from five bins to four bins on the angle to ease the requirement for the robots field of view based on

camera availability. cEach of these trends was individually removed from the simulation (by using the control values for the relevant parameters) and

the change in the shelter-to-control path length is reported here as an indication of the importance of that trend in this shape arena, with Figures 11

and 13 representing the most important trends and the parameters in Figures 7 and 8 not strongly affecting performance. 1000 simulated trials for

each shelter location. ± 6% accuracy at 95% bootstrap confidence.

Daltorio et al. 417

(such as escape stimuli). This behavioral study couldcomplement brain modeling such as the control archi-tectures suggested by Wessnizer and Webb (2003) andBeer (2003) and insect-inspired wall-following robots(Chapman and Webb, 2006; Cowan et al., 2006). Thebehavior we characterize in cockroaches is quite differ-ent from the computer science ‘‘Bug algorithms’’ whichare generally straight-line approaches to goals andobstacle skirting (Kamon, Rivlin, & Rimon, 1996). Infuture work, it might be interesting to generalize thecockroach decision-making process for different typesof stimuli.

Funding

This work was supported in part by an NSF GSRP (#DGE-0951783), an NDSEG fellowship, SPUR grant from HHMI,a NASA Office of the Chief Technologist’s Space TechnologyResearch Fellowship (Grant number #NNX11AN15H), NSFgrant IOS-1120305, AFOSR grant FA9550-10-1-0054 toRER, grant W91CRB-11-1-0003 from DARPA’s M3 pro-gram, and an AAUW dissertation fellowship.

References

Asadpour, M., Tache, F., Caprari, G., Karlen, W., & Sieg-

wart, R. (2006). Robot-animal interaction: Perception and

behavior of Insbot. International Journal of Advanced

Robotics Systems, 3(2), 93–98.Azuma, S., Sakar, M. S., & Pappas, G.J. (2010). Nonholo-

nomic source seeking in switching random fields. 49th

IEEE Conference on Decision and Control, Atlanta, GA,

15–17 December (pp. 6337–6342).Bartumeus, F. (2009). Behavioral intermittence, Levy pat-

terns, and randomness in animal movement. Oikos, 118(4),

488–494.Beer, R. D. (2003). The dynamics of active categorical percep-

tion in an evolved model agent. Adaptive Behavior, 11,

209–243.Bender, J. A., Simpson, E. M., Tietz, B. R., Daltorio, K. A.,

Quinn, R.D., & Ritzmann, R. E. (2011). Kinematic and

behavioral evidence for a distinction between trotting and

ambling gaits in the cockroach, Blaberus discoidalis. Jour-

nal of Experimental Biology, 214, 2057–2064.Benichou, O., Coppey, M., Moreau, M., & Voituriez, R.

(2006). Intermittent search strategies: When losing time

becomes efficient. Europhysics Letters, 75(2), 349–354.Bobadilla, L., Martinez, F., Gobst, E., Gossman, K., &

LaValle, S.M. (2012). Controlling wild mobile robots

using virtual gates and discrete transitions. American Con-

trol Conference, Montreal, Canada, 27–29 June.Borger, L., Dalziel, B. D., & Fryxell, J. M. (2008). Are there

general mechanisms of animal home range behaviour? A

review and prospects for future research. Ecology Letters,

11, 637–650.Borst, A., & Egelhaaf, M. (1989). Principles of visual motion

detection. Trends in Neurosciences, 12(8), 297–306.Branson, K., Robie, A., Bender, J., Perona, P., & Dickinson,

M. H. (2009). High-throughput ethomics in large groups

of Drosophila. Nature Methods, 6, 451–457.

Camhi, J.M., & Johnson, E.N. (1998). High-frequency steer-

ing maneuvers mediated by tactile cues: Antennal wall-

following in the cockroach. Journal of Experimental Biol-

ogy, 202, 631–643.Canonge, S., Sempo, G., Jeanson, R., Detrain, C., & Deneu-

bourg, J. L. (2009). Self-amplification as a source of inter-

individual variability: Shelter selection in cockroaches.

Journal of Insect Physiology, 55, 976–982.Chapman, T. P., & Webb, B. (2006). A model of antennal

wall-following and escape in the cockroach. Journal of

Comparative Physiology A, 192(9), 949–969.Collet, T. S., & Land, M. F. (1975). Visual spatial memory in a

hoverfly. Journal of Comparative Physiology A, 100(1), 59–84.Cowan, N. J., Lee, J., & Full, R. J. (2006). Task-level control

of rapid wall following in the American cockroach. Journal

of Experimental Biology, 209, 1617–1629.Creed, R. P., & Miller, J. R. (1989). Interpreting animal wall-

following behavior. Celluar and Molecular Life Sciences,

46(7), 758–761.Daltorio, K.A., Tietz, B.R., Bender, J.A., Webster, V.A.,

Szczecinski, N.S., Branicky, M.S., Ritzmann, R.E., &

Quinn, R.D. (2012). A stochastic algorithm for explorative

goal seeking extracted from cockroach walking data. Pro-

ceedings of ICRA 2012 (pp. 2261–2268).

Dorigo, M., & Gambardella, L. M. (1997). Ant colony sys-

tem: A cooperative learning approach to the traveling

salesman problem. IEEE Transactions on Evolutionary

Computation, 1, 53–66.Franz, M. O., & Mallot, H. A. (2000). Biomimetic robot navi-

gation. Robotics and Autonomous Systems, 30, 133–153.Gautrais, J., Jost, C., Soria, M., Campo, A., Motsch, S.,

Fournier, R., Blanco, S., & Theraulaz, G. (2009). Analyz-

ing fish movement as a persistent turning walker. Journal

of Mathematical Biology, 58(3), 429–445.Guo, P., & Ritzmann, R. E. (2013). Neural activity in the central

complex of the cockroach brain is linked to turning beha-

viors. Journal of Experimental Biology, 216(6), 992–1002.Halloy, J., Sempo, G., Caprari, G., Rivault, C., Asadpour,

M., Tache, F., Saıd, I., Durier, V., Canonge, S., Ame, J.

M., Detrain, C., Correll, N., Martinoli, A., Mondada, F.,

Siegwart, R., & Deneubourg, J. L. (2007). Social integra-

tion of robots into groups of cockroaches to control self-

organized choices. Science, 318, 1155–1158.Harley, C. M., English, B. A., & Ritzmann, R. E. (2009).

Characterization of obstacle negotiation behaviors in the

cockroach, Blaberus discoidalis. Journal of Experimental

Biology, 212(3), 1463–1476.Hoffmann, G. (1983). The random elements in the systematic

search behavior of the desert isopod, Hemilepistus reau-

muri. Behavioral Ecology and Sociobiology, 13, 81–92.Huber, S. A., Franz, M. O, & Bulthoff, H. H. (1999). On

robots and flies: Modeling the visual orientation behavior

of flies. Robotics and Autonomous Systems, 29, 227–242.Jeanson, R., Blanco, S., Fournier, R., Deneubourg, J-L.,

Fourcassie, V., & Theraulaz, G. (2003). A model of animal

movements in a bounded space. Journal of Theoretical

Biology, 225, 443–451.Jost, C., Garnier, S., Jeanson, R., Asadpour, M., Gautrais,

J., & Theraulaz, G. (2004). The embodiment of cockroach

behaviour in a micro-robot. Proceedings of 35th Interna-

tional Symposium on Robotics.

418 Adaptive Behavior 21(5)

Kamon, I., Rivlin, E., & Rimon, E. (1996). A new range-sensor based globally convergent navigation algorithm formobile robots. IEEE International Conference on Robotics

and Automation.Lewinger, W. A., Harley, C. M., Ritzmann, R. E., Branicky,

M. S., & Quinn, R. D. (2005). Insect-like antennal sensingfor climbing and tunneling behavior in a biologically-inspired mobile robot. Proceedings of the International

Conference on Robotics and Automation (pp. 4176–4181).Liu, S.-J., & Krstic, M. (2010). Stochastic source seeking for

nonholonomic unicycle. Automatica, 46, 1443–1453.Muller, M., & Wehner, M. (1994). The hidden spiral: Sys-

tematic search and path integration in desert ants, Cata-glyphis fortis. Journal of Computational Physiology, 175(5),525–530.

Nicolau, D.V., Armitage, J.P., & Maini, P.K. (2009). Direc-tional persistence and the optimality of run-and-tumble che-

motaxis. Computational Biology and Chemistry, 33, 269–274.Okada, J., & Toh, Y. (1998). Shade response in the escape

behavior of the cockroach, Periplaneta americana. Zoolo-gical Science, 15(6), 831– 835.

Patton, P., Windsor, S., & Coombs, S. (2010). Active wall fol-lowing by Mexican blind cavefish (Astyanax mexicanus).Journal of Comparative Physiology A, 196, 853–867.

Schaefer, P.L., & Rtizmann, R.E. (2001). Descending influ-ences on escape behavior and motor pattern in thecockroach. Journal of Neurobiology, 29(1), 9–28.

Stankovic, M. S., & Stipanovic, D. M. (2009). Discrete time

extremum seeking for autonomous vehicle target tracking

in stochastic environment. 48th IEEE Conference on Deci-

sion and Control (pp. 4541–4546).Straw, A. D., & Dickinson, M. H. (2009). Motmot, an open-

source toolkit for realtime video acquisition and analysis.

Source Code for Biology and Medicine, 4(9).Tourtellot, M.K., Collins, R.D., & Bell, W.J. (1991). The prob-

lem of movelength and turn definition in analysis of orienta-

tion data. Journal of Theoretical Biology, 150, 287–297.Webster V. A., Szczecinski N. S., Tietz B. R., Daltorio K. A.,

Porr D., Richards J., Ritzmann R.E., & Quinn R. D.

(2012). Barrier navigating and shelter seeking robot

inspired by cockroach behavior. 15th International Confer-

ence on Climbing and Walking Robots, Baltimore, MD.Wehner, R. (2003). Desert ant navigation: How miniature

brains solve complex tasks. Journal of Comparative Phy-

siology A, 189, 579–588.Wessnizer J., & Webb, B. (2003). Multimodal sensory integra-

tion in insects—Toward brain control architectures. Bioin-

formation and Biomimetics, 1(3), 63.Yu, S.-E., Lee, C., & Kim, D. (2012). Analyzing the effect of

landmark vectors in homing navigation. Adaptive Beha-

vior, 20(5), 537–359.

Zill, S. (2010). Invertebrate neurobiology: Brain control of

insect walking. Current Biology, 20(10), R438–R440.

About the Authors

Kathryn A Daltorio received her PhD in mechanical engineering from Case Western ReserveUniversity in Cleveland, Ohio in 2013. She won NDSEG, NSF, Zonta, and AAUW graduate stu-dent fellowships. As a member of the biologically inspired robotics laboratory, she earned herMasters in 2007 on climbing robot foot design and her BSE in 2005 at Case Western ReserveUniversity. Address: Department of Mechanical and Aerospace Engineering, Case Western ReserveUniversity, Glennan Building, Room 814, 10900 Euclid Avenue, Cleveland, OH 44106-7222, USA.E-mail: kathryn.daltorio@gmail.edu.

Brian R Tietz is a graduate student in the biologically inspired robotics laboratory at CaseWestern Reserve University. Brian spent 3 years studying cockroach shelter seeking behavior in col-laboration with the Ritzmann laboratory, culminating in earning his Masters in mechanical engi-neering in 2012, where he also received his BS in 2011. He is currently pursuing a PhD at CWRUon a NASA space technology research fellowship and is collaborating with the intelligent roboticsgroup at NASA Ames Research Center. Address: Case Western Reserve University, GlennanBuilding Room 813, 10900 Euclid Avenue, Cleveland OH 44106, USA. E-mail:brian.tietz@case.edu

John A Bender is director of technology at General UI, a software firm in Seattle, Washington. Hereceived a BS in biochemistry from Montana State University, where he did undergraduate researchunder John P Miller on neural coding in the cricket cercal sensory system. He attended graduateschool at the California Institute of Technology, earning a PhD in Michael Dickinson’s laboratorystudying the neural control of flight in Drosophila melanogaster. He worked as a post-doctoral associ-ate in Roy Ritzmann’s laboratory at Case Western Reserve University from 2007–2011, where hedeveloped software to extract three-dimensional patterns of leg movements from high-speed video ofwalking cockroaches. He used in-depth analysis to show that cockroaches may control their legs inqualitatively different ways at different speeds and that activity in certain brain areas predicts changesin walking speeds. While in the Dickinson lab, he helped produce Ctrax, an open-source tracking sys-tem for behavioral biology, and he is the current maintainer of this software. Address: 3417 EvanstonAvenue N, Suite 501, Seattle, WA 98103, USA. E-mail: johnabender@gmail.com.

Daltorio et al. 419

Victoria A Webster is a NSF graduate fellow at Case Western Reserve University (CWRU). Shereceived her BS and Masters in mechanical engineering from CWRU in 2012 and 2013 respectivelyand is currently pursuing her PhD as a member of the biologically inspired robotics laboratory.Address: Department of Mechanical and Aerospace Engineering, Case Western Reserve University,Glennan Building, Room 813, 10900 Euclid Avenue, Cleveland, OH 44106-7222, USA. E-mail:vaw4@case.edu.

Nicholas S Szczecinski received his MS in mechanical engineering from Case Western ReserveUniversity in Cleveland, Ohio in 2013. He also received a BS in mechanical engineering from Casein 2012. He is currently pursuing his PhD in mechanical engineering as a NASA space technologyresearch fellow. Address: Department of Mechanical and Aerospace Engineering, Case WesternReserve University, Glennan Building, Room 813, 10900 Euclid Avenue, Cleveland, OH 44106-7222, USA. E-mail: nicholas.szczecinski@case.edu.

Michael S Branicky is professor and chair of electrical engineering and computer science at CaseWestern Reserve University (CWRU). He received a BS (1987) and MS (1990) in electrical engi-neering and applied physics from CWRU and an ScD in electrical engineering and computer sci-ence from the Massachusetts Institute of Technology (1995). He re-joined CWRU as a facultymember in 1997. He has held research positions at MIT’s AI laboratory, Wright-Patterson AFB,NASA Ames, Siemens Corporate Research (Munich), and Lund Institute of Technology’s depart-ment of automatic control. His research interests include hybrid systems, intelligent control, andlearning, with applications to robotics, manufacturing, control over networks, and biology. FromJuly 2008 to June 2010, Branicky was a program director at the National Science Foundation inComputer Systems Research and Cyber-Physical Systems, where he won the NSF Director’sSuperior Accomplishment Award. Currently, he is serving NSF as an expert in the NationalRobotics Initiative (NRI). Address: Department of Electrical Engineering and Computer Science,Case Western Reserve University, Glennan Building, Room 327, 10900 Euclid Avenue, Cleveland,OH 44106- 70711, USA. E-mail: mb@case.edu.

Roy E Ritzmann is a professor of biology at Case Western Reserve University in Cleveland, Ohio. Hereceived his undergraduate degree in zoology at the University of Iowa and his PhD in biology at theUniversity of Virginia, working with DeForest Mellon Jr. Ritzmann was a postdoctoral researcher atCornell University in the laboratory of Jeffrey M. Camhi, where he worked on the neural circuitry thatunderlies the escape system of the cockroach. Ritzmann has been a member of the faculty of the biologydepartment at Case since 1977. In recent years, the Ritzmann laboratory has exploited parallel analysisstrategies in examining how insects deal with barriers to forward locomotion. This has led to an analysisof a group of midline neuropils in the insect brain that process large amounts of sensory informationand that the Ritzmann laboratory has shown is involved in descending motor control. Address:Department of Biology, DeGrace Hall, Room 220, Case Western Reserve University, 10900 EuclidAvenue, Cleveland, OH 44106-7080, USA. E-mail: roy.ritzmann@case.edu.

Roger D Quinn is the Arthur P Armington professor of engineering at Case Western ReserveUniversity in Cleveland Ohio. Quinn received his PhD in engineering science and mechanics fromVirginia Tech in 1985 and his MS and BS in mechanical engineering from the University of Akronin 1983 and 1980, respectively. He joined the mechanical and aerospace engineering department atCase in 1986. Quinn has directed the biorobotics laboratory since its inception in 1990 and has beenworking with Roy Ritzmann since that time. His research, in collaboration with biologists includingRitzmann, is devoted to the development of robots and control strategies based upon biologicalprinciples. Quinn is the lead inventor of a group of robotic vehicles that benefit from abstractedbiological principles. Quinn has more than 200 publications and holds several patents. His colla-borative work on behavior-based distributed control, autonomous robot climbing, human–machineinterfacing, and soft robots has earned awards from the IEEE. Address: Department of Mechanicaland Aerospace Engineering, Case Western Reserve University, Glennan Building, Room 818,10900 Euclid Avenue, Cleveland, OH 44106-7222, USA. E-mail: rdq@case.edu.

420 Adaptive Behavior 21(5)