ECOLOGICAL PSYCHOLOGY, 10(3-4), 177-219 Copyright © 1998, Lawrence Erlbaum Associates, Inc.

Visually Controlled Locomotion: 40 Years Later

William H. Warren, Jr. Department of Cognitive and Linguistic Sciences

Brown University

Gibson's article, "Visually Controlled Locomotion and Visual Orientation in Ani­mals" (1958/this issue), is the leading statement of a nonrepresentational, informa­tion-based approach to visual control. The core ideas he introduced 40 years ago re­surface, explicitly or implicitly, in much contemporary work on perception and action in humans, insects, robots, and autonomous agents. The purpose of this special issue is to assess the continuing pertinence of these insights and illustrate current directions in research on visually controlled locomotion. In this article, I locate the 1958 article in the context of Gibson's emerging theory of perception, contrast information-based control with standard model-based and cybernetic control architectures, evaluate the current status of Gibson's visual control formulae, and situate visual control within an informational-dynamical approach to agent--environment systems.

Locomotion is a biologically basic function, and if that can be accounted for then the problem of human space perception may appear in a new light. The question, then, is how an animal gets about by vision.

-Gibson (1958, p. 183/this issue)

How do animals get about by vision? The question is of fundamental importance to our understanding of both perception and behavior. The capacity for locomotion is a defining characteristic distinguishing the kingdoms Animalia and Plantae, and the imperative of controlling locomotion has clearly shaped the evolution of the visual system. The function of the most primitive visual organs such as invertebrate ocelli is to regulate locomotion and orientation, including orienting to the vertical, phototaxis, and shadow escape responses. Insects have specialized visual-motor systems that support locomotor behavior such as the optomotor response and so­phisticated flight control. We are only beginning to learn about cortical motion ar-

Requests for reprints should be sent to William H. Warren, Jr., Department of Cognitive and Linguis­tic Sciences, Brown University, Box 1978, Providence, R1 02912. E-mail: bill_warren@brown.edu

178 WARREN

eas in mammals that appear to be specialized for the detection of optic flow patterns. The significance of vision for locomotor guidance has also become apparent with re­cent attempts to build autonomous robots and intelligent vehicles, although com­puter vision systems remain unequal to this most basic perceptual task. However, when Gibson posed the question in 1958, it was quite novel to the field of percep­tion, and in its pedestrian familiarity, locomotion is something most psychologists and cognitive scientists continue to take for granted. It is typically assumed that the perception of space will provide a general basis for behavior, but seldom considered that the latter might condition the former. In reversing this formulation, Gibson ar­gued that visual perception might better be understood in the context of its biologi­cal function of guiding action, with locomotion as a paradigmatic example.

In many ways, the field is only now catching up with Gibson circa 1958. The ideas he proposed 40 years ago are resurfacing, explicitly or implicitly, in much of the recent work on perception and action in humans, insects, and machines. The current interest in this area is leading to a new appreciation of the deep relations between behavior and vision (Aloimonos, 1993; Milner & Goodale, 1995). The purpose of this special anniversary issue is to draw attention to this lineage, assess the continuing pertinence of Gibson's insights, and illustrate current directions in research on visually controlled locomotion. In this article, I begin by locating the 1958 article in the context of Gibson's emerging theory of perception and high­lighting what I view as its core contributions. Gibson's account of informa­tion-based control is contrasted with standard model-based control and cybernetic control architectures. My own research program can be viewed as an attempt to work through Gibson's "formulae" for the visual control oflocomotion experimen­tally, and the heart of this article is an effort to evaluate and update his hypotheses in light of recent findings. Finally, I discuss how our understanding of visual control might be advanced by situating it within an informational-dynamical approach to agent--environment systems.

THE 1958 ARTICLE AND ITS ANTECEDENTS

The article "Visually Controlled Locomotion and Visual Orientation in Animals" was drafted during 1955-1956 while Gibson was on a Fulbright fellowship at Oxford, following a year at Berkeley in the company of Edward Tolman, Egon Brunswik, and Gordon Walls. The fmal version appeared in the BritishloumalofPsychology in 1958, when Gibson was 54. It represents a milestone in the gradual development of Gib­son's pragmatic, information-based theory of perception. The "animal" in the title is indicative of his ambitions-to propose control principles generic to any mobile agent, from aerial bird to aquatic fish to terrestrial human to, potentially, Martian ro­bot. Although the article itself may not have had a high "impact factor" by the some­what cynical standards of today, his biographer Ed Reed (1988b) cited it as among his most important. Core concepts such as the optic array, affordances, visual kinaesthe­sis, and visual control had their first full expression therein and were subsequently

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LAlER 179

elaborated in Gibson's last two influential books (1966; 1979/1986). In each of these threads of thought, his engagement with the problem of locomotion played a catalyz­ing role and reilluminated the problem of spatial perception.

Gibson's early interest in locomotion was manifested in an analysis of automobile driving (Gibson & Crooks, 1938). Arguing even then that "locomotion is guided chiefly by vision" (p. 454), Gibson and Crooks, an automotive engineer, offered a functional description of the task of driving based on the concept of the field of safe travel, defmed as the possible paths a car can currently take given the destination of the driver, the array of stationary and moving obstacles, and the speed of the vehicle. Following Lewin (1936), they asserted that the environment possesses positive and negative valences that attract one toward a destination and repel one from obstacles, and they suggested that the sum of these two force fields yields the path oflocomotion and might even account for detour behavior-anticipating current potential field models of robot navigation (Khatib, 1986). However, whereas Lewin conceived of valences as subjective values attributed to objects, Gibson and Crooks stated that the field of safe travel existed objectively, independent of the driver's awareness, as a field of "behavioral opportunities." This clearly contains the germ of the later notion of affordances. Further, they claimed that "When they [fields of possible behavior! are perceived as such by the driver, the seeing and the doing are merged in the same experience" (p. 460), an early assertion of the mutuality of perception and action. Here Gibson began the transmutation of the motor theory of perception he inherited from his professor, the behaviorist Edwin Holt (see Reed, 1988b, pp. 67-73), accord­ing to which spatial experience is determined by patterns of motor responses to the environment, into a theory in which perception and action are complementary as­pects of adaptive behavior. As he stated in 1950, "Spatial behavior and spatial per­ception are coordinate with one another" (Gibson, 1950, p. 225). Notably absent, however, was any conception of the visual information for the field of safe travel, but by 1958 Gibson was prepared to argue that the optic array "specifies" goals, obstacles, and paths of locomotion.

During Wodd War II, Gibson turned his attention to the practical problems of pilot testing and training, an experience that greatly influenced his thinking about perception (Gibson, 1947,1950). As he recounted it, tests of depth cues for points in empty space, such as the 3-point test of stereo acuity, failed to predict an individ­ual pilot's ability to land an airplane. This realization led Gibson to reject the long-standing "air theory" of spatial perception and propose a reconceived "ground theory," in which perception depends on a continuous ground surface and associ­ated optical gradients. His work on flying also led to the discovery of the informa­tion that had been missing from his analysis of driving-the optic flow field (see also Langewiesche, 1944, which Gibson cited). Optic flow was the offspring of a meet­ing between Helmholtz' idea of motion parallax and Gibson's ground theory: the generalization of local parallax to a gradient of optical velocities surrounding the moving observer, which he termed motion perspective and represented as a vector field (Figure 1). First, the gradient of vector magnitudes specifies the structure of

180 WARREN

the environment, including continuous distance along the ground and the orienta­tion of surfaces. Second, the pattern of vector directions specifies one's own path of locomotion, for they radiate outward from a focus of expansion that corresponds to the current direction of self-motion or heading. Optic flow thus revealed that we not only perceive a stable environment but co-perceive self-motion through that environment, which Gibson termed visual kinaesthesis. Third, Gibson realized that optic flow might reciprocally be used to control locomotion. This was only hinted at in 1950, when he suggested that steering is "a matter of keeping the focus of expan­sion in the direction one must go" (Gibson, 1950, p. 128). This circular causality of self.produced stimulation being used to guide action contributed to Gibson's rejec­tion of the causal theory of perception and stimulus-response (S-R) theories of be­havior, in which perception and behavior are responses to stimuli, in favor of active perception that seeks out information and purposive action that is visually guided but not causally determined. In 1955, Paul Olum and Frank Rosenblatt (later of perceptron fame) helped Gibson perform a mathematical analysis of optic flow and characterize the information it contains for relative distance, direction of locomo­tion, and time-to-contact (Gibson, Olum, & Rosenblatt, 1955).

aX osnsss as

-....- --.. -- --.. /'

,/" ~

/ "'" FIGURE 1 Gibson's representation of optic flow as a two-dimensional velocity field, for observer rranslation. Each vector represents the optical motion of the corresponding environmental element. All vectors radiate outward from a focus of expansion in the direction of heading. Note. From The Ecological Approach to Visual Perception (p. 124), by J. J. Gibson, 1986, Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. (Origi­nal work published 1979). Copyright 1986 by Lawrence Erlbaum Associ­ates, Inc. Reprinted by permission.

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 181

The 1958 article represents the culmination of Gibson's hard-won insights about perception and locomotion. Foremost among them is the first exposition of the optic array, his crucial move away from the retinal image as the foundation for understand­ing perception. During the 1940s and 1950s, Gibson wrestled with the fundamental problem of perceptual constancy-how a moving observer can see a rigid world with objects of constant size and shape despite continually varying stimulation and a ka­leidoscopic flux of visual sensations. The traditional explanation was that mental op­erations are required to reconstruct constant objects from inadequate sense-data. Yet if the perceived world is an artifact of the mind, how is it that we are able to get about in the real one as accurately and precisely as we do? And how is it that different individuals perceive the same world well enough to interact and communicate? By 1950 Gibson arrived at the opposite conclusion-that perception is not a mental construction and constancy must be based on invariant properties of stimulation. Although Gibson relied on the retinal image in 1950, shortly thereafter he realized that it suffered from the same problem as sensations-it was highly variable, fluctuat­ing with movement of the eyes, head, and body, and historically had not yielded up correlates for the constant properties of objects. In the 1958 article, he took the step to the optic array, the pattern of "different intensities (and frequency compositions) in different directions" in the light available at a point of observation, independent of the optics or physiology of any particular eye. The higher order variables he sought, confounded in the retinal image, would be more readily revealed in the optic ar­ray-specifically, the family of continuous transformations that corresponded to the observer's path oflocomotion, and the mathematical invariants that corresponded to the constant structure of the environment. Subsequent formal work on structure from motion has shown that information for three-dimensional structure and the ob­server's path is indeed available in the pattern of optical motion.

The second notable contribution of the 1958 article is the first articulated ac­count of affordances (although only the verb form afford is used). Gibson carefully distinguished between visual control reactions based on optic flow, such as steering toward an object, and identifying reactions based on object perception, such as per­ceiving whether the object is a bump-into-able obstacle or edible food, a mate-with-able conspecific or a dangerous predator. The important new claim was that these higher order properties of objects are specified by higher order variables in the optic array. To the extent that their textures, colors, and motions discrimi­nate behaviorally significant classes of objects, discriminative reactions are possi­ble. However, betraying a lingering behaviorist influence, each object was still conceived as belonging to only one behavioral class rather than possessing multiple affordances, temporarily salvaging an "S-R theory of identifying reactions," with stimulus-response links between objects and appropriate actions. With regard to locomotion, features of the ground surface were described in functional terms as paths, barriers, jumping-off places, and falling-off places, all of which "depend on the locomotor capacities of the animal" (p. 192). To the extent that the arrangement of surfaces was specified in the optic array, so were these functional features, al-

182 WARREN

though their scaling with respect to the actor was not yet incorporated into the ac­count (see Gibson, 1979/1986; Warren, 1984; Warren & Whang, 1987). The claim that an animal "perceives the possibilities of locomotion surrounding him" (Gibson, 1958, p. 192) clearly echoed the definition of the field of safe travel 20 years before, but these new details anticipated the full theory of affordances to come 20 years later. When Gibson went on to say, "And this is probably what should be meant by asserting that an animal has visual space perception" (p. 192), the implica­tion was that perceiving the layout of surfaces and what it affords for locomotor be­havior is the evolutionarily primary function of spatial perception.

Third, Gibson developed his 1950 observations about visual kinaesthesis. The notion that vision, in addition to vestibular, joint, and muscle proprioception, could detect bodily movement subsequently led Gibson (1966) to question the tra­ditional anatomical classification of the senses and argue that all perceptual sys­tems serve both proprioceptive and exteroceptive functions. His classic examples of the fish swimming upstream and the bird flying upwind first appeared in 1958 to make the case that visual kinaesthesis is essential for sensing movement relative to the environment, although Gibson also allowed that terrestrial animals who push against the ground may also exploit touch and muscle-joint proprioception as well. The "sea of energy" around us, he said, contains "redundant information" that can be picked up in multiple ways and in various combinations. Far from being inade­quate, information provides an embarrassment of riches, allowing for behavior to be robust over a wide range of conditions.

However, the central hypothesis of the 1958 article was that the optic array and optic flow provide information for the control of locomotion relative to the visible environment. Gibson argued that visual kinaesthesis is "an invariable accompani­ment of locomotor behavior and therefore provides 'feedback' stimulation for the control and guidance of locomotor behavior" (1958, p. 185/this issue)-a clear statement of optic flow as self-produced stimulation and the circular causality of vi­sual control. It also revealed a cybernetic turn in his thinking, with its concepts of control loops, feedback, and error-correction, which I believe colored his framing of the formulae. I I Most important for present purposes, Gibson offered a fully con­ceived set of five descriptive "formulae" for the visual control oflocomotion, which govern starting and stopping, steering toward a goal, braking, obstacle avoidance, and pursuit of and flight from moving objects. He later called them "rules" while cautioning that, "The rules are not commands from a brain; they emerge from the animal-environment system" (Gibson, 1979/1986, p. 232); I subsequently called them "laws" (Warren, 1988) to emphasize their principled basis in the systematic relations between information and behavior. It is to the problem of visual control that I now turn.

I He later noted that visual kinaesthesis does not, strictly speaking, constitute feedback about purposive movement because it is produced by passive as well as active self-motion (Gibson, 1979/1986).

VISUALLY CONlROLLED LOCOMOTION: 40 YEARS LATER 183

VISUAL CONTROL ARCHITECIURES

Locomotion and manipulation are controlled not by the brain, but by information. Control lies in the animal-environment system. Behavior is regular without being reg­ulated. The question is how this can be.

--Gibson (1979/1986, p. 225)

One approach to the guidance oflocomotion common in mobile robotics (Moravec, 1981) and human navigation (Loomis & Beall, this issue) is model-based control (see Figure 2a). On this view, input from vision and other sensors is used to construct a metric three-dimensional model of the world, which is in turn used to explicitly plan a path of movement, which is finally executed. This approach is consonant with the prevailing cognitivist paradigm in which all behavior is based on internal representa­tions such as models and plans, and consequently also confronts that paradigm's in­herent foundational problems (see Clark & Toribio, 1994; Epstein, 1993; Putnam, 1994; Searle, 1980; Shaw, Turvey, & Mace, 1981; vanGelder, 1995). The approach also assimilates the standard view that the function of perception is to create and up­date a general-purpose three-dimensional scene description, independent of the be­havior it serves--ironically, a view that owes much to Marr's (1982) interpretation of Gibson (1950). However, considered in the biological context of the survival of the organism, the function of perception is better stated as creating contact between the animal and its environment that supports adaptive behavior. The reification of the environment as an internal representation is not necessary for such contact, and if analogous knowledge is required for other purposes, it is likely to be derived from this primary contact.

Gibson's article was the leading statement of an alternative nonrepresentational approach, which I call information-based control. Rather than being governed by an internal model and a precomputed plan, behavior is controlled "online," if you will, by occurrent information2 about the relation between the observer and the environ­ment. Steering through a field of obstacles, for instance, could be controlled on the basis of optic flow by perceiving heading and time-to-contact with respect to them, without computing a model of their three-dimensional layout. The realized path of locomotion is thus emergent rather than planned in advance, resulting from the dy­namic interaction between the structure of the environment and the animal's strate­gies for steering and obstacle avoidance, as suggested in the quote from Gibson (1979/1986) at the beginning of this section.

There was, however, a tension lurking in Gibson's (1958/this issue) discussion of information-based control, specifically, whether it is based on general structural in­formation for the objects and three-dimensional layout of the environment, or spe­cific control information found in optic flow patterns (see also Loomis & Beall, this issue). On the one hand, the central thrust of his emerging theory of perception was

21nformation that is currently available to the observer.

184 WARREN

(a) ~~~~I~ 3D World H Pa~ f::Ac~onl Info: Model Planmng Action 2

(b)

(c) Info I ~ Controllaw1 --. Action I Info 2 ~ Controllaw2 --. Action 2

FIGURE 2 Possible architectures for visual control. (a) Model-based control. (b) Informa­tion-based control, predicated on generalsttuctural information. (c) Information-based control, predicated on task-specific control information.

to account for the human ability to perceive the layout of the environment, by means of a rich array of static and dynamic information. Relating this account of perception to the problem of visual control, he emphasized at several points that behavior is ori­ented to the environment, based on the perception of constant objects. Further­more, Gibson explicitly criticizes the notion that proximal stimulus patterns serve as '''cues' for behavior" (p. 191), reflecting his struggle with an increasingly creaky be­haviorism. This view echoes Holt's insistence on the "recession of the distal stimu­lus," the idea that mature behavior is no longer a reflexive response to proximal stimuli but an integrated response to distal objects. I have schematized this in Figure lb, where a variety of information, such as texture gradients, disparity gradients, mo­tion perspective, dynamic occlusion, and eye-height information may contribute to the perception of the three-dimensional environment surrounding the observer, which guides adaptive action. The proposal that behavior is oriented to the environ­mentratherthan composed of stereotyped responses to stimuli allows for action to be flexible and context-conditioned, so that it is functionally specific to the situation at hand (Reed, 1988a). However, although based on occurrent information rather than an internal representation, it is also reminiscent of the general-purpose view that vision recovers the three-dimensional structure of the world, incidentally pro­viding a common basis for any action.

On the other hand, when it comes to detailing control principles for locomo­tion, Gibson's formulae read very much as though behavior is controlled by prox­imal stimulation-in the form of optic flow patterns-rather than by perceived environmental structure. Particular actions, such as steering to a goal or making contact without collision, are each governed by particular informational vari­ables (or combinations of variables) according to corresponding control laws, as schematized in Figure lc (Warren, 1988). This view seems to sacrifice a general

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 185

basis for functional specificity and lead to a proliferation of control laws for every conceivable situation. In sum, there appears to be a rift between the role of infor­mation in specifying the environment and its use in controlling behavior.

I think this tension between specification and control is dissolved by the under­standing that optic flow patterns constitute not merely proximal stimulation, but in­formation about the ecological state of affairs encompassing both the environment and the observer. In his deSCription of visual kinaesthesis, Gibson emphasized that optic flow specifies the relative motion between the observer and surfaces in the world and conversely can be used to control that relative motion. Thus, locomotion is neither a response to proximal stimulus cues (as in both S-R and cybernetic the­ory), nor governed by perception of the environment per se (exteroception), but is controlled by information about the motion of the observer relative to that environ­ment (which David Lee called ex-proprioception) and motion of the environmentrel­ative to the observer (which Bob Shaw has christened pro-exteroception). AB I have argued previously, laws of control are reciprocals of laws of specification (Warren, 1988). If information adequately specifies the state of the econiche, corresponding control laws are avaUable that wUl allow behavior to be functionally specific to the situation at hand.

An account of visual control in the spirit of Gibson (1958/this issue) thus im­plies three specific claims: (a) control laws are nonrepresentational, in that visual control does not employ an internal model of the environment; (b) control laws are predicated on special-purpose control information, rather than general-purpose structural information about the three-dimensional layout of the environment; and (c) control laws are task-specific, such that different actions may be regulated by different sets of informational variables (which may overlap). Control laws for obstacle avoidance, braking, and pursuing a moving object, for example, might in­volve different relations with different optic flow variables.

The adequacy of the control architectures sketched in Figure 2 is ultimately an empirical matter. The first divide, between model-based and information-based control (Figure 2a vs. 2b,c) , poses the question of whether locomotion is based on an internal representation or occurrent information. Although this is largely a metatheoretical issue that is difficult to submit to direct experimental test, certain implications may be drawn from experiments that force participants to rely on mem­ory for the environment or a preplanned action sequence. To the extent that perfor­mance deteriorates under such conditions, it would imply that occurrent information is ordinarily used for visual control and would undermine reliance on a memory representation or advanced path planning; however, it might be argued that this does not rule out some sort of transient world model that depends on occurrent input. Conversely, maintaining accurate performance under such conditions would imply the persistence of spatial knowledge or an action plan sufficient to gUide be­havior; however, it could be of a partial and task-specific variety as opposed to a rich world model, and it would not follow that online visual control is normally based on such knowledge. Evidence for a more complete memory representation might in-

186 WARREN

volve showing that, after blindfolding, action can be accurately redirected to unex­pected target locations, even after partial execution to expected locations. For example, research on blind walking has shown that people can walk to targets or point to a target during walking after brief visual exposure to the layout (Fukusima, Loomis, & Da Silva, 1997; Loomis, Da Silva, Fujita, & Fukusima, 1992), suggesting that they can preserve and update some knowledge of spatial position. On the other hand, recent work on change blindness and saccadic integration, which shows that observers fail to notice large changes in a scene during a blank interval or a saccade, suggests that people do not have a very complete memory representation for unat­tended parts of the world (Rensink, O'Regan, & Clark, 1997).

The second divide for visual control is between general structural infonnation about the three-dimensional layout of the scene and specific control infonnation for particular tasks (Figure 2a,b vs. 2c). One way to approach this is to manipulate three-dimensional information and control infonnation selectively, to detennine which variables are tracked by perfonnance. To the extent that (a) task-specific variables are sufficient for behavior in the absence of three-dimensional infonna­tion and (b) they predict behavior even when three-dimensional infonnation is available, this would constitute evidence in favor of task-specific control infonna­tion and vice versa. However, given that optic flow provides infonnation both for control and for three-dimensional layout, the experimental dissociation of the two may be difficult; for example, motion perspective specifies both self-motion and surface orientation, and relative time-to-contact with an array of objects in the scene is fonnally equivalent to a relative depth map of the scene. A second chal­lenge is that, even in the presence of general three-dimensional infonnation, sys­tematic distortions in the perception of metric structure are frequently reported, such as compression along the depth axis (Todd, Tittle, &Nonnan, 1995), so one must consider whether these perceptual effects could also account for the observed behavior. Finally, the task-specificity of control infonnation can be tested by deter­mining whether different tasks, such as obstacle avoidance, braking, and pursuit, actually depend on different infonnational variables.

We recently probed the first divide by manipulating the visibility of objects during a joystick steering task (Duchon & Warren, 1997). Participants were instructed to steer through a slalom course of "gates" (pairs of vertical posts) while we manipulated (a) the number of gates that were visible and (b) the number of gates immediately in front of the observer that disappeared from view before the observer reached them. For example, of the four upcoming gates, the farthest two might be visible while the closest two are invisible. Steering accuracy was as high when participants could see only one visible gate as when they could see three, suggesting that they do not take advantage of preview infonnation in this demanding task.J However, when two or more gates became invisible directly in front of them, forcing them to plan ahead, ac-

3 A stronger test of this question is planned by varying the position of far gates to see whether they influence current behavior.

VISUALLY CONlROLLED LOCOMOTION: 40 YEARS LATER 187

curacy plummeted so that, on average, they missed every one. This implies that par­ticipants control steering with respect to the upcoming gate and cannot accurately plan their actions more than one gate ahead, either because they cannot remember and update the metric position of more than one gate (impoverished world model) or because they cannot plan an action sequence longer than one steering maneuver (impoverished path planning). They do, however, usually tum in the right general direction even with two invisible gates, suggesting the persistence of some crude spa­tial knowledge or left/right tum sequence. The results appear contrary to model­based control and consistent with online control, at least for the normal guidance of steering with respect to visible objects.

Even if an information-based view is granted for online visual control with respect to the visible environment, it is often argued that when it comes to long-range navi­gation with destinations beyond the range of vision, mental representations such as cognitive maps become necessary. However, in 1958 Gibson began to extend his approach to what he called way-finding. He suggested that we remain oriented to re­mote parts of our environment because we are linked to them via sequences of transfor­mations of the optic array, which specify both the invariant structure of the environment and the paths we have traveled. The orderly opening up and closing of vistas over time offers an informational basis for apprehending the three-dimensional environmental layout, which Gibson (1979/1986, p. 198-199) distinguished from both an S-R chain oflocations and actions and a plan-view cognitive map, but is more like "being everywhere at once." Most controversially, he stretched the traditional definition of perception to cover the awareness of existing environmental surfaces that are both visible and currently occluded, which thereby "includes within percep­tion a part of memory, expectation, knowledge, and meaning," (1979/1986, p. 255) . In some instances purported evidence for metric cognitive maps has turned out to depend on the presence of occurrent information for orientation, such as long-distance landmarks or vistas (Dyer, 1991; Gould, 1986). In collaboration with Michael Tarr and Leslie Kaelbling, we are currently setting up a visual navigation laboratory at Brown University that will use virtual reality techniques to investigate this form of spatial knowledge and its dependence on exploratory behavior, in both humans and robots.

The past decade has seen a reaction against the model-based approach in both computer vision and robotics. Work in animate vision (Aloimonos, 1993; Bajcsy, 1988; Ballard & Brown, 1992) has rejected the aim of constructing a gen­eral-purpose world model, and uses active vision systems to extract information for particular behavioral goals. Work in behavior-based robotics (Braitenberg, 1984; Brooks, 1991) seeks to build adaptive robots using a repertoire of simple behaviors based on direct sensory-motor mappings, without reference to a central model. Complex behavior emerges from the interaction between the agent and the envi­ronment rather than being prescribed by a plan. I view these projects as contempo­rary avatars of Gibson (1958/this issue), a vindication (often unacknowledged) of his earlier insights. The empirical adequacy and range of applicability of informa-

188 WARREN

tion-based control, however, remains to be determined. Let us consider his formu­lae for the visual control of locomotion in detail.

LAWS OF CONTROL FOR LOCOMOTION

Gibson (1958/this issue) articulated five control formulae for guiding locomotion with respect to stationary and moving objects, which remained substantively similar in 1979. There are 12 generic situations: the global array may be flowing outward, static, or flowing inward, specifying whether the observer is moving forward, sta­tionary, or backing up with respect to the environment; and an object's contour may expand, remain constant, or contract, specifying whether the object is ap­proaching, remaining a constant distance, or retreating with respect to the ob­server. If the local expansion is consistent with the global flow, then one is ap­proaching a stationary object; otherwise, one is dealing with a moving object. The 12 combinations correspond to various forms of goal seeking (approaching, keeping a constant distance from, or avoiding a stationary object), pursuit (gaining on, shad­owing, or falling behind a moving object), flight (being gained on, shadowed by, or outdistancing a moving object), and vigilance (a stationary observer with an ap­proaching, stationary, or retreating object).

Although they represent an important new way of thinking about visual con­trol, the specific formulae were based on an intuitive analysis of flow patterns and should be treated as empirical hypotheses open to experimental test. Alter­native hypotheses abound, some of them not based on optic flow at all. Despite his early exploitation of film and shadow-caster displays, Gibson lamented the lack of adequate techniques for studying optic flow and active perception, which have recently become available in the form of real-time computer graphics and virtual realiry technology. Over the past 10 years, our Perception and Action Laboratory at Brown has carried out a number of passive perception experiments on optic flow, demonstrating that humans can judge their direction of heading from global radial flow patterns with an accuracy on the order of 10 of visual an­gle (see Warren, 1998, for a review). This is theoretically sufficient for guiding a variety of locomotor behaviors. The ultimate agenda of this research is to deter­mine how optic flow is actually used to control posture and locomotion, and we have recently begun to tackle this problem directly using interactive displays and virtual environments. In what follows, I try to assess the current status of Gib­son's five formulae. In each case, the "structural" alternative is to perceive the metric structure of the scene from spatial information and choose a path based on the three-dimensional locations and motions of goals and obstacles.

Starting and Stopping

To begin locomotion, Gibson proposed, one should move so as to make the optic ar­ray flow outward; to stop, make the flow cease; and to back up, make the array flow in-

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 189

ward. The best evidence for such effects is Lee's classic work on postural sway in the "moving room" (Lee & Aronson, 1974; Lee & Lishman, 1975), where the task of standing still is analogous to stopping by making the flow cease. When the visual sur­round was moved toward the observer along the anterior-posterior axis, creating a radial outflow pattern in the optic array, it induced compensatory backward sway, whereas radial inflow induced compensatory forward sway. Note that in the moving room, other spatial information about the distance of the walls and the participant's position in the room is also available, but similar effects have been observed with large-screen computer displays that isolate monocular optic flow, albeit with a some­what lower gain. In addition, experiments with stroboscopic illumination indicate that the visual system is responding to optical motion per se rather than the position of features in the scene (Amblard, Cremieux, Marchand, & Carblanc, 1985; van Asten, Gielen, & van der Gon, 1988b).

There is considerable evidence that such postural responses are functionally spe­cific to the ecological state of affairs specified by optic flow. Their directional specific­ity indicates that the system responds appropriately to both outflow and inflow, consistent with the hypotheses about starting and backing up (Delorme, Frigon, & Lagace, 1989; van Asten, Gielen, & vanderGon, 1988a); directional specificity has also been observed along lateral and diagonal axes for postural sway during walking (Warren, Kay, & Yilmaz, 1996). The type of postural response (horizontal move­ment or body tilt) also depends on the class of optic flow (translation or pitch/roll stimulation; van Asten et al., 1988b; Warren et al., 1996). Consistent with motion perspective, optic flow generated by a three-dimensional environment is more effec­tive than that generated by a flat frontal plane, indicating that differential motion is used to stabilize posture (Bardy, Warren, & Kay, 1996). Further, behavior is not sim­ply driven by the current flow stimulation, for with oscillatory displays postural sway is adapted to match the display frequency (Dijkstra, Schaner, Giese, & Gielen, 1994; Schaner, Dijkstra, &Jeka, this issue). There is also evidence that for terrestrial ani­mals such as ourselves, postural sway is not determined solely by vision but is also in­fluenced by somatosensory information from the feet and ankles (Diener, Dichgans, Bruzek, &Selinka, 1982; Diener, Dichgans, Guschlbauer, & Mau, 1984; Howard, 1986) , indicating that we take advantage of typically redundant information (but see Stoffregen & Riccio, 1988, for a different interpretation).

Gibson further hypothesized that to speed up one should act so as to increase the flow rate, and to slow down, decrease it. This is appropriate if environmental structure is constant, but a change in the distance of surrounding surfaces such as walls or objects can also affect the flow rate. There is indeed some evidence that the overall flow rate affects the speed of walking. When a "moving hallway" trans­lates at a constant velOCity opposite the direction of walking, Konczak (1994) re­ported that participants reduced their speed, but they did not systematically speed up with hall motion in the same direction as walking. Analogous results were found with an apparatus that projected moving spots of light on a stationary floor (Pailhous, Ferrandez, Fliickiger, & Baumberger, 1990).

190 WARREN

Steering Toward a Goal

The basic function of locomotion is to get to a destination, often some goal object with positive affordances. Thus, the most essential of Gibson's formulae deals with steering toward a stationary goal. He assumed that the object projects a "form" in the optic array, an optical contour with internal texture. Of course, given that back­ground surfaces are textured, too, Gibson (197911986) later rephrased this in terms of dynamic occlusion. As the observer moves, an object is specified by the gain or loss of texture outside a closed contour, whereas an aperture is specified by gain or loss of texture inside a closed contour.

The focus of expansion hypothesis. To aim locomotion at a goal, move so as to keep the focus of expansion close to the object's form in the optic array (Figure 3a). This deceptively simple formulation is one of Gibson's most elegant proposals, a solution that will bring the observer to the goal regardless of environmental condi­tions. Self-motion through the world generates a global flow pattern with a focus of expansion in the direction of travel, so keeping this focus of expansion on a stationary goal will take one to it. He also pointed out that when making a turn, the focus of ex­pansion shifts in the scene through a visual angle that is equivalent to the angle of the turn. This implies that to steer toward a goal, the required turning angle is specified by the current visual angle between the focus of expansion and the goal; thus, turning might be controlled either continuously or discretely. The cybernetic interpretation of this hypothesis is straightforward, for the visual angle between the focus of expan­sion and the object can be treated as an error signal in a closed-loop system that is re­duced to achieve a desired state (0°). Because the focus of expansion is a fixed point in the flow field, the goal will remain in a roughly constant position in the optic array.

However, there are a number of alternative control laws that might also govern steering toward a goal. Those I have identified are sketched in Figure 3 with optic flow strategies4 in the top row and positional strategies in the bottom row, although surely there are others, such as strategies based on the tau function that Lee ana­lyzes in this issue.

The heading hypothesis. This is essentially the same as the preceding hypothesis, except that one's perceived heading, rather than the focus of expan­sion per se, is kept near the goal. In recent experiments, we found that perceived heading can be biased away from the focus of expansion by a few degrees when an independently moving object crosses the path of travel (Royden & Hildreth, 1996; Warren & Saunders, 1995a, 1995b). The observer's locomotion gener­ates a focus of expansion for stationary background surfaces (circle in Figure 4b), but a moving object can create a secondary focus of expansion (square in

41 use this term as shorthand for a possible law of control, and do not mean to imply an explicit cognitive strategy.

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 191

goal goal goal

'~I ~ " .,., t§~ -;,e ~ FOE - ---- ,-,,---+- ~

VL VR ~ ....... .......

! :gent ! :gent ! ~gent (a) FOE/heading (b) Magnification (c) Equalization

goal goal goal

D D D , , , , , , , , , , ~ , ~

~ " bT .6, T",,,, agent

"" agent :~

+F (d) Thrust (e) Constant bearing (f) Centering

FIGURE 3 Diagram of visual control hypotheses for steering toward a goal or aperture. FOE = focus of expansion.

Figure 4b) such that the perceived heading determined by the total flow field lies in between them (X in Figure 4b). Interestingly, this suggests that the visual sys­tem does not resolve the structure and motion of all objects in the scene to per­ceive heading, but rather determines heading directly from the global flow pat­tern. Thus, this control law states that to steer toward a goal, keep the perceived heading close to the goal.

The magnification hypothesis. To steer toward a goal, move so that its form is magnified or expands symmetrically (Figure 3b). This strategy can be distin­guished from the focus of expansion hypothesis because it depends on the flow of the object itself, rather than the global flow pattern, and does not rely on the radial directions of flow vectors, but on the pattern of their speeds or magnitudes. There

192 WARREN

(a) Stationary scene

(b) Obstacle texture motion

(c) Sidewall texture motion

o Actual heading

X Perceived heading

• Obstacle FOE

/

~ Texture motion

Resultant flow

FIGURE 4 Schematic of corridor displays to test the heading and equal­ization strategies for goal seeking and obstacle avoidance. (a) Optic flow due to self-motion with no added motion in scene. (b) Optic flow resulting from added obstacle texture motion to the right: perceived heading shifts to the left, flow rate increases on the right. (c) Optic flow resulting from added texture motion on side wall: no shift in perceived heading, flow rate increases on the left. The symmetrical conditions were also tested.

are cases in which the two strategies diverge, for example, when heading toward a flat surface at an oblique angle, there is radial flow from the heading point but the expansion is not symmetrical. In most cases, however, keeping the focus of expan­sion on the goal is accompanied by approximately symmetrical expansion, and a vi­sual system tuned to such flow patterns would respond similarly to radial flow or symmetrical expansion.

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 193

The thrust hypothesis. On the other hand, one need not use optic flow at all to steer to a goal-one could use positional information instead. The next three strategies are all based on the egocentric direction of the goal, its position with re­spect to the observer's body. Most simply, one might detect the position of the goal and locomote toward it; specifically, one could perceive its egocentric direction and apply thrust force in the opposite direction (Figure 3d). This hypothesis assumes that the visual direction of a target can be related to the direction of effector move­ment, for which there is ample evidence; one can point accurately to a light in the dark, for instance. It also relies on the fact of terrestrial locomotion that there is typ­ically a unique relation between a push against the ground and the resulting dis­placement of the body, so a known thrust force will move the observer in the visually specified direction. The optical result is that the goal remains in a roughly constant position in the optic array.

The constant bearing hypothesis. Similarly, one could steer toward a goal by moving so that it expands at a constant bearing in the optic array (Le., a fixed egocentric direction, assuming no body rotation; Figure 3e). This differs from the previous hypothesis because it does not rely on a known mapping from visual direc­tion to direction of thrust. Instead, it capitalizes on a local feature of optic flow: If the goal is at the focus of expansion, it will remain at a fixed egocentric position, othetwise it will drifr. A simple way to do this is to move so as to match the optical velocity of the goal, such that drift goes to zero. Once again, the optical result is that the goal remains in a roughly constant position in the optic array.

The centering hypothesis. A closely related hypothesis is to move so as to keep the goal in the center of the visual field or at the midline of the body (Figure 3f). This is essentially the same as the previous strategy but with a privileged ego­centric direction of "straight ahead." Its effect is to line up body orientation and for­ward locomotor progression with the direction of the goal. This strategy would move the goal to a particular egocentric position.

Research on strategies for steering to a goal is only now beginning to appear. A major obstacle is that most of these hypotheses are redundant. Whether relying on the focus of expansion, perceived heading, symmetrical expansion, egocentric di­rection, or constant bearing, they generally predict that the goal will end up at the focus of expansion in a fixed egocentric position. We have recently obtained evi­dence consistent with the heading strategy in an active joystick task, in which par­ticipants steered toward a target in the presence of a moving object (J urn, Duchon, & Warren, 1998). The moving object induces steering biases that were consistent with those observed in passive heading judgments, implying that joystick control

194 WARREN

has a basis in perceived heading, rather than the focus of expansion per se or posi­tional infonnation. This provides evidence for a task-specific optic flow strategy.

To examine steering control in legged locomotion, we have recently begun teas­ing apart some of these hypotheses using a "virtual treadmill" that allows us to ma­nipulate the optic flow of a virtual environment during actual treadmill walking. A computer display is projected on a large screen in front of the treadmill, and an electromagnetic tracker on the participant's forehead is used to update the center of projection in real time. The results are quite preliminary and what follows should be regarded as a tentative progress report. In our initial experiments CW arren & Kay, 1997a, 1997b), we simulated motion toward a doorway surrounded by a tex­tured frontal wall, ground plane, and ceiling. The door could appear directly in front of the participant or 5° on either side, and the task on each trial was simply to walk through the center of the doorway. To dissociate the thrust strategy from the optic flow strategies, we rotated the tracker's coordinate system so that the focus of expansion was offset by 5° from the actual direction of walking. Thus, when the participant walked toward the doorway, the scene did not expand from the door­way, but rather from a point 5° to the left or right. If, as predicted by the thrust strategy (Figure Sa), the participant walked in the egocentric direction of the door, the heading error (~) between the walking direction and the doorway would tend to zero. (The door would subsequently drift, but the participant would chase it across the screen, keeping ~ near zero.) If, on the other hand, the participant walked so as to keep the focus of expansion within the doorway (or make the door­way expand symmetrically), then they would have to walk 5° away from the door, so beta would go to 5° (Figure 5b). The mean-time series of heading error (Figure 5c) indicated that participants walked in the egocentric direction of the door for the first second (~ = 0°), but then switched to an optic flow strategy (~ = 5°).

However, a similar pattern of behavior could also be produced by either the constant bearing or centering strategies, for keeping the doorway in a fixed egocen­tric direction would have also yielded walking at a 5° angle to the door. We thus tested a control condition in which a vertical target line was present at the same lo­cation as the doorway, but all other optic flow was eliminated. The mean time se­ries of heading error was quite similar to the doorway condition, such that participants initially walked in the egocentric direction of the target (~ = 0°) but then, as it began to drift, heading at an angle to it (~ = 5°). Further analysis indi­cated that they may have been tracking the horizontal motion of the target in front of them on the screen, for their lateral position was within 2 cm of the horizontal position of the target, consistent with the centering strategy. However, this behav­ior could be an artifact of the screen and treadmill apparatus, so we plan to repeat the study with a head-mounted display. At this point in our work, it seems that steering toward an expanding doorway might be guided by either an optic flow strategy or the centering strategy.

Rushton, Harris, Lloyd, and Wann (1998) recently reported steering behavior that is consistent with the centering strategy. They asked participants to walk to a

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 195

10 (c)

5 0;

CD ~ 0

~ al -5

N=10

-10 0

(a) Thrust force hypothesis

-t,t-: ~=5·

iT (b) Optic flow hypothesis

Offset

... - - - -.. ;" - ~ - - ... - ------- --........... - --Control

2 3

t (s)

FIGURE 5 Steering toward a doorway in the virtual tread­mill. (a) Prediction of thrust hypothesis. (b) Prediction of optic flow hypotheses. (c) Mean-time series of heading error (Ii)

when focus of expansion is offset from direction of walking by 50 (Offset) and with no offset (Control). N = 10.

4

target -a small ball 1 0 to 15 m away-while wearing prism glasses that displaced the entire visual scene by 16°. The prisms acted to displace both the egocentric direction of the target and the flow pattern together. so that if observers kept the focus of ex­pansion near the target they would walk on a straight path to its actual location. On the other hand. if they kept the target in a fixed egocentric direction. they would walk with a constant heading error of 16°. yielding a curved path to the target (with an in­creasing curvature). Videotapes revealed curved rather than straight paths. consis­tent with any of the positional strategies; assuming that participants walked with forward progression. the results were indicative of centering. However. a small target held in the air may not have generated significant flow until the last seconds before contact. and even if it did. the prism deflection was large enough to create a notice­able discrepancy between the optic flow and somatosensory information for walking direction. which could have led participants to discount the flow.

The results on steering to a goal are thus suggestive but not definitive. At this early stage. it remains possible that both egocentric initial position and optic flow playa role. Similar studies of obstacle aYOidance in the virtual treadmill. to be dis­cussed next. clearly indicate a dependence on optic flow.

196 WARREN

Obstacle Avoidance

Steering away from an obstacle is the converse of steering toward a goal, and Gibson (1958/this issue) proposed a reciprocal formula. Another way of framing the task is to steer toward an opening or bounded apenure, in which case the control laws for obstacle avoidance are identical to those for goal seeking. Gibson described both.

The focus of expansion/heading hypothesis. To steer away from an ob­stacle, keep the focus of expansion (or more likely, one's perceived heading) outside its optical contour and within a region of the optic anay that specifies an opening. This strategy thus relies on the focus of expansion (or perceived heading) by placing it in openings between obstacles.

The magnification hypothesis. In the same breath, Gibson described the magnification hypothesis. To steer away from an obstacle, move so that it undergoes a skewed magnification (i.e., an asymmetrical expansion). The alternative formula­tion is to move so that the optical contour of an aperture expands symmetrically. This strategy is based on the local expansion of a bounded contour rather than the global flow pattern.

The equalization hypothesis. A different approach is suggested by the ob­servation that once a goal is specified, all other objects become obstacles. Conse­quently, the observer need not steer with respect to them individually, or even seg­ment them from the background, but could simply lump the flow from all surfaces together. The equalization hypothesis thus proposes that one move so as to equate or balance the global rate of flow on either side of the goal, the focus of expansion, or the midline of the body (Figure 3c). This strategy leads the observer to steer away from regions of high flow and toward regions oflow flow. It is surprisingly effective for obstacle avoidance because when one nears an object or surface, its rate of flow increases, leading one to tum away from it. If the observer is aimlessly wandering, the equalization strategy would prevent collisions with objects, but if the agent is seeking a goal it would dovetail nicely with the focus of expansion/heading strategy. While the focus of expansion (or heading) is kept near the goal, obstacles would be avoided without having to explicitly reset the focus of expansion for each opening.

The equalization strategy was originally discovered in honeybees, based on the observation that they tend to fly through the middle of apenures (as in Figure 3c) and down the center of corridors (Srinivasan, this issue; Srinivasan, Lehrer, Kirchner, & Zhang, 1991). To test the hypothesis, Srinivasan and his colleagues added a longitudinal motion to one side wall of the corridor, and found that bees flew down the hallway on a trajectory that precisely balanced the speed of flow in the two eyes (Srinivasan, this issue, Figure 2). Specifically, when the side wall moved against the bee's direction of travel, this increased the flow rate on that side and the bee took up a position fanher from the moving wall, but when it moved in

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LA1ER 197

the same direction as the bee this decreased the rate of flow on that side and the bee flew nearer to the moving wall. This implies that the bee steers by means of a task-specific control law that equalizes the rate of flow, rather than using other structural information about the three-dimensional environment and its position in the corridor.

The positional hypotheses. Each of the three positional hypotheses also has a straightforward reciprocal that could apply to avoiding obstacles. The thrust hypothesis can be rephrased as, "apply force opposite the direction of an opening"; the constant bearing hypothesis as, "move so as to keep an opening in a fixed ego­centric position"; and the centering hypothesis as, "move so as to keep an opening in the center of the visual field (or at the midline of the body) and progress forward." In each case, these strategies assume segmentation of objects and openings.

There is even less experimental evidence regarding human control laws for obsta­cle avoidance than there is for goal seeking, let alone the usual combination of the two. The most systematic work on this problem has been carried out by Andrew Duchon at Brown. Duchon initially evaluated the adequacy of the equalization strat­egy by implementing it in a mobile robot and testing model agent simulations (Duchon & Warren, 1994; Duchon, Warren, & Kaelbling, 1998); similar work was done at about the same time by other robotics groups (Coombs, Herman, Hong, & Nashman, 1998; Sandini, Santos-Victor, Curotto, & Girabaldi, 1993; Weber, Venkatesh, &Srinivasan, 1997). The robot had a fixed camera with an optical axis in the direction of translation, so the focus of expansion was generally at the center of the visual field. The control law simply determined the robot's turning rate as a func­tion of the difference between the summed magnitudes of optic flow on the left and right sides of the visual field (normalized by the total sum). This simple strategy was surprisingly robust, allowing the robot to wander for extended periods while success­fully avoiding walls, chairs, people, and potted plants without collision, as long as there was sufficient lighting and surface texture for optic flow extraction. This con­firmed that equalization could function as an effective obstacle-avoidance strategy without performing object segmentation. But do people use it?

A recent perceptual experiment by Dyre and Anderson (1996) bears on this question, for they reported that heading judgments indeed depend on the rate of optic flow, not just vector directions. They found that greater flow magnitudes on one side of the focus of expansion bias perceived heading toward the opposite side. However, this predicts that compensatory steering should be biased back toward the side with greater flow, contrary to the equalization hypothesis. A different in­terpretation of this fmding, consistent with participants' ratings, is that the asym­metrical flow pattern was perceived as movement on a curved path.

In his dissertation work, Duchon (1998; Duchon & Warren, 1998) first repli­cated the honeybee experiment in humans, using both steering with a joystick and walking in the virtual treadmill. Somewhat to our surprise, he found that people be-

198 WARREN

have just like bees in both set-ups. They steer to the predicted position in the corridor when added side wall motion is 0.5 times observer speed (in the opposite direction), although they undershoot slightly to about 80% of the predicted position when wall motion is 1.0 times observer speed. This indicates that humans rely on the equaliza­tion strategy rather than geometric position information such as texture gradients, texture scale, or perspective, at least until extreme positions are specified, consistent with task-specific control information. The result is also contrary to what one would expect for active steering based on Dyre and Anderson's (1996) perceptual data.

Duchon then turned to the fundamental question of integrating obstacle avoidance with goal seeking. An intriguing possibility suggested earlier is that one may simulta­neously use the heading strategy to steer toward a goal and the equalization strategy to avoid obstacles. To examine this question, a target line was added at the far end of the corridor and an obstacle was placed in the path, a frontal surface that ran from the right wall almost to the center of the corridor (symmetrical conditions were also tested). Fig­ure 4a is a schematic observer's-eye view of the scene, with the optic flow due to self-motion represented by black vectors (the target is behind the obstacle). Presumably, parricipants would try to head toward the target but detour around the obstacle and through the opening. To do so, they might use (a) the equalization strategy, equating the global rate of flow on either side of the opening, (b) the heading strategy, placing the per­ceived heading in the middle of the opening, or (c) any of the positional strategies, keep­ing the middle of the opening in a fixed egocentric direction. To dissociate these strategies, two conditions were tested in a crossed design. First, we added a horizontal motion to the texture on the obstacle, say to the right (gray vectors in Figure 4b) ; when summed with the optic flow due to self-motion, the resultant flow vectors (dashed lines) created a secondary focus of expansion (square). Consider the consequences for the heading strategy. AI, mentioned earlier, we previously showed that such independent motion biases perceived heading by a couple of degrees in the opposite direction (to the left; X in Figure 4b), 5 which thus predicts a compensatory steering bias in the direction of texture motion (to the right). Now consider the equalization strategy. The added texture motion increases the rate offlowon the right side of the visual field, which should lead the observer to steer away from the obstacle, thus predicting a steering bias opposite the di­rection of texture motion (to the left). The two flow strategies thus make opposite predic­tions. Finally, the positional strategies predict no bias from texture motion, because the geometric structure of the obstacle and opening remains constant. The second manipu­lation tested the equalization strategy by adding longitudinal motion of the texture on the left wall, opposite the direction of travel (gray vectors in Figure 4c). Motion that is

generally parallel to the observer's path does not shift the focus of expansion or perceived heading, for it only affects the magnitudes of the resultant flow vectors. This increase in the rate of flow on the left predicts a steering bias to the right if the equalization strategy is

in play, but not if the heading strategy alone governs steering.

5We used speeds of lateral texture motion that, when added to the optic flow produced by self-motion, yielded this bias (technically, by creating a path angle of 8°).

E

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 199

SideWall 6 Texture

5

4

3

2

p < .001

1 ~ stationary --.- moving

obstacle

O+--,---r--.-~~~~ -0.25 -0.2 -0.15 -0.1

7 Obstacle

6 Texture p < .001

5

4

3

2

i-left --e- stationary __ right

obstacle

O+--'---r--.--'--~~ -0.25 -0.2 -0.15 -0.1 -0.05 0 0.05

lateral posttion on treadmill (m)

FIGURE 6 Mean time series of lateral position on the treadmill with corridor displays (Figure 4). Left panel: Effect of side wall texture motion against the direction of travel. Right panel: Effect of obstacle texture motion to the right and left. N = 12.

All panicipants detoured around the obstacle even though a straight path to the target was possible, and the mean time series oflateral position reveal a consis­tent steering pattern. For active walking on the treadmill, added motion of the tex­ture on the obstacle significantly biased steering in the same direction as the texture motion (Figure 6, right panel), consistent with the heading strategy. In ad­dition, added side wall motion also induced a significant steering bias away from the side wall (Figure 6, left panel), consistent with the equalization strategy. The same pattern of results was obtained in the joystick task. It thus appears that both strategies were running simultaneously and may have partially canceled each other out; the positional strategies, on the other hand, cannot account for these effects of texture motion. Duchon modeled these results with a simulated agent-environ­ment system using both strategies simultaneously and linearly summing their out­puts (turn angle). To reproduce the data, it was necessary for the heading strategy to explicitly set the goal point in the middle of the opening until the agent passed the obstacle, and then reset it to the target. These results are consistent with the idea that steering toward a goal is controlled by the heading strategy, whereas ob­stacle avoidance involves both the heading and equalization strategies, and behav­ior is determined by the linear combination of the two. It remains to be seen whether this interpretation will generalize to more complex situations.

Braking

Gibson described braking in terms of approaching an object so as to make contact without collision. When approaching at a constant velocity, the visual angle of the object will expand at an accelerating rate, at first indicating a "looming" object until

200 WARREN

the final explosive expansion specifies an "imminent collision." The moment of the eye's contact with the surface occurs when the visual angle reaches 1800

• Thus, Gib­son proposed, to make contact without collision, slow down so as to cancel this ex­pansion at the moment of contact. The intuition here is that one can use the rate of optical expansion to control one's deceleration, but there are a number of ways in which this might be done, and several versions have been proposed.

The tau-dot hypothesis. Lee (1976) first formalized this idea in terms of the tau-dot variable (refer to Figure 7 top). Tau denotes the current visual angle of an object (9) divided by its rate of expansion (9), which corresponds to the time-to-contact with an object if observer velocity is constant. Tau-dot is the time-derivative of tau, which can be thought of as expressing the rate of change in time-to-contact.A tau-dot of -1.0 corresponds to a constant velocity approach with no deceleration. Lee showed that if i < -0.5, one's current deceleration is too low and will result in a collision, so one should increase braking; if i > -0.5, the cur­rent deceleration is too high and one will stop short of the object, so one should de­crease braking; and if i = -0.5, holding the current deceleration constant will bring one to a stop at the moment of contact. Thus, one could control braking by adjust-

v, d

Q .. :~~--J-_~_~~-_-_-_-___ -_-_-_] t = a I e z

x ~(])----.

brake: d = ex

tau-dot i = (zdlv2 )-1

Displacement

Z = zu-.5t 2 d '" v = vo-td brake

d=ex

point attractor

Braking action x = xo+b(tm-i)+E

FIGURE 7 Hypothesized dynamics of braking. t.. = margin value of i; £ = noise tenn; x = brake position; d = observer deceleration; \I = observer velocity; z = distance from ob­server to object.

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LA 'fER 201

ing deceleration so that tau-dot is near the margin value of -0.5, either by holding tau-dot constant at that value, by treating it as a set-point in a control loop, or by us­ing the current value of tau-dot to control the direction and magnitude ofbraking.

The rate of expansion hypothesis. Alternatively, one might keep the sim­ple rate of expansion theta-dot near a constant positive value (Yilmaz & Warren, 1995). Any such margin value will lead to a smooth decrease in velocity that ap­proaches zero at contact, with higher values of expansion rate requiring larger ini­tial decelerations. This strategy would produce positive values of tau-dot near + 1.0 for most of the approach, so it is empirically distinguishable from the preceding hy­pothesis. Flach, Stanard, and Smith (in press) made a persuasive case that the per­ception of time-to-contact may be based on the rate of expansion rather than tau, thereby accounting for data showing that time-to-contact judgments are influ­enced by object size and velocity, and they favor the idea that braking is governed by optical expansion as well. They propose that braking could be initiated at a ctitical boundary value of optical expansion, but do not show how deceleration would actu­ally be regulated by this variable.

The deceleration hypothesis. Finally, there are a number of ways that the deceleration (d) required to stop at an object could be computed from spatial infor­mation about object distance (z), object size (r), and observer velocity (v), together with optical variables such as tau or the visual angle of the object (Yilmaz & War­ren, 1995); forexample,d = v2/2z (refer to Figure 7 top). Of course , these strategies assume that the corresponding spatial properties can be accurately perceived. The computed deceleration is identical to that produced by holding tau-dot constant at -0.5, given the same initial conditions, and conversely the unique correct decelera­tion will yield a tau-dot of -0.5. However, the deceleration hypothesis can be tested by selectively removing spatial information about distance, size, and velOcity.

The data on braking strategies are hotly contested. Lee and his colleagues have reported several naturalistic studies that are consistent with the tau-dot strategy (Lee, Davies, Green, &vanderWeel, 1993; Lee, Reddish, & Rand, 1991). For ex­ample, Lee et al. (1991) found that, during the last 100 msec of approach to a feeder, hummingbirds decelerate with a mean tau-dot of -0. 71 (computed for the tip of the bird's bill), a slightly low value consistent with the fact that the bill does not stop at "contact" but enters the feeding tube. However, it is questionable whether active visual regulation is occurring over such a short 100 msec interval. In tasks such as slowing down to grasp a doorknob, Wann, Edgar, and Blair (1993) found that adults have a mean tau-dot of -0.45 to -0.5 for most of the approach (computed for the eye), but switch to a rapid deceleration and a high tau-dot in the fmal350 msec. This is understandable, because the hand (not the eye) must make contact with the door, and if the eye stops short, tau-dot necessarily escalates to in­fmity. Such results are consistent with a tau-dot strategy. However, it is a general

202 WARREN

weakness of observational studies that participants could be using a spatial strategy that yields the appropriate deceleration and coincidentally generates mean tau-dot values near -0.5. To distinguish competing hypotheses the information available for the control of braking must be manipulated experimentally and the time-series of tau-dot (rather than just mean values) must be analyzed.

Tau-dot information was experimentally isolated by Kim, Turvey, and Carello (1993), who presented computer displays of an approaching surface at various con­stant tau-dot values and had observers judge whether they would result in a "hard" or "soft" collision. They found a category boundary centered at a tau-dot of -0.5, consis­tent with the successful perception of collision states. Kaiser and Phatak (1993) criti­cized this finding because holding tau-dot constant at any value greater than -1.0 will, in the limit, result in a smooth stop at the object, and thus -0.5 does not demar­cate a critical point between hard and soft collisions. However, if tau-dot is viewed as information about the appropriateness of one's current deceleration, the value of -0.5 does indeed specify a critical point dividing adequate from inadequate decelera­tion, which distinguishes different adaptive actions. Thus, I believe that Kim et al.'s (1993) results show that observers can reliably judge a pragmatic boundary between crash states and safe states, providing evidence for reliance on tau-dot.

In an attempt to study the active control of braking, we manipulated the visual information available in interactive displays that simulated an approach to a set of three diamond-shaped "road signs" (Yilmaz & Warren, 1995). Participants tried to stop at the object using a hand-held spring-loaded brake in which deceleration was a linear function of brake position (with a maximum of 0.7 g), modeled on the lin­ear range of an automobile brake. On half the trials, a checkerboard ground surface provided information about object distance, size, and observer velocity; when it was absent, only optical expansion and tau-dot information were available. The data reveal that mean tau-dot during the approach was -0.51, as predicted by the tau-dot strategy. Further, the presence of the ground surface made little differ­ence,6 suggesting that a tau-dot strategy was used whether or not spatial informa­tion was available. A detailed analysis of the time-series data showed that, in both conditions, each brake adjustment acted, on average, to bring tau-dot to a critical value of -0.52 (Figure 8), indicative of discrete adjustments to a tau-dot margin value. These data also clearly contradict a constant expansion strategy, for the ob­served tau-dot values are nowhere near the predicted value of + 1.0.

In sum, our results are consistent with the use of tau-dot to control the direction and magnitude of braking, whether or not spatial information is available, at least for the simplified case of a linear brake with no plant dynamics. The tau-dot strat­egy may be particularly appropriate when the controller regulates deceleration di­rectly, as in an automobile or on foot. Kaiser and Phatak (1993) argued that a

6'fhe only exception was a slightly lower mean tau-dot (...{).43) with the checkerboard ground surface on the shottest high-velocity trials (initial time-to-contact 3 sec), suggesting that panicipants used the fast ground motion as a cue to decelerate faster early in the trial.

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LAlER 203

FIGURE 8 Mean change in tau-dot on each brake adjustment as a function of tau-dot at the onset of the adjust­ment. The regression line has a slope of -1.04 and a zero-crossing at i = 4l.52, with r = 0.98. Shaded area represents ± 1 SE (the mean within-subject SE). Nate. From "Visual Control of Braking: A Test of the i Hypothesis" by E. H. Yilmaz and W. H. Warren, 1995,JouT­nal of Experimental Ps,chology: Human PerceptionandPe!j'armance, 21, p.l01O. Copyright e 1995 by the American Psychological Association. Reprinted by permission.

~~ ·c S ,,-"'O~ . .,...~ <l "

0.8

0.6

0.4

0.2

0

-0.2

-0.4

-0.6

-0.8 +-~---.-~~-r-~---.-~~.,..-~-l

-1.0 -0.8 -0.6 -0.4

-1 at onset of adjustment

-0.2 o

simple tau-dot strategy cannot account for deceleration profiles observed in tasks such as helicopter landing, raising the possibility that there may be task-specific braking strategies that depend on the nature of the controller and the dynamics of the vehicle. As I emphasize later, ecosystem dynamics provide an important con­straint that may influence visual control strategies.

Pursuit and Flight

Gibson cast pursuit and flight in terms of predator-prey relations, but this formula is relevant to the general problem of steering with respect to a moving object. In prin­ciple, given that optic flow is determined by the relative motion between the ob­server and any surface, steering with respect to that surface could be controlled on the basis of its flow alone, whether it is stationary or moving. Thus, Gibson's solu­tion was to rely on the local flow of the object itself.

The magnificationlminification hypothesis. To pursue a moving object, move so as to magnify its contour in the field of view; reciprocally, to flee, move so as to minify its contour. As before, this hypothesis relies on the symmetrical expansion or contraction of the object, and hence on the speeds of its flow vectors. There are, of course, a number of alternatives.

204 WARREN

The object focus of expansion hypothesis. The local focus of expansion, defined by the flow vector directions for the object alone, specifies whether one is currently on a collision course. Thus, to pursue a moving object, keep the local focus of expansion within the object's form in the optic array. Conversely, to avoid a mov­ing object, shift the local focus of expansion away from it, and to flee, keep the local focus of contraction near it. Gaining on, shadowing, or falling behind the object would be specified by whether it is expanding, constant, or contracting.

The background focus of expansion hypothesis. Given that the focus of expansion defined by optic flow from background surfaces specifies one's current di­rection of heading in the environment, one could use this information to steer with respect to moving objects. Specifically, to pursue a moving object, keep the back­ground focus of expansion near its form in the optic arrayj to flee, keep the focus of contraction from the background near it. The limitation of this hypothesis is that it does not take the object's motion into account and is thus not predictive about whether one is on a collision course. The resulting closed-loop behavior over time would usually yield successful pursuit, but would not guarantee effective avoidance.

The perceived heading hypothesis. The preceding strategies assume that the flow of a moving object can be segmented from the flow of the surrounding background surfaces. However, the finding that perceived heading is biased by a moving object indicates that this might not be the case rw arren & Saunders, 1995b). In addition, when pursuing a moving object that is embedded in a global ra­dial flow field, detection latencies for expansion are affected by the surrounding flow (Probst, Krafczyk, Brandt, & Wist, 1984). Thus, a more likely hypothesis is: To pursue a moving object, keep one's perceived heading near its form in the optic ar­raYj to flee, keep one's perceived point of origin near it.

The constant bearing hypothesis. There is, however, a strategy familiar from sailing that is based on predictive information about a collision course. To intercept a moving object, move so that its form expands at a constant bear­ing in the optic array, or equivalently, at a constant visual angle to one's head­ing direction (the object-heading angle). This can be achieved by translating so as to match its horizontal optical velocity. Reciprocally, to flee, move so that its form contracts at a constant bearing, or constant object-heading angle. To avoid a moving object, move so that its bearing or object-heading angle is not constantj if the object-heading angle is increasing, one will pass in front of the object, and if it is decreasing, one will pass behind it (Cutting, Vishton, & Braren, 1995). This strategy is predicated on two assumptions: that both the object and the observer are traveling on linear paths and moving at constant velocities.

To test the perceptual basis for this strategy, Cutting et al. (1995) presented displays of an approaching pedestrian on a ground surface and asked participants

VISUALLY CON1ROLLED LOCOMOTION: 40 YEARS LATER 205

to judge whether they would pass in front, behind, or collide with him. The dis­plays contained a component of rotation to simulate observer fixation of the pe­destrian, such that he remained at the center of the screen. The pedestrian thus always had a constant bearing with respect to the observer, forcing participants to use the object-heading angle. Judgments of one's path with respect to the pedes­trian were quite accurate, suggesting that observers can use the object-heading angle to perceive when they are on a collision course. Paradoxically, however, judgments of heading with respect to the scene were highly inaccurate, raising the question of whether collision judgments were actually based on some other information. In a post hoc analysis, the authors found that differential motion parallax of the ground surface about the pedestrian correlated with passing in front, behind, or colliding, but this hypothesis was not tested experimentally. Nevertheless, these results indicate that humans can perceive future hits and misses with a moving object with sufficient accuracy to control steering, although the informational basis for this ability remains unclear.

The thrust and centering hypotheses. Finally, one could pursue a moving object by applying force opposite its current egocentric direction, or moving so as to keep it centered in the visual field. The centering strategy has been observed in in­sects, where it is referred to as fixation or tracking (Land & Collett, 1974; Reichardt & Poggio, 1976). When presented with a moving bar or a flying conspecific, a fly will turn to fixate the object as a function of both its retinal position and velocity. This is equivalent to the constant bearing strategy with a privileged egocentric position in the center of the visual field. When implemented in a mobile robot, the centering strategy yields quite robust pursuit behavior (Duchon et al., 1998).

Lane Following

There is a special case of steering that should also be considered. which I call lane fol­lowing. Here the task is to follow a preexisting path defined by lane markings or edges, as when driving down a road, landing on a runway, or walking on a sidewalk, down a hallway, or along a wall. One could think of this as a form of goal seeking, where the far end of the lane is the goal, or a form of obstacle avoidance, where the edges of the lane are barriers, but the geometrical features of edges or markings also allow for specialized strategies.

The heading hypothesis. To travel in a lane, keep the perceived heading centered on the far end of the lane, such that the radial flow vectors line up with the edges of the lane. A change in position in the lane is specified by a heading error. This strategy requires radial optic flow from texture or other structure on the ground surface.

206 WARREN

The splay angle hypothesis. The splay angle of an edge line is defined as its optical angle with respect to the vertical in the field of view. Lateral translation of the observer, such as a change of position in the lane, is accompanied by a change in splay angle (optical rotation of the edge lines), whereas a pure rotation of the ob­server is accompanied by a simple displacement of the edge lines in the field of view (Calvert, 1954; Riemersma, 1981). Hence, the splay angle can be used to control one's lateral deviation in a lane. Specifically, to travel in the center of a lane, keep the splay angle of its left and right edges symmetrical about the vertical; or to main­tain a constant distance from one edge, as when driving on the right side of an un­marked road, keep its splay angle at a fixed margin value. This strategy is available when only edge lines are visible and there is no other optic flow, as when driving at night.

There is empirical evidence for both strategies. In Mclean and Hoffmann's (1973) study of driving a straight lane, cross-cortelations with the steering wheel angle indicated that heading angle and heading rate were continuously controlled, whereas lateral deviation in the lane, specified by splay angle, was adjusted only in­termittently. Similarly, in a driving simulator study with lateral cross-wind distur­bances, Weir and Wojcik (1971) found that the best model of the data included a short high-frequency control loop that reduced heading ertors and a long low-frequency loop that occasionally corrected for accumulated lateral deviation. These results indicate that heading is more important than splay information, but other results suggest that splay angle is dominant. Beall and Loomis (1996) found that performance in simulated driving with cross-winds was the same with edge lines alone, providing only splay information, as it was with both edge lines and random dots on the ground surface, which added heading information. Beusmans (1995) added optic flow to the ground texture independent of the edge lines, which should have shifted the perceived heading, but found no cross-correlation between the texture motion and steering adjustments. However, he only tested two participants, who directly controlled lateral position in the lane (i.e., splay an­gle itself) rather than actual steering (which has a rotational component), which may have biased them toward the splay angle strategy.

In Duchon's (1998) corridor studies, he also included a condition in which the side walls ended at an implicit black floor, so that the base of the walls provided lane edges and, hence, splay angles. In each of Duchon's experiments, the pres­ence of splay information attenuated but did not eliminate the previously described effects of optic flow. Notably, in the corridor with the highest side wall motion (1.0 times observer speed), participants responded in the first 6 sec just as they had without the floor, consistent with the equalization strategy, but then they stopped their lateral movement and at 12 sec swerved back to the center of the corridor, consistent with the splay strategy. This is reminiscent of a short loop based on optic flow and a long loop based on splay angle. Finally, other studies that have exam­ined curve-taking when following a lane indicate that, in addition to this

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LA1ER 207

closed-loop control, there is a long-range preview that anticipates steering adjust­ments based on the upcoming curvature of the road (Land & Horwood, 1995; Land & Lee, 1994; McRuer, Allen, Weir, & Klein, 1977). In sum, existing results suggest that both optic flow and splay angle, when it is available, are exploited in lane following.

Finally, Gibson (1958/this issue) also noted the special complexities of terres­trial locomotion, including the first reference I know of to the "rough terrain" problem of finding footing (Patla, Robinson, Samways, & Armstrong, 1989; Warren & Yaffe, 1989; Warren, Young, & Lee, 1986). Patla and his colleagues continue to be one of the few groups studying how vision actually modulates hu­man gait with respect to the contingencies of the terrain, such as stepping over obstacles, changing direction, or choosing foot placement (Patla, 1991; Patla, this issue; Patla, Martin, Rietdyk, & Prentice, 1996). New work by Adolph and her colleagues (Adolph, 1997; Adolph & Eppler, this issue) examines the devel­opment of visually guided locomotion in varying terrain such as slopes, illuminat­ing the task-specificity of the coupling of perception to action (see also Gibson, Riccio, Schmuckler, & Stoffregen, 1987).

THE DYNAMICS OF VISUAL CONTROL

Up to this point my focus, like Gibson's, has been on the informational basis for vi­sual control. However, the broader point with which I wish to close is that a theory of visually controlled behavior is going to require both informational and dynamical concepts. An agent perceiving and acting in an environment together form a non­linear dynamical system, and the next advance in our understanding of adaptive be­havior must take this as a starting point. Behavior is not simply determined by a con­trol formula or action plan in a straightforward manner, but corresponds to stable attractors in the dynamics of the ecosystem as a whole. The control problem should thus be reformulated in terms of using available information to tweak the ecosystem dynamics so as to yield an attractor for the desired behavior. Rather than a central­ized command-and-control architecture, a more apt metaphor is surfing. One reads the waves and makes adjustments so as to get in the groove and ride the dynamics as far as possible. But in this metaphor the actions of the surfer can reciprocally shape the perfect wave, creating a recurrent system with its own interaction dynamics.

We have observed this kind of behavioral complexity already in the discus­sion of visual control laws. Even though the heading strategy and the equaliza­tion strategy are based on simple informational variables and are conceptually quite straightforward, to predict the result of simply summing them together lin­early, simulations of the agent~nvironment interaction were needed (Duchon, 1998; Duchon et al., 1998). The situation is further complicated by control laws that are nonlinear. The tau-dot strategy, for instance, is described by a simple set of second-order differential equations that characterize the relations between

208 WARREN

tau-dot and the state variables of position, velocity, and deceleration, yet the re­sulting behavior can be wildly unintuitive. These observations exemplify the tenet that combinations of simple rules can generate surprisingly complex behav­ior. This argues for simulation analysis of agent-environment dynamics and the collection of active control data when ttying to evaluate competing visual con­trol strategies.

In my current version of such a framework (see Figure 9), the agent and the en­vironment are treated as a pair of coupled dynamical systems. This is a truism for the environment. Changes in environmental state variables (e) are a function of their initial state (e) and any external forces if) applied by the agent (or other ac­tors), according to the laws of physics. For purposes of analysis, the state variables are defined as those relevant to a given task, and may include geometric, kine­matic, and kinetic properties of the world's furniture, such as positions, masses, spring properties, and so on, as well as the position of the observer. But it is equally the case that the agent can be considered a dynamical system, parameterized by task-relevant state variables of the action system (a), with changes in these vari­ables (a) being a function of their initial state and occurrent information (i). In the absence of occurrent information, the dynamics of the action system are referred to as its intrinsic dynamics, and the biomechanical properties of the body contribute to their determination. Kelso, Turvey, and their colleagues devoted the better part of the past two decades to showing that the action system exhibits stable movement patterns that can be identified with attractors of a dynamical system and transition behavior that can be identified with bifurcations in the dynamics (Kelso, 1995; Kugler & Turvey, 1987; Schmidt, Shaw & Turvey, 1993). The high dimensionality of the action state space is reduced to a low-dimensional, task-specific, dissipative dynamic, with stable attractors and only a few free parameters. Some of these pa­rameters act to tune the attractor states, and hence the preferred behavior (its am­plitude, frequency, etc.), whereas others may act as control parameters that yield a qualitative change in the layout of attractors, and hence transitions in behavior (e.g. from one coordination pattern to another). The role of information is to modulate the free parameters of an action system that is organized for a particular task, thereby shaping these dynamics.

The agent and environment are coupled in two ways. A mechanical coupling expresses the fact that the agent's movements translate into forces that affect the state of the environment (including the position of the agent), by coupling ac­tion variables into the environmental force term. An informational coupling cap­tures the laws of ecolOgical optics, acoustics, haptics, olfaction, and so forth, expressing Gibson's essential point that the environment structures energy arrays for a stationary or moving observer in a lawful manner; hence, environmental variables are coupled into the information term. Thus, following the percep­tion-action cycle, forces applied by the agent change the state of the environ­ment (including the position of the agent), making new information available, which acts to modulate the free parameters of the action system and affect the

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 209

Environment e = <l>(e, f) .. ,,.------

mechanics f=9(a)

behavioral attractor

Agent it = 'P(a, i)

FIGURE 9 A framework for the dynamics of the agent-envi­ronment system. a = state variables of action system; e = state variables of environment; i = informational coupling; f = me­chanical coupling.

movements of the agent. The resulting behavior is emergent, determined by the dynamics of the coupled system and corresponding to its attractor states. Thus, to govern its behavior, the agent cannot simply dictate a path or trajectory, but must use the levers at its disposal, using available information to adjust the pa­rameters of the action system in a way that creates an attractor in the ecosystem dynamics for the intended behavior.

Similar general characterizations of the dynamics of agent-environment sys­tems have been offered independently in the literature on autonomous agents (Beer, in press; Smithers, 1994). The most systematic and rigorous approach has been developed by Schoner and his colleagues, applied both to modeling biological behavior (Schoner, 1991, 1994; Schoner et aI., this issue) and to path planning and control in mobile robots (Schoner & Dose, 1992; Schoner, Dose, & Engels, 1995). Although the visual information is not elaborated, those of us seeking to analyze and model the dynamics of visual control can learn from his programmatic formal strategy.

Although this is merely a sketch, the present framework does make several conceptual commitments. First, the dynamics are defined over both the agent and the environment, thereby attributing the structure of behavior to the ecosys­tem rather than the cognitive or neural system alone. Second, the dynamics are embodied, such that the physics of the environment and the biomechanics of the body contribute to constrain stable solutions. This reflects the belief that biologi­cal systems exploit physical and informational regularities to order their behav­ior. For example, the moment of inertia of a limb constrains the eigenfrequency of rhythmic behavior, which in tum constrains phase transitions in coordination. This also suggests that, when it comes to vehicular control, the dynamics of the

210 WARREN

vehicle and the controller may condition the observed control strategies. Third, information is also taken to be a fundamental constituent of the ecosystem dy­namics. The central problem for visual control is how information couples into the dynamics so as to affect behavior (Schaner et al., this issue; Warren, 1988). Part of the solution lies in characterizing occurrent informational variables-op­tic flow patterns, chemical gradients, and so on-in the same units and coordi­nates as behavioral variables, so they can modulate the action system directly. Another part lies in understanding exactly what influence informational vari­ables have on the action system, such as driving the state of the system, or tun­ing a parameter on the system, or allowing the action system to adapt its own parameters to attain a certain informational state, or perhaps all of the above. Fourth, behavior involves both performatory and exploratory activity, in Gib­son's sense. Movements in the service of performing a task will reveal informa­tion, and movements may also be performed solely to obtain information, which in both cases can be used to guide further behavior.

As an example, consider Gibson's basic task of steering toward a goal (Figure 10). The dynamics of the environment are relatively trivial in this case, involving the effect of locomotor forces on the displacement of the observer. The observer's motion (described by vector f) with a heading error (P) with respect to the goal creates a focus of expansion at a visual angle (a) from the goal (a = p, normally). Following the perception-action cycle, this visual angle governs the agent's turn­ing rate according to a law of control, determining the change in the direction of thrust force applied in the environment. This in turn alters the direction of ob­server motion and the rate of change in heading error (13), which determines a new value of visual angle (a). The dynamics of this simple system can be parameterized in intrinsic coordinates by beta, the current direction of heading with respect to the goaL The goal functions as an asymptotically stable fixed point (Strogatz, 1994) that attracts the agent's heading direction:

~ = a sin(P-1t) (1)

where the fixed point is at P = 0 and the slope (a) determines the strength of the at­tractor.7 The behavior of this system can be represented in a phase portrait, which plots the rate of change in beta as a function of beta (see Figure 10 inset). Zero-crossings with negative slopes correspond to stable point attractors, those with positive slopes to unstable repellors. This simple idea makes testable predictions, for example, that turning rate should increase monotonically with heading error from the goal and heading should be stable under perturbation. Further, the observed be­havior will depend on the underlying control strategy and the available informa­tion. In the virtual treadmill, for example, we created two competing attractors, one

7Schoner, Dose, and Engels (1995) proposed a sinusoidal form for this function, because angle is a circular variable.

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 211

goal

o "t: FOE ..... ,-...

vJl,~\

, :z ,

, , , a : \f:

T~ agent

" ;,., ....... :y~ F

FOE angle a=(3

Displacement

T = g(-F) thrust angle

A

F==y

. -'"

(3 = T/z

. ~

point attractor

Turn rate y = -bsin( a+Jt)

FIGURE 10 Hypothesized dynamiCS of steering toward a goal.\i = heading error; <X = visual angle between goal and focus of expansion (FOE); 'Y = angle between goal and thrust vector; F = thrust force, t = observer translation.

fixed point at ~ = 0° for the thrust strategy (Figure 5a), and another at ~ = 5° for the optic flow strategies (Figure 5b).

The cybernetic approach was the first to analyze such visual-motor behavior by describing it in terms of closed-loop control, borrowing heavily from classical con­trol theory. It was this potential for characterizing the circular causality of percep­tion and action, lacking in S-R formulations, that appealed to Gibson in 1958 and

212 WARREN

lent a cybernetic flavor to his control formulae. Perhaps its greatest achievement is the empirical analysis and modeling of insect flight as a control system (Heisenberg & Wolf, 1993; Reichardt & Poggio, 1976), incorporating efference copies, reafferent loops, and error correction. Such linear models are, in fact, a subclass of dynamical systems, formalized in similar terms as sets of differential equations. Al­though there is much to be learned in these analyses, from the present point of view the cybernetic approach has two limitations.

First, on the cybernetic view, consistent with its engineering roots, behavior is governed by uninterpreted sensory and control signals (e.g., error signals), rather than by meaningful information. Consequently, the system functions adaptively by virtue of its design, including its components, their relations in control loops, the values of reference levels, and so on, rather than by virtue of its basis in laws of specification and control that relate the agent to ecological states of affairs.

Consider, as another example, visual control of braking. On the one hand, the margin value of i == -0.5 can be treated as a set-point in a control loop. But one can go beyond this description by grounding the margin value in an analysis of the agent-environment interaction, which reveals it to be a critical point demarcating adequate from inadequate levels of deceleration. A simple model of the dynamics of braking appears in Figure 7 (bottom). Following the perception-action cycle, the change in brake position (x) is governed by the difference between the current value of tau-dot and the margin value of -0.5. The resulting deceleration (d) is de­termined by the characteristics of the linear brake, and in tum alters the observer's velocity (v) and distance (z) from the obstacle, generating a new value of tau-dot. The dynamics of this system are again characterized by a simple attractor with a fixed point at i = -0.5 (inset in Figure 7). The data in Figure 8, which plot the change in tau-dot as a function of tau-dot, have the characteristics of a point at­tractor: a phase portrait with a negative zero-crossing at -0.5. This does not imply that viewing the margin value as a set-point in a control loop is incorrect, only that this description fails to capture the richness of the situation, particularly the physi­cal regularity that determines the critical value.

The second limitation is that the cybernetic approach emphasizes linear control systems and their conditions for stability, and has difficulty with discontinuous or nonlinear phenomena. The advantage of this approach is that linear sensor-driven control achieves a given behavior reliably and stably. On the other hand, it limits the agent's flexibility and adaptiveness, particularly when facing what are referred to as the problems of "action selection" or adopting a new action mode, and so-called "reflex interaction" or interference between action modes. For example, the control model for a behavior such as target tracking in insects is based on a con­tinuous mapping from sensory input to motor output. Although this results in sta­ble orienting to the target, it is not clear how to handle discrete switching between continuous modes, whether it be switching to a different goal object (e.g., another conspectlic), a different control strategy (e.g., from the centering to the object fo­cus of expansion strategy), a different behavior (e.g., foraging), or integrating two

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 213

or more behaviors to perform several tasks at once (e.g., tracking and the optomotor response).

One approach is to extend linear control principles. Consider the problem of competition between tracking and the optomotor response in insects. If a target is moving against a stationary background, the target will induce a tracking response. However, this yields global motion of the background, which will in tum induce an opposite optomotor response. Various researchers have suggested that the two control modes are simultaneously active and sum linearly (Collett & Land, 1975), as we observed for the heading and equalization strategies in steering and obstacle avoidance, or that they operate with different gains or temporal frequencies (Egelhaaf, 1987; Wagner, 1986), or that an efference copy of the tracking com­mand is used to correct the optomotor response (Collett, 1980). However, there are clearly limits to how far this kind of analysis can be extended to more complex interactions and nonlinear switching.

A second approach is to add a discrete lOgic atop an otherwise continuous con­trol system, such as a behavioral hierarchy (Tyrrell, 1993), a subsumption architec­ture (Brooks, 1986), a set of state transition rules (Duchon et al., 1998), or some other process that selects appropriate action modes. Gibson's (1958/this issue) so­lution was of this sort, implied by his distinction between "identifying reactions" for affordances and "control reactions" for visual guidance, and I adopted a similar view in Warren (1988). Given an intentional constraint that defines a task ("get coffee"), the affordances of objects specify immediate goals (the door) and obsta­cles (the chairs) and allow one to select an appropriate action (goal seeking and ob­stacle avoidance) and corresponding control laws (heading and equalization strategies). The problem of discontinuous switching is thus regressed onto inten­tions and affordances, which effectively function as switches between continuous control modes. Although this may prove to be adequate, it is conceptually unsatis­fying because it conflates the languages of discrete logic and continuous dynamics. One would like to cast both linear and nonlinear phenomena within a common framework.

A nonlinear dynamics approach offers a natural set of concepts that encompass not only stability but the instability that underlies flexible behavior, transitions, and the formation of new action patterns. Although this does not solve the prob­lem of intentionality, it does provide a context in which the selection of tasks, goals, action modes, and control strategies can be framed in terms of competition between attractors, within intentional and informational constraints. Assuming that a task and an immediate goal object have been specified, for example, consider the phenomenon of switching between control laws. In the case of steering, one in­terpretation consistent with our data on walking through doorways is that steering is initially governed by the thrust strategy, but then switches to the heading strat­egy. As just noted, this can be modeled dynamically in terms of competition be­tween two attractors, one at ~ = 0° and one at ~ = 5°, whose relative strength is influenced by a variable such as the rate of optic flow. As the observer approaches

214 WARREN

the doorway, its flow rate increases, raising the competitive advantage of the head­ing strategy, until a bifurcation occurs corresponding to a transition between strat­egies. An occurrent variable (flow rate) thus acts as a control parameter so that switching can be accounted for within the dynamics of the agent-environment sys­tem. Schoner and Dose (1992) proposed a nonlinear form for such competitive dy­namics in a different context (competition between goals and obstacles in robot navigation), and with Tjeerd Dijkstra's help we successfully adapted it to model our data on strategy switching. One can envision similar analyses of switching be­tween goals or action modes that incorporate information about the affordances of objects, specifying the behavior they afford for a given task. The problem of scaling up such dynamical descriptions to more complicated tasks and a larger repertoire of behaviors while avoiding local minima was recently addressed by Large, Christensen, and Bajcsy (in press).

In principle, both the continuous control modes that Gibson (1958/this issue) proposed in his formulae for locomotion and the discontinuous switching between them required for flexible behavior can be accommodated by a nonlinear dynami­cal framework for visual control. I believe that such an informational-dynamical approach will ultimately allow us to understand how it can be, in the deepest sense, that control lies not in the brain, but in the animal-environment system.

ACKNOWLEDGMENT

This research was supported by grants from the National Eye Institute (EYI0923) and the National Institute of Mental Health (MH01353).

REFERENCES

Adolph, K. E. (1997). Learning in the development of infant locomotion. Morwgraphs of the Society for Research in Child Dwelopmem, 62 (Serial No. 251).

Aloimonos, Y. (1993). Actil1e perception. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Amblard, B., Cremieux, J., Marchand, A. R., & Carblanc, A. (1985). Lateral orientation and stabili­

zation of human stance: Static versus dynamic visual cues. Experimental Brain Research, 61, 21-37.

Bajcsy, R. (1988). Active perception. Proceedings of the IEEE 76, 8, 966-1005. Ballard, D., & Brown, C. (1992). Principles of animate vision. Computer Vision, Graphics, and Image Pro­

cessing, 56, 3-21. Bardy, B., Warren, W. H., & Kay, B. A. (1996). Motion parallax is used to control postural sway during

walking. Experimental Brain Research, 1 J I, 271-282. Beall, A. C., & Loomis, J. M. (1996). Visual control of steering without course information. Perception,

25,481-494. Beer, R. (1995). The dynamics of adaptive behavior: A research program. Robotics andAulOllOTllOUS S:ys­

tems, 20, 257-289. Beusmans, J. (1995). Center of outflow is not used to control locomotion (Tech. Rep. CBR TR 95-5). Cam­

bridge, MA: Cambridge Basic Research.

VISUALLY CONlROLLED LOCOMOTION: 40 YEARS LATER 215

JkaitenhetJ1. V.Jl~\' V.l'hide~'~'.h>.5'ftllil!'.tir,b~~ .c.amh~ .M~.·.MT;.rRt.'lI1f. Brooks, R. A. (1986). A robust layered control system for a mobile robot.IEEEJournal of Robotics and Au-

tomation, RA-2, 12-23. Brooks, R. A. (1991). Intelligence without representation. ArtificiallnteUigence, 47, 139-160. Calvert, E. S. (1954). Visual judgements in motion. Institute of Navigation Journal, 7, 233-251. Clark, A., & Toribio, J. (1994). Doing without representing? Synthese, 101,401-431. Collett, T. S. (1980). Angular tracking and the optomotor response: An analysis of visual reflex interac­

tion in the hoverfly.Joumal of Comparative Ph,5io1ogy, 140, 145-158. Collett, T. S., & Land, M. F. (1975). Visual control of flight behavior in the hoverfly, S,nttlJ pipiens. Jour­

nal 0/ Comparative PhY5iology, 99, 1-66. Coombs, D., Herman, M., Hong, T., &Nashman, M. (1998). Real-time obstacle avoidance using central

flow divergence and peripheral flow. IEEE Transactions on Robotics and Automation, 14,49-59. Cutting, J. E., Vishton, P. M., & Braren, P. A. (1995). How we avoid collisions with stationary and with

moving obstacles. P5:ychological Review, 102,627-651. Delorme, A., Frigon, J.-Y., & Lagace, c. (1989). Infants' reactions to visual movement of the environ­

ment. Perception, 18,667-673. Diener, H. c., Dichgans, J., Bruzek, W., & Selinka, H. (1982). Stabilization of human posture during in­

duced oscillations of the body. Experimenral Brain Re5eQrch, 45, 126-132. Diener, H. c., Dichgans, J., Guschlbauer, B., & Mau, H. (1984). The significance of proprioception on

postural stabilization as assessed by ischemia. Brain Ruearch, 296, 103-109. Dijkstra, T. M. H., Schoner, G., Giese, M. A., & Gielen, C. C. A. M. (1994). Frequency dependency of

the action-perception cycle for postural control in a moving visual environment: Relative phase dy­namics. Biological Cybernetics, 71, 489-501.

Duchon, A. (1998). Vi.!ual5tTategies for the control of locomotion. Unpublished doctoral dissertation, De­partment of Cognitive and linguistic Sciences, Brown University, Providence, RI.

Duchon, A., & Warren, W. H. (1994). Robot navigation from a Gibsonian viewpoint. In Proceedings of the IEEE InteTTUUional Conference on Systems, Man, and C:ybemetics (pp. 2272-2277). Piscataway, NJ: IEEE.

Duchon, A. P., & Warren, W. H. (1997). Path planning vs. on-line control in visually guided locomo­tion. Investigative Ophthalmolog;y and Vi.!ual Science, 38, S79.

Duchon, A. P., & Warren, W. H. (1998) . Interaction of two strategies for controlling locomotion.lnves­tigative Ophthalmology and Vi.!ual Science, 39, S892.

Duchon, A. P., Warren, W. H., & Kaelbling, L. P. (1998). Ecological robotics. Adaptive Behavior, 6, 473-507.

Dyer, F. C. (1991). Bees acquire route-based memories but not cognitive maps in place learning. Animal Behavior, 41, 239-246.

Dyre, B. P., & Andersen, G. J. (1996). Image velocity magnitudes and perception of heading. Journal 0/ Experimental P5:ychology: Human Perception and PerfO'tTRa11Ce, 23, 546-565.

Egelhaaf, M. (1987). Dynamic properties of two control systems underlying visually guided turning in house-flies. Journal of Comparative Physiology, A, 161, 777-783.

Epstein, W. (1993). The representational framework in perceptual theory. Perception & Psychophysics, 53, 704-709.

Flach, J. M., Stanard, T., & Smith, M. R. H. (in press). Perception and control of collisions: An alterna­tive to the tau hypothesis. In M. McBeath (Ed.), Spatial navigational principles used by humans, animals, and machines. Thousand Oaks, CA: Sage.

Fukusima, S. S., Loomis, J. M., & Da Silva, J. A. (1997). Visual perception of egocentric distance as as­sessed by triangulation. Journal of Experimental Psychology: Human Perception and PerfDmlllnce, 23, 86-100.

Gibson, E. J., Riccio, G., Schmuclder, M. A., & Stoffregen, T. A. (1987). Detecting the traversability of surfaces by crawling and walking infants. Journal of Experimental P5:ychology: Human Perception and Perfonmance, 13,533-544.

216 WARREN

Gibson, J. J., & Crooks, L E. (1938). A theoretical field analysis of automobile-driving. AmericanJournal of Psychology, 51, 453-471.

Gibson, J. J. (1947). Motion picture testing and research (AAF Aviation Psychology Research Report 7). Washington, DC: U.S. Government Printing Office.

Gibson, J. J. (1950). Perception of the visual WOTId. Boston: Houghton Mifflin. Gibson, J. J. (1966). The senses considered as peTCeptual rystems. Boston: Houghton Mifflin. Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associ­

ates, Inc. (Original work published 1979) Gibson, J. J. (this issue). Visually controlled locomotion and visual orientation in animals. BritishJournal

of Psychology, 49,182-194. (Original work published 1958) Gibson, J. J., Olum, P., & Rosenblatt, F. (1955). Parallax and perspective during aircraftlandings. Ameri­

can Journal of Psychology, 68, 372-385. Gibson, E. J., Riccio, G., Schmuckler, M. A., & Stoffregen, T. A. (1987). Detecting the traversability of

surfaces by crawling and walking infants. Journal of Experimental Prychology: Human Perception and Peiformance, 13,533-544.

Gould, J. L (1986). The locale map of honey bees: Do insects have cognitive maps? Science, 232, 861-863. Heisenberg, M., & Wolf, R. (1993). The sensory-motor link in motion-dependent flight control of flies.

In F. A. Miles &J. Wallman (Eds.) , Visual mvtion and its role in thestabilimtionofga~e (pp. 265-283). Amsterdam: North-Holland, Elsevier.

Howard,1. P. (1986). The perception of posture, self-motion, and the visual vertical. In K. R. Boff, L Kaufman, & J. P. Thomas (Eds.), Handbook of peTCeption and human peifOTTllilnce, Vol. 1: Sensory pro­cesses and peTCeption (pp. 18-1 to 18-62). New York: Wiley.

Kaiser, M. K., & Phatak, A. V. (1993). Things that go bump in the light: On the optical specification of contact severity. Journal of Experimental Prychology: Human PeTCeptionand PeifOTTllilnce, 19, 194-202.

Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behallior. Cambridge, MA: MIT Press.

Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. InternationalJour­nal of Robotics Research, 5, 90-98.

Kim, N.-G., Turvey, M. T., & Carello, C. (1993). Optical information about the severity of upcoming contacts. Journal of Experimental Psychology: Human PeTCeption and Peiformance, 19, 179-193.

Konczak, J. (1994). Effects of optic flow on the kinematics of human gait: A comparison of young and older adults. Journal of Motor Behavior, 26, 225-236.

Kugler, P. N., & Turvey, M. T. (1987). InfOTTlliltion, natural law, and the self-assembly of rh'Jthmic motIe­ment. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Land, M. F., & Collett, T. S. (1974). Chasing behavior of houseflies (fannia canicularis).Journal of Com-parative Ph'Jsiology, 89, 331-357.

Land, M., & Horwood, J. (1995). Which parts of the road guide steering? Nature, 377, 339-340. Land, M. E., & Lee, D. N. (1994). Where we look when we steer. Nature, 369, 742-744. Langewiesche, W. (1944). Stick and rudder. New York: McGraw-HilI. Large, E. W., Christensen, H. 1., & Bajcsy, R. (in press). Scaling the dynamic approach to path planning

and control: Competition among behavioral constraints. International Journal of Robotics Research. Lee, D. N. (1976). A theory of visual control of braking based on information about time-to-collision.

Perception, 5, 437-459. Lee, D. N., & Aronson, E. (1974). Visual proprioceptive control of standing in human infants. Perception

& PS'Jchoph'Jsics, 15,529-532. Lee, D. N., Davies, M. N. 0., Green, P. R., &vanderWeel, F. R. (1993). Visual control of velocity ofap­

proach by pigeons when landing. Journal of Experimental Biology, 180, 85-104. Lee, D. N., & Lishman, J. R. (1975). Visual proprioceptive control of stance. Journal of Human MOtiement

Studies, I, 87-95. Lee, D. N., Reddish, P. E., & Rand, D. T. (1991). Aerial docking by hummingbirds. Naturwissenschaften,

78, 52~527.

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 217

Lewin, K. (1936). Principle3 of topological psychology. New Yode: McGraw-Hill. Loomis, J. M., Da Silva, J. A., Fujita, N., & Fulcusima, S. S. (1992). Visual space perception and visually

directed action.JournalofExperimental Psychology: HumanPerceptionandPe1jomumce, 18, 906-921. Marr, D. (1982). Vision. San Francisco: Freeman. Mclean, J. R., & Hoffmann, E. R. (1973). The effects of restricted preview on driver steering control and

performance. Human Factors, 15,421-430. McRuer, D. T., Allen, R. W., Weir, D. H., & Klein, R. H. (1977). New results in driver steering control

models. Human Factors, 19,381-397. Milner, A. D., & Goodale, M. A. (1995). The visual bmin in action. Oxford, England: Oxford University

Press. Moravec, H. P. (1981). Rover vehicle obstacle avoidance. In Proceedings of the Seventh Interna!ionalJoin!

Conference on ArtificiallnteUigence (pp. 785-790). Los Altos, CA: Kautmann. Pailhous, J., Ferrandez, A. M., Fliickiger, M., & Baumberger, B. (1990). Unintentional modulations of

human gain by optical flow. Behavioral Brain Research, 38, 275-281. Patla, A. (1991). Visual control of locomotion: Strategies for changing direction and for going over ob­

stacles. Journal of Experimental Psychology: Human Perce{ltion and Performance, 17, 603-634. Patla, A. E., Martin, C., Rietdylc, S., & Prentice, S. (1996). Locomotor patterns of the lead and the trail­

ing limbs as solid and fragile obstacles are stepped over: Some insights into the role of vision during locomotion. Journal of MaIOT Behavior, 28, 35-47.

Patla, A. E., Robinson, C., Samways, M., & Armstrong, C. J. (1989). Visual control of step length during overground locomotion: T asic-specific modulation of the locomotor synergy. Journal at Experirnen!al Psychology: Human Perce{ltion and Performance, 15,603-617.

Probst, T., Krafczyk, S., Brandt, T., & Wist, E. R. (1984) . Interaction between perceived self-motion and object-motion impairs vehicle guidance. Science, 225, 536-538.

Putnam, H. (1994). Sense, nonsense, and the senses: An inquiry into the powers ofthe human mind. Journal of Philosophy, 91, 445-517.

Reed, E. S. (l988a). Applying the theory of action systems to the study of motor slcills. In O. G. Meijer & K. Roth (Eds.), Complex 7IlOtIe1l1e7l! behatlior: The molOT-action con!rOtlersy (pp. 45-86). Amsterdam: North-Holland.

Reed, E. S. (I 988b). James]. Gibson and the psychology of perce{ltion. New Haven, CT: Yale University Press.

Reichardt, W., & Poggio, T. (1976). Visual control of orientation behavior in the fly: 1. A quantitative analysis. Quarterly Review of Biophysics, 9, 311-375.

Rensinlc, R. A., O'Regan,J. K., &Clarlc,J. J. (1997). To see or not to see: The need for attention to per­ceive changes in scenes. Psychological Science, 8, 368-373.

Riemersma, J. B. J. (1981). Visual control during straight road driving. Acta Psychologia, 48,215-225. Royden, C. S., & Hildreth, E. C. (1996). Human heading judgments in the presence of moving objects.

Perception & Psychophysics, 58, 836-856. Rushton, S. K., Harris, J. M., Uoyd, M., & Wann, J. P. (1998). The control oflocomotor heading on foot:

The role of perceived location and optical flow. Investiga!We Ophthalmology and Visual Science, 39, S893.

Sandini, G., Santos-Victor, J., Curotto, F., & Girabaldi, S. (1993). Robotic bees. In Proceedings oflROS '93. New Yorlc: IEEE.

Schmidt, R. c., Shaw, B. K., & Turvey, M. T. (1993). Coupling dynamiCS in interlimb coordination. Journal of Experimen!al Psychology: Human Perception and Performance, 19, 397-415.

Schoner, G. (1991). Dynamic theory of action-perception patterns: The "moving room" paradigm. Bio­logical Cybernetics, 64, 455-462.

Schoner, G. (1994). Dynamic theory of action-perception patterns: The time-before-contact paradigm. Human Mooement Science, 13,415-439.

Schemer, G., & Dose, M. (1992). A dynamical systems approach to task-level system integration used to plan and control autonomous vehicle motion. Robo!ics and Au!Ollo7l\oUS Systems, 10, 253-267.

218 WARREN

Schoner, G., Dose, M., & Engels, c. (1995). Dynamics of behavior: Theory and applications for autono­mous robot architectures. Robotics and Autonomous Systems, 16, 213-245.

Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3, 417-457. Shaw, R. E., Turvey, M. T., & Mace, W. M. (1981). Ecological psychology: The consequence of a com­

mitment to realism. In W. Weimer & D. Palermo (Eds.), Cognition and the symbolic processes (Vol. 2, pp. 159-226). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Smithers, T. (1994, December). What the dynamics of adaPtive behavior and cognition might look like in agent~lIJironment interaction systems. Paper presented at the Workshop on the Role of Dynamics and Representation in Adaptive Behavior and Cognition, San Sebastian, Spain.

Srinivasan, M. V., Lehrer, M., Kirchner, W., & Zhang, S. W. (1991). Range perception through appar­ent image speed in freely-flying honeybees. Visual Neuroscience, 6, 519-535.

Stoffregen, T. A, & Riccio, G. E. (1988). An ecological theory of orientation and the vestibular system. Psychological Review, 95, 3-14.

Strogatz, S. H. (1994). Nonlinear dynamics and chaos. Reading, MA: Addison-Wesley. Todd, J. T., Tittle, J. S., & Norman, J. F. (1995). Distortions of3-dimensional space in the perceptual

analysis of motion and stereo. Perception, 24, 75--86. Tyrrell, T. (1993). The use of hierarchies for action selection. Adaptive Behavior, 1,387-420. van Asten, W. N. J. c., Gielen, C. C. AM., & van derGon, J. J. D. (1988a). Postural adjustments in­

duced by simulated motion of differently structured environments. Experimenral Bmin Research, 73, 371-383.

van Asten, W. N. J. c., Gielen, C. C. AM., & van der Gon, J. J. D. (1988b). Postural movements in­duced by rotation of visual scenes. Journal of the Optical Society of America, A, 5, 1781-1789.

vanGelder, T. (1995). What might cognition be, if not computation?Journal of Philosophy, 92, 345-381. Wagner, H. (1986). Flight performance and visual control of flight in the free-flying housefly (Musca

domestica): III. Interactions between angular movement induced by wide- and small-field stimuli. Philosophical Transactions of the Royal Society of London, B, 312, 581-595.

Wann, J. P., Edgar, P., & Blair, D. (1993). Time-to-contact judgment in the locomotion of adults and preschool children. Journal of Experimenral Psychology: Human Perception and Performance, 19, 1053-1065.

Warren, W. H. (1984). Perceiving affordances: Visual guidance of stair climbing. Journal of Experimenral Psychology: Human Perception and Performance, 10,683-703.

Warren, W. H. (1988). Action modes and laws of control for the visual guidance of action. In O. Meijer & K. Roth (Eds.), Movement behavior: The motor-action contTOlJeT'SY (pp. 339-380). Amsterdam: North-Holland.

Warren, W. H. (1998). The state of flow. In T. Watanabe (Ed.), High-level motion processing (pp. 315-358). Cambridge, MA: MIT Press.

Warren, W. H., & Kay, B. A. (1997a). Control law switching during visually guided walking. Abstracts of the Psychonomic Society, 2, 52.

Warren, W. H. & Kay, B. A (1997b). The focus of expansion is used to control walking. In M. A Schmuckler & J. M. Kennedy (Eds.), Studies in Perception and Action N, 207-210.

Warren, W. H., Kay, B. A., & Yilmaz, E. H. (1996). Visual control of posture during walking: Functional specificity. Journal of Experimenral Psychology: Human Perception and Performance, 22, 818--838.

Warren, W. H., & Saunders, J. A (1995a). Perceived heading depends on the directionoflocal object motion. Investigative Ophthalmology and Visual Science, 36 (ARVO Suppl.), S8229.

Warren, W. H., & Saunders, J. A (1995b). Perception of heading in the presence of moving objects. Per­ception, 24, 315-331.

Warren, W. H., & Whang, S. (1987). Visual guidance of walking through apertures: Body scaled infor­mation for affordances. Journal of Experimental Psychology: Human Perception and Performance, 13, 371-383.

Warren, W. H., & Yaffe, D. M. (1989). Dynamics of step length adjustment during running. Journal of Experimenral Psychology: Human Perception and Performance, 15,618-623.

VISUALLY CONTROLLED LOCOMOTION: 40 YEARS LATER 219

Warren, W. H., Young, D. 5., & Lee, D. N. (1986). Visual control ofstep length during running over ir­regular terrain.lournal of Experimental Ps,chololO: Human Perception and Performance, 12, 259-266.

Weber, K., Venkatesh, S., & Srinivasan, M. V. (1997). Insect inspired behaviours for the autonomous control of mobile robots. In M. V. Srinivasan & S. Venkatesh (Eds.), From lilling eyes to seeing ma­chines (pp. 226--248). Oxford, England: Oxford University Press.

Weir, D. H., & Wojcik, C. K. (1971). Simulator studies of the driver's dynamic response in steering con­trol tasks. Highway [Transportation} Research Record, 364, 1-15.

Yilmaz, E. H., & Warren, W. H. (1995). Visual control of braking: A test ofthe tau-dot hypothesis.lour­nal of Experimental PsychololO: Human Perception and Performance, 21, 996--1014.

Yum, J. S., Duchon, A. P., & Warren, W. H. (1998). A moving object biases both active steering and per­ceived heading. In\leStigati\le OphthalmololO and Visual Science, 39, 51095.

Copyright © 2002 EBSCO Publishing