Locality, Delays, and Internal Feedback in Sensorimotor ControlPart 1: Motivation and Introductory Theory

Josefin Stenberg1, Jing Shuang (Lisa) Li2,a, Anish A. Sarma3,a, Terrence J. Sejnowski4,5,Gwyneth Card6, Elizabeth S. Heckscher7, Narayanan Kasthuri8, John C. Doyle2

Abstract— Neural architectures in organisms support effi-cient and robust control that is beyond the capability ofengineered architectures. Unraveling the function of such ar-chitectures is highly challenging; they contain massive diver-sity and heterogeneity in components and cryptic internalfeedback pathways (IFPs). We introduce the key concepts ofspeed-accuracy tradeoffs (SATs), diversity-enabled sweet spots(DESS), and controller architecture which aim to help deciphercomplex neural architecture. As a case study in diversity andcomplex architectures, we consider nervous system control ofmovement. In this system, diversity in sensors corresponds todiverse morphology, physiology, and biochemistry of neurons.which can be interpreted as SATs in component specification.While such tradeoffs are often implicit in controller design,we aim here to make them explicit in the canonical LQRsetting. We consider an LQR problem with two types of sensors,one fast but sparse and one dense but slow. The sensorsachieved substantially lower performance costs when combinedthan when separate, demonstrating a DESS. The DeSS is alsoobserved for the dual problem in actuation, corresponding todiverse muscle fibers. Controller-internal dynamics, sometimescalled internal feedback pathways (IFPs) in biology, werenecessary to achieve these combinations. This paper, intendedas the first of two parts, bridges familiar control theoryproblems and novel applications of the System Level Synthesisframework to problems in biology and cyber-physical systems.We additionally outline future experimental plans which willutilize this new theory.

I. INTRODUCTION

Neural architectures and body plans have evolved to sup-port exceptionally efficient and robust sensorimotor control– despite the fact that biological components communicateand compute more slowly and more noisily than engineeredcomponents. These differences suggest that neural controlsystems are unlike those currently found in engineeringtheory. Unfortunately, the neural architectures by whichenergy-efficient, distributed action selection is implementedin biological systems are unclear; conventional control the-ory models are unable to accommodate critical featuresof biological control. We require new theory that doesencompass critical features of biological control.

A feature of neural architectures is the presence of IFPs,whose role in sensorimotor control is ambiguous. Strik-

1Engineering Physics, KTH Royal Institute of Technology, 2Computingand Mathematical Sciences, Caltech, 3Computation and Neural Sys-tems, Caltech, 4Computational Neurobiology Laboratory, The Salk In-stitute for Biological Studies, 5Biological Sciences, University of Cal-ifornia San Diego, 6HHMI Janelia Research Campus, 7Molecular Ge-netics and Cell Biology, University of Chicago, 8Neurobiology, Univer-sity of Chicago, ato whom correspondence may be addressed: {jsli,aasarma}@caltech.edu

ing behaviors, like reflexive escape behaviors and object-tracking, are typically thought to be generated by fast feedfor-ward pathways (FFP), as illustrated in Fig 1 for the primatemotion system in visual cortex and the larval and adultescape systems. Meanwhile, anatomical and physiologicalevidence tell us that the IFPs are more complex and diversein speed and composition, far more so than can be shownin a single figure. Such massive and diverse IFPs are amystery since they have no obvious function for fast reflexes.Nonetheless, massive and diverse IFP are found in almostnervous systems across species and life stages. We requirenew theory that explains the presence, magnitude, andfunction of this IFP.

In our companion paper [2], we apply System LevelSynthesis (SLS) [3] theory to show that given the featuresof biological systems (i.e. local, delayed, distributed), IFPin fact enables the robust and optimal action selection thatis typical in nature, thus uniting the two problems statedabove. In this paper, we discuss these concepts in thewell-understood LQR context, which captures some of thecomplexity of the problem but is less descriptive of the realbiology than the more involved SLS model. In Section II,we describe key concepts and theory regarding diversity-enabled sweet spots (DeSS) [1] and controller architecturethat facilitate improved understanding of biological control.In Section III, we set up a basic system with delayedsensors. We then demonstrate the presence and importance ofDESS (Section IV) and IFP (Section V) in this system, anddiscuss the implications for general architecture. We discusscandidate future experiments that aim to advance both theoryand experimental understanding of neural architectures, aswell as concluding remarks, in Section VI.

II. BACKGROUND & MOTIVATION

A. Key concepts

Fig. 1 demonstrates some examples of the enormous IFPpresent in brains. However, it is also a simplified figurethat hides the enormous complexity of the underlying cells,neurons, and networks. We introduce four key conceptsnecessary to begin to untangle this complexity.

1) Levels: The most familiar concept is that biology hasdistinct levels, or scales (e.g. from molecules synapses, cells,circuits). Fig. 1 hides lower level implementations in neuralhardware.

2) Speed-accuracy tradeoffs (SATs): Biological hardwarehas severe speed-accuracy tradeoffs (SATs), a familiar con-cept that is expanded and extended in our theory and

1

arX

iv:2

109.

1175

2v1

[ee

ss.S

Y]

24

Sep

2021

Fig. 1. Schematic system level architectures for bottom up and top down processing streams in the brains of primates, adult flies and larval flies.Fast feedforward pathways (FFP) shown in black carry salient sensory information that gives rise to numerous internal feedback pathways (IFP) shown inred, which can influence feedforward processing in myriad ways. Single arrows represent axon bundles with a wide range of sizes having a diversity ofspeeds and accuracies. Arrows labelled with ”?” are predictions made by SLS that will be examined experimentally via connectomic reconstructions. Asimplified diagram of the motion processing circuit in primates (A) is optimized for making rapid saccades to salient targets such as a bright flash. Thiscircuit demonstrates the existence of massive IFPs. However, it has been difficult to make further progress because of experimental difficulties and lackof a theoretical framework. The fruit fly offers a model system in which we can directly align the mathematical components of SLS theory with specificneural components. The escape circuits in both adult (B) and larval flies (C) already bear a striking resemblance to predictions made by SLS. Note thatin contrast to the visual cortex diagram in which the blocks represent whole brain areas, the blocks in the fly diagrams represent individual cells or celltypes, with exact chemical synaptic connectivity known from connectomic EM studies.

Fig. 2. Controllers for sensorimotor modeling. (A) Layered controller adapted from a model of bike riding [1] that illustrate DESS. This controlleroptimally layers diverse speed- and accuracy-limited components from axons to muscle to produce optimal behavior with good speed and accuracy in boththeory and experiments. (B) SLS controller diagram emphasizing the role of delays in sensing and internal feedback pathways (IFP), both central featuresof brains. (C) SLS controller with internal communications delays and localized memory and even greater IFP. Control systems in brains have layers anddiversity, sparsity, locality, and delay everywhere, combining all three diagrams. These are the simplest SLS causes of IFP but there are many others thatwill be explored in explaining the IFP found in flies. Similar to this controller, neurons also process information in highly distributed and localized ways;each neuron shares information with only a very tiny subset of the organism’s total neurons. Note: The SLS controller simultaneously contains delays anddistributed processing, but the combined image would be difficult to visually decipher (as are neuronal connectomes), so we have depicted each separately.

experiments. SATs are prevalent at the hardware level, butless so at the behavioral level.

3) Diversity-enabled sweet spots (DESS): [1] Highlydiverse hardware-level components (which obey SATs) en-able performance sweet-spots which overcome the hardware-level SATs. Extreme diversity in axon sizes, receptors, andneurotransmitters are abundant but largely hidden in Fig. 1.By itself, diversity of components only enables sweet spots;to achieve these sweet spots requires specific architecturesto maximize the utility of each component. We present anumerical example in Section IV.

4) Layered architecture: Layered architecture (e.g. fastreflex layer vs. slow planning layer) help organisms createDESS [1]. In general, we claim that DESS is universal,architecture remains an ad hoc and domain-specific subject,something we aim to change.

Fig. 2a highlights the essential concepts of layered ar-chitecture, SATs, and DESS but with the simplest possibleexperiments and theory [1], [4]. Naively, success seems torequire speed and accuracy that this hardware lacks, makingnon-layered solutions infeasible. Our layered brain breaks the

overall problem into a high “trails” layer of slow but accuratevision with trail look ahead for advanced warning, and a low“bumps” layer that uses fast but inaccurate muscle spindlesand proprioception to sense and reject the bump disturbances.Their motor commands to the muscles simply add. Theresulting layers naturally multiplex well in experiments andtheory. Effective architectures (e.g. layering) creates a DESSwhere extremely diverse hardware enable a sweet spot thatis both fast and accurate despite having no hardware thatis. Across a wide range of conditions, theory and data agree,and all are remarkably robust to assumptions and details (e.gworst-case versus stochastic). All experiments and theoryrequire only high-school level math, and the formulas foroptimal controllers and the resulting errors are analytic,simple, and transparent.

B. SATs, diversity, and IFP in biology

The primate sensorimotor system uses components withSATs and diversity in response speeds. Roughly speaking,we can divide the sensorimotor system into a fast layer(i.e. proprioception, or the ability to sense the position and

2

movements of our body) and a slow layer (i.e. visual cortex).Proprioceptive reaction time is around 95.6 ± 10.6 ms [5],while visual reaction time is much slower at 247.6±18.54 ms[6]. Although proprioception is faster, it is much less accuratethan vision; when performing sensorimotor tasks, we rely onvision for precision and accuracy [7]. On the motor side, wealso possess diverse muscle fibers that obey SATs; inaccuratefast-twitch muscles, and accurate slow-twitch muscles.

The immune system response also contains diverse partsobeying SATs. The immune system can roughly be di-vided into two parts: innate and adaptive immunity. Innateimmunity provides fast but not very accurate protection,and is able to limit infections to some extent. Here cellslike macrophages, neutrophils and basophils respond withinminutes to hours with a rapid response. Adaptive immunityis accurate but slow, providing specialized protection againstpathogens. Adaptive immune responses can take up to weeks,but are very effective; T lymphocytes and B lymphocyteswork to produce efficient and long lasting protection.

Within vision, IFP is abundant and well-documented [8]–[10]. This is also shown in Fig. 1a, and also discussed atlength in the previous sections. IFP is also present in theimmune system, where T cells can be further divided intoCD4+ T cells, or T helper and regulatory cells, and CD8+T cells, or cytotoxic T cells, which function as the namessuggest. While CD8+ T cells can be thought of as actuators inthe control sense, helper and regulatory cells mediate muchmore complicated dynamics that suppress or amplify CD8+cells as well as innate immune sensory circuits [11]–[13].Collectively, these elements suggest that IFP is abundantlypresent in the immune system in order to recognize and besensitive to changes and irregularities.

C. Architectures in controllers

To explain the ubiquity of IFP and to begin to integratethe above concepts, we will use theory from standard controland System Level Synthesis [3].

Biological control uses components (i.e. cells and neu-rons) that are sparse, local, delayed, saturating, noisy, anddistributed throughout. We shall refer to these as SLD+constraints; they are essential for producing realistic models.Additionally, the inclusion of such constraints allow us tomodel diverse components and the resultant DESS. However,existing theories from optimal and robust control becomecomputationally intractable even with minimal combinationsof SLD+ constraints. In contrast, SLS explicitly allows forthe implementation of control systems with SLD+ constraintson sensing, communication, and actuation. This allows usto model SATs at various levels of biology. SLS not onlyproduces optimal controllers under SLD+ constraints, butalso yields novel architectures for controller implementationwith striking similarities (i.e. abundant IFP) to biologicalarchitectures [2]. This in turn inspired us to return tostandard, “pre-SLS” controllers and dig deeper into theirunderlying architecture. We focus on pre-SLS controllerswith delayed sensing in Sections IV and V, and also give

a brief discussion of what such controllers can tell us aboutIFP and architecture.

SLS and SLD+ constraints are complementary to thenotion of DESS and layered architecture; a full model ofbiological control includes both, as suggested in Fig. 2.Though Fig. 2a produces a key finding regarding layeredarchitectures, DESS, and SATs, it abstracts away underlyingarchitectural details such as massive IFP. Boxes and wiresin this figure are, as in Fig. 1, schematic drawings ofsystems whose lower levels are implemented in hardwarelevel neurons, sensors, muscles, all of which obey SATsand SLD+ constraints. To expand the model and uncoverthe underlying IFP, SLS is required.

Fig. 2b expands the green reflex box from 2a. Here, wesense the full state with delay, creating IFP in the optimalcontroller. This is explored in a more concrete examplein Section V. This figure presents a simple case in whichrich IFP arises from a single, neurally plausible mechanism:sensing delay. An orthogonal direction in Fig 2c imposeslocal communication constraints (i.e. locality) on the internalcommunications and computing hardware. The result is anexplosion in IFP and internal memory that is difficult tovisualize even in the simplest case; we refer the reader to[2] for more detailed visualizations. This figure demonstratesanother simple case wherein rich IFP arises from a single,neurally plausible mechanism: this time, local communica-tion. We emphasize that SLS is able to accommodate delayand locality constraints easily.

IFP arising from both delay and locality can be looselyinterpreted as prediction; it is dominated by the known effectsof our actions and how they propagate through the plantdynamics, which includes the body and its environment. Italso depends crucially on what the objective of our futureactions are, and most importantly how delays, including ininternal communications and actuation limit such actions(not shown). This is much more than Bayesian prediction,which is pedagogically important given its popularity, but isa very special and ultimately minor source of IFP in reality;we briefly discuss this in Section V. Our view of IFP isconsistent with recent experiments [14], [15] done withoutthe insights from SLS, but with limited connection with pre-SLS theory.

We have considered locality and delay constraints; thereare further realistic constraints beyond those in Fig. 2 (e.g.actuator sparsity, actuator saturation) that cause further ex-plosions in IFP that are nevertheless entirely comprehensiblewith SLS. Superficially, the SLS “physiology” of Fig. 2b-chas many of the features of Fig. 1; we anticipate that incor-porating additional constraints will strengthen the similarity.

Additionally, much of the above theory is not only essen-tial to biological architectures but to engineering architec-tures as well. Fast electronic parts allow us to avoid mostSLD+ constraints in current engineered systems. However,this will become less and less true in networked engineeringsystems (e.g. smart grids), which are highly complex, requirerobust and efficient performance, and in which SLD+ con-straints sometimes become unavoidable due to the sheer size

3

Fig. 3. The ring formation withn = 5 states where each state isconnected to (and interacts with)two neighbors.

of networks.

III. CASE STUDY

We now present the setup of a simple system with diversesensing, in which we will analyze DESS (Section IV) andIFP (Section V). The goal of this analysis is not to modelthe aforementioned biological systems in fine detail, but tostudy simple, human-interpretable examples that give rise tofundamental theory on the existence and characteristics ofDESS and IFP using standard control theory.

We consider a system composed of n nodes in a ringformation, as shown in Figure 3. State dynamics in thisring can be described by the difference equation x(t+ 1) =A1x(t), where x ∈ Rn represents ring states and A1 ∈ Rn×n

is:

A1 =a

3∗

1 1 0 . . . 11 1 1 0 . . .

0. . . . . . . . .

......

. . . . . . . . . 11 0 . . . 1 1

(1)

Here, a is a scaling parameter that determines the spectralradius (i.e. maximum-magnitude eigenvalue) of A1; a =1 corresponds to a neutrally stable system, and a > 1corresponds to an unstable system.

We consider two types of sensors: fast and sparse, andslow and dense, motivated by the examples discussed inSection II. These sensors lie on different ends of the speed-accuracy spectrum. We focus on the case of perfect actuationand diverse sensing, i.e. full control (FC), and assume zerosensor noise. All analysis and findings naturally apply tothe dual case of perfect sensing and diverse actuation, i.e.state feedback (SF). We will also briefly discuss the caseof neither perfect actuation nor perfect sensing, i.e. outputfeedback (OF) at the end of Section V.

Consider the discrete-time linear time-invariant system

x(t+ 1) = Ax(t) +B2u(t) +B1w(t)

y(t) = Cx(t)(2)

where x, u, w, and y are the state, control action, disturbance,and output, respectively.

To model sensing delays, we introduce additional, artificialstates representing delay. Assume we sense each state, butwith a delay of d timesteps; then, we introduce d copiesof each ring state to represent information internal to thedelayed sensors. The resulting state vector x is of size

n(d + 1). Here, the first n entries of x represent realstates on the ring, and the rest denote the internal delayedstates. We remark that most delays in biology are causedby limitations in communication between components. Forthis simple example, we ‘push’ all delay to the sensor. Moresophisticated modeling of sensor and communication delaysis described in [2].

The state matrix A for the vector x including delay stateshas dimension n(d+ 1)× n(d+ 1), and can be written as

A =

[A1 0I 0

](3)

where I has dimension nd. A has equal spectral radius asA1; its stability is also determined by a.

For full control (FC), we require full actuation, i.e. B2 =I . Additionally, we only consider disturbances that affect thering states, i.e. B1 =

[I 0

], where I has dimension n and

the zero block has dimension nd.For fast and sparse sensing, instead of sensing each state,

we choose to sense along eigenvectors of the ring statematrix A1. We will choose some m eigenvectors, typicallycorresponding to the m highest-magnitude eigenvalues ofA1. Cf has dimension n(d+ 1)×m, and is written as:

Cf =

[E0

](4)

where E ∈ Rn×m has as columns the m selected eigenvec-tors of A1. When we choose the eigenvectors correspondingto unstable eigenvalues, Cf represents the instant but sparsesensing which senses along the m “worst directions” ofimpulse propagation.

For slow and dense sensing, we sense each state with adelay of d timesteps. Cs is of the size n(d+1)×n, and canbe written as Cs =

[0 I

], where I is of size n× n

We will study the system with fast-only, slow-only, anddiverse (i.e. both fast and slow) sensors, corresponding toC = Cf , C = Cs, and C =

[Cf Cs

], respectively.

We consider a standard LQR cost, with state penalty Q =B1B

>1 and zero input penalty1. Note that we only penalize

the n ring states and not the delayed sensing states. We solvethe DARE to obtain the Riccati matrix S, which is used tofind the optimal controller gain L = A>SC(C>FC)−1. Theoptimal controller is u(t) = −Ly(t). Assuming no sensornoise, the LQR cost can be calculated via Tr(B>1 FB1).

We will now study the behavior of this system anddemonstrate substantial presence of both DESS and IFP.

IV. DIVERSITY-ENABLED SWEET SPOT (DESS)

As described above, we consider two types of sensors,fast and sparse, and slow and dense. We first study systemperformance with uniform sensors, i.e. fast-only or slow-only. We show that with uniform sensing, our controllerperforms terribly and incurs extreme and sometimes infinitecosts. We then show that by allowing the system to use both

1To avoid issues with numerical solvers, we may need to set an inputpenalty R = εI for some ε� 1

4

Fig. 4. System behavior with fast-only sensing. (Left) Cost as a functionof stability a for different system sizes n. In all cases, the cost eventuallydiverges to infinity as the system becomes less stable and the controller failsto stabilize it. (Right) Maximum instability a that can be stabilized by theoptimal controller, as a function of system size n. As system size grows,stabilization is increasingly challenging and we are only able to deal withsystems with small instability.

fast and slow sensors, performance is dramatically improved.This demonstrates the existence of a striking DESS.

For the remainder of this and the next section, unlessotherwise specified, we use parameters n = 5, a = 1.856(to be explained), m = 1, and d = 3.

A. Fast-and-sparse sensing

For a controller with only fast-and-sparse sensing, weshow the relationship between the cost and the stabilityvalue a in the left panel of Fig. 4. For each line, there isclearly a breaking point at which fast-only sensing resultsin instability and infinite cost. These breaking points occurwhen we reach a value of a that gives more than m = 1unstable eigenvalues in A1; in this case, the fast-only sensingis no longer sufficient for the controller to reject all directionsof unstable impulse propagation. Thus, certain states neverconverge to zero, yielding infinite cost.

The right panel of Fig. 4 shows the breaking point valuesof a for which fast-only sensing results in infinite costs(i.e. fails to stabilize). As the system size n increases,the controller is less able to stabilize unstable open-loopsystems. For large system sizes, the controller can onlyhandle marginally unstable systems (i.e. a very close to 1).

For n = 5, the minimum value of a that causes instabilityand incurs infinite cost is a = 1.856. The correspondingimpulse response is show in on the left in Figure 5, wherewe note that the response does not get attenuated to zero, asexpected.

B. Slow-and-dense sensing

Consider a controller with only slow-and-dense sensing:the impulse response is shown in the middle panel of Fig. 5.In the first three timesteps, the system behaves as if it wereopen-loop; no action is taken because due to delayed sensing,the controller does not yet know about the disturbance. Oncethe delayed information reaches the controller, the impulsepropagation is immediately killed off, thanks to the denseinformation provided by the slow sensor.

C. Diverse sensing

We now consider a controller with diverse sensors: thiscontroller uses both the fast-and-sparse and slow-and-dense

Fig. 5. Impulse responses for the ring system. For the center and rightpanels, the first 5 states correspond to the true states on the ring, whereaslater states represent internal, delayed sensor states. (Left) Fast-only, cost =∞. Notice that the state does not decay to zero. (Center) Slow-only, cost= 13.726. The impulse response propagates and amplifies until the delayedinformation about the impulse reaches the controller, at which point allstates become zero. (Right) Diverse, cost = 2.279. The impulse responseappears to be a mix of the fast-only and slow-only cases; some portion ofthe impulse is rejected right away, and all states become zero after delay d.

Fig. 6. Cost of controllers using fast-only, slow-only, and diverse (i.e. fastand slow) sensing for varying values of delay. The fast-only setup incurseffectively infinite cost across all values of delay. At low delay, the slow-only setup incurs cost on the same order of magnitude as the diverse setup.As delay is increased, the slow-only setup incurs dramatic, exponentiallyincreasing costs, while the diverse setup maintains a low cost.

sensors. The impulse response is shown in the right panelof Fig. 5. The diverse controller displays a mix of behaviorsfrom the fast-only and slow-only setups. In the first threetimesteps, the behavior is identical to the fast-only system;as soon as information from the slow sensor reaches thecontroller, all impulse propagation is completed attenuatedand the state values go to zero. This controller also has much-improved cost; it incurs a performance cost that is nearly sixtimes smaller than the best non-diverse setup. This displaysa dramatic DESS.

We vary the delay parameter and show that this DESSonly becomes more dramatic for increased values of delay;this is shown in in Fig. 6. Clearly, the diversity in sensorsachieves a sweet-spot; while fast-only sensing incurs effec-tively infinite costs and slow-only sensing incurs dramatically

5

exponentially increasing costs, diverse sensing allows us tokeep the cost below 101. We conclude that delayed sensing isacceptable if paired with a fast-but-sparse sensor; similarly,sparse sensing is acceptable if paired with dense-but-slowsensor. This sweet spot is consistent with what we expectfrom the sensorimotor system and neural systems in general;neurons and cells typically obey SATs, but a combination ofdiverse neurons (i.e. both fast and slow) allows the systemto achieve a DESS.

Overall, this striking but simple example suggests thatDESS is a fundamental feature in the presence of diversecomponents. Even in our very simple system with only twotypes of sensors, we observe a strong DESS. We expect thatin systems with more diversity between components (i.e.biological and neural systems), even more extreme DESSmay be found.

We remark that similar DESS are obtained for largervalues of n and a. In fact, larger systems with more in-stability display even more dramatic DESS. This is becausethe fast-only sensing becomes more sensitive to instabilityas n increases, as suggested in Fig. 4. On the other hand,increasing m improves the performance of the fast-onlysystem. As long as m is less than the number of unstableeigenvalues in A1, we observe dramatic DESS. When m isgreater or equal to the number of unstable eigenvalues in A1,we still observe a DESS, albeit a slightly less dramatic one.Also, all analysis and results in this section apply to the dualcase of diverse actuation, in which dramatic DESS may alsobe observed.

In our example, considering the terrible performance ofthe non-diverse setups, it is safe to say that the diverse setupof sensors performs far better than the sum of its individualparts. This illustrates a basic principle at work: by cleverlyusing architecture, we can combine individual componentsthat together perform better than the sum of their parts (i.e.achieve a DESS). We now study a fundamental architecturalfeature that enables this DESS – internal feedback pathways(IFP).

V. INTERNAL FEEDBACK PATHWAYS (IFP)

We now observe the structure of the controllers that givethe dramatic DESS found above. We show that IFP is anessential architectural feature that enable DESS and thatremoving IFP destroys system performance.

We define IFP in controllers as information traveling fromthe actuation output toward the sensor input of the controller.We show the presence of IFP in our FC controller withdelayed sensing and its dual (SF controller with delayedactuation) in Fig. 7. In our setup, our definition of the delaystates as internal, sensor states means that all control actionto them (i.e. the orange arrow exiting from L2 in Fig. 7)constitute IFP. This control action corresponds to internalwires in the controller rather than physical actuation uponthe external environment. Conversely, in a system with nointernal states, a static gain controller u = Kx or u = Lywould contain only feedforward paths from sensor input (xfor SF and y for FC) to actuation output u, with no IFP.

Fig. 7. Block diagrams displaying IFP in standard controllers with delay.(A) IFP for the full control problem with delayed sensing is shown bythe orange arrows. The optimal FC controller L =

[L1> L2>

]> ispartitioned into two parts representing FFP and IFP, respectively. (B) IFPfor the (dual) state feedback problem with delayed actuation is shown bythe blue arrows. The optimal SF controller K =

[K1 K2

]is partitioned

into two parts representing FFP and IFP, respectively.

Fig. 8. The optimal controllers L for setups that include the slow sensor(and therefore IFP). For both controllers, the first 5 rows correspond tofeedforward processing, and the subsequent rows represent IFP. In bothcontrollers, IFP is abundantly present. (Left) Controller with slow-onlysensing. (Right) Controller with diverse (i.e. fast and slow) sensing.

In our model, IFP only appears in the model when theslow sensor (i.e. delayed internal states) is present, whetheron its own or in a diverse-sensor setup. For the optimal FCcontroller

L =

[L1L2

](5)

where L1 ∈ Rn×n, entries in L2 correspond to IFP. Weshow our slow-only and diverse controllers from the previoussection in Fig. 8. For the slow-only controller, the shape ofthe A1 matrix is clearly visible within both the feedforwardand IFP blocks. This is not a coincidence, since both feedfor-ward and IFP paths utilize A1 to predict and counteract thebehavior of the ring states. For the controller with both fastand slow sensors, the shape of the blocks are still similar,

6

Fig. 9. Impulse responses of controllers with removed IFP. Prior toremoving IFP, both controllers were able to stabilize the system. Afterremoving IFP, neither can. (Left) Slow-only sensing, cost =∞. (Right) Fast& slow sensing, cost = ∞.

although including the fast sensor changes the magnitudesof the entries in the controller corresponding to the slow anddense sensor.

In both controllers, it is clear that the system uses IFPto produce systems with optimal performance – in the caseof the diverse controller, IFP is required to give the DESSdiscussed in the previous section. To further study the impactof IFP, we remove the IFP in setups with slow and diversesensing and observe the resulting performance. This is doneby setting L2 to be zero in the optimal controller.

A. System performance without IFP

The removal of IFP causes dramatically worse perfor-mance, as shown in Fig. 9. With IFP, both the slow-only anddiverse controller delivered stable, finite-cost performance.After removing IFP, both controllers give unstable perfor-mance and infinite cost.

Removal of IFP causes a gap in the system informationabout previously implemented actions and current state.This creates a system that only acts based on what hasbeen sensed d time steps ago without taking previous butmore recent actions into account. This is apparent in theimpulse response of the slow-only setup on the left in Fig.9. The impulse is attenuated perfectly after d+1 time steps,when the controller has finally accessed the delayed sensedinformation. However, immediately in the next timestep, thestates receive more attenuating control action, causing thezero states to become negative (due to the positive sensedstate from d timesteps ago). Later, these negative states willbe sensed, and positive control action will be taken. Zero-value states will be achieved again, but additional attenuatingcontrol action (this time negative) will be taken, causingthe zero states to become positive. The entire system willfluctuate between positive-value and negative-value states,

each time growing in magnitude. Overall, the removal ofIFP causes the system with slow-only sensing to becomeunstable.

Removing IFP in the diverse sensing case yields similarlybad results; the impulse response again fluctuates such thatthe amplitude increases to infinity. Compared to the slow-only setup, the initial state magnitudes are smaller. However,over long intervals of time, the amplitude of the statesstill increases to infinity, demonstrating similarly unstablebehavior to the slow-only case.

An interesting observation: if we reduce the instabilityvalue a of the system, then a setup with fast-only sensingis sufficient to stabilize the system. If we add slow sensingto this to create a diverse sensing setup, then remove IFP,the resulting system still fluctuates but eventually decays tozero instead of blowing up. In this case, the diverse setupis worse than the fast-only setup. This again emphasizesthe importance of IFP, not only for performance but alsoto facilitate DESS in systems where delays are present.

B. IFP in Standard Controllers

In the above sections, we used standard full control (FC)models to describe a simplified model with sensing delays.These results also apply to the dual state feedback (SF)case with actuation delays; both produce IFP connections,as shown in Fig. 7. An additional model to consider fromstandard controls is output feedback (OF), which inherentlycontains IFP whether or not delays are present (see “IFP-Sense-2”, “IFP-State”, and “IFP-Act-2” in Fig. 10). We canuse additional fictitious delay states to introduce sensingand actuation delay into an OF controller, which introducesadditional sources of IFP that look similar to our IFP fromFC and SF. Together with the original IFP present in OF,these comprise the main sources of IFP that can be modelledusing standard control models. These are shown in Fig.10. We refer to them as ‘Pre-SLS’ sources of IFP; in ourcompanion paper [2], we use SLS to produce IFP modelswith greater physiological fidelity.

VI. DISCUSSION

The results of this paper show that dramatic diversity-enabled sweet spots in systems with diverse sensing and ac-tuation can exist, where two poorly performing componentsseparately create a well-functioning system when combined.The results also show the importance of internal feedbackwithin delayed components, where the system performancebecomes unstable without the correct feedback information.These theoretical considerations can be applied to sensori-motor control and to other complex structures that carry outlarge computations.

A. Experimental Predictions

New theory, motivated by biology and cyber-physicalsystems and enabled by System Level Synthesis, allows us tostudy the internal structure of complex control systems withconstrained hardware. By computing optimal controllers withhardware (SLD+) constraints, we can design increasingly

7

Fig. 10. All IFP sources available to standard ‘Pre-SLS’ controllers: fullcontrol (FC), state feedback (SF), and output feedback (OF). IFP-Sense-1 isIFP induced by delayed sensing, and resembles IFP from the pure FC model(Fig. 7a). IFP-Act-1 is IFP induced by delayed actuation, and resemblesIFP from the pure SF model (Fig. 7b). These two IFPs have dimensiondetermined by the magnitude of delay (i.e. number of steps) multipliedby number of physical states. The remaining three IFPs are intrinsic tothe OF Kalman filter, and are comparatively smaller in size; IFP-State andIFP-Sense-2 have dimension equal to the number of physical states, whileIFP-Act-1 has dimension equal to the number of physical actuators.

complicated architectures for which we can precisely explainthe function of each component. These optimal controllersmay not perfectly match neurobiology, but will lead tomore refined experiments and constraints. Distinctly frommodeling and simulation, but consistent with traditions incontrol systems engineering, our “models” are constraints:from the top down on behavior and performance, includinguncertainty in the environment, and from the bottom up onhardware components, as in Fig 2. This means comparisonscan be made with all layers and levels of experimental datain a way that conventional modeling and simulation can’t,and makes connections between levels, layers, tradeoffs, andbehavior more transparent. This kind of iteration betweenexperiments and both simulation and theory is essentialin engineering, but rarer and more challenging in biology.To illustrate the applicability of theory in this setting, wepropose concrete experiments below. While many of thetheoretical concepts are expected to be general across organ-isms, biological experiments necessarily require engagementwith details of a specific model system.

Neuronal circuits for escape behaviors in the fruit flyDrosophila melanogaster have long been important modelsfor unraveling the neural control of behavior because theycontain experimentally tractable large axons, specialized forrapid conduction, and hard-wired dedicated circuits with fewsynapses from sensor to motor that drive critical escapeactions [16]. This has led to a model that behavior iscontrolled by dedicated feedforward pathways, with mutualinhibition to arbitrate between them [17]. This view is toosimple, as such a system could not be robust to the widerange of attack scenarios that real animals encounter. Re-

cently, dynamical frameworks have been applied to complexbehaviors, for example, replacing processing in individualneurons with attractor states of neural population activity[17], [18]. However, what is still missing from our current bi-ological understanding is how nervous systems make flexibledecisions at the component level, and in particular, how IFP,not just feedforward processing, from many different parts ofthe nervous system may be integrated to achieve behavioralgoals. We expect that the theory described in this paperand the companion paper [2] will surmount these currentgaps in understanding of action selection and sensorimotortransformations during naturally-occurring behaviors.

Concretely, a unique descending neuron specialized forrapid conduction, the large-axon Giant Fiber, integrates inputfrom these two looming cell types and projects to the nervecord where it synapses directly on the motor neuron thatdrives the “jump” muscle. The timing of a single spike inthe Giant Fiber determines whether the fly will choose speed(fast takeoff) or accuracy (slow takeoff) in its escape [19]–[21]. Thus in adult flies we have identified a feedforwardpathway from photoreceptor to muscle underlying rapidescape choice, for which we know the identity of each neuronand, at least conceptually, the computation performed acrosseach synapse. This pathway participates in a multi-layeredvisual processing system in which component diversity (axondiameter) is a prominent feature. The theory described aboveand in [2] offers an initial theoretical explanation not onlyfor what the Giant Fiber system should do but for whatinformation must be carried by IFPs to the dendrites of thethe Giant Fiber in order to make rapid responses possible.In larvae, this leads to the more speculative prediction thatpreviously unidentified feedback circuits and subsytsemsmediate escape behaviors.

Central to the goal of uniting biological mechanisms withstructured control components (derived as here through mod-ified LQR theory or through SLS) is our ability to measurein complete and exquisite detail the neuronal ‘hardware’ thatunderlies specific behaviors, i.e. the map of how neuronsconnect with each other. Until recently, mapping neuronalconnections completely and comprehensively was difficult.However, the field of large volume serial electron microscopy(EM), the gold standard for identifying and quantifyingneuronal connections, has recently undergone a renaissance.This renaissance is enabled by the automation of ultra-thin sectioning of brains, fast EM imaging (e.g., gigahertzimaging rates), and machine learning algorithms for tracingneurons and connections comparable to human annotators.These data sets are publicly available and a large communityhas formed around shared annotation and scientific discoveryleveraging this data. These data sets are a potential avenueto test specific hypotheses about circuits, neuronal morphol-ogy, or the cellular composition of the neurons involved.Data about the cell biological level is critically relevant tobehavior, but has not yet been incorporated into models thatexplain behavior – in part because of limitations of traditionaltheory.

8

REFERENCES

[1] Y. Nakahira, Q. Liu, T. J. Sejnowski, and J. C. Doyle, “Diversity-enabled sweet spots in layered architectures and speed–accuracy trade-offs in sensorimotor control,” Proceedings of the National Academy ofSciences of the United States of America, vol. 118, no. 22, pp. 1–11,2021.

[2] J. S. Li and J. C. Doyle, “Locality, delays, and internal feedback insensorimotor control part 2: System level synthesis and architecture,”2021.

[3] J. Anderson, J. C. Doyle, S. H. Low, and N. Matni, “System levelsynthesis,” Annual Reviews in Control, vol. 47, pp. 364–393, 2019.

[4] Q. Liu, Y. Nakahira, A. Mohideen, A. Dai, S. Choi, A. Pan, D. M.Ho, and J. C. Doyle, “Experimental and educational platforms forstudying architecture and tradeoffs in human sensorimotor control,” inProceedings of the American Control Conference, 2019, pp. 483–488.

[5] C. Manning, S. Tolhurst, and P. Bawa, “Proprioceptive reaction timesand long-latency reflexes in humans,” Experimental brain research,vol. 221, pp. 155–66, Jul 2012.

[6] A. Jain, R. Bansal, A. Kumar, and K. Singh, “A comparative study ofvisual and auditory reaction times on the basis of gender and physicalactivity levels of medical first year students,” International Journal ofApplied and Basic Medical Research, vol. 5, pp. 124–127, 2015.

[7] F. R. Sarlegna and R. L. Sainburg, “The roles of vision and propriocep-tion in the planning of reaching movements,” Advances in experimentalmedicine and biology, vol. 629, pp. 317–335, 2009.

[8] D. J. Felleman and D. C. V. Essen, “Distributed hierarchical processingin the primate cerebral cortex,” Cereb Cortex, pp. 1–47, 1991.

[9] E. M. Callaway, “Feedforward, feedback and inhibitory connections inprimate visual cortex,” Neural Networks, vol. 17, no. 5-6, pp. 625–632,2004.

[10] L. Muckli and L. S. Petro, “Network interactions: non-geniculate inputto V1,” Current Opinion in Neurobiology, vol. 23, no. 2, pp. 195–201,2013.

[11] D. Busse, M. de la Rosa, K. Hobiger, K. Thurley, M. Flossdorf,A. Scheffold, and T. Hofer, “Competing feedback loops shape il-2signaling between helper and regulatory t lymphocytes in cellular mi-croenvironments,” Proceedings of the National Academy of Sciences,vol. 107, no. 7, p. 3058–3063, Jan 2010.

[12] J. M. Lund, L. Hsing, T. T. Pham, and A. Y. Rudensky, “Coordinationof early protective immunity to viral infection by regulatory t cells,”Science, vol. 320, no. 5880, pp. 1220–1224, May 2008.

[13] N. W. Palm and R. Medzhitov, “Not so fast: adaptive suppression ofinnate immunity,” Nature Medicine, vol. 13, no. 10, pp. 1142–1144,Oct 2007.

[14] S. Musall, M. T. Kaufman, A. L. Juavinett, S. Gluf, and A. K.Churchland, “Single-trial neural dynamics are dominated by richlyvaried movements,” Nature Neuroscience, vol. 22, no. 10, pp. 1677–1686, 2019.

[15] C. Stringer, M. Pachitariu, N. Steinmetz, C. B. Reddy, M. Carandini,and K. D. Harris, “Spontaneous behaviors drive multidimensional,brainwide activity,” Science, vol. 364, 2019.

[16] T. J. Carew, “Startle circuits: Neural mechanisms of startle behavior,”Science, vol. 229, no. 4709, pp. 158–159, 1985.

[17] W. B. Kristan, “Neuronal decision-making circuits,” Current Biology,vol. 18, no. 19, pp. R928–R932, 2008.

[18] X.-J. Wang, “Probabilistic decision making by slow reverberation incortical circuits,” Neuron, vol. 36, no. 5, pp. 955–968, 2002.

[19] C. R. von Reyn, A. Nern, W. R. Williamson, P. Breads, M. Wu,S. Namiki, and G. M. Card, “Feature integration drives probabilisticbehavior in the drosophila escape response,” Neuron, vol. 94, no. 6,pp. 1190–1204.e6, 2017.

[20] J. M. Ache, J. Polsky, S. Alghailani, R. Parekh, P. Breads, M. Y.Peek, D. D. Bock, C. R. von Reyn, and G. M. Card, “Neural basisfor looming size and velocity encoding in the drosophila giant fiberescape pathway,” Current Biology, vol. 29, no. 6, pp. 1073–1081.e4,2019.

[21] C. R. von Reyn, P. Breads, M. Y. Peek, G. Z. Zheng, W. R. Williamson,A. L. Yee, A. Leonardo, and G. M. Card, “A spike-timing mechanismfor action selection,” Nature Neuroscience, vol. 17, no. 7, pp. 962–970,2014.

9