functional models of neurobiological processes

Marh[ Compur. Mode//kg, Vol. 14, pp. 317-321, 1990 Printed in Great Britain

0895-7177/90 %3.00 + 0.00 Pergamon Press plc

FUNCTIONAL MODELS OF NEUROBIOLOGICAL PROCESSES

William P. Coleman MIEMSS, UMAB; Baltimore, MD 21201 USA,

Department of Mathematics & Statistics, UMBC; Baltimore, MD 21226 USA,

and St. John’s College; Annapolis, MD 21404 USA

David P. Sanford

Aeronautical Radio, Inc.; 2551 Riva Road, Annapolis, MD 21401 USA

Andrea De Gaetano

C.N.R. Centro di Studio per la Fisiopatologia dello Shock, Istituto di Clinica Chirurgica, Universitl Cattolica de1 Sacro Cuore; Via Pineta Sacchetti, 644, I-00168 Roma, Italia

Fred Geisler

Division of Neurosurgery, UMAB; Baltimore, MD 21201 USA,

and Department of Neurosurgery, Patuxent Medical Group; Columbia, MD 21045 USA

Abstract. This paper uses avery general theory of computation to develop a set of working hypotheses and an accompanying mathematical formalism that model the semantics and structure of cognitive processes. We can think of the brain a8 a modular system of neural networks, each producing a sequence of output states in response to input states furnished by others, and sometimes by the outside world. We use category theory to specify, and reason about, a model consisting of an appropriately connected set of functions.

Keywords. Neurons; Neural Nets; Cognition; Sense Perception; Category Theory; Logic.

INTRODUCTION

This paper is an interim report in our continuing effort [5] [6] [4] to use a very general theory of computation, called functional logic, to develop a set of working hypotheses and an accompanying mathematical formalism that model the semantics and structure of cognitive processes. We use category theory to specify, and reason about, a model consisting of an appropriately connected set of functions.

In our model, we have tried to incorporate certain general architectural features.

0 Using fast-slow approximations familiar from non- linear mathematics, we can think of the brain aa a modular system of neural networks, each sub- network producing a sequence of output states in response to input states furnished by others, and sometimes by the outside world.

. Accordingly, the network can be viewed with different degrees of granularity. We should be able to see the “same” computation taking place simply, or as a feature of a more complex computation.

. The architecture should be layered. Semantically, the lowest layers refer to the outside world, while the higher layers refer to lower layers.

One would hope to use these structures in such a way as to be consistent both with neurobiology and with psychology: this is not intended to be strictly a mathe- matician’s model.

FUNCTIONAL ORGANIZATION

We try to analyze the structure of computations in the network by using algebraic techniques to represent the logic of the functions it can compute. We illustrate this by the simplified network N shown in figure 1. U has three neurons: f. g and h. The axon of neuron f termi- nates in a set b of synapses to g and a set d of synapses to h. Similarly, neuron g has a set e of synapses to h. There is a set of synapses a to f, and a set c to g.

Each set of synapses can assume various states, depend- ing on activity in the presynaptic neuron. We call such a set of states a type. For example, in ,h/, the set of synapses a can assume states in the type a, and the other sets of states similarly correspond to types. In

317

318 Proc. 7th Int. Conf: on Mathematical and Computer Modelling

this light, we see that f computes a function

f :a+bxc,

while g computes a function

g:bxc+e.

At a coarser level of granularity, f and g can be lumped together and considered as a subnet [f.g] that computes a function

lf~!Jl : axc+dxe.

Thus, at any level of granularity, we regard a subnet as a black box for computing output states from input states, and we use the same principles to analyze the logic of the network - the system of functions that it is capable of computing.

There is a third way of looking at the functions computed by N. Given a fixed b E 6, g computes a function

gb : c -+ e,

given by

5’b(c) = g(b,c). (1)

Thus there is an ambiguity between the use of b as the name of a state of the synapses b and as the name of the function gb computed by g. This tends to confuse us when we try to analyze the semantics of the network. To say that an object is rectangular, for example, is to

Figure 1: A simple network N tions it computes.

f :a+bxc,

g:bxc+e.

(f,g]:axc--tdxe.

For fixed b E 6, gb : c + e,

gb(c) = g(b,c).

of neurons and the func-

ascribe to it a function that, together with some functions that describe the relative spatial position of the object, computes the apparent shape as seen by the ob- server. The apparent shape, probably some trapezoid, can be any of a repertory of lower level functions, one of them confusingly also called “rectangular.” The effect of the output of any subnet is to modify the function computed by the other subnets to which it is connected.

ANALYTICAL METHOD

We can state our general strategy in light of the pre- ceding discussion of functional organization. We take it as given that a neuron or a subnet can, in virtue of its anatomical configuration, compute an output as a function of its inputs, and we do not inquire closely into the biophysics of how this happens. Rather, we are concerned to analyze the logical organization of the network. We are concerned with syntax - Starting with given inputs, what sequences of computations are possible? - and with semantics - How do these computations have cognitive meaning?

Functional logic [3] is a very general theory that at- tempts to analyse the logic of natural and computational processes. The point of view of this article, that we are studying a system of function computations and their compositions, is a prototypical example. There is an extended review, with many examples, in [2]. Func-

tional logic is based on the mathematical theory of cat-

egories (121, discovered by EILENBERG and MAC LANE,

which has been used, particularly in work deriving from

LAWVERE, both explicitly [ll] and implicitly [1], to pro-

vide a foundation for logic. (There is an accessible pre-

sentation in [9].) The automata theory books [8] [7] are

also very relevant to the present work.

CATEGORIES

A category is a system of objects and arrows that obeys certain rules. For each object e, there is an identity

function i, :c+c,andforarrows/:c+dandg:b+c, there is the composition (f o g) : b --) d. We have

and

idof=f=fOi,, (21

f 0 (9 0 h) = (f 0 9) oh, (3)

for any h : a -+ 6. An arrow goes from its domain to its

codomain.

A category might be just a suitable system of letters and arrows. For example, start with the diagram,

f h ZZYY%

!?

and add enough arrows to make it a category. That is, . . add in the identItles t., i, and i, and also all the com-

positions g o f, f o g, g 0 f 0 g, h 0 f 0 g and so on. This category C is the category generated by the diagram.

Proc. 7th Int. Conf. on Mathematical and Computer Modelling 319

The original arrows in the diagram are the basis arrows of the category. We often indicate a category just by showing a diagram that generates it. An different example is Set, the category of sets. Its objects are sets, and its arrows are ordinary functions between sets.

A ]unctor is a mapping between categories, carrying objects and arrows in one category correspondingly to objects and arrows in the other. Structure must be pre- served:

F(i,) = iF(,), and (4)

F(f 0 9) = F(f) 0 F(s), (5)

for composable f and g. For example, let D be generated by the diagram,

m z----,3

n

There is is a functor F from D to the category C discussed above that sends i H 5 and j H z, and that sendsm- (ho’f) andn++ (hofogof).

SCHEMATA

In a neural network, we have a system of types, which are sets of synaptic states, and the functions that the subnets compute between types. We can now see these types and functions as a fragment of Set, the category of sets and functions. Further, we can use functors to show how a particular logical structure might be embedded in such a system.

We begin by looking at a neural network. Let C be a network with three neurons f, g, and h. The possible inputs to f are a, b and c, which constitute the type X = {a, b,c}. The outputs of f are in the type Y = {p,q,r}. The neuron g maps Y to X, and h maps Y to 2 = {s, t}. These functions are shown in the diagram,

The actions of f, g and h are given by the tables,

x E x f(x) E Y

b” P

-1 ~~~~~~~

q '

C I-

YE y g(y) 6 x h(Y) E z

P b 8

9 b t . I- b 8

There is an obvious functor C from the category C, discussed above, to Set, taking zr to X and so on. It models the neural network C by showing how a certain logical structure, that of C, takes on form in a certain set of types. We call such a functor a schema.

The network D is simpler. It has only two types, I = {d,e} and J = {u,v}, and two neurons m and n,

m

Z-t J

The functions they compute are given by the table,

i~J~rn(i~~J n(i)EJ v .

V

As before, there is an obvious schema D : D --( Set that models the network D.

MODEL THEORY

Hidden inside of network C is a copy of the structure of network D. To see this, one thinks of the states of C as relabelings of states of D. Let 4 be the function that does this: it maps X to I by +(a) = 4(c) = d, q%(b) = e, and it maps Z to J by d(s) = u, 4(t) = v. With this relabeling, the process h o f of C acts on X in the same way that the process m of D acts on I:

m(6(x)) = 4((h 0 j)(x)), for x E X. (6)

Thus h o f covers m. The same relabeling also allows h o f o g o f to cover n. More intuitively: to discover the effect of a process in D on a state i E I, find a state x E X that i labels, do a process covering it in C, and then relabel again to find the result in J.

C is in every way more complex than D - it has more types, more states and more subnets -, yet contained in it is a copy of D that does not mislead us as long as we interpret it consistently. This notion of “covering” points out a way in which we can understand how a simplified, and more abstract, mental process can sym- bolically represent a complicated, and more concrete, one, or how a mental process can represent a physical process in the world outside the mind. The methods just described are a step towards a semantic interpretation of D in C.

If we replace the networks C and D by their corresponding schemata, we see that there are two parts to the notion of “covering.” First, there is the functor F : D + C, the same one discussed above, that shows how the logic of the two systems is related. Second, for each type of D there is function, coming from the corresponding type of C, that shows how the states are to be relabeled. In category theory, such a system of functions is a natural transformation,

0. : (C o F) -L D.

It is a way of converting the functor C o F : D -+ Set into the functor D : D + Set between the same categories.

Formally, a model M = (F,a) of D : D -+ Set in C : C 4 Set is given by a functor F : D --t C, and a natural transformation a : (C 0 F) -h C’,

320 Proc. 7th Int. Conf: on Mathematical and Computer Modelling

C C -- Set

t F

I

a

D B Set D

We say that C realizes D, or that D represents C.

DEVELOPMENTS AND PARTICLES

Next, we have to distinguish between a schema, which is a template according to which certain processes might occur, and an instance in which such a process actually does occur. A development of the schema C : C --t Set

is a functor P : P -+ C, where P is a finite totally ordered category, that maps basis arrows of P to basis arrows of C. Such a development induces a schema (C 0 P) : P + Set.

For example, the categories P,

and Q,

91 fl hl Yl c x1 --A y2 --4 t1

can both be made into developments of C. The functors P and Q are the obvious ones that just forget the subscripts.

It is evident that a development of C corresponds to a path of C. As any instance, or particle, of the schema unfolds, it corresponds to successively longer developments.

CONCEPTUAL SCHEMATA

Usually, something that we would want to dignify by the term “concept” is more complex than just a schema. It is, rather, a system of schemata that are given a com- mon interpretation by representation in a single category. Thus, even a humble concept like the letter “F” is a system of horizontal and vertical bars that are them- selves made up of pixels in a cortical map. “Horizontal bar” and “vertical bar” are two schemata whose types are sets of pixels. “F” is also a schema whose types are sets of pixels, but it is specified by requiring that its states be made up of instances of horizontal and vertical bars, with certain required incidence relations.

A conceptual schema is a schema C : C -+ Set together with a system {C, : Ci + C} of functors to C. Each of the Ci has a subschema Co C,. A development of a con-

ceptual schema is a development of one of the schemata (C 0 Cj) 1 Ci -+ Set.

SEMANTICS

Some cognitive processes describe natural processes outside the mind. Some cognitive processes describe other cognitive processes. We use the notion of ‘model’ to capture how one process can describe another. Let C : C -+ Set and D : D -+ Set be conceptual schemata. An interpretation of D in C is a model M = (F, a), together with some extra data. For each subschema Di of D there must be given a subschema Ci of C and a functor F; : Di + Ci, and there must be a function as- signing to each particle of C’; a particle of C,. We think of the particles of D as linguistic objects corresponding to, and describing, events that are particles of C.

SENSORY PROCESSING

Using these ideas, we can begin to see a way to ex- press how the influences of sensory data, working from bottom up, and of conceptual schemata, working from top down, are worked out in the neural network. We hypothesize that there is a relatively small number of schemata in the network’s repertory at any one time, and from these can be generated a relatively large number of states, but by no means all the states that would be mathematically possible. The states that are possible are semantically linked by the relationships of the schema processes that can generate them - the brain doesn’t find them by searching, but by using a pattern to generate them. Thus a presented object is recognized as being of a certain sort, i.e. as corresponding to a certain conceptual schema, by the parallel selection of a small number of choices at each level of processing.

There are two kinds of learning. The brain memo- rizes information about specific instances of schemata - facts, physical and mental. These facts are then re- employed as input data for further recognition tasks. When the brain doesn’t have schemata to fit the data at hand, it has to build new ones. The successive replace- ment of schemata, as well as the integration of schemata into successively more powerful representations, constitute a different type of learning.

This approach to cognition can be contrasted with that of artificial intelligence, in that schemata - rules - are not found by sequential search, but are generated by the data that they apply to. Further the state space that has to be searched is small and structured. (The structures can evolve in time through a larger space, but at any one time they are sparse.) On the other hand, this approach can be contrasted with that of non-modular neural networks in that the modularity provides a struc-

ture for the state space, and this structure gives the

states and the processes their semantic meaning. The

meaning of any state is determined by the set of com-

putations into which it can enter.

Proc. 7th Int. Conf. on Mathematical and Computer Mode&

REFERENCES

[l] M. Barr and C. Wells. Toposes, LQiples and Theo-

ries. Springer-Verlag, New York, 1985.

[2] W. P. Coleman. Computational logic of network processes. In M. D. Fraser, editor, Advances in Control Networks and Large Scale Parallel Dis- tributed Processing Models, Ablex Publishing Com- pany, 1989. In press.

131 W. P. Coleman. Models of computational processes. Journal of Symbolic Logic. (Presented at the Spring Meeting 1989 of the Association for Symbolic Logic. Manuscript in progress.).

[4) W. P. Coleman. Semantics of cognitive processes. Cognition, 1989. (Submitted).

[5] W. P. Coleman, D. P. Sanford, A. De Gaetano, and F. Geisler. Logical structure of neurobiological information processing. In Proceedings of the lst

Creut bkes Computer Science Conference, 1080

(in press).

(61 W. P. Coleman, D. P. Sanford, A. De Gaetano, and F. Geisler. Modularity of neural network architecture. In Proceeding8 of the International Joint

Conference on Neural Networka, 1000, Washington

DC (in press), 1990.

[7] S. Eilenberg. Automata, Languages, and Machines. Volume B, Academic Press, New York and London, 1976.

[E] S. Eilenberg. Automata, Languages, and Machines. Volume A, Academic Press, New York and London, 1974.

[Q] R. Goldblatt. Topoi: The Categorial AndySi8 of

Logic. North-Holland Publishing Company, Ams- terdam, revised edition, 1984.

[lo] S. Grossberg, editor. Neural Networks and Natural

Intelligence. MIT Press, Cambridge, MA, 1988.

[ll] J. Lambek and P. J. Scott. Introduction to Higher Order Categorical Logic. Cambridge University Press, Cambridge, 1986.

[12] S. Mac Lane. Categories for the Working Mathe-

matician. Springer-Verlag, New York, 1971.

321

functional models of neurobiological processes

Documents