laurent itti: cs564 - brain theory and artificial intelligence. introduction and brain overview 1...

1Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence

Lecture 1. Introduction and Brain Overview

Reading Assignments:*TMB2: Chapters 1; 2.4

HBTNN:I.1. Introducing the Neuron (Arbib)

* Unless indicated otherwise, the TMB2 material is the required reading, and the other readings supplementary.

2Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

CS 564: Brain Theory and Artificial Intelligence

URL: http://iLab.usc.edu/classes/2002cs564/ for syllabus, instructor and TA information, handouts, homework and grades

Instructor: Laurent Itti; itti@pollux (Office Hour: Mon 3-5,

HNB30A)TA:

Yoo-Hee Shin yooheesh@usc.edu

This course provides a basic understanding of brain function, and of artificial neural networks which provide tools for a new paradigm for adaptive parallel computation.

No background in neuroscience is required, nor is specific programming expertise, but knowledge of C++ will be useful for homeworks.

3Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Texts and Grading

Text: M.A. Arbib, 1989, The Metaphorical Brain 2: Neural Networks and Beyond, Wiley-Interscience.

Supplementary reading: M.A. Arbib, Ed., 1995, The Handbook of Brain Theory and Neural Networks, MIT Press (paperback).

One mid-term and a final will cover the entire contents of the readings so far as well as the lectures.

The final exam will cover all of the course, but emphasizing material not covered in the mid-term.

Distribution of Grades:

Homeworks: 40%; Mid-term: 30%; Final Exam: 30%.

4Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Syllabus Overview

Introduction

Overview

Charting the brain

The Brain as a Network of Neuronsx (t)1

x (t)n

x (t)2

y(t+1)

w1

2

n

w

w

axon

5Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Syllabus Overview

Introduction (cont.)

Experimental techniques

Introduction to Vision

Schemas

6Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Syllabus Overview

Basic Neural Modeling & Adaptive Networks

Didday Model of Winner-Take-All

Hopfield networks

Adaptive networks: Hebbian learning;Perceptrons; landmark learning

E = - ½ ij sisjwij + i sii

7Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Syllabus Overview

Neural Modeling & Adaptive Networks (cont.)

Adaptive networks: gradient descentand backpropagation

Reinforcement learning

Competition and cooperation

Visual plasticity; self-organizing feature maps; Kohonen maps

8Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Syllabus Overview

Examples of Large-scale Neural Modeling

System concepts

Model of saccadic eye movements

Feedback and the spinal cord; mass-spring model of muscle

)(0

)(

)(

00

10

)(

)()(

1tu

tx

tx

tx

txtq

m

PP

SC

vm

sm

ms

FOn

FEF

vm

sm

vs

CD

sm

vs

SNR

vs

ms

qv

TH

qv

LGN

VisCx

wta

Brainstem Saccade Generator

eye movement

RetinaVisInput

delay

FEFvsFEFms

FEFvsFEFms

swit

chPPctr

delay

FOn

9Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Syllabus Overview

Large-scale Neural Models of Vision

Early visual processing

Depth perception

Optic flow

C

A B

D

L R

OA OA

-qmax

-qmax +qmax

+qmax

qRqL

qLB, qLD

q0 q0

qLA, qLC qRB, qRC

qRA, qRD

10Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Syllabus Overview

Large-scale Neural Models of Vision (cont.)

Visual attention

Object recognition

Scene perception

11Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

AIP

F5

dorsal/ventral streams

Task Constraints (F6)

Working Memory (46)

Instruction Stimuli (F2)

Task Constraints (F6)Working Memory (46?)Instruction Stimuli (F2)

AIP

DorsalStream:Affordances

IT

VentralStream:Recognition

Ways to grab this “thing”

“It’s a mug”PFC

Syllabus Overview

Other Advanced Neural Modeling

Reaching, grasping and affordances

Cerebellar adaptation

Memory and consciousnessVisualCortex

ParietalCortex

InferotemporalCortex

How (dorsal)

What (ventral)

reach programming

grasp programming

12Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Syllabus Overview

Applications and Outlook

Towards highly-capable robots

Overview and summary

13Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Three Frameworks

Artificial intelligence (AI): build a “packet of intelligence” into a machine

Cognitive psychology: explain human behavior by interacting processes (schemas) “in the head” but not localized in the brain

Brain Theory: interactions of components of the brain -

- computational neuroscience

- neurologically constrained models: e.g., networks of neurologically localized schemas

and abstracting from them as both Artificial intelligence and Cognitive psychology:

- connectionism: networks of trainable “quasi-neurons” to provide “parallel distributed models” little constrained by neurophysiology

- abstract (computer program or control system) information processing models

14Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

The Aim of the Course

To gain an understanding of biological neurons as the basis for:

Brain Theory: modeling interactions of components of the brain, especially more or less realistic biological neural networks localized in specific brain regions

Connectionism in both Artificial intelligence (AI) and Cognitive psychology: modeling artificial neural networks -- networks of trainable “quasi-neurons” -- to provide “parallel distributed models” of intelligence in humans, animals and machines

This lecture: A tourist’s guide to the brain ;-)

15Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

A motivating theme: Vision

Vision as a progressive change in representation

Marr (1982): through 2 ½ D primal sketch

Because vision is by far the most studied sense (because it is easy to experiment with), we will use it as a basis for many examples of models studied in this course.

16Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Vision and the brain

Roughly speaking, about half ofthe brain is concerned with vision.Although most of it is highly auto-mated and unconscious, vision henceis a major component of brain function.

Ryback et al, 1998

17Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Vision, AI and robots

1940s: beginning of Artificial Intelligence

McCullogh & Pitts, 1942i wixi

Perceptron learning rule (Rosenblatt, 1962)

BackpropagationHopfield networks (1982)Kohonen self-organizing maps…

18Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Vision, AI and Robots

1950s: beginning of computer visionAim: give to machines same or better vision capability as

oursDrive: AI, robotics applications and factory automation

Initially: passive, feedforward, layered and hierarchical process

that was just going to provide input to higher reasoning

processes (from AI)

But soon: realized that could not handle real images

1980s: Active vision: make the system more robust by allowing the

vision to adapt with the ongoing recognition/interpretation

19Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

A tourist’s guide to the brain

Gross anatomy

Non-neural structures

Major cortical areas

20Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Central vs. Peripheral Nervous System

The brain is not the entire nervous systems; there is also the spinal cord, many peripheral “ganglia” (small clusters of neurons), and neurons extend connections to locations all over the body (e.g., sensory neurons, motor neurons).

21Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Autonomic Nervous System

22Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

23Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Axes in the brain

24Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

The “Bauplan” for the Mammalian Brain

25Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Medical Orientation Terms for Slices

26Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Main Arterial Supply to the Brain

27Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Arterial Supply is Segmented

Occlusion/damage to one artery will affect specific brainregions. Important to remember for patient studies.

28Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Ventricular System

Ventricules: Cavities filled with fluid inside and around the brain. One of their functions is to drain garbage out of the brain.

29Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Cortical Lobes

Sulcus (“fissure” if very large): Grooves in folded cortexGyrus: cortex between two sulci

1 sulcus, many sulci; 1 gyrus, many gyri

30Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

31Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

32Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

33Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

34Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

35Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

36Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Neurons

Cell body (soma): where computation takes placeDendrites: input branchesAxon: unique output (but may branch out)Synapse: connection between presynaptic axon and

postsynaptic dendrite (in general).

37Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Electron Micrograph of a Real Neuron

38Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Neurons and Synapses

39Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Grey and White Matters

Grey matter: neurons (cell bodies), at outer surface of brain

White matter: interconnections, inside the brainDeep nuclei: clusters of neurons deep inside the brain

40Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Major Functional Areas

Primary motor: voluntary movementPrimary somatosensory: tactile, pain, pressure, position, temp., mvt.Motor association: coordination of complex movementsSensory association: processing of multisensorial informationPrefrontal: planning, emotion, judgementSpeech center (Broca’s area): speech production and articulationWernicke’s area: comprehen-

sion of speechAuditory: hearingAuditory association: complex

auditory processingVisual: low-level visionVisual association: higher-level

vision

41Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Major Functional Areas

42Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

A View of the Monkey Brain

43Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

44Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

http://www.radiology.wisc.edu/Med_Students/neuroradiology/fmri/

45Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

46Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

47Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

48Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

49Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

50Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

51Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

52Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Limbic System

Cortex “inside” the brain.Involved in emotions, sexual behavior, memory, etc(not very well known)

53Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Major Functional Areas

54Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Visual Input to the Brain

55Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Eye and retina

56Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Human Visual System

57Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Primary Visual Pathway

58Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Layered Organization of Cortex

Cortex is 1 to 5mm-thick, folded at the surface of the brain (grey matter), and organized as 6 superimposed layers.

Layer names:1: Molecular layer2: External granular layer3: External pyramidal layer4: internal granular layer5: Internal pyramidal layer6: Fusiform layer

Basic layer functions:Layers 1/2: connectivityLayer 4: InputLayers 3/5: Pyramidal cell bodiesLayers 5/6: Output

59Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Layered Organization of Cortex

60Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Slice through the thickness of cortex

1

2

3

4

5

6

61Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Columnar Organization

Very general principle in cortex: neurons processing similar “things” are grouped together in small patches, or “columns,” or cortex.

In primary visual cortex… as in higher (object recognition) visual areas…

and in many, non-visual, areas as well (e.g., auditory, motor, sensory, etc).

62Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Retinotopy

Many visual areas are organized as retinotopic maps: locations next to each other in the outside world are represented by neurons close to each other in cortex.Although the topology is thus preserved, the mapping typically is highly non-linear (yielding large deformations in representation).

Stimulus shown on screen… and corresponding activity in cortex!

63Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Retinotopy

64Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Mammalian and Frog Visual Systems

65Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

66Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

67Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Felleman & Van Essen, 1991

Interconnect

68Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Interconnect… (other source)

69Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

More on Connectivity

70Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Frog

Snake

Horse

Primitive Mammal

Catfish

Alligator

Goose

Varieties of Vertebrate Brains

71Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview

Outlook

There is a lot to learn about the brain!… but don’t feel overwhelmed, we will smoothly

introduce all new concepts.

Principled theoretical and engineering methods will allow us to abstract some of these complications.

Starting with fundamental techniques, we will then study fairly complex, large-scale neural models.