mandyam srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › l8... ·...

37
Queensland Brain Institute and Institute of Electrical and Electronic Engineering University of Queensland and ARC Centre of Excellence in Vision Science Mandyam Srinivasan Vision and navigation in bees and birds and applications to robotics

Upload: others

Post on 31-May-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Queensland Brain Institute and Institute of Electrical and Electronic

Engineering University of Queensland

and ARC Centre of Excellence in Vision Science

Mandyam Srinivasan

Vision and navigation in bees and birds and applications to robotics

Page 2: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Sam Baker, Daniel Bland, Natalie Bland, Nikolai Liebsch, Richard Moore, Gavin Taylor, Saul Thurrowgood, Dean Soccol

Page 3: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Peculiarities of insect vision

Small interocular separation

Therefore, stereo vision is difficult

Insects rely heavily on image motion cues to infer object distance, perceive the world in 3-D and navigate in it

Page 4: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Dr. Miriam Lehrer University of Zürich

Page 5: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Kirchner & Srinivasan Naturwissenschaften (1988)

Srinivasan, Lehrer, Kirchner & Zhang Vis. Neurosci. (1991)

Bees negotiate narrow gaps by balancing the image velocities in the two eyes

Page 6: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Centering response in budgerigars P. Bhagavatula, C. Claudianos, M. Ibbotson, M.V. Srinivasan Current Biology (2011)

N.S.

Page 7: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Visual control of flight speed - bees

~300 deg/sec

Baird, Srinivasan, Zhang and Cowley, J Exp Biol (2005)

Page 8: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Speed of flight is regulated by holding the global image velocity constant

Control of flight speed

Srinivasan, Zhang, Lehrer & Collett J. Exp. Biol. (1996)

Page 9: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

0

0.1

0.2

0.4

0.3

0.5

Battling headwind

Barron and Srinivasan, J. Exp. Biol. (2006)

0 40 80 120 160

0.9 m/s 1.6 m/s 2.8 m/s 3.8 m/s

Ground speed m/s

Volts

Headwind speed

Headwind

Page 10: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Landing

How does a bee perform a smooth, grazing landing ?

Page 11: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

h

(height above surface)

(Forward speed)

(Descent speed) V d

V f !=Tan-1"

V f

V d

!"(Descent angle)

Landing parameters

Page 12: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Horizontal flight speed versus height

Srinivasan, Zhang, Chahl, Barth & Venkatesh, Biol. Cybern (2000)

Page 13: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec
Page 14: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Landing on a vertical surface

Page 15: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

With Emily Baird, Norbert Boeddeker, Michael Ibbotson

Page 16: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

0 100 200 300 400 500 600 700-300

-200

-100

0

100

200

300

400

Time (t), msec

Posi

tion

(Z),

mm

S61

Z=-273.48+519.7*exp(-0.0028*t)

Stat 6 arms

11 flights

Distance decreases exponentially as a function of time, suggesting that approach speed is controlled by holding rate of image expansion constant

Honeybee landing

Baird, Boeddeker, Ibbotson, Srinivasan: PNAS (in press)

Page 17: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

0 200 400 600 800 1000 1200-250

-200

-150

-100

-50

0

50

100

150

200

250Expanding 1 rpsContracting 1rpsStatic

Posi

tion

(Z),

mm

Time (t), msec

Test of hypothesis: Expanding spiral reduces approach speed; contracting spiral increases it

Baird, Boeddeker, Ibbotson, Srinivasan: PNAS (in press)

Page 18: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

200 deg/sec

Constant approach speed

Baird, Boeddeker, Ibbotson, Srinivasan: PNAS (in press)

Page 19: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

The waggle dance

Honeybee odometry

Wag

gle

dura

tion

(sec

)

Q: How does a bee work out how far she has flown?

Karl von Frisch

Page 20: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec
Page 21: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Round dance

(feeder distance < 50m)

Waggle dance

(feeder distance > 50m)

Page 22: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Honeybee navigation: How does a bee estimate how far it has flown?

Page 23: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

6 m

Feeder

6 m

Feeder

Hive

Hive

Feeder Hive

6 m

Dance signal

~ 0 m

~ 200 m

Round dance

Waggle dance

Round dance ~ 0 m

Srinivasan, Zhang, Altwein & Tautz, Science (2000) Esch, Zhang, Srinivasan & Tautz Nature (2001)

Page 24: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

1 msec of waggle corresponds to 18o of image movement in the eye during flight

18o

Visual calibration of the honeybee’s odometer and dance

Srinivasan, Zhang, Altwein & Tautz, Science (2000)

Page 25: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Scouts returning from 8 m tunnel signal a distance of 140 m

Recruits search at 140 m

8 m

How do the recruits respond to the dancing tunnel bees?

Esch, Zhang, Srinivasan & Tautz Nature (2001)

Page 26: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Image courtesy Jeff Wilson and Judith Reinhard

Honeybee perception and ‘cognition’

Page 27: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Maze learning

Page 28: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec
Page 29: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Maze navigation: following a colour mark

Page 30: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec
Page 31: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec
Page 32: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Music in the wind

Bev Doolittle

Page 33: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Rewarded Unrewarded 49.0% 51.0%

63.1% 36.9%

61.2% 38.8%

(n=1099, P > 0.50

(n=141, P < 0.005)

(n=509, P < 0.001) 62.6% 37.4%

(n=238, P < 0.001)

“Top-down” processing in honeybees

Zhang & Srinivasan Nature (1994)

Page 34: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

Associative recall

Page 35: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

2m

Rose-scented feeder

Hive

21 individually marked bees

Lemon-scented feeder

Training paradigm

Can scent trigger memories of locations?

Reinhard, Srinivasan and Zhang, Nature (2004)

Page 36: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec

2m

Former “rose” feeder

Former “lemon” feeder

rose scent

4

41 40

5

Hive

21 individually marked bees

lemon scent

Test

Can scent trigger memories of locations?

Reinhard, Srinivasan and Zhang, Nature (2004)

Page 37: Mandyam Srinivasanrobotics.itee.uq.edu.au › ~metr4202 › 2014 › lectures › L8... · 2014-10-04 · 0 100 200 300 400 500 600 700-300-200-100 0 100 200 300 400 Time (t), msec