Colouran algorithmic approach

Thomas Bangertthomas.bangert@qmul.ac.uk

http://www.eecs.qmul.ac.uk/~tb300/pub/PhD/ColourVision2.pptx

PhD Research Topic

understanding how natural visual systems process information

Visual system: • about 30% of cortex• most studied part of

brain• best understood part

of brain

This research is abut the information produced by the early visual system Information which goes from front of

brain to higher levels at rear of brain

Image sensors Binary sensor array

monochromatic ‘external retina’

Luminance sensor arraydichromatic colour

Multi-Spectral sensor arraytetrachromatic colour

Sensors: what is measured and what information is sent?

What is Colour?

Visible Spectrum

Visual system must measure and represent light within this zone.

We start with Luminance – how bright?(we measure how much light)

What information does colour add?How do we code this information.

the stimulus

Any ideas?

Lets hypothesise … When an astronomer looks at a star, how does he code the information his sensors produce?

It was noticed that parts of spectrum were missing.

Looking our own star – the sun

• x

Each atomic element absorbs at specific frequencies …

We can Code for these elements …

We can imagine how coding spectral element lines could be used for visual perception … by a creature very different to us… a creature which hunts by ‘tasting’ the light we reflect… seeing the stuff we are made of

Colour in this case means atomic structure and chemistry…

Sensor we build cannot do spectral analysis

Crude system to reproduce colour

3 colour values, usually 8-bit: RGB

What does RGB mean? lights for colour reproduction

The Standard Observer

CIE1931 xy chromaticity diagramprimaries at: 435.8nm, 546.1nm, 700nm

XYZ – a 3 sensors model of human vision

xxx y z

1 central luminance sensor: Yand colour information are 2 difference measurements ... from YThe Math:

yyx y z

… z is redundant

Understanding CIE chromaticity

White in center

Saturated / monochromatic colours on the periphery

Best understood as a failed colour circle

Everything in between is a mix of white and the colour

xxx y z

yyx y z

Does it match?The problem of

‘negative primaries’

But does it blend?

Monochromatic Colours

The Human Visual System (HVS) does things differently!

?

Human Visual

System (HVS)

Coding Colour

The Sensor2 systems: day-sensor & night-sensor

To simplify: we ignore night sensor system

Cone Sensors very similar to RGB sensors we design for cameras

sensor array

arrangement is random

note:very few blue sensors, none in the centre

sensor pre-processing circuitry

First Question: What information is sent from sensor array

to visual system?

Very clear division between sensor & pre-processing (Front of Brain) andvisual system (Back of Brain) connected with very limited communication link

starting with the sensor:Human Sensor Response

to non-chromatic light stimuli

350 400 450 500 550 600 650 7000

10

20

30

40

50

60

70

80

90

100

Wavelength (nm)

Abso

rptio

n (%

)

RGB

HVS Luminance Sensor IdealizedSe

nsor

Valu

e

Wavelength(λ)λ

0.8

0.6

0.2

0.0

1.0

0.4

λδλ−

A linear response in relation to wavelength.Under ideal conditions can be used to measure wavelength.

Spatially Opponent

HVS:Luminance is always measured by taking the difference between two sensor values.Produces: contrast value

Sens

or V

alue

Wavelength(λ)λ

0.8

0.6

0.2

0.0

1.0

0.4

λδλ−

Sens

or V

alue

Wavelength(λ)λ

0.8

0.6

0.2

0.0

1.0

0.4

λδλ−

 

Which is done twice, to get a signed contrast value

Colour Sensorresponse to monochromatic light

350 400 450 500 550 600 650 7000

10

20

30

40

50

60

70

80

90

100

Wavelength (nm)

Abso

rptio

n (%

)

RGB

370 nm 445 nm 508 nm 565 nm

700 nm330 nm 400 nm 500 nm 600 nm

1.0

0.5

0.0

Human

Bird

4 sensorsEquidistant on spectrum

if we make a simplifying assumption:our light is monochromatic!

 Then:

Wavelength

0.8

0.6

0.2

0.0

1.0

0.4

λ-Δ λ λ+Δ

RG

a shift of Δfrom a known reference point

the ideal light stimulusSe

nsor

Val

ue

Wavelength

0.8

0.6

0.2

0.0

1.0

0.4

λ-δ λ λ+δ

RG Monochromatic Light

Allows frequency to be measured in relation to reference.

Problem:natural light is not ideal

Sens

or V

alue

Wavelength

0.8

0.6

0.2

0.0

1.0

0.4

λ-δ λ λ+δ

RG

• Light stimulus might not activate reference sensor fully.

• Light stimulus might not be fully monochromatic.

ie. there might be white mixed in

Sens

or V

alue

Wavelength(λ, in nm)400300 430 460 490 520 550 580 610 640 670 700

0.8

0.6

0.2

0.0

1.0

0.4

Solution:

A 3rd sensor is used to measure equiluminance.

Which is subtracted.

Then reference sensor can be normalized

a 4 sensor designSe

nsor

Val

ue

Wavelength(λ, in nm)400300 430 460 490 520 550 580 610 640 670 700

0.8

0.6

0.2

0.0

1.0

0.4

2 opponent pairs• only 1 of each pair can be active• min sensor is equiluminance

,R G B y

What is Colour?What is the information?• Luminance• Equi-Luminance• Colour

Colour channels are: RGByellow

4 primaries.

Purpose of Colour is to code wavelength!

Information = Luminance + Wavelength

Any Stimuli can be reduced to:

Equi-Luminance Location on Spectrum Luminance

Complex Spectrum is reduced to very simple equivalent

Colour often involves further high level processing …Examples of real world colour:

Colours are often computed, not measured!

… an extreme example

What is the colour?

http://www.eecs.qmul.ac.uk/~tb300/pub/PhD/ColourVision2.pptx

ReferencesPoynton, C. A. (1995). “Poynton’s Color FAQ”, electronic preprint.http://www.poynton.com/notes/colour_and_gamma/ColorFAQ.html

Bangert, Thomas (2008). “TriangleVision: A Toy Visual System”, ICANN 2008.

Goldsmith, Timothy H. (July 2006). “What birds see”. Scientific American: 69–75.

Neitz, Jay; Neitz, Maureen. (August 2008). “Colour Vision: The Wonder of Hue”. Current Biology 18(16): R700-r702.

Questions?