Modelling, calibration and correction of nonlinear

illumination-dependent fixed pattern noise in logarithmic

CMOS image sensors

Dileepan Joseph and Steve Collins

Department of Engineering Science

University of Oxford, England

May 21-23, 2001 IMTC, Budapest (IEEE) 2

Outline

• Logarithmic image sensors

• Pixel modelling

• Fixed pattern noise (FPN)

• Sensor calibration

• Image correction

• Summary and conclusions

May 21-23, 2001 IMTC, Budapest (IEEE) 3

Logarithmic image sensors

• CMOS versus CCD image sensors– Electronics placed on same die as pixels– Cost, power consumption, size, weight– Quantum efficiency, yield, price pressure

• Logarithmic versus linear pixels– Continuous sensing, random access– High dynamic range, low bit rate– Resolution, sensitivity, frame rate

May 21-23, 2001 IMTC, Budapest (IEEE) 4

Pixel modelling

• Physical model • Abstract model

y)uncertaint (noise,error

leakage) e(photodiod bias

slope) old(subthreshgain

voltages)(thresholdoffset

response

eilluminanc

ln

c

b

a

y

x

xcbay

May 21-23, 2001 IMTC, Budapest (IEEE) 5

Fixed pattern noise (FPN)

• Offset (aj) variation (1 j N pixels)

• Offset and gain (bj) variation

• Offset, gain and bias (cj) variation

ijij xcbay lnˆ

ijjij xcbay lnˆ

ijjjij xcbay lnˆ

May 21-23, 2001 IMTC, Budapest (IEEE) 6

Sensor calibration

• Calibrate the sensor with images yij of M uniform illuminances xi (e.g. white paper)

• Extract parameters by minimising the mean square error

M

i

N

jijij yy

MNMSE

1 1

2ˆ1

May 21-23, 2001 IMTC, Budapest (IEEE) 7

Sensor calibration cont’d

• Calibration error is 3.9, 1.9 and 0.9 for one, two and three parameter models

• Calibration error versus illuminance differs markedly

M

i

N

jijij yy

DOF 1 1

22 ˆ1

May 21-23, 2001 IMTC, Budapest (IEEE) 8

Image correction

• Left to right: original plus one, two and three parameter FPN corrected images

• Top to bottom: apertures of 1.8, 4, 8 and 16 f-stops

• Inter-scene plus intra-scene dynamic range equals 67 dB

May 21-23, 2001 IMTC, Budapest (IEEE) 9

Image correction cont’d

May 21-23, 2001 IMTC, Budapest (IEEE) 10

Summary and conclusions

• Physical and abstract pixel model– Offset, gain, bias and error

• Parameter variation causes FPN– Calibration with uniform illuminance– Results indicate FPN is nonlinear

• FPN correction is necessary– Digital correction of images– More robust analogue circuits

May 21-23, 2001 IMTC, Budapest (IEEE) 11

Acknowledgements

• Many thanks to the Natural Sciences and Engineering Research Council of Canada and the Engineering and Physical Sciences Research Council of Britain for their generous support.