Temperature Temperature Dependence of FPN in Dependence of FPN in Logarithmic CMOS Logarithmic CMOS Image SensorsImage Sensors

Dileepan JosephDileepan Joseph¹¹ and Steve and Steve CollinsCollins²²

¹University of Alberta, Canada¹University of Alberta, Canada

²University of Oxford, UK²University of Oxford, UK

May 1–3May 1–3 IMTC 2007IMTC 2007 22

OutlineOutline

Image SensorsImage Sensors– CCD versus CMOSCCD versus CMOS– Linear versus logarithmicLinear versus logarithmic

Logarithmic ImagersLogarithmic Imagers– Fixed pattern noise (FPN)Fixed pattern noise (FPN)– Colour renditionColour rendition– Temperature dependenceTemperature dependence

ConclusionConclusion

May 1–3May 1–3 IMTC 2007IMTC 2007 33

Image SensorsImage Sensors

A digital camera consists of many A digital camera consists of many components, of which the image components, of which the image sensor is considered principalsensor is considered principal

Typical imagers may be charge Typical imagers may be charge coupled device (CCD) sensors or coupled device (CCD) sensors or complementary metal-oxide-complementary metal-oxide-semiconductor (CMOS) sensorssemiconductor (CMOS) sensors

May 1–3May 1–3 IMTC 2007IMTC 2007 44

CCD Image SensorsCCD Image Sensors

March photo-March photo-generated charge generated charge systematically from systematically from an array of pixels to an array of pixels to an output amplifieran output amplifier

Established Established technologytechnology

High resolution, high High resolution, high sensitivity, low noisesensitivity, low noise

Fabrication process is Fabrication process is optimised for imagingoptimised for imaging

May 1–3May 1–3 IMTC 2007IMTC 2007 55

CMOS Image SensorsCMOS Image Sensors

Work like memory Work like memory with photosensitive with photosensitive pixels inside each cellpixels inside each cell

Signal processing may Signal processing may be incorporated on be incorporated on the same die as pixelsthe same die as pixels

High yield and good High yield and good video performancevideo performance

May be fabricated by May be fabricated by microchip makersmicrochip makers

May 1–3May 1–3 IMTC 2007IMTC 2007 66

Linear PixelsLinear Pixels

Linear pixels (either Linear pixels (either CCD or CMOS type) CCD or CMOS type) “count” photons over “count” photons over a discrete period of a discrete period of timetime

They produce a They produce a voltage directly voltage directly proportional to the proportional to the light intensitylight intensity

Unfortunately, the Unfortunately, the response may saturate response may saturate white or black easilywhite or black easily © IMS Chips

http://www.ims-chips.de/

May 1–3May 1–3 IMTC 2007IMTC 2007 77

Logarithmic PixelsLogarithmic Pixels

Logarithmic pixels Logarithmic pixels (CMOS only) measure (CMOS only) measure the “rate” of photon the “rate” of photon incidence continuouslyincidence continuously

They produce a They produce a voltage directly voltage directly proportional to the proportional to the logarithm of the light logarithm of the light intensityintensity

The response is similar The response is similar to that of human to that of human visionvision © IMS Chips

http://www.ims-chips.de/

May 1–3May 1–3 IMTC 2007IMTC 2007 88

The ProblemThe Problem

Logarithmic pixels are Logarithmic pixels are great for high dynamic great for high dynamic range video but…range video but…

FPN is worse compared FPN is worse compared to typical linear pixelsto typical linear pixels

Colours are worse than Colours are worse than for typical linear pixelsfor typical linear pixels

Impact of temperature Impact of temperature on the image quality is on the image quality is poorly understoodpoorly understood

May 1–3May 1–3 IMTC 2007IMTC 2007 99

The Solution…The Solution…

IMTC 2001IMTC 2001– We fixed the fixed We fixed the fixed

pattern noisepattern noise IMTC 2002IMTC 2002

– We improved the We improved the colour renditioncolour rendition

IMTC 2007IMTC 2007– We considered We considered

temperaturetemperature

May 1–3May 1–3 IMTC 2007IMTC 2007 1010

Fixed Pattern NoiseFixed Pattern Noise

Two photodetectors in Two photodetectors in the human eye or in a the human eye or in a digital camera are not digital camera are not going to be identicalgoing to be identical

A varying response to A varying response to light stimulus causes light stimulus causes “fixed pattern noise”“fixed pattern noise”

The eye uses motion The eye uses motion to factor out the FPN; to factor out the FPN; not practical for not practical for camerascameras

May 1–3May 1–3 IMTC 2007IMTC 2007 1111

Fixed Pattern NoiseFixed Pattern Noise

Modelling the FPN of Modelling the FPN of logarithmic pixels, we logarithmic pixels, we improved calibrationimproved calibration

Responses to uniform Responses to uniform stimuli were used to stimuli were used to define correctionsdefine corrections

Our correction reduced Our correction reduced the FPN to the same the FPN to the same order as the random order as the random temporal noisetemporal noise

May 1–3May 1–3 IMTC 2007IMTC 2007 1212

Colour RenditionColour Rendition

We have shown how We have shown how to render accurate to render accurate colours with colours with logarithmic pixelslogarithmic pixels

A colour mapping was A colour mapping was defined using images defined using images of a reference chartof a reference chart

Perceptual error of the Perceptual error of the rendered colours was rendered colours was comparable to that of comparable to that of consumer camerasconsumer cameras

Rendered

Rendered

Reference

Reference

May 1–3May 1–3 IMTC 2007IMTC 2007 1313

Temperature Temperature DependenceDependence Unlike with humans, Unlike with humans,

digital cameras do not digital cameras do not regulate temperatureregulate temperature

Temperature affects Temperature affects the response of a pixel the response of a pixel to a light stimulusto a light stimulus

A “new” FPN appears A “new” FPN appears when the temperature when the temperature dependence varies dependence varies from pixel to pixelfrom pixel to pixel

May 1–3May 1–3 IMTC 2007IMTC 2007 1414

Temperature Temperature DependenceDependence The dark response The dark response

of a pixel depends of a pixel depends only on temperatureonly on temperature

Thus, it may be Thus, it may be used to correct FPN used to correct FPN due to temperature due to temperature in the light responsein the light response

We validated this We validated this idea by simulation idea by simulation with real mismatch with real mismatch datadata

May 1–3May 1–3 IMTC 2007IMTC 2007 1515

ConclusionConclusion

Logarithmic CMOS image sensors are ideal Logarithmic CMOS image sensors are ideal for capturing high dynamic range videofor capturing high dynamic range video

Our research aims to improve the image Our research aims to improve the image quality of these cameras from machine quality of these cameras from machine grade to consumer grade and bettergrade to consumer grade and better

The dark response of the image sensor may The dark response of the image sensor may be used to correct temperature-dependent be used to correct temperature-dependent fixed pattern noise in the light responsefixed pattern noise in the light response

Future work will simplify our methods and Future work will simplify our methods and implement them in a complete prototypeimplement them in a complete prototype

May 1–3May 1–3 IMTC 2007IMTC 2007 1616

AcknowledgementsAcknowledgements

The authors gratefully acknowledge the The authors gratefully acknowledge the support of the Natural Sciences and support of the Natural Sciences and Engineering Research Council of Canada and Engineering Research Council of Canada and the Engineering and Physical Sciences the Engineering and Physical Sciences Research Council of the United KingdomResearch Council of the United Kingdom