B. Hoefflinger (Ed.)

High-Dynamic-Range (HDR) Vision Microelectronics, Image Processing, Computer Graphics

With 172 Figures

Sprin ger

Contents

1 The Eye and High-Dynamic-Range Vision Bernd Hoefflinger 1 References 12

2 The High-Dynamic-Range Sensor Bernd Hoefflinger and Verena Schneider 13 2.1 General Considerations 13 2.2 The HDRC (High-Dynamic-Range CMOS) Pixel 19 2.3 The HDRC Sensor 27 2.4 Fixed-Pattern Correction of HDR Imagers 32

2.4.1 Physical Background of Logarithmic OECF 32 2.4.2 Parameter Extraction with Software 33 2.4.3 Effects of Parameter Variation on the OECF 35 2.4.4 Presentation of Three Correction Algorithms 37 2.4.5 New Parameterized Correction Algorithm 38 2.4.6 Masking Process 40 2.4.7 Algorithm Including Temperature 41 2.4.8 Correction Procedure and Runtime 46 2.4.9 Summary 47

2.5 HDRC Dynamic Performance 47 2.6 HDRC Sensor with Global Shutter 53 References 56

3 H D R Image Noise Bernd Hoefflinger 57 References 63

4 High-Dynamic-Range Contrast and Color Management Bernd Hoefflinger 65 References 71

X Contents

5 HDR Video Cameras Markus Strobel and Volker Gengenbach 73 5.1 Introduction 73 5.2 HDRC CamCube Miniaturized Camera Module 75

5.2.1 Features of the HDRC CamCube 76 5.2.2 Assembly Techniques 76 5.2.3 System Design 77 5.2.4 Application Example 78

5.3 HDRC Camera Front-End 78 5.4 Digital HDRC Camera Link™ System 82

5.4.1 Features of the HDRC Camera Link Camera 84 5.4.2 Features of the "IP3 Control" Software 84 5.4.3 Application Example 84

5.5 Intelligent HDRC GEVILUX CCTV Camera 85 5.5.1 Features of the Camera 85

5.6 HDR Video-Based Aircraft Docking Guidance 90 5.6.1 Summary 90 5.6.2 Introduction 91 5.6.3 Operation 92 5.6.4 Challenges to the Sensor System 93 5.6.5 HDR Camera with Improved Sensitivity 94

5.7 Conclusion 97 References 97

6 Lenses for HDR Imaging Hans-Joerg Schoenherr 99

7 HDRC Cameras for High-Speed Machine Vision Bela Michael Rohrbacher, Michael Raasch and Roman Louban 107 7.1 General Requirements 107 7.2 Special Characteristics 108 7.3 Methods for Obtaining the Specific Image Information 109 7.4 Optoelectronic Transfer Function

(Lookup Table, LUT) 110 7.4.1 Mode 1:1 111 7.4.2 Mode Rec. 709 111 7.4.3 Mode Stretched 112 7.4.4 Mode CatEye 112 7.4.5 Mode CatEye2 112

7.5 Application Example Surface Inspection 113 7.6 Evaluation Algorithms 114 7.7 Robot ControUed Image-Processing System for Fully Automated

Surface Inspection 118 References 121

Contents XI

8 H D R Vision for Driver Assistance Peter M. Knoll 123 8.1 Introduction 123 8.2 Components for Predictive Driver Assistance Systems 124

8.2.1 Ultrasonic Sensors 124 8.2.2 Long Range Radar 77 GHz 125 8.2.3 Video Sensor 125

8.3 Driver Assistance Systems for Convenience and for Safety 127

8.4 Video-Based Driver Assistance Systems 128 8.4.1 Video System 128 8.4.2 Image Processing 130

8.5 Night Vision Improvement System 130 8.6 Night Vision Enhancement by Image Presentation 131 8.7 Night Vision Warning 132 8.8 Sensor Data Fusion 133

8.8.1 Lane Detection and Lane Departure Warning 134 8.8.2 Trafhc Sign Recognition 134

8.9 Conclusion 135 References 136

9 Min ia tu re HDRC Cameras for Endoscopy Christine Harendt and Klaus-Martin Irion 137 References 139

10 H D R Sub-ret inal Implant for t he Vision Impai red Heinz-Gerd Graf, Alexander Dollberg, Jan-Dirk Schulze Spüntrup and Karsten Warkentin 141 10.1 Introduction 141 10.2 Electronic HDR Photoreceptors 142 10.3 The Differential Principle 143 10.4 The Complete Amplifier Cell 143 10.5 The Retinal Implant 145 References 145

11 H D R Tone Mapp ing Grzegorz Krawczyk, Karol Myszkowski, and Daniel Brosch 147 11.1 Taxonomy 148

11.1.1 Spatially Invariant Operators 149 11.1.2 Spatially Variant Operators 153

11.2 HDR Video: Specific Conditions and Requirements 159 11.3 Tone Mapping for HDR Video 161

11.3.1 Response Curve Compression 161 11.3.2 Local Details Enhancement 162 11.3.3 Temporal Luminance Adaptation 163

XII Contents

11.3.4 Key Value 164 11.3.5 Tone Mapping 165

11.4 Simulating Perceptual Effects 166 11.4.1 Scotopic Vision 166 11.4.2 Visual Acuity 167 11.4.3 Veiling Luminance 168 11.4.4 Tone Mapping with Perceptual Effects 169

11.5 Bilateral Tone Mapping for HDRC Video 170 11.6 Summary 175 References 175

12 HDR Image and Video Compression Rafal Mantiuk 179 12.1 Introduction 179 12.2 Device-Referred and Scene-Referred Representation of Images . . . . 180 12.3 HDR Image and Video Compression Pipeline 180 12.4 HDR Image Formats 181

12.4.1 Radiance's HDR Format 181 12.4.2 LogLuv TIFF 182 12.4.3 OpenEXR 183 12.4.4 Subband Encoding - JPEG HDR 183

12.5 HDR Extension to MPEG Video Compression 184 12.6 Perceptual Encoding of HDR Color 187 12.7 Software for HDR Image and Video Processing 191 References 191

13 HDR Applications in Computer Graphics Michael Goesele and Karol Myszkowski 193 13.1 Introduction 193 13.2 Capturing HDR Image Data 194

13.2.1 Multiexposure Techniques 194 13.2.2 Photometrie Calibration 194

13.3 Image-Based Object Digitization 196 13.3.1 Image-Based Capture of Spatially Varying BRDFs 196 13.3.2 Acquisition of Translucent Objects 197

13.4 Image-Based Lighting in Image Synthesis 199 13.4.1 Rendering Techniques for Image-based Lighting 200 13.4.2 A CAVE System for Interactive Global Illumination

Modeling in Car Inferior 203 13.4.3 Interactive Lighting in Mixed Reality Applications 205

13.5 Requirements for HDR Camera Systems 206 References 208

Contents XIII

14 High-Dynamic Range Displays Helge Seetzen 211 14.1 HDR Display Requirements 211 14.2 HDR Display Design 213

14.2.1 LED Backlight 215 14.2.2 LCD Panel 216 14.2.3 Image Processing Algorithm 216

14.3 HDR Display Performance 221 14.4 Alternative Implementation 222 14.5 Conclusion 222 References 222

15 Appendix 225

15.1 Symbols 225 15.2 Abbreviations 229 15.3 Glossary 230 15.4 Some Useful Quantities and Relations 231 15.5 Trademarks 231

Index 233