1. Introduction to CameraBy: Sergio Aguirre

2. Basics Camera: Device that records and storesimages. Still Images: Videos or movies: 3. Basics But essentially, what defines a camera device? Recording medium Lens Light Lens Recording Medium Light 4. Basics Now, what about a typical digital camera? Image sensor (CCD or CMOS) Lens Flash ISP: Image Signal Processor Image and video Encoders (To save space) Storage Media (SD Cards, USB drives, Etc.) 5. Image sensors 6. Image sensor technology CCD: Charge Coupleddevice. Moves electricalcharges to an areawhere it can bemanipulated. CMOS: Complementarymetal-oxidesemiconductor. Array of pixel sensors,each one containing aphotodetector and anactive amplifier. 7. Image sensor pixel arrays Foveon: 100% Red 100% Green 100% Blue Bayer: 25% Red 50% Green 25% Blue 8. Image sensor formats (Most typical) RAW 10-bits. 10-bit digital value obtained thru an ADC (Analog toDigital Converter) of the analog voltage comingfrom each pixels associated photodetector charge-to-voltage converter. YUV 4:2:2 8-bits. Represent pairs of horizontally contiguous pixels. 8-bits per component (Y: Luminance, UV:Chrominance). Order is usually YUYV, or UYVY.Croma components (UV) are subsampled at halfthe sample rate than luma (Y). 9. Image sensor formats (Most typical) YUV 4:2:0 8-bits. Similar to 4:2:2, but the difference being that the UVcomponents are also vertically subsampled by 2.This format contains interleaved data, whichmeans, odd lines contain only Y elements, andeven lines contain packed UYVY elements, similarlyto 4:2:2 subsampling. NV12 (YUV 4:2:0 8-bits, planar) Exact same subsampling as above format, but thedifference being that one array contains only Ycomponents, and a second array contains packedUV components, 2x2 subsampled. 10. Image sensor core Typical block diagram of an image sensor corecomponents:ColumnSample/hold Digital gain Row select Black level calibrationImage 10-bit arrayAMPADC ... Gaincontrol 11. Image sensor core functions Analog gain: Amplifies constant voltage per-pixel for entire array. Done before ADC. Black level: Level of pixel voltage at the darkest(black) part of the incoming image. Digital gain: Adds a constant number to thedigitally converted pixel value, after goingthrough the ADC. Skipping: Certain pixels rows/columns areavoided (skipped) on sampling. Binning: Averaging groups of neighborhoodpixel charges. 12. Image sensor interfaces Overall, theres at least 2 communication links: Control: Typically I2C, but not necessarly the case.This is used by a master IC to initialize/program thesensor for desired operation and data outputconfiguration. Data: This can be Parallel or Serial.SensorData Master IC Control 13. Image sensor interfaces - Control I2C: Inter-Integrated Circuit Multi-master serial single-ended computer bus,used to attach low-speed peripherals to a host IC. Uses 2 open-drain lines, w/pull-up resistors: SCL (Serial CLock). SDA (Serial DAta). 14. Image sensor interfaces - I2C I2C: Inter-Integrated Circuit START and STOP conditions to claim/release bus control. First byte: 7-bit device address space, 1 bit used to indicate Read(HI) or Write (LOW). ACK bit pulled low by slave after each byte sent, used tohandshake transmission between master and slave ICs. If ACK bitis high, master should stop transmission. 15. Image sensor interfaces - Parallel Parallel interface (a.k.a. Digital Video Port) Data signals: 1 digital output pin per bit. Sync signals: PCLK: Pixel Clock. HSYNC: Horizontal Sync. VSYNC: Vertical Sync. Horizontal BlankingD[9:0]Image data SensorPCLK Master ICHSYNCVSYNC Vertical Blanking 16. Image sensor interfaces - CSI2 MIPI CSI2 (Camera Serial Interface 2). Data signals: Up to 4 differential lanes (+/- signals). Clock signals: 1 differential lane (+/- signals). Speeds up to 1 Gbit/s per lane (500 MHz) CLK (+/-) DAT0 (+/-)Sensor DAT1 (+/-) Master IC DAT2 (+/-) DAT3 (+/-) 17. Image sensor interfaces - CSI2 CSI2 Receiver components: 18. Image sensor interfaces - CSI2 D-PHY layer: 19. Image sensor interfaces - CSI2 CSI2 Protocol Layer: 20. Image sensor interfaces - CSI2 Data Lane Merger 21. Lens 22. Lens Could be either a simple convex lens, or acompound lens made up of a number of lenselements, used to correct optical aberrations,and keep good image sharpness on a specificobject distance from the camera. There are fixed-focus lens, and lens whoseposition can be adjusted by a VCM (voice coilmotor). 23. Lens For moving lens, position can be eithercontrolled as part of the image sensor registers,or as a separate I2C device in the bus. This is usually done as a result of the AutoFocus algorithm, after analyzing the capturedimages. It can also be due to manual control, ifdesired. 24. Flash 25. Flash Used in photography to compensate naturalambient light, and illuminate a dark scene. Syncronized with start of capture imagecommand, with adjusted flashing duration, andintensity. Usually triggered with GPIOs and configuredwith I2C commands. 26. Thanks! 27. Extra slides 28. Image sensor interfaces - Data MIPI CSI (Camera Serial Interface). Data signals: 1 differential lane (+/- signals). Clock signals: 1 differential lane (+/- signals). Speeds between 1->208 Mbps CLK (+/-)Sensor DAT (+/-) Master IC 29. Image sensor interfaces - Data SMIA CCP2 (Compact Camera Port 2).Data signals: 1 differential lane (+/- signals).Clock signals: 1 differential lane (+/- signals).Class Data transfer capacity Signaling methodCLK, or(sustain data rate)STROBE (+/-)Class 0