Camera Driver Porting

MTK SZ

06/08/02

2

Outline

IntroductionBase band Camera featureMT6217/18+BackendMT6219+SensorCode Review

3

CameraBase band feature(MT6217)

Jpeg DecoderHas to decode full imageDestination has to be sequential

Multi-layer 4 layers

LCD ControllerNeed an extra Camera Backend

OSDJpeg DecoderLCD ControllerSensorImage ProcessorJpeg Encoder

4

Camera

Base band feature(MT6219)Jpeg Decoder/Encoder

May decode partial image

Mpeg 4 Decoder/EncoderMulti-layer

4 layers

LCD ControllerImage ProcessorNeed an extra Sensor

Support up to 1.3 Mega Pixel (1280x1024)

5

Camera

Camera MT6217 Slave backend

MT6217

PAP1301 (PixelArt)

LCMBase band

Sensor

Camera Backend

6

Camera

CameraMT6217 Master Backend

MT6217

VC0558(ViMicro)

LCM

Base band

Sensor

Camera Backend

7

Camera

Camera MT6219 structure

MT6219

LCM

Base bandSensor

8

Camera Software ArchitectureMMI Task

Camera ApplicationLogic, Interrupt handling

MDI_CameraAPI interface to use camera features

GDIMulti-layer (frame buffer) management

Media TaskInterface to access camera driverCamera Driver

Control Camera backend

LCD InterfaceControl LCM

MMI Task

MDI Camera GDI

LCD InterfaceCamera Driver

HW- (LCM, ISP, MT6218, MT6219...)

MEDIA Task

Camera MMI

9

Camera

Slave backend

MMI Task

MDI Camera GDI

LCD InterfaceCamera Driver

MEDIA Task

Camera MMI

LCMAIT103

Layer 0

MT6218B

Layer 1

Layer 2

10

Camera

Master backendMMI Task

MDI Camera GDI

LCD InterfaceCamera Driver

MEDIA Task

Camera MMI

LCM

Layer 0

Layer 1

MT6218B

Layer 0Layer 1

AIT700

Intermediate Buffer

AIT700 OSD Buffer

11

Camera

MMI Task

MDI Camera GDI

LCD InterfaceCamera Driver

MEDIA Task

Camera MMI

LCM

ISP

Layer 0blt_layer_flag == LAYER 0 | LAYER 1 | LAYER 2

preview_layer_flag == LAYER 0

Layer 1

Layer 2

MT6219

Layer 0Layer 1Layer 2

12

Camera (MT6217)

Files:\custom\drv\cam_module\[prj]\cam_module.c\custom\drv\cam_module\[prj]\cam_module.h

Function:Function:

13

Camera (MT6219)

Camera Task6219 ISP

isp_if.c.h,ae_awb.c.h,

Driverimage_sensor.c.h

Tuningcamera_para.c

AE AWB

6219 ISP

control tuning

14

HW Interface

Signal Bus

VSYNC

HSYNC

SCLK

SDATA

XCLK

PCLK

8 or 10

DATA

PWDN

POWER

6219ISP

Sensor

15

SW Interface

void (* sensor_init)(void)Initialize sensor and power on sensor module.Sensor default Configuration.The first executed API.

void (* sensor_power_off) (void)Clean sensor state and power off sensor module.

16

SW Interfacevoid (* get_sensor_id)

kal_uint8 *sensor_write_idkal_uint8 *sensor_read_id

To report sensor R/W ID.SCCB(I2C) R/W has been customized.

17

SW Interface

void (* get_sensor_size)kal_uint16 *sensor_widthkal_uint16 *sensor_height

To report image sensor resolution.IMAGE_SENSOR_SXGA_WIDTHIMAGE_SENSOR_SXGA_HEIGHT

18

SW Interface

void (* get_sensor_period)kal_uint16 *pixel_numberkal_uint16 *line_number

To report the period of image sensor in a frame.VGA_PERIOD_PIXEL_NUMSVGA_PERIOD_LINE_NUMS

These are the parameters of frame rate, base-shutter and exposure control.

Base-shutter affects the banding effect.

19

SW Interfacevoid (* sensor_capture_setting)void (* sensor_preview_setting)

image_sensor_exposure_window_struct *image_windowkal_uint16 grab_start_x;kal_uint16 grab_start_y;kal_uint16 exposure_window_width;kal_uint16 exposure_window_height;kal_uint16 image_target_width;kal_uint16 image_target_height;kal_uint16 exposure_pixel;kal_uint16 exposure_line;kal_uint8 digital_zoom_factor;

kal_uint8 image_mirrorNormal, H-Flip, V-Flip, HV-Flip.

20

SW Interface

void (* write_sensor_reg)kal_uint32 addr;kal_uint32 para;

kal_uint32 (* read_sensor_reg)kal_uint32 addr;

SCCB R/W interface for CCT access.Sensor address/parameter bit width ?

Micron(9M011,9M111)8bits/16bits(9M111 has 3 page address i.e. 3*0xFF)

OV(OV9650,OV7660)8bits/8bits

21

SW Interfacevoid (* set_sensor_eshutter)

kal_uint16 eshutter;

To set sensor exposure time.Important API associated with AE algorithm, banding.Depends on sensor characteristic.

22

SW Interface

void (* set_camera_night_mode)kal_bool enable;

To turn ON/OFF sensor night mode.Double MAX_EXPOSURE_LINES.Increase sensor global gain.

23

SCCB(I2C)Micron 8bit

Write(PARA to ADDR)

Read(PARA from ADDR)

Micron 16bitWrite(PARA to ADDR)

Read(PARA from ADDR)

S WID PARA1 EA ADDR A S AA WID A 0xF1 PARA2 A

S WID A ADDR A S RID A PARA1 N

S WID A 0xF1 A S RID A NPARA2 E

S WID A ADDR A PARA1 A PARA2 A E

S WID A ADDR A S RID A PARA1 NPARA2A E

24

Image sensor data input

V _Sync

H _Sync

Clk

CMOS sensor data input timing

Fram e cycle

Line cycle

25

Sensor&ISP timing

Sync. Signal

Set VSYNC polarity LOWSet HSYNC polarity LOW.

VSYNC

HSYNC

26

Sensor&ISP timingTiming adjustment

ENABLE_CAMERA_PIXEL_CLKIN_ENABLE;//MT6226 or laterSET_TG_OUTPUT_CLK_DIVIDER(1);//XCLK=MCLK/(1+1)=26MSET_CMOS_RISING_EDGE(0); //PCLK rise at MCLK 0SET_CMOS_FALLING_EDGE(1); //PCLK fall at MCLK 1SET_TG_PIXEL_CLK_DIVIDER(3);//PCLK=MCLK/(3+1)=13MSET_CMOS_DATA_LATCH(2); // PCLK latch at MCLK 2

PCLK

XCLK

MCLK0 1 2 3

27

Sensor&ISP timing

First Valid Pixel/Line

VSYNC

HSYNC

First Valid Pixel

XCLK

PCLK

image_window->grab_start_x = dummy 的 lines,

base on datasheetFirst Valid Line

28

Sensor&ISP timing

Bayer pattern

SET_BAYER_PATTERN(FIRST_GRAB_COLOR)#define FIRST_GRAB_COLOR XX

Gr00

R01

B10

Gb11

Base on datasheet

29

Sensor&ISP timing

Valid Frame

VSYNC Valid Frame

SENSOR_PERIOD_PIXELSHSYNC

HBLANKING

VBLANKING

30

AE/AWB Algorithm

AE/AWB Window

AE “Window size”, “region” and “weighting” are configurable, and reports the luminance.AWB window reports the R, G, B accumulation.

w1 w2 w3

w4 w5 w6

w7 w8 w9

31

AE/AWB Algorithm

AE Process(FW)AE banding factor.

Base-shutter should be at lease 1/2f.Base-shutter=PIXEL_CLK/(2f*SENSOR_PERIOD_PIXELS)

VSYNC ValidFrame

SENSOR_PERIOD_PIXELSHSYNC

32

AE/AWB Algorithm

AE Process(FW)AE computation.

Exposure time(shutter), pre-gain.Linear interpolation.

avg_lum = weighted average of 9 windows.

Exposure time 和 Luminance 不是线性的，要逐次计算AE逼近

Exposure time

Luminance

Y1

Y2Yt

T2 T1Tt

Desired Luminance

33

AE/AWB Algorithm

AE/AWB parameter setting timing(HISR)

Three of them should be set precisely, or AE vibration occurred.

AE_AWB_CAL_PERIOD=3 AE_AWB_CAL_PERIOD=3

VSYNC

AE_GAIN_DELAY_PERIOD=2AE_SHUTTER_DELAY_PERIOD=1

34

Code ReviewPreview

ISP/Sensor CLK setting.Mode/Resolution configurationActive window configuration

CaptureAll you did in “Preview”Mode switching

VGA/SXGAzoom<2x and 2x<=zoom<=4x.

Set ShutterControl the exposure time precisely.Depends on sensor characteristic.

35

Case Study

Step 1: define sensor power enable GPIO pin in Camera_para.c

36

Case Study

Step 2: set I2C read ID and write ID

37

Case Study

Step 3: Implement write_cmos_sensor() and read_cmos_sensor()

38

Case Study

Step 4: set Vsync/Hsync polarity and first grab color

39

Case Study

Step 5: insert initial code to CAMERA_PARA_DEFAULT_VALUE or write it directly in init_OV9650()

40

Case Study

Step 6: implement OV9650_preview()

MCLK=13MHzPCLK = 3.25MHz

If latch point wrong pic will be rad/blue

41

Case Study

Step 7: implement OV9650_capture()

VGA & zoom<20PCLK = MCLK/2MCLK=13MHz

PCLK = 6.5MHz

VGA & zoom > 20PCLK = MCLK/4MCLK=13MHz

PCLK = 3.25MHzincrease dummy

42

Case Study

Step 7: implement OV9650_capture()

SXGA & zoom<20PCLK = MCLK/2MCLK=13MHz

PCLK = 6.5MHz

SXGA & zoom > 20PCLK = MCLK/4MCLK=13MHz

PCLK = 3.25MHzincrease dummy

decrease shutter

43

Case Study

Step 8: implement set_OV9650_shutter()set_OV9650_gain()OV9650_night_mode()power_off_OV9650()

44

Q&A