understanding digital systems - electrical and computer ...duarte/elec542/notes_3_20.pdf ·...

1

Understanding Digital Systems

ECE541Spring 2005

Rice University

2

Real-Time Video Systems

Current challenges

Today’s Video/Imaging systems

Standards and their Usage

3

Agenda• General Market Trends in digital video systems• Understanding Video and Imaging Systems

– Analog Video Formats– Hardware Codecs– Digital Video Formats– Software Codecs

• Choosing a Video and Imaging Solution• Conclusions• Wrap-up / Q&A

4

Requirements are drastically changing your way to solve your application challenge.

Versatility, performance and programmability are key careabouts

Programmable DSP are providing these features

Any Content, Any Format, Any Time, Any Place…

Video Market TrendsYC4

YC4 This foil or the next????Y. Cager, 4/2/2003

5

Trends in the Video space

• Non-PC-based Internet appliances are one of the fastest-growing areas of network applications.

• Streaming video is becoming a reality• Standards are still under definition• Increasing number of formats and rate of innovation

> Many requests for multi-format capability• New innovations like H.26L offer ½ bit rate of MPEG2 at same quality

> High quality video over IP at under 1Mbps is now possible• Market is truly emerging with a variety of EE • End-customers want

– Multi-standard flexibility and upgradeability– Video quality and channel density– System cost, especially on the client side

YC5

YC5 This slide or previous.Y. Cager, 4/2/2003

6

Any Content

Entertainment Distribution- Content Production Services, Studio, Networks, Sports, News- Published Media On DVD, Game Consoles- Entertainment Terminals (TV Sets, STB..)- Entertainment Personal Computers (PVRs, Juke Boxes..)- the Internet In All Its Various Aspects

Personalized Multimedia- Non-PC-based Internet appliances are one of the fastest-growing

areas of network applications.

Corporate Information Distribution- Video Conferencing- Video Security

Still Image

Traditional Voice Communications

7

Any Format

Battle of the Codecs

- Increasing number of formats and rate of innovation> Many requests for multi-format capability> Standards are still under definition

- New innovations like H.264 offer ½ bit rate of MPEG2 at same quality> High quality video over IP at under 1Mbps is now possible

- New formats bring increased algorithm complexity

MPEG-2 MPEG-4 H.264DIVX

8

Anytime, Any Place

Deliver the content WHEN the user wants..- Streaming media- Home gateway- Video On Demand- Personalization of viewing- Interactive communication- Digital Ads insertion

… And WHERE the user wants.- Adapt content to available bandwidth

Transrating- Transport content through Heterogeneous network

Transcoding- Support Full Duplex Media Communication

Encoding

9

Three Video Market Segments

Portable Client Infrastructure

Multi-mediaedge router

StreamingServer

Video Phone

Video Surveillance

IP Personal Video Recorder

Statisticalremultiplexor

Digital Photo Albums

PDA

eBooks

DVC

10

Agenda• General Market Trends in Video Imaging systems• Understanding Video and Imaging Systems



11

Typical Video System Chain

Source - Ingenient

OutputDisplay

InputCapture

Video Palette

NTSC/PAL Video

encoder

S-Video orComposite Out

RGB Out

NTSC/PALVideo

Decoder

S-Video orComposite In

Real-timeSignal Processing

System Control and Communications

CompressionDecompression

EncryptionFormatting

Transmission

IP Network or Interface

IP Network or Interface

12


OutputDisplay

InputCapture

Video Palette

NTSC/PAL Video

encoder


RGB Out

NTSC/PALVideo

Decoder






Transmission

Analog Formats

Hardware Codecs

Software Codecs

Digital Formats

13




14


Analog Formats

Interlaced vs. progressiveColor spaceComposite vs. component vs. S video

OutputDisplay

InputCapture

Video Palette

NTSC/PAL Video

encoder


RGB Out

NTSC/PALVideo

Decoder






Transmission

15

TV – Interlace

• Video Signals– voltage varying wrt time– Cathode Ray Tube

• Non-Linear – “GAMMA”– Gamma Correction

• Computer Monitors– Progressive Scan

• 2:1 interlaced scanning– Alternate Odd/Even Lines– Half Required Bandwidth– Artifacts

16

Color Space

• 3 sensors in camera– RGB– Max Accuracy

• Eye– Relies on brightness to

convey detail

• RGB Matrixed– Luminance

• MonoChrome compatible

– Chrominance– Eye sensitivity

• Luminance - Weighted sum

• Savings in Bandwidth by color difference

17

Composite Video Component Video

• Component video is native format for digital TV since MPEG compression operates on component video

• Component video does away with several artifacts of composite video

Composite vs Component

18

fsc= 3.58 MHz= 227.5 fh

with fh= 15.734kHz linerate(NTSC-system M)fsound = 4.5MHz

U = modulated (B’-Y’) color componentV = modulated (R’-Y’) color component

LUMINANCE CHROMINANCE

NTSC Frequency Spectrum(Composite)

SOUND

19




YC6

YC6 THIS SECTION IS BEING REVISED BY THE ANALOG TEAM.Y. Cager, 4/2/2003

20


Hardware Codecs

Video DecodersVideo Encoders

OutputDisplay

InputCapture

Video Palette

NTSC/PAL Video

encoder


RGB Out

NTSC/PALVideo

Decoder






Transmission

21

Hardware Codecs

• Video Decoders– Analog video inputs

• NTSC/PAL composite & S-Video.– Input video decoded to YPbPr component video– VBI data slicer enables NABTS / WST teletext and close

captioned processing on chip.– Multiple host interfaces (I2C/VMI/VIP) provides flexibility to

interface with host processors.

• Video Encoders– Digital Component Video Input– Analog Component / Composite Output– Colour Space Conversion– Copy Protect Management etc

22

DTAC/RDYDTAC/RDY

VI_1AVI_1A

XTAL2XTAL2XTAL1XTAL1

RW/WRRW/WRDS/RDDS/RD

D[7:0]D[7:0]

SCLKSCLKPCLKPCLKPREFPREFGLCOGLCO

CHROMACHROMA

LUMALUMAAGCAGC

VI_1BVI_1B

VI_2AVI_2A

VI_2BVI_2B

FORFOR

MATMAT--

TERTER

1010--bitbitADCADC

LumaLuma

Chroma Chroma

Sep. Sep.

Y[9:0]Y[9:0]

UV[9:0]UV[9:0]

OEBOEB

VBIVBI

PROCESSORPROCESSORCSCSA0A0A1A1

SYNCSYNC

PROCESSORPROCESSOR

HSYNHSYNVSYNVSYNFIDFIDPALIPALIGPCLGPCLRSTINBRSTINB

AGCAGC 1010--bitbitADCADC

HOSTHOST

I/FI/F

CLOCKCLOCK

GEN.GEN.

Video Decoder

23

• DAC Features : (THS8135)– Speed up to 240 MSPS / 10

bit – Can support video &

graphics formats up to UXGA

– 1.8V digital supply– Pin-compatible with

THS8133– Improved video functionality

DACINPUT

FORMATTER

SYNC/BLANK

CONTROL

CONFIG.CONTROL

BANDGAPREF.

DAC

DAC

R/Pr

G/Y

B/Pb

Component Video/PC Graphics

24

• Features : (THS8200)

– Analog backend for all VESA formats up to UXGA@75Hz

& ATSC/EIAformats for DTV

– Programmable VESA & DTV sync generation, color space conversion, video interpolation, CGMS-A insertion & 525P copy protection

VESA Member Product Notebook IDF 2001

VIDEO

ASIC/DSP

PC, PVR,STB,… end equipment

VESA/EIA - Compliant interfaceto TV/monitor

11-bit205MSPS

DAC

11-bit205 MSPS

DAC

11-bit205 MSPS

DAC

COLORSPACE

CONVERTER

INTER-POLATION

FILTER

SYNCTIMING

GENERATOR

4:2:2to

4:4:4

I2CHOST

INTERFACE

D A T A

M A N A G E R

'All-format' VESA & ATSCcomponent video/ PC graphics

encoder IC

SYNC/DATA

I N S E R T E R

525PCOPY

PROTECTIONTHS8200

Video Encoder

25




26


Digital Formats

Pixel FormatsPixel RepresentationStandards

OutputDisplay

InputCapture

Video Palette

NTSC/PAL Video

encoder


RGB Out

NTSC/PALVideo

Decoder






Transmission

27

Digital Video Formats• Picture Formats

YC7

YC7 This chart needs to be reworked. It is blurry.Y. Cager, 4/2/2003

28

Pixel Representation• RGB: Color Scheme Using Primary Colors

– Red, Green, Blue– Useful for display

• YCbCr: Color encoding scheme for digital video– Y: Luminance– Cb: Blue Chrominance– Cr: Red Chrominance– Human eye more sensitive to grayscale (Y) than to color– Useful for compression

29

Chroma Format

Chroma Format: Ways of coding YCrCb Pixels

–4:2:0• Two chroma per four luma: YYYYCbCr• Odd lines only• Most common format: Mainstream TV,

consumer applications, videoconferencing

–4:2:2• Four chroma per four luma:

YYYYCbCrCbCr• Used in studios and professional video

–4:4:4• Full Pixel Representation:

YYYYCbCrCbCrCbCrCbCr• Used in high-quality video applications

–Others exist but are less common

4:2:0 format

4:2:2 format

Luma Sample (Y)

Chroma Sample (Cr,Cb)

30

Common Video Standards

ITU-BT.656 • Interface standard for 525-line (NTSC) and 625-line (PAL) digital component

video encoded using the 4:2:2 level of ITU-BT.601 • 8-bit or 10-bit words conveyed as 27 Mwords/s muxed parallel or serial data

stream• Embedded timing reference signals to indicate Field, Horizontal Blanking and

Vertical Blanking• Common interface for SDTV video decoder and encoder ICs

ITU-BT.1120• Interface Standard for the following HDTV format specified in ITU BT.709

– 1125/60/2:1(1152 total lines, 60 fields/s, 2:1 interlaced, 1035 active lines, 1920 active samples per line)

– 1250 50/2:1(1250 total lines, 50 fields/s, 2:1 interlaced, 1152 active lines, 1920 active sampels per line)

– 1920 x1080 HD-CIF including 60i, 60p, 50i, 50p, 25p, 24p etc.• Uniformly quantized PCM sampling for each of the video component

signals(Y/Cb/Cr or R/G/B) 8- or 10-bit/sample(10-bit preferable)• Bit-parallel or serial interface • Embedded timing reference signals to indicate Field, the start and end of

Horizontal and Vertical Blanking.• Common interface for HDTV video decoder and encoder ICs

31




32


Software Codecs

CompressionStandards

OutputDisplay

InputCapture

Video Palette

NTSC/PAL Video

encoder


RGB Out

NTSC/PALVideo

Decoder






Transmission

33

Why Compression?

• Needed to store video within storage constraints and/or transmit video over limited bandwidth

• 30 Frames/s D1 Resolution at 4:2:0 Translates to:– ~15.5 Mbytes/s (124.4 Mbits/s)– 83.7 GBytes to store 90 minute video

• 30 Frames/s CIF Resolution at 4:2:0 Translates to:– ~4.5 Mbytes/s (36.5 Mbits/s)– 23.3 GBytes to store 90 minute video

34

Compression Application Examples

• Bandwidth example– 1.5 Mbps (CD) requires 83:1 compression for D1– 384 kbps (2-way ADSL) requires 95:1 compression for CIF

• ISDN (128kbits/s) video conferencing– Allocate 112 kbits/s– CIF@30 fps 4:2:0 -> 36.5 Mbits/s– Requires ~323:1 compression

• PVR or DVR system today includes 40 to 80 GBytes of storage– Quality options for recording

• Typically low, medium, and high quality options ranging from 2 Mbps to 6 Mbps

– Storage requirements for 1 hour program• Uncompressed D1: 55.8 Gbytes

– 0 shows on 40 Gbyte HDD• 6 Mbps (MPEG-2 high quality): 2.7 Gbytes

– ~15 shows on 40 Gbyte HDD• 2 Mbps (low quality): .9 Gbytes

– ~44 shows on 40 Gbyte HDD

35


OutputDisplay

InputCapture

Video Palette

NTSC/PAL Video

encoder


RGB Out

NTSC/PALVideo

Decoder






Transmission

Videodecoder DSP

OutputDisplay

InputCapture

Video encoder


36

Benefits of Programmability

• First to market with new codecs– UB Video on 64x first to support real-time main profile H.264

decoding on an embedded platform

• Support for multiple standards/algorithms– Landscape changing to larger number of formats

• Future proofing– Videoconferencing vendors almost all use DSPs for future

upgadeability• Systems able to add support for H.263 after initial deployment

• Full algorithm control– Ability to tailor encoder for needs of the applications

• E.g., use MPEG-2 with only Intra frames for surveillance• Access motion vectors to determine activity

• System consideration – system cost and design complexity

37

TI’s Broad Video and Imaging Portfolio

Hundreds of video and imaging products in production todayHundreds of products available today,

•TMS320C6000TM DSP platform for video client and infrastructure applications

•High-Perf. Digital Media

• DM642 Digital Media Processor

•TMS320C5000TM DSP and OMAP platforms•

•Portable Digital Media

• DSC21, DSC24, DSC25 SoCs for DSCs• DM310 Digital Media Processor• OMAP5910 Dual core, RISC/DSP processor

•TVP5200 first all-format video processor•TVP5145 Best-in-class mixed-signal decoder for video applications•THS8200 all format video backend for video & PC applications•Other high-performance data converters and power management products

•Analog

•DSL Modems•Cable Modems•Wireless LAN

•Complementary TI Technologies

38

Video and Imaging Domain Roadmap

DM642

C6211

C6412C6415

C5509 C5510 C6205

C5402 C5410A

C6411C6204

Portable Client Infrastructure

OMAP5910

Video server

WirelessGateway

StatisticalRemux

Multi-mediarouter

Digital VideoCamera

DSC

eBooks

PDA

Digital PhotoAlbum

inte

grat

ion Perform

ance

DVR

PVR

Smart Securitysystem

Video Conf.

Video Phone

39

Media encoders/decoders

Video on demand set top boxDigital media center

Surveillance systemsDigital video recordersMultiplexors

Target End Equipments

• Video Compression H.261,H.263,H.26L codecs• Speech Compression G.722, G.722.1, G.728• Networking Stack TCP/IP, UDP

• Video Compression MPEG-2, MPEG-4, H.26L, WMT, Real Networks, On2 VP5 decoders

• Audio Compression AAC, AC3 decoders• Networking Stack TCP/IP, UDP, RTP, RTSP• Middleware Browser, OSD, Encryption

• Video Compression H.261,H.263,H.26LMPEG-4, JPEG codecs

• Speech Compression G.722, G.722.1, G.728• Networking Stack TCP/IP, UDP, RTP, RTSP

• Video Compression MPEG-2, MPEG-4, H.26L, WMT, Real Networks, On2 VP5 codecs

• Audio Compression AAC, AC3 codecs• Networking Stack TCP/IP, UDP, RTP, RTSP

IP video phoneIP video conferencing

40

Benchmarks

85%44%42%22%90%46%Video Encoder Loading

28%16%14%8%29%17%Video Decoder Loading

CIFCIFCIFCIFCIFCIFProcessing Resolution

301530153015Frame Rate (fps)

384128384128384128Video Bit Rate (Kbps)

C6211-167MHzC6203-300MHzC6205-200MHz

Performance data as of March 2001Performance data is average across 5 public domain video test sequences:

Foreman, News, Coast Guard, Silent, Akiyo

Ingenient Technologies: TI Third party providing optimized MPEG-4 solutionsFurther details at: www.ingenient.com

TMS320C6000 MPEG-4 Encoder/Decoder

YC8

YC8 Need C64x BenchmarksY. Cager, 4/2/2003

41

DVR/Surveillance System

42

Video Encoder/Decoder BoxVideo encoder/decoder’s are appliance type client-end boxes that create MPEG from analog (encode) & play(decode) video content. Encoder boxes are usually small & compact-size, sometimes connect to PC or work as standalone network boxes (configured for ethernet IP connectivity). Enc/Dec boxes features high quality full motion (NTSC/PAL) MPEG video.

43

IP-Video Phone

IP Videophones feature real-time multi-channel video integration to existing IP-voice phones, providing 3-way video conferencing at 30 fps, real-time sync of video with voice across IP-networks

44

SUMMARY

• Advances and variety in video codec algorithms creates an opportunity for programmable digital media processors

• New codecs provide significant improvement in compression at cost of increasing computational cost

• New standards often provide a toolkit of coding techniques that can be applied in many different ways

• Profiles attempt to classify combinations that make sense for different applications

• Encoder implementations can vary significantly based on target application

• We need to capitalize on our early ability to support advanced codecs with early design-ins and comprehensive roadmap to retain these sockets as they go to high volume production

45

Conclusions

• Explosion of Standards for Different Applications– Multi-standard flexibility and upgradeability

– Video quality and channel density

– System cost, especially on the client side

• Shown the Pieces of the Puzzle and how Fit Together

• Majority of Algorithms are Core Kernel Functions

• Need to be able to adapt to Standards /New Annexes

46

Acknowledgements and References

• The Art of Digital Video -John Watkinson• Digital Television -H. Benoit• Video Demystified -Keith Jack• Video Compression Demystified -Peter Symes• http://www.diffuse.org• UB Video / Ateme / Ingenient• MPEG information website: www.mpeg.org• JPEG/JBIG Committees website: www.jpeg.org• DM642 information: www.ti.com/dm642intro.com

understanding digital systems - electrical and computer ...duarte/elec542/notes_3_20.pdf ·...

Documents