choosing the best ip for compressing images & video
DESCRIPTION
Introduces some of the key factors to consider when choosing IP cores -- and a supplier -- for image and video compression applications. Covers compression algorithms, hardware implications, and evaluation methods for JPEG, JPEG 2000, H.264, and more.TRANSCRIPT
Choosing the Best IPfor CompressingImages and Video
Choosing the Best IPfor CompressingImages and Video
CAST, Inc.+1 [email protected]
Slide 2
Selecting the Best IPSelecting the Best IP
Algorithms Image Quality Formats
Hardware Area Speed Memory System Integration Application Requirements
Evaluation
Expertise
JPEGJPEG
JPEG-LSJPEG-LS
JPEG 2000JPEG 2000
Lossless JPEGLossless JPEG
H.264H.264
Motion JPEG
Motion JPEG
Slide 3
Compression AlgorithmsCompression Algorithms Lossy
discards image data but looks good enough
Losslessdiscards no image data
Near-Losslessvisually identical, ornumerically acceptable data loss
Slide 4
JPEGGood quality
< 20:1
Moderate quality 20:1 – 50:1
Poor quality > 50:1
JPEG 2000Good up to 80:1
H.264Best for > 50:1
Image Quality: LossyImage Quality: Lossy
Slide 5
LJPEG Compresses 2:1
Not good for text, computer graphics
JPEG-LS 2.3:1
Usually best
JPEG 2000 2.1:1
Flexible (lossy, too)
Image Quality: Lossless Image Quality: Lossless
Slide 6
Input Format: Image TypesInput Format: Image Types
Still images vs. motion video Still images — JPEG, LJPEG, JPEG-LS, JPEG 2000 Motion video — Motion JPEG, H.264
H.264 Motion Estimation Predicts where motion will occur Represents motion, not entire frame: fewer bits
Slide 7
CGA 320 x 20064,000 pixels
CGA 320 x 20064,000 pixels
QSXGA 2560 x 2048 5,242,880 pixels
QSXGA 2560 x 2048 5,242,880 pixels
Input Format: Data FactorsInput Format: Data Factors
Number of bits/pixelJPEG — 8LJPEG — up to 16JPEG-LS — up to 16JPEG 2000 — up to 16
Number of pixels: frame size
Representation method
Slide 8
Input Format: SubsamplingInput Format: Subsampling
Eye more sensitive to changes in brightness (luma) than color (chroma)
Can use just some of the chroma info (a subsample) to compress an image
Encoders use different subsample formats JPEG — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale
LJPEG — 4:4:4, grayscale
JPEG-LS — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale
JPEG 2000 — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale
H.264 — 4:2:0
Color SpaceNotation
Y:Cb:Cr
Y = luma value
Cb:Cr together
= chroma value
Can include fewer chroma values in various ways.
Another popular color space is similar:
Y:U:V
Slide 9
Hardware & Algorithm Trade-OffsHardware & Algorithm Trade-Offs
Area
Speed
Memory
System Integration
Application Requirements
Slide 10
AreaArea
JPEG 2000 Parameters
No. of Entropy Encoders
Filter Types & Levels
Quantization Tables
Block Size
Whole Frame or Tiles
Slide 11
SpeedSpeed
JPEG fastest on the market, 1 clock/sample
JPEG 20001.4 clocks/sample, depending on configuration
H.2642.5 clocks/sample
LJPEG and JPEG-LSvery fast, 1 clock/sample
Max clock
rate of the
technology
limits the
frame size
and fps
Slide 12
MemoryMemory
Dependent on frame size
H.264can use SDRAM, DDR or DDR2(more flexible than MPEG4)
JPEG 2000large demands, but can reduce by tiling (at expense of speed)
WVGA480 x 854
1024 x 768
1080p 1920 x 1080
Slide 13
IntegrationIntegration
Latency Time from first input sample to first compressed output
CPU required? Core may have a CPU for configuration,
or read from input stream
Memory Interface Best not embedded in compression core to allow flexibility
Slide 14
Application RequirementsApplication Requirements
Rate Control
Error Resilience
Special FeaturesMultiple ResolutionsQuality LayersRectangular Regions of Interest
Slide 15
Rate ControlRate Control
Trades off transmission bit rate and video quality
Essential for applications with constrained channel bandwidth or storage capacity
Must: Guarantee channel will never be overflowed Maximize quality for even very low bit rates Be adaptable to any type of scene or motion Provide a fast response to changes in video
characteristics Allow simple system integration
Hardware Bit Rate Control is best
Slide 16
Multiple Resolution LevelsMultiple Resolution Levels
JPEG 2000 can store Images at multiple resolutions
Thumbnails quickly pulled from storage and analyzed
Higher resolution images then requested from storage for in-depth analysis
Excellent for reconnaissance& surveillance
From: http://www.centurionalarm.com/j2kvsmpeg4vsjpegvsh264.pdf
Slide 17
Error ResilienceError Resilience
JPEGRestart markers, but works on blocks
JPEG-LSNone
JPEG 2000Packet headers in stream, other features
H.264Reference frame errors extend to dependent frames; CAST adds multiple slices to increase error resilience
Slide 18
Original Lossless710 Kbytes20 Quality Layers
Original Lossless710 Kbytes20 Quality Layers
8:1 Compression88.76Kbytes10 Quality Layers
8:1 Compression88.76Kbytes10 Quality Layers
16:1 Compression44.28 Kbytes5 Quality Layers
16:1 Compression44.28 Kbytes5 Quality Layers
256:1 Compression2774 bytes1 Quality Layer
256:1 Compression2774 bytes1 Quality Layer
Quality Layers with JPEG 2000Quality Layers with JPEG 2000
Military applications: thermal IR & satellite imagery
Multiple layers with different compression ratios
Slide 19
Rectangular Regions of InterestRectangular Regions of Interest
Specify an area is more important than the rest of the image
Vary quality of the region and the background per: Desired
compression ratio Relative
importance of the region
50:1 CompressionRegion: no special importance50:1 CompressionRegion: no special importance
50:1 CompressionRegion: slightly important50:1 CompressionRegion: slightly important
50:1 CompressionRegion: moderately important50:1 CompressionRegion: moderately important
50:1 CompressionRegion: extremely important50:1 CompressionRegion: extremely important
Slide 20
Image & Video IP EvaluationImage & Video IP Evaluation
Get a Bit Accurate Model (BAM) to run your images
Try a Prototype Board with the IP in hardware
Get references in your market sector (military is tough)
Study representative synthesis results for area and max operating frequency
Make sure you will have access to compression engineers
Slide 21
CAST Compression Reference Design SystemCAST Compression Reference Design System
Complete FPGA System Board: Learn compression Evaluate our cores with
your media Get a system head start
Combines multiple cores and essential software
Slide 22
Vendor ExpertiseVendor Expertise
CAST:
Delivering IP for 16 Years Privately-held, independent 1,000 sales to 600 customers
Expert Development Partners
Extreme Customer Focus 24/7, fast-response culture Global team of 100 people
Airborne Reconnaissance
Audio/Video Platform
Broadcast Equipment
Camera Module
Cell Phone Camera Image Processing Chip
Digital Still Camera
Digital Video Recorder
Earth Observation Satellite
High-Definition Monitor
High-Resolution Airborne Reconnaissance Sensor
High-Speed Video Camera
Imaging Chips
Military - Classified
Mobile Camera SoC
Mobile Digital Video Recorder Platform
Mobile Phone SIM Card
Multiple Kill Vehicle
Office Class Copier/printer
Prototype 3D Camera
Satellite - Classified
Satellite Camera and Measurement Device
Security and Surveillance Camera
Security LCD Monitor
SoC-on-panel Display
Surveillance Camera
Technology Proof-of-Concept Camera
TV Surveillance Camera Display
USB Module
Video Compression
Video Conferencing System
Video Image Acquisition System for the Military
Video Surveillance for Road Tolling
Weather Satellites
Wireless Rear-view Camera
X-ray Dental Project
Some Customer Multimedia Applications
Slide 23
CAST Broad IP Cores LineCAST Broad IP Cores Line
MultimediaJPEGJPEG Scalado SpeedtagsJPEG Ext. 12-bitJPEG LosslessJPEG-LS JPEG 2000 H.264 VideoAudio: I2S, SPDIFImage Conversion
System IPSubsystems
8051/USB, 8051/MACSoC Platforms
ARM, eASICAMBA Infrastructure
ReplacementsProcessors, Bit SlicesControllers, Peripherals, Timers, UARTs, TTL Parts
EncryptionAES DES, Triple DES MD5 SHA-1, SHA-256
CommunicationsConsumer IR ControllersSynchronous UARTS SDLC, HDLC
InterfacesPCI & PCI ExpressEthernet MACs USB & OTG CAN, LIN, I2C, SPIECP
Memory & PeripheralsMemory Controllers
DDR1/2 SDRAMSDIO HostSDR mobile SDRAMNAND flash
Serial flashStorage Controllers
ATA/IDE Interface Device Controllers
Smart Card Reader, TV Display, Hi-Res
Processors8-bit: 8051 fast, tiny, entry
Z80, C650216-bit: 68000, 80186xl,
80186ec, 387L32-bit 68000 AHBDSP: 32025, 32025TX
See more at www.cast-inc.com
Slide 24
CAST Image & Video IP CoresCAST Image & Video IP Cores
MultimediaJPEGJPEG Scalado SpeedtagsJPEG Ext. 12-bitJPEG LosslessJPEG-LS JPEG 2000 H.264 VideoAudio: I2S, SPDIFImage Conversion
Cameras, Printers, Copiers, etc.
Pro Video,Medical Imaging
Best Lossless
Surveillance,Medical Imaging
Replacing MPEG4
Fast Cameras
High QualityImaging
Block to RasterRaster to BlockDownscalingColor Space Conversion