![Page 1: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/1.jpg)
Image Sequence Codingby Split and Merge
Patrice Willemin, Todd R. Reed and Murat Kunt
Presented by: Idan Shatz
![Page 2: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/2.jpg)
Outline
• Split and Merge Coding– For Still Images
• Split and Merge Coding– For Image Sequence
• Conclusions
![Page 3: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/3.jpg)
Image Coding
• First Generation– Pels coding (prediction or transform)
• Second Generation– Object Based coding– Region based coding
• Split and Merge Method
![Page 4: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/4.jpg)
Split & Merge
• Split…
• Recursive algorithm– Approximate a region by a 2D polynomial– If (approximation error > threshold)
• Split the region into 4 sub-regions• Approximate each sub-region
![Page 5: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/5.jpg)
Split Example
Original Image Split, depth=1
![Page 6: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/6.jpg)
Split Example
Original Image Split, depth=2
![Page 7: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/7.jpg)
Split Example
Original Image Split, depth=3
![Page 8: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/8.jpg)
Split Example
Original Image Split, depth=4
![Page 9: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/9.jpg)
Split Example
Original Image Split, depth=5
![Page 10: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/10.jpg)
Split Example
Original Image Split, depth=6
![Page 11: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/11.jpg)
Split Example
Original Image Split, depth=7
![Page 12: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/12.jpg)
Split Example
Original Image Split, Final resultsThreshold=100,000
![Page 13: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/13.jpg)
The threshold100,000 50,000 20,000 10,000 5,000
2,000 1,000 500 200 100
![Page 14: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/14.jpg)
5,000
100,000
100
102
104
106
108
0
500
1000
1500
2000
2500
3000
3500
4000
4500
# of
reg
ions
Threshold
The threshold
![Page 15: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/15.jpg)
Split & Merge
• An improvement of the Split algorithm– At each step decide between x-split and y-split
• produces fewer regions
![Page 16: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/16.jpg)
Split Example (2)
Original Image Split, depth=1
![Page 17: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/17.jpg)
Split Example (2)
Original Image Split, depth=2
![Page 18: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/18.jpg)
Split Example (2)
Original Image Split, depth=3
![Page 19: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/19.jpg)
Split Example (2)
Original Image Split, depth=4
![Page 20: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/20.jpg)
Split Example (2)
Original Image Split, depth=5
![Page 21: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/21.jpg)
Split Example (2)
Original Image Split, depth=6
![Page 22: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/22.jpg)
Split Example (2)
Original Image Split, depth=7
![Page 23: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/23.jpg)
Split Example (2)
Original Image Split, depth=8
![Page 24: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/24.jpg)
Split Example (2)
Original Image Split, depth=9
![Page 25: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/25.jpg)
Split Example (2)
Original Image Split, depth=10
![Page 26: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/26.jpg)
Split Example (2)
Original Image Split, depth=11
![Page 27: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/27.jpg)
Split Example (2)
Original Image Split, depth=12
![Page 28: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/28.jpg)
Split Example (2)
Original Image Split, depth=13
![Page 29: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/29.jpg)
Split Example (2)
Original Image Split, Final resultsThreshold = 100,000
![Page 30: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/30.jpg)
Split Example (Comparison)
Split – 4 sub regionsThreshold = 100,000# regions = 613
Split – 2 sub regionsThreshold = 100,000# regions = 387
![Page 31: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/31.jpg)
Comparison
100
102
104
106
108
0
500
1000
1500
2000
2500
3000
3500
4000
4500
Threshold
# of
reg
ions
4-split2-split
![Page 32: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/32.jpg)
Image Coding
• Each region is codes by:– The Border– 2D polynomial coefficients
![Page 33: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/33.jpg)
The borders of the regions
• BSP-Tree– Binary space partition tree– Tree Nodes:
• “X” – mark an X-split• “Y” – mark an Y-split• “0” – mark an Region
![Page 34: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/34.jpg)
Regions Borders
• Example of BSP-Tree
Y
X 0
0 Y
X
0 0
0
![Page 35: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/35.jpg)
Storing a BSP Tree
• N regions 2N-1 BSP-Nodes
• Storing the tree in preorder
Y
X 0
0 Y
X
0 0
0
Y X Y X 00 0 0 0
![Page 36: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/36.jpg)
Image Coding
• For N regions– Border (BSP) – N*3[bit]– 2D polynomial coefficients – N*24[bit]
• 8[bit] x 3 – gray level.• 8[bit] x 3 x 3 - for RGB.
![Page 37: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/37.jpg)
Image Coding
• Without Compression– 196,608[Byte]
• With Compression– # regions = 387– 3628[Byte]
• Compression Raito:– 1 to 54
Split – 2 sub regions
![Page 38: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/38.jpg)
Split & Merge
• Merge…– For each region:
• 3[bit] border << 54[bit] Coefficients
• Reduce the number of regions– Merging neighbor regions with similar
coefficients.
![Page 39: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/39.jpg)
Merge Example (Comparison)
Split – 4 sub regionsThreshold = 100,000# regions = 613After Merge# region = 411
Split – 2 sub regionsThreshold = 100,000# regions = 387After Merge# regions = 348
![Page 40: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/40.jpg)
Comparison
100
101
102
103
104
105
106
107
0
500
1000
1500
2000
2500
3000
3500
4000
4500
Threshold
# of
reg
ions
split4split2split4 + mergesplit2 + merge
![Page 41: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/41.jpg)
The borders of the regions
• The borders are more complex– Simple BSP Tree is not enough
• Therefore– BSP Tree– Regions labels
![Page 42: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/42.jpg)
Regions Labeling
• How many bits are needed for labeling?– ~1000 regions ~10[bit]
• Non-neighbor regions can have the same labeling– ~3[bit] for labeling– Does not depend on the
number of regions
![Page 43: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/43.jpg)
Image Coding
• For N regions made from M BSP-regions– Border (BSP) – M*3[bit]– Labeling – M*3[bit]– 2D polynomial coefficients – N*24[bit]
• 8[bit] x 3 – gray level.• 8[bit] x 3 x 3 - for RGB.
![Page 44: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/44.jpg)
Image Coding
• Without Compression– 196,608[Byte]
• With Compression– # BSP regions = 387– # regions = 348– 3446[Byte]
• Compression Raito:– 1 to 57 (>54)
Split – 2 sub regionsAnd merging
![Page 45: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/45.jpg)
• Same Idea.– Still Image (x,y)– Video (x,y,t)
• Split Stage– Decide between x-split, y-split and t-split.
• Merge Stage– Same
Image Sequence Coding
y
x
t
![Page 46: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/46.jpg)
Real Time Transmission
• Compress all movie at once– Achieving best compression.– However
• can’t be transmitted on-line• Very high complexity
• Deal with chunks of 5~25 frames– Compression ratio decreases– Can be transmitted on-line
![Page 47: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/47.jpg)
Conclusions
• Still Images compression– Achieves good results with good compression
ratio.– 2 major defects due to region boundaries:
• Disappearance of important boundaries• Border artifacts between regions.
– Merge Regions• using the same threshold has little effect on performance• adaptive threshold for achieving a giving compression
ratio
![Page 48: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/48.jpg)
Conclusions
• Image Sequence compression– Achieves good results
• At the article: 1 to 200 comp. ratio.
– But, compression ratio decreases on high movement scenes.
• compression ratio still images.• Merging regions can be used to balance the buffer size.
– Hard to decode at real-time.• Must be coded at frame chunks.• Decoding process is hard to implement in real time
environment.
![Page 49: Image Sequence Coding by Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz](https://reader034.vdocuments.mx/reader034/viewer/2022051618/56649d365503460f94a0f037/html5/thumbnails/49.jpg)
Go To…• Still Images
– Split• to 4 sub-regions• to 2 sub-regions• Threshold (2)• Compression
– Merge• Threshold• Compression
• Image Sequence• Conclusions