adaptable applications towards balancing network and terminal resources to improve video quality
DESCRIPTION
Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality. D. Jarnikov. Contents. Introduction (www) New solutions Subjective evaluation Conclusions and future plans. Introduction of in-home network. Problem description. Video data transmission. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/1.jpg)
1
TU/e
Adaptable applicationsTowards Balancing Network and Terminal Resources to Improve Video Quality
D. Jarnikov
![Page 2: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/2.jpg)
2
TU/e
Contents
• Introduction (www)
• New solutions
• Subjective evaluation
• Conclusions and future plans
![Page 3: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/3.jpg)
3
TU/e
Introduction of in-home network
![Page 4: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/4.jpg)
4
TU/e
Problem description
Video data transmission
Video data decoding
Network condition
good bad
Pe
rce
ive
d
vid
eo
qua
lity
bad
good
Pe
rce
ive
d
vid
eo
qua
lity
bad
good
Resource consumptionlow high
![Page 5: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/5.jpg)
5
TU/e
Objective: networked terminal
• Resource-constrained terminal : CPU• Resource-constrained network : bandwidth
• Wireless network has fluctuations
Source Terminal
Wireless network
![Page 6: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/6.jpg)
6
TU/e
Solution
• Scalable video technique– Choose size of base layer such, that we can almost guarantee the
transmission
– The enhancement layer is transmitted if there is available bandwidth
– The number of layers to be decoded can be chosen for every frame
• Controller– Choose how much video data (e.g. layers) should be processed.
– Optimizes perceived quality when looking at available input data AND available CPU power
![Page 7: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/7.jpg)
7
TU/e
Past: Conclusions & Future plans
• The usage of scalable video enables trade-offs between user perceived quality and network and terminal resources
• A controller can be used to optimized perceived quality with respect to the available CPU power and amount of input data
• We developed the controller that doesn’t depend on the scalability technique
• The correctness of controller behavior depends on rightness of parameters
• Take into account other terminal resources
• Organize a feedback from the terminal to the source
• Create MPEG2 to scalable video transcoder
![Page 8: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/8.jpg)
8
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 9: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/9.jpg)
9
TU/e
System: Present view
Source Terminal
Wireless network
TranscoderNetwork
Sender-ReceiverTerminal --
?
![Page 10: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/10.jpg)
10
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 11: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/11.jpg)
11
TU/e
Transcoder: General Info
Separates video stream on most important information and least important information (refinement)
• Input: MPEG2 video• Output: Scalable MPEG2 video• Parameters: number of output layers, sizes of the layers
VLDInverse Quantization
Q-1
stream QuantizationQ
VLC
-
ELstream
BLstream
QuantizationQ’
VLC
![Page 12: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/12.jpg)
12
TU/e
Transcoder: New enhancement layers
From I to B frames:• empty macroblocks can be skipped• variable length coding tables of B frames are better suited to
encode residual values
Advantages: • lower importance of base layer size• less syntax overhead
Both approaches are compliant with new MPEG System proposal
![Page 13: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/13.jpg)
13
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 14: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/14.jpg)
14
TU/e
Controller: General view
• Controller chooses how much video data (e.g. layers) should be processed. Takes into account:– amount of available resources (CPU)– amount of video data available (e.g. how many layers have we
received)
• Objective: maximize perceived quality– MAX number of layers to be processed– MIN deadline misses– MIN quality changes
Terminal
Scalable VideoInput
Decoder
Controller
Post-processing
Controller
![Page 15: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/15.jpg)
15
TU/e
Controller: Markov decision process
State: progress w.r.t. deadline
number of layers decoded
maximal number of layers for the next frame
Revenue:Reward: number of layers
Penalty: deadline misses
Penalty: quality change
Penalty: quality change, caused be the network
![Page 16: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/16.jpg)
16
TU/e
Controller: Challenges
Processing times- depends on content (stochastic)- depends on layer size
Probabilities change for every possible layer size
=> Unique strategy for every layer size
Maximal number of layers for the next frame- network-dependent parameter (stochastic)
We need appropriate network behavior model Unique strategy for every network condition
Number of layers in total Unique strategy for every number of layers
![Page 17: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/17.jpg)
17
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 18: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/18.jpg)
18
TU/e
Network sender-receiver (NSR)
• Streams layers over the network
• Takes care about layer prioritization
• Sends feedback about receiving conditions
Streaming conditions + Receiving conditions = Network conditions
μ, BER / PER
![Page 19: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/19.jpg)
19
TU/e
NSR: Channel Model
• Using μ, BER / PER we can build a simple channel model [example: Gilbert-Elliott channel (GEC)]
• We run a simulations for different network conditions and layer configurations
• For every pair μ, BER / PER we can estimate what is an optimal layer configuration (lookup table)
Layer configuration is a run-time input for the transcoder
![Page 20: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/20.jpg)
20
TU/e
NSR: Controller input
Processing times- depends on content (stochastic)- depends on layer size
Probabilities change for every possible layer size layer size
=> Unique strategy for every layer size
Maximal number of layers for the next frame- network-dependent parameter (stochastic) number of layers
We need appropriate network behavior model layer size Unique strategy for every network conditions μ, BER / PER
Number of layers in total Unique strategy for every number of layers number of layers
BUT! Layer sizes and number of layers have one-to-one relation with μ, BER / PER
![Page 21: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/21.jpg)
21
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 22: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/22.jpg)
22
TU/e
System: Global view
TranscoderNetwork
Sender-ReceiverTerminal --
μ, BER / PER
number of layerslayers sizes
strategy
Calculatedoffline
Alt
erna
tive
![Page 23: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/23.jpg)
23
TU/e
Contents
• Introduction (www)
• New solutions
• Subjective evaluation
• Conclusions and future plans
![Page 24: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/24.jpg)
24
TU/e
User tests
How does proportion between sizes of BL and EL influence perceived quality?
What is better: a large EL with high losses or small EL with low losses?
RESULTS: • BL size is very important• With decrease of overall bit-rate the importance of BL size
increases• With increase of EL size married to a lower frequency is
perceived slightly worse
![Page 25: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/25.jpg)
25
TU/e
Contents
• Introduction (www)
• New solutions
• Subjective evaluation
• Conclusions and future plans
![Page 26: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/26.jpg)
26
TU/e
Conclusions
• Made a transcoder that produces less bit per EL for the same quality
• Made a network simulation that allows a better choice of layer configuration
• Enhanced a terminal controller with realistic network behavior model
• User tests were performed for perceived quality evaluation of a scalable video
![Page 27: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/27.jpg)
27
TU/e
Future plans
• Take into account other terminal resources
• Enhance the feedback mechanism from the terminal to the source with terminal capabilities information
• Allow source-terminal resource negotiations
• Perform subjective test for dynamic behavior of scalable video scheme
• Implement transcoder on CE platform
![Page 28: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/28.jpg)
28
TU/e
?
![Page 29: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/29.jpg)
29
TU/e
Transcoder: Comparison of the approaches
0
2
4
6
8
10
12
14
BL BL&EL
PS
NR
dif
fere
nc
e w
ith
re
fere
nc
e (d
B)
1 + 4 2 + 3 2.5 + 2.5 3 + 2 4 + 1
0
2
4
6
8
10
12
14
BL BL&EL
PS
NR
dif
fere
nce
wit
h r
efe
ren
ce
(dB
)
1 + 4 2 + 3 2.5 + 2.5 3 + 2 4 + 1
Difference in PSNR between one-layer reference and two-layer scalable coding
(the overall bitrate is 5 MBps)
I-frame approach
B-frame approach
![Page 30: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/30.jpg)
30
TU/e
NSR: Channel Model
• Using μ, BER / PER we can build a simple channel model [example: Gilbert-Elliott channel (GEC)]
• We model the channel as a two-state discrete time Markov chain (DTMC) with states G (good) and B (bad) and four probabilities P, Q, EG, and EB
G B
P
Q
1-P 1-Q
)QP( 1
1
50 P.BER
![Page 31: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/31.jpg)
31
TU/e
NSR: Channel Model Simulation
![Page 32: Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022062517/56813e7c550346895da8a87e/html5/thumbnails/32.jpg)
32
TU/e
NSR: Channel Model Outcome
For every pair μ, BER / PER we can estimate what is an optimal layer configuration (lookup table)
Layer configuration is a run-time input for the transcoder
Example: maximal BL bitrate as a function of network conditions