1 adaptable applications towards balancing network and terminal resources to improve video quality...
TRANSCRIPT
![Page 1: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/1.jpg)
1
TU/e
Adaptable applicationsTowards Balancing Network and Terminal Resources to Improve Video Quality
D. Jarnikov
![Page 2: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/2.jpg)
2
TU/e
Contents
• Introduction (www)
• New solutions
• Subjective evaluation
• Conclusions and future plans
![Page 3: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/3.jpg)
3
TU/e
Introduction of in-home network
![Page 4: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/4.jpg)
4
TU/e
Problem description
Video data transmission
Video data decoding
Network condition
good bad
Pe
rce
ive
d
vid
eo
qua
lity
bad
good
Pe
rce
ive
d
vid
eo
qua
lity
bad
good
Resource consumptionlow high
![Page 5: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/5.jpg)
5
TU/e
Objective: networked terminal
• Resource-constrained terminal : CPU• Resource-constrained network : bandwidth
• Wireless network has fluctuations
Source Terminal
Wireless network
![Page 6: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/6.jpg)
6
TU/e
Solution
• Scalable video technique– Choose size of base layer such, that we can almost guarantee the
transmission
– The enhancement layer is transmitted if there is available bandwidth
– The number of layers to be decoded can be chosen for every frame
• Controller– Choose how much video data (e.g. layers) should be processed.
– Optimizes perceived quality when looking at available input data AND available CPU power
![Page 7: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/7.jpg)
7
TU/e
Past: Conclusions & Future plans
• The usage of scalable video enables trade-offs between user perceived quality and network and terminal resources
• A controller can be used to optimized perceived quality with respect to the available CPU power and amount of input data
• We developed the controller that doesn’t depend on the scalability technique
• The correctness of controller behavior depends on rightness of parameters
• Take into account other terminal resources
• Organize a feedback from the terminal to the source
• Create MPEG2 to scalable video transcoder
![Page 8: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/8.jpg)
8
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 9: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/9.jpg)
9
TU/e
System: Present view
Source Terminal
Wireless network
TranscoderNetwork
Sender-ReceiverTerminal --
?
![Page 10: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/10.jpg)
10
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 11: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/11.jpg)
11
TU/e
Transcoder: General Info
Separates video stream on most important information and least important information (refinement)
• Input: MPEG2 video• Output: Scalable MPEG2 video• Parameters: number of output layers, sizes of the layers
VLDInverse Quantization
Q-1
stream QuantizationQ
VLC
-
ELstream
BLstream
QuantizationQ’
VLC
![Page 12: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/12.jpg)
12
TU/e
Transcoder: New enhancement layers
From I to B frames:• empty macroblocks can be skipped• variable length coding tables of B frames are better suited to
encode residual values
Advantages: • lower importance of base layer size• less syntax overhead
Both approaches are compliant with new MPEG System proposal
![Page 13: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/13.jpg)
13
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 14: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/14.jpg)
14
TU/e
Controller: General view
• Controller chooses how much video data (e.g. layers) should be processed. Takes into account:– amount of available resources (CPU)– amount of video data available (e.g. how many layers have we
received)
• Objective: maximize perceived quality– MAX number of layers to be processed– MIN deadline misses– MIN quality changes
Terminal
Scalable VideoInput
Decoder
Controller
Post-processing
Controller
![Page 15: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/15.jpg)
15
TU/e
Controller: Markov decision process
State: progress w.r.t. deadline
number of layers decoded
maximal number of layers for the next frame
Revenue:Reward: number of layers
Penalty: deadline misses
Penalty: quality change
Penalty: quality change, caused be the network
![Page 16: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/16.jpg)
16
TU/e
Controller: Challenges
Processing times- depends on content (stochastic)- depends on layer size
Probabilities change for every possible layer size
=> Unique strategy for every layer size
Maximal number of layers for the next frame- network-dependent parameter (stochastic)
We need appropriate network behavior model Unique strategy for every network condition
Number of layers in total Unique strategy for every number of layers
![Page 17: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/17.jpg)
17
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 18: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/18.jpg)
18
TU/e
Network sender-receiver (NSR)
• Streams layers over the network
• Takes care about layer prioritization
• Sends feedback about receiving conditions
Streaming conditions + Receiving conditions = Network conditions
μ, BER / PER
![Page 19: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/19.jpg)
19
TU/e
NSR: Channel Model
• Using μ, BER / PER we can build a simple channel model [example: Gilbert-Elliott channel (GEC)]
• We run a simulations for different network conditions and layer configurations
• For every pair μ, BER / PER we can estimate what is an optimal layer configuration (lookup table)
Layer configuration is a run-time input for the transcoder
![Page 20: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/20.jpg)
20
TU/e
NSR: Controller input
Processing times- depends on content (stochastic)- depends on layer size
Probabilities change for every possible layer size layer size
=> Unique strategy for every layer size
Maximal number of layers for the next frame- network-dependent parameter (stochastic) number of layers
We need appropriate network behavior model layer size Unique strategy for every network conditions μ, BER / PER
Number of layers in total Unique strategy for every number of layers number of layers
BUT! Layer sizes and number of layers have one-to-one relation with μ, BER / PER
![Page 21: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/21.jpg)
21
TU/e
Contents
• Introduction (www)
• New solutions
– System view
– Transcoder
– Controller
– Network sender-receiver
– Summary
• Subjective evaluation
• Conclusions and future plans
![Page 22: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/22.jpg)
22
TU/e
System: Global view
TranscoderNetwork
Sender-ReceiverTerminal --
μ, BER / PER
number of layerslayers sizes
strategy
Calculatedoffline
Alt
erna
tive
![Page 23: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/23.jpg)
23
TU/e
Contents
• Introduction (www)
• New solutions
• Subjective evaluation
• Conclusions and future plans
![Page 24: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/24.jpg)
24
TU/e
User tests
How does proportion between sizes of BL and EL influence perceived quality?
What is better: a large EL with high losses or small EL with low losses?
RESULTS: • BL size is very important• With decrease of overall bit-rate the importance of BL size
increases• With increase of EL size married to a lower frequency is
perceived slightly worse
![Page 25: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/25.jpg)
25
TU/e
Contents
• Introduction (www)
• New solutions
• Subjective evaluation
• Conclusions and future plans
![Page 26: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/26.jpg)
26
TU/e
Conclusions
• Made a transcoder that produces less bit per EL for the same quality
• Made a network simulation that allows a better choice of layer configuration
• Enhanced a terminal controller with realistic network behavior model
• User tests were performed for perceived quality evaluation of a scalable video
![Page 27: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/27.jpg)
27
TU/e
Future plans
• Take into account other terminal resources
• Enhance the feedback mechanism from the terminal to the source with terminal capabilities information
• Allow source-terminal resource negotiations
• Perform subjective test for dynamic behavior of scalable video scheme
• Implement transcoder on CE platform
![Page 28: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/28.jpg)
28
TU/e
?
![Page 29: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/29.jpg)
29
TU/e
Transcoder: Comparison of the approaches
0
2
4
6
8
10
12
14
BL BL&EL
PS
NR
dif
fere
nc
e w
ith
re
fere
nc
e (d
B)
1 + 4 2 + 3 2.5 + 2.5 3 + 2 4 + 1
0
2
4
6
8
10
12
14
BL BL&EL
PS
NR
dif
fere
nce
wit
h r
efe
ren
ce
(dB
)
1 + 4 2 + 3 2.5 + 2.5 3 + 2 4 + 1
Difference in PSNR between one-layer reference and two-layer scalable coding
(the overall bitrate is 5 MBps)
I-frame approach
B-frame approach
![Page 30: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/30.jpg)
30
TU/e
NSR: Channel Model
• Using μ, BER / PER we can build a simple channel model [example: Gilbert-Elliott channel (GEC)]
• We model the channel as a two-state discrete time Markov chain (DTMC) with states G (good) and B (bad) and four probabilities P, Q, EG, and EB
G B
P
Q
1-P 1-Q
)QP( 1
1
50 P.BER
![Page 31: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/31.jpg)
31
TU/e
NSR: Channel Model Simulation
![Page 32: 1 Adaptable applications Towards Balancing Network and Terminal Resources to Improve Video Quality D. Jarnikov](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ef45503460f94c06b01/html5/thumbnails/32.jpg)
32
TU/e
NSR: Channel Model Outcome
For every pair μ, BER / PER we can estimate what is an optimal layer configuration (lookup table)
Layer configuration is a run-time input for the transcoder
Example: maximal BL bitrate as a function of network conditions