Euro NF Ph D Course on QoEEuro-NF Ph.D. Course on QoE

--Quantitative RelationshipsBetween QoE and QoS

Markus FiedlerMarkus Fiedler

How to derive QoE QoS relationships?How to derive QoE–QoS relationships?• Experiments

– (Groups of) {naïve|expert} user(s)– Well-defined conditions and network stimuli

• Data collection– Subjective: questionnaires, observations– Objective: Measurementsj

• Analysis– Trends relationships thresholds– Trends, relationships, thresholds– Regressions, hypotheses, impact analysis

DreamhackDreamhack Winter 2007 Winter 2007 –– GamingGaming QoEQoEDreamhackDreamhack Winter 2007 Winter 2007 GamingGaming QoEQoE

DreamhackDreamhack Winter 2007 Winter 2007 –– ShapingShaping / / QoSQoS measurementsmeasurementsDreamhackDreamhack Winter 2007 Winter 2007 ShapingShaping / / QoSQoS measurementsmeasurements

QoE in ITU T Rec G 1080QoE in ITU-T Rec. G.1080• Minimum level of transport layer performance

required to provide satisfactory QoE for H.262 encoded HDTV services – Jitter < 50 ms – Max. duration of single error event: ≤ 16 ms

• Corresponding loss period in IP packets: < 24 @ 15 0 Mbit/s < 29 @ 18 1 Mbit/s< 24 @ 15.0 Mbit/s … < 29 @ 18.1 Mbit/s

– Loss distance: ≤ 1 error event per 4 hours• Corresponding average IP video stream• Corresponding average IP video stream

packet loss rate: ≤ 1.17⋅10–6

Mobile videos with loss introducedMobile videos with loss introducedMOS vs. loss ratio

5MOS vs. loss ratio

QoEMoVi PhD students +

4supervisorsDreamhack, > 20 years

D h k 203

Dreamhack, < 20 years

Summer school

2 BTH students

avg10.0% 0.2% 0.4% 0.6% 0.8% 1.0%

avg

Fundamental QoE QoS relationshipsFundamental QoE–QoS relationships• Dependencies on network conditions typically addressed• Dependencies on network conditions typically addressed

by parameter vectors [QoSi, …, QoE]– Results from questionnaires, observations, measurementsq , ,– Several impact factors: QoE = f(QoS1, QoS2, …)

• We focus on one impact factor at a time• Description by partial differential equations• Consider fundamental relationships of the typep yp

( , )ii

QoE g QoE QoSQoS∂

=∂

• Maximise/minimise QoE to interval [1, 5] afterwards

Fundamental QoE QoS relationshipsFundamental QoE–QoS relationships• Investigated set

– Linear QoE ∂ QoSi

Logarithmic QoE log(QoS )– Logarithmic QoE ∂ log(QoSi)– Exponential log(QoE) ∂ QoSi

– Power log(QoE) ∂ log(QoSi)g(Q ) g(Q i)

• Nice properties– Seen from regressions on linear vs. logarithmic scales– Reasoning behind each relationship

• Examples– Most formulae from [Shaikh et al., 2010]– Work supported by Euro-NF

Linear relationship QoE QoS∂ ∝ ∂Linear relationship iQoE QoS∂ ∝ ∂

QoE axis QoS axis• Linear scale Linear scale• Additive change Additive change

• QoE gradient independent of QoE and QoSQoE gradient independent of QoE and QoS• Linear regression often the first choice• Local approximation• Local approximation• Example:

– Download time perception as function of lossDownload time perception as function of loss24.3 31 ( 0.99)QoE PLR≈ − ℜ >

Logarithmic relationship iQoSQoE ∂∂ ∝Logarithmic relationship

i

QoEQoS

∂ ∝

QoE axis QoS axis• Linear scale Logarithmic scale• Additive change Multiplicative change

• QoE gradient proportional to reciprocal QoSQoE gradient proportional to reciprocal QoS• Weber-Fechner Law (1834):

– Just noticeable differences, multiplication on stimuli sideJust noticeable differences, multiplication on stimuli side

• Utility functions (implicit proportional fairness)• Example:Example:

– Download time perception as function of bandwidth21 2 3 3lg( /Mbps) ( 0 99)QoE R≈ + ℜ >1.2 3.3lg( /Mbps) ( 0.99)QoE R≈ + ℜ >

Exponential relationship QoE QoS∂∝ ∂Exponential relationship iQoS

QoE∝ ∂

QoE axis QoS axis• Logarithmic scale Linear scale• Multiplicative change Additive change

• QoE gradient proportional to actual QoEQoE gradient proportional to actual QoE– Nuclear decay– Human memory– IQX hypothesis [Hossfeld et al., 2007; Fiedler et al., 2010]

• Examples– Image quality perception as function of blur, blockiness, … (QoP)– Download time perception as function of response time (QoD)

24.8exp( 0.15 /s) ( 0.99)QoE RT≈ − ℜ >

Power type relationship iQoSQoE ∂∂∝

Q E i Q S i

Power-type relationshipiQoE QoS

∝

QoE axis QoS axis• Logarithmic scale Logarithmic scale• Multiplicative change Multiplicative change

• Long tails on both axesLong tails on both axes• Examples

– Session volume asy = 14.613x-0.386

R² = 0.6803OS vs PDV (ms)

0.39 214.6( /ms) ( 0.68)QoE PDV −≈ ℜ ≈Session volume as function of bandwidth

– Video perception as function of jitter

4

5

6

function of jitter

1

2

3OS

0

1

0 100 200 300 400 500 600 700 800PDV(ms)

Provisioning delivery hysteresisProvisioning-delivery hysteresis• Different types of QoE and QoS parameter –

Distinguish between– Success and failure– Resource and problem– Promise-making and promise-keeping

• Examplesp• Provisioning-delivery hysteresis

– DerivationDerivation– Examples

Different types of QoE and QoS parametersDifferent types of QoE and QoS parameters

Q E Q S• Success rating – The higher, the better

Mean Opinion Score (1 5)

• Resource measure– The higher, the better

Throughput

QoE rQoS

– Mean Opinion Score (1..5)

• Failure rating

– Throughput

• Success measure– Availability (e g 99 99 %)QoE

sQoS• Failure rating

– The higher, the worse– Cancellation rate

– Availability (e.g. 99.99 %)– Packet success ratio

QoE

Cancellation rate– Churn rate • Failure measure

– The higher, the worseP k t l ti

fQoS

• Watch the signs!– Packet loss ratio– Delay jitter– ReorderingReordering

Skype: MOS = f(packet loss ratio)Skype: MOS = f(packet loss ratio)[Fiedler et al., 2010]

QoE[ ]

fQoS

Skype: MOS = f(reordered ratio)Skype: MOS = f(reordered ratio)[Fiedler et al., 2010]

QoE[ , ]

fQoS

G 1030/download: MOS = f(session time)G.1030/download: MOS = f(session time)[Fiedler et al., 2010]

QoE[ , ]

fQoS

Web: Cancel rate = f(delivery bandwidth)Web: Cancel-rate = f(delivery bandwidth)[Khirman & Henriksen, 2002]

[Fiedler et al., 2010]

QoE[ , ]

rQoS

Web: MOS = f(throughput)Web: MOS = f(throughput)[Shaikh et al., 2010]

QoE[ ]

rQoS

Basic shapesBasic shapesQ E QoE QoEQoE QoE QoE

FLIP

QoS QoS QoSresource problem

rQoS fQoS

QoEFLIP

sQoS

P

Relationship?

rQoSp

Provisioning delivery hysteresis for webProvisioning-delivery hysteresis for web

1 – PLR

[Fi dl & H f ld 2010][Fiedler & Hossfeld, 2010]

Provisioning delivery hysteresis for videoProvisioning-delivery hysteresis for videoPotential forPotential forPotential for Potential for

savingssavings

To beTo beTo be To be avoidedavoided

[Zi t l 2010][Zinner et al., 2010]

ConclusionConclusion• Given you are to study QoE–QoS relationships

– Get users involved – it’s fun!Try objective measurements of user action– Try objective measurements of user action

• Check out [ETSI STF 354]– Play with the logscale button on both axesy g

• Eventually, offsets need to be taken away– Interprete the outcome

Additi lti li ti h ? I li ti ?• Additive or multiplicative change? Implications?

• Please let me know of new fundamental relationships• Please let me know of new fundamental relationshipsand examples you publish: markus.fiedler@bth.se