Packet audio Packet audio playout delay playout delay adjustmentadjustment

Performance bounds and Performance bounds and algorithmsalgorithms

Moon, Kurose, TowsleyMoon, Kurose, Towsley

Overall IdeaOverall Idea

Because of packet jitter/delay Because of packet jitter/delay changes, we need a changes, we need a playout bufferplayout buffer

The bigger, the betterThe bigger, the better But, a large buffer hinders But, a large buffer hinders

responsive transmission of audioresponsive transmission of audio 400ms/5% loss for voice conversation400ms/5% loss for voice conversation Interactive media/video conferencing Interactive media/video conferencing

needs the smallest buffers possibleneeds the smallest buffers possible

A solution, and an A solution, and an approximationapproximation

In the first part of the paper, they give In the first part of the paper, they give bounds on the size of playout buffer bounds on the size of playout buffer needed under certain lossesneeded under certain losses Not an online algorithm, computationally Not an online algorithm, computationally

expensive (the idea is to focus on expensive (the idea is to focus on percentages)percentages)

Inelastic mediumInelastic medium In the second part of the paper, they In the second part of the paper, they

present an on-line algorithm that is present an on-line algorithm that is computationally feasible to adjust computationally feasible to adjust talkspurt playout delaytalkspurt playout delay

Related WorkRelated Work

Playout delay adjustmentsPlayout delay adjustments Per-packet and per-talkspurt Per-packet and per-talkspurt

(assumptions… speech, or music?)(assumptions… speech, or music?) Network level observationsNetwork level observations

Three graphs, probe compressionThree graphs, probe compression Baseline doesn’t change much—real Baseline doesn’t change much—real

advantages in adjusting delay advantages in adjusting delay playout occurs in multi-talkburst playout occurs in multi-talkburst delay spikesdelay spikes

Problem StatementProblem Statement

For a given set of losses at the For a given set of losses at the receiver, we get to set the playout receiver, we get to set the playout delays of each talkspurt anyway we delays of each talkspurt anyway we wantwant Which assignment is the best?Which assignment is the best? For 1 packet lost? 2? 3? 134?For 1 packet lost? 2? 3? 134?

First, let’s fix some notationFirst, let’s fix some notation

NotationNotation

ttkki – i – sender timestamp of sender timestamp of iith packet of th packet of kkth th

talkspurttalkspurt aakk

ii – receiver timestamp of – receiver timestamp of iith packet of th packet of kkth th talkspurttalkspurt

nnkk – num packets in – num packets in kkth talkspurt (received)th talkspurt (received)

N – total number of packets in trace (N – total number of packets in trace (ΣΣkknnkk))

ppkkii(A) – playout time under algorithm A(A) – playout time under algorithm A

Delay: pDelay: pkkii(A) – t(A) – tkk

ii, loss if p, loss if pkkii(A) < a(A) < akk

ii

Indicator if packet is played:Indicator if packet is played: rrkk

ii(A)(A)

Notation, (con’t)Notation, (con’t)

Total # packets played under ATotal # packets played under A N(A) = N(A) = ΣΣkk

M M ΣΣiinknk r rkk

ii(A)(A)

Average playout delay:Average playout delay: 1/N(A) 1/N(A) ΣΣkk

M M ΣΣiinknk r rkk

ii(A)(p(A)(pkkii(A) – t(A) – tkk

ii))

Loss rate:Loss rate: l = (N – N(A)) / N * 100l = (N – N(A)) / N * 100

Notation (con’t)Notation (con’t)

d’d’kkii: delay between sending and : delay between sending and

receivingreceiving d’: min (d’d’: min (d’kk

ii))

ddkkii: normalized delay = d’: normalized delay = d’kk

ii – d’ – d’

ddkk(i)(i): ith smallest normalized delay: ith smallest normalized delay

Off-line solution w/o Off-line solution w/o collisionscollisions

To play To play i i packets from the packets from the kkth th talkspurt, the playout delay must be at talkspurt, the playout delay must be at least (the unknowable) dleast (the unknowable) dkk

(i)(i)

Remember that if algorithm A uses a Remember that if algorithm A uses a large playout delay for one talkspurt, large playout delay for one talkspurt, it could delay subsequent talkspurts it could delay subsequent talkspurts (collisions)(collisions)

Let’s ignore them for nowLet’s ignore them for now Time: O(MNTime: O(MN22) Space: O(MN)) Space: O(MN)

Off-line solution w/o Off-line solution w/o collisionscollisions

We assume percentages of loss, not actual We assume percentages of loss, not actual loss patterns (to simplify the complexity)loss patterns (to simplify the complexity)

D(k,i) is min playout delay for i packets D(k,i) is min playout delay for i packets lostlost

D(k,i) =D(k,i) = 0 if i = 00 if i = 0 ddkk

(i)(i) if k = M and i <= n if k = M and i <= nMM

inf if k = M and i > ninf if k = M and i > nMM

min (((i-j)D(k+1,i-j) + jdmin (((i-j)D(k+1,i-j) + jdkk(j(j))/i)))/i)

Proof by contradictionProof by contradiction

Offline algorithm with Offline algorithm with collisionscollisions

We might have to adjust the playout times of We might have to adjust the playout times of some of the talkspurts due to collisions, so D some of the talkspurts due to collisions, so D must now take those into accountmust now take those into account

We define a vector S (captures length of silence)We define a vector S (captures length of silence) We can capture the sum of the increasesWe can capture the sum of the increases Now D includes C as well (C tracks packets Now D includes C as well (C tracks packets

played out at every step of the computation)played out at every step of the computation) D now differs from the old D only in the extra D now differs from the old D only in the extra

delays incurred by the collisionsdelays incurred by the collisions The new D does not capture the optimal, though The new D does not capture the optimal, though

(why?)(why?) Time: O(MTime: O(M22NN22) Space: O(M) Space: O(M22NN22))

An online algorithmAn online algorithm

Algorithm 1: LinearAlgorithm 1: Linear Slow to catch up, good at maintaining a solid Slow to catch up, good at maintaining a solid

valuevalue Algorithm 2: Depends on spike detectionAlgorithm 2: Depends on spike detection

Quick at catching up, but sometimes Quick at catching up, but sometimes overzealousoverzealous

Algorithm 3: Two ModesAlgorithm 3: Two Modes Track spikes when they are detectedTrack spikes when they are detected Otherwise update delay and delay varience (q)Otherwise update delay and delay varience (q) Switch when you have a multiple of the delaySwitch when you have a multiple of the delay

Evaluation / ConclusionEvaluation / Conclusion

They instrument the senders and the They instrument the senders and the receiversreceivers

Plot average playout delay vs packet loss Plot average playout delay vs packet loss raterate

Results seem to show that Algorithm 3 gets Results seem to show that Algorithm 3 gets very close to the optimalvery close to the optimal

However, the results are very close much However, the results are very close much of the timeof the time

Sometimes 1 is much worse, sometimes 2, Sometimes 1 is much worse, sometimes 2, but 3 seems to always be pretty stablebut 3 seems to always be pretty stable

Queue Queue MonitoringMonitoring

A Delay Jitter Management A Delay Jitter Management PolicyPolicy

Stone, JeffayStone, Jeffay

Display and e2e JitterDisplay and e2e Jitter Recall the steps for transmitting video:Recall the steps for transmitting video:

Acquire, digitize, compress, transmit, Acquire, digitize, compress, transmit, decompressed, buffer, displaydecompressed, buffer, display

Display LatencyDisplay Latency is acquire to display is acquire to display e2e latencye2e latency is acquire to buffer is acquire to buffer

What problems can affect this process?What problems can affect this process? Delay Jitter (variance in e2e latency)Delay Jitter (variance in e2e latency)

Can we ensure constant e2e latency?Can we ensure constant e2e latency? Even with Isochronous service models?Even with Isochronous service models?

We’re going to adjust the display latency We’re going to adjust the display latency insteadinstead

Audio vs videoAudio vs video

Recall the audio applicationRecall the audio application Talkspurts vs Silence PeriodsTalkspurts vs Silence Periods Analog for video?Analog for video?

Are gaps ok during the transmission?Are gaps ok during the transmission? Display perceptionDisplay perception Network congestionNetwork congestion

Video as a datatypeVideo as a datatype Can we repeat frames, leave black spaces, Can we repeat frames, leave black spaces,

etc?etc?

Late policiesLate policies

I-policy:I-policy: DiscardDiscard All frames now have the same display All frames now have the same display

latencylatency StaticStatic

E-policy:E-policy: Play at earliest conveniencePlay at earliest convenience Increases latency for subsequent framesIncreases latency for subsequent frames Keeps getting higher than observed e2e Keeps getting higher than observed e2e

delaydelay

Example 1Example 1

I-Policy

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 10

Queue Length Display Latency

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10

E-Policy

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 10

Queue Length Display Latency

Example 2Example 2

I-Policy

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 10

Queue Length Display Latency

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10

E-Policy

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 10

Queue Length Display Latency

I-vs-EI-vs-E

I policy’s advantageI policy’s advantage Low jitter and burstsLow jitter and bursts

E policy’s advantageE policy’s advantage Good during high latency and low Good during high latency and low

latency, but not good after burstslatency, but not good after bursts Hybrid approach: Hybrid approach: Queue MonitoringQueue Monitoring

Queue MonitoringQueue Monitoring When displaying a frameWhen displaying a frame

Thresholding operationThresholding operation If qlen is If qlen is mm, then counters 1 through , then counters 1 through mm-1 are -1 are

incrementedincremented All others are resetAll others are reset

When the counter exceeds a value, the oldest When the counter exceeds a value, the oldest frame is discardedframe is discarded

If the queue has contained more than If the queue has contained more than nn frames, frames, then we can reduce the latency (the jitter is then we can reduce the latency (the jitter is stable)stable)

Large variations occur infrequently and smaller Large variations occur infrequently and smaller variations occur more frequently (still true variations occur more frequently (still true today)?today)?

EvaluationEvaluation

The inherent difficultyThe inherent difficulty Gaps vs display latencyGaps vs display latency

Lexocographic ordering for two axesLexocographic ordering for two axes Average gap rateAverage gap rate Average display latencyAverage display latency

Experimental DesignExperimental Design ““academic computer science” networkacademic computer science” network

Time of day, workload seenTime of day, workload seen

Evaluation ResultsEvaluation Results Comparison between I2, I3, and EComparison between I2, I3, and E Usually the same or betterUsually the same or better

Except for incomparable resultsExcept for incomparable results In comparison to the E-policy, it seems to In comparison to the E-policy, it seems to

be workload/network dependentbe workload/network dependent Instantaneous gap rate, delay policy Instantaneous gap rate, delay policy

would be better (perhaps)would be better (perhaps) More adaptive I-policyMore adaptive I-policy More tests, of courseMore tests, of course Addressing ad-hoc quality measuresAddressing ad-hoc quality measures