carousel: scalable traffic shaping at end hosts · carousel: scalable traffic shaping at end hosts...
TRANSCRIPT
![Page 1: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/1.jpg)
Carousel: Scalable Traffic Shaping at End Hosts
Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli, and Amin Vahdat
1
![Page 2: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/2.jpg)
2
![Page 3: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/3.jpg)
3
Rate limiting and isolation between thousands of flows per machine [BwE - SIGCOMM ’15]
![Page 4: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/4.jpg)
4
New protocols that require per-flow pacing [TCP BBR and TIMELY - SIGCOMM ’15]
Rate limiting and isolation between thousands of flows per machine [BwE - SIGCOMM ’15]
![Page 5: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/5.jpg)
Packet sources
Sch
edul
er To NICRate1
Rate2
Rate3
Shaper
Traffic Shaping
Cla
ssifi
er
5
![Page 6: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/6.jpg)
Packet sources
Sch
edul
er To NICRate1
Rate2
Rate3
Shaper
Traffic Shaping
Cla
ssifi
er
5
![Page 7: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/7.jpg)
Packet sources
Sch
edul
er To NICRate1
Rate2
Rate3
Shaper
Traffic Shaping
Cla
ssifi
er
5
![Page 8: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/8.jpg)
Packet sources
Sch
edul
er To NICRate1
Rate2
Rate3
Shaper
Traffic Shaping
Cla
ssifi
er
5
![Page 9: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/9.jpg)
Packet sources
Sch
edul
er To NICRate1
Rate2
Rate3
Shaper
Traffic Shaping
Cla
ssifi
er
6
Overhead of managing a queue per configured rate
![Page 10: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/10.jpg)
Packet sources
Sch
edul
er To NICRate1
Rate2
Rate3
Shaper
Traffic Shaping
Cla
ssifi
er
6
Overhead of managing a queue per configured rate
![Page 11: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/11.jpg)
Packet sources
Sch
edul
er To NICRate1
Rate2
Rate3
Shaper
Traffic Shaping
Cla
ssifi
er
6
Overhead of managing a queue per configured rate
![Page 12: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/12.jpg)
Packet sources
Sch
edul
er To NICRate1
Rate2
Rate3
Shaper
Traffic Shaping
Cla
ssifi
er
6
Overhead of managing a queue per configured rate
![Page 13: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/13.jpg)
We need new traffic shapers that can handle tens of thousands of flows and rates
7
![Page 14: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/14.jpg)
Main Idea Replace the many queues with a single low-overhead queue
8
![Page 15: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/15.jpg)
Contributions
To NIC
Shaper
Cla
ssifi
er
Sch
edul
erRate1
Rate2
Rate3
Processing Overhead
Synchronization
9
![Page 16: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/16.jpg)
Contributions
To NIC
Shaper
Synchronization
Timing Wheel
Tim
esta
mpe
r
Single, O(1), Queue
9
![Page 17: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/17.jpg)
Contributions
To NIC
Shaper
Timing Wheel
Tim
esta
mpe
r
Single, O(1), Queue
10
![Page 18: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/18.jpg)
Contributions
To NIC
Shaper
Timing Wheel
Tim
esta
mpe
r
Single, O(1), QueueOne Shaper per Core
10
![Page 19: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/19.jpg)
Contributions
To NIC
Shaper
Timing Wheel
Tim
esta
mpe
r
Single, O(1), QueueOne Shaper per Core
Apply backpressure
10
![Page 20: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/20.jpg)
Outline
● Problems with Current Shapers
● Carousel Overview
● Single Queue Shaping
● Backpressure
● Evaluation
11
![Page 21: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/21.jpg)
Problems with Current Shapers
12
![Page 22: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/22.jpg)
● Implements per TCP flow pacing
● Requires a queue per flow
○ Flows are kept in order of their scheduled transmission time
○ Flows are dequeued in order
● O(log n) operations per packet to operate on a sorted list of flows
Flow pacing
Flow delays (red-black tree on timestamps)
F1 t=now+5
F2 t=now+3
Source flows
F7 t=now+2
F4 t=now+9
time_next_packet
Flow state lookup
Hashtable of red-black trees on flow ids
F6 t=now+4
F5 t=now+6
F3 t=now+8
NIC13
FQ/Pacing
![Page 23: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/23.jpg)
CPU utilization for FQ/pacing and a NOOP Qdisc for the same load
14
![Page 24: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/24.jpg)
FQ/Pacing introduces 10% more CPU overhead
15
![Page 25: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/25.jpg)
Carousel Overview
16
![Page 26: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/26.jpg)
Carousel Overview
● Relies on a single queue for all packets from all flows
● Requires a high frequency timer or busy polling
● Pinned to a single core 17
![Page 27: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/27.jpg)
Single Queue Shaping
18
![Page 28: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/28.jpg)
Single Queue Shaping
19
● All packet are sorted by their transmission time in one data structure
● A single queue for all traffic will need to handle tens of thousands of packets
● Challenge: Enqueue and dequeue in a data structure of sorted elements at line rate
![Page 29: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/29.jpg)
Single Queue Shaping
● All packet are sorted by their transmission time in one data structure
● A single queue for all traffic will need to handle tens of thousands of packets
● Challenge: Enqueue and dequeue in a data structure of sorted elements at line rate
20
Timing Wheel
Tim
esta
mpe
r
![Page 30: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/30.jpg)
Timing Wheel [Varghese et al. SOSP ’87]
● Bucket sort approach to Calendar Queue covering a time horizon
● Relies on having a minimum rates
● Implemented as an array of buckets each a linked list of packets
● Each bucket represents a certain time range
Time Horizon (h)
T T+r
21
T+2r T+3r T+4r T+h
![Page 31: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/31.jpg)
Timing Wheel Benchmark
● Measured overhead per enqueue/dequeue pairs
● Overhead per element is between 21-22 nanoseconds
● Fixed for 2000 to 2 million sorted elements
● 21 nanoseconds per packet = 500 Gbps (for 1500 byte packets)
22
![Page 32: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/32.jpg)
Timestampers
● Packets are timestamped by policy enforcers in their transmission path ● TCP timestamps a packet based on its
pacing rate ● Bandwidth enforcer timestamps a
packet based on its policy-based aggregate rate
● Carousel picks the largest timestamp
● NextTimestamp = LastTimestamp + 23
SizeOfPacketConfiguredRate
Timestamper Timestamper
Timestamper
Pacing rate
Aggregate rate
Timing Wheel
![Page 33: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/33.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
Each bucket represents 1 second
Example of Shaping using Carousel
![Page 34: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/34.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
T=1
Each bucket represents 1 second
Example of Shaping using Carousel
A time step 0
![Page 35: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/35.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
T=2
Each bucket represents 1 second
Example of Shaping using Carousel
A time step 0
![Page 36: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/36.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
T=3
Each bucket represents 1 second
Example of Shaping using Carousel
A time step 0
![Page 37: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/37.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
Each bucket represents 1 secondT=2
Example of Shaping using Carousel
A time step 0
![Page 38: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/38.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
Each bucket represents 1 secondT=4
Example of Shaping using Carousel
A time step 0
![Page 39: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/39.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
Each bucket represents 1 second
Example of Shaping using Carousel
A time step 0
![Page 40: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/40.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
Each bucket represents 1 second
Example of Shaping using Carousel
A time step 1
![Page 41: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/41.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
Each bucket represents 1 second
Example of Shaping using Carousel
A time step 2
![Page 42: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/42.jpg)
24
Timing Wheel
Tim
esta
mpe
r
Rate (1 pps)
Rate (0.5 pps)
Each bucket represents 1 second
Example of Shaping using Carousel
![Page 43: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/43.jpg)
Backpressure with Deferred Completion
25
![Page 44: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/44.jpg)
The Value of Backpressure
● Without backpressure shaper queues get full with small number of flows causing
26
![Page 45: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/45.jpg)
The Value of Backpressure
● Without backpressure shaper queues get full with small number of flows causing● Unnecessary drops (when the queue is full the queue tail drops)
26
![Page 46: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/46.jpg)
The Value of Backpressure
● Without backpressure shaper queues get full with small number of flows causing● Unnecessary drops (when the queue is full the queue tail drops)● Head of Line Blocking
26
![Page 47: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/47.jpg)
The Value of Backpressure
● Without backpressure shaper queues get full with small number of flows causing● Unnecessary drops (when the queue is full the queue tail drops)● Head of Line Blocking
● Backpressure allows shapers to control sender rate and avoid overwhelming the shaper
26
![Page 48: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/48.jpg)
The Completion Signal
● Completions are signals from the NIC to the network stack to inform it that a packet has been transmitted
27
NIC
Network Stack
Packet
![Page 49: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/49.jpg)
The Completion Signal
● Completions are signals from the NIC to the network stack to inform it that a packet has been transmitted
27
NIC
Network Stack
Completion
![Page 50: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/50.jpg)
The Completion Signal
● Completions are signals from the NIC to the network stack to inform it that a packet has been transmitted
27
NIC
Network Stack
![Page 51: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/51.jpg)
The Completion Signal
● Completions are signals from the NIC to the network stack to inform it that a packet has been transmitted● Completions are typically delivered in order
27
NIC
Network Stack
![Page 52: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/52.jpg)
The Completion Signal
● Completions are signals from the NIC to the network stack to inform it that a packet has been transmitted● Completions are typically delivered in order● Completion should be controlled by the
hypervisor not the virtual NIC
27
NIC
Network Stack
![Page 53: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/53.jpg)
The Completion Signal
● Completions are signals from the NIC to the network stack to inform it that a packet has been transmitted● Completions are typically delivered in order● Completion should be controlled by the
hypervisor not the virtual NIC
● Completions should be delivered out of order and completely controlled by Shapers
27
NIC
Network Stack
![Page 54: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/54.jpg)
Backpressure with Deferred Completion
28
Without Backpressure
![Page 55: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/55.jpg)
Backpressure with Deferred Completion
29
With Backpressure
Without Backpressure
![Page 56: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/56.jpg)
Backpressure with Deferred Completion
30
![Page 57: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/57.jpg)
Backpressure with Deferred Completion
31
Queue saturated
![Page 58: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/58.jpg)
Backpressure with Deferred Completion
32
Deferred completions limits the number of packets in shaper reducing its memory footprint
![Page 59: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/59.jpg)
Evaluation
33
![Page 60: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/60.jpg)
Evaluation Setup
● Carousel deployed within a Software NIC
34
Kernel Networking Stack
Application
NIC
Software NIC
![Page 61: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/61.jpg)
Evaluation Setup
● Carousel deployed within a Software NIC
● Evaluation on Youtube servers comparing Carousel and FQ/Pacing
34
Kernel Networking Stack
Application
NIC
Software NIC
![Page 62: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/62.jpg)
Evaluation Setup
● Carousel deployed within a Software NIC
● Evaluation on Youtube servers comparing Carousel and FQ/Pacing
● Each server handles up to 50k sessions concurrently
34
Kernel Networking Stack
Application
NIC
Software NIC
![Page 63: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/63.jpg)
Evaluation Metric
● Measures Gbps served per CPU utilization ● Metric used is Gbps/CPU (higher is better) ● Compare machines with similar CPU utilization ● Measurements performed during peak 12-hours per day
● Evaluation is performed for: ● Overall CPU utilization ● Software NIC utilization
35
![Page 64: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/64.jpg)
Overall CPU Utilization
36
![Page 65: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/65.jpg)
Overall CPU Utilization
37
Carousel saves up to 8.2% of overall CPU utilization(5.9 cores on a 72 core machine)
![Page 66: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/66.jpg)
SoftNIC Utilization
38
![Page 67: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/67.jpg)
SoftNIC Utilization
39
Carousel improves even Software NIC utilization by 12% by increasing size of batches of packets enqueue in the Software NIC
![Page 68: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/68.jpg)
Evaluation Summary
40
Performance improvement when Carousel starts on 5 different machines
![Page 69: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/69.jpg)
Conclusion
● Carousel allows networks operators for the first time to shape tens of thousands of flows individually
● Carousel advantages make a strong case for providing single-queue shaping and backpressure in kernel, userspace stacks, hypervisors, and hardware
41
![Page 70: Carousel: Scalable Traffic Shaping at End Hosts · Carousel: Scalable Traffic Shaping at End Hosts Ahmed Saeed, Nandita Dukkipati, Vytautas Valancius, Vinh The Lam, Carlo Contavalli,](https://reader033.vdocuments.mx/reader033/viewer/2022050604/5fabf28a2bf59b2209376ec6/html5/thumbnails/70.jpg)
Questions?
42