Linux TCP/IP Tuning

17 March 2015

Presented by

Binan AL Halabi

Master in Communications

For Real Time Communication

Blog: voipmagazine.wordpress.com/

Why Tuning ?

Service Availability (Signaling) Media Quality (Low Delay, Low Loss and Low Jitter)

* Low Jitter = Packets arrives in a nice steady stream (not bursts)

Signaling /Media

Packets

Interrupts Tuning

Interrupts Distribution Fair: All CPU Cores have the same priority.

Using “ irqbalance” daemon. Problem: Time is needed to load the interrupt

handler to the cache of the fresh CPU core. Not Fair:

Interrupt Affinity = CPU Cores (As a Bit Mask) that handle the interrupts from a Specific Device.

Interrupt Coalescing

Some NIC drivers support interrupt coalescing (multiple interrupts can be coalesced into one kernel interrupt).

Transmission & Receiving Queues

“txqueuelen”: Transmission Queue Length related to the interface. The kernel stores the departing frames in this queue (the frames are not loaded into the NIC’s buffer yet).

The “netdev_max_backlog” is the corresponding receive queue.

Socket Buffers Sizes

Sizes allowed to be allocated by the application Read/Receive Buffer: rmem_max and rmem_default Write /Send Buffer: wmem_max and wmem_default The values are in Byte.

Protocol-Specific Parameters

Example ” TIME_WAIT” Interval (tcp_fin_timeout)

– TCP socket state is “closing”

– Waiting to ensure the ACK (of respective FIN) it sent was received.

Small value is recommended for high loaded servers (e.g. 1-10 seconds)

Port Range & Number Of File Handles

Port Range “net.ipv4.ip_local_port_range” “fs.file-max” is the maximum number of file

handles on the entire system

SHELL Limits

“ulimit” is used to limit the resources allocated to the user, among other things, the number of files that can be opened by the user:

• The user can change its limit:

# ulimit -n New_Value• The root set the range for the user's

value. “/etc/security/limits.conf”

Packet Marking & Kernel Traffic Control (TC/TCNG) Packets Marking using IPtables: target “MARK”

with the option “--set-mark” Example: Mark RTP packets with 6:

# iptables -t mangle -A PREROUTING -p udp -m udp --dport 16384:32768 -i eth0 -j MARK --set-mark 6

Kernel Traffic Control (TC/TCNG)

Bind Marks with Defined Classes

Example: Makes packets marked with 6, to be processed by the 1:10 class:

# tc filter add dev eth1 protocol ip parent 1:0 prio 1 handle 6 fw flowid 1:10

Define the classes (class based queueing)# tc class add dev eth1 parent 1:1 classid 1:10 htb

rate XMbps ceil YMbps burst 10k

Kernel Connection Tracking System

Conntrack Example: Udp 17 20 src=192.168.1.2 dst=192.168.1.5 sport=137 dport=1025 [UNREPLIED] src=192.168.1.5 dst=192.168.1.2 sport=1025 dport=137 use=1

Connection Tracking“nf_conntrack” Hash Table

Error : “nf_conntrack: table full, dropping packet Change The value: net.netfilter.nf_conntrack_buckets

And net.netfilter.nf_conntrack_max

What We Need ?

Testing Lab: Case Definition Traffic Generator Measurement Tools Results Documentation

Virtual Network Node

The network node is going virtual Follow the packet in its path in the

system: Policy Network Virtualization Solution's QoS

Support

And Much More Parameters

But Remember the Whole Picture

(All Parameters Together)