live partition mobility: networking best practices

Live Partition Mobility: NetworkingBest Practices

2

Live Partition Mobility (LPM) is a feature of the PowerVM Enterprise Edition that can move live AIX, Linux, and IBMi logical partitions from one frame to another, without any downtime. It can also be used to move inactive logical partitions. The mobility process transfers the logical partition environment, including the processor state, memory, attached virtual devices, and connected users.

3

Case 1: IP address on Physical Adapter (Highly recommended for LPM)

VEA = Virtual Ethernet Adapter

REA = Real Ethernet Adapter

PEA = Physical Ethernet Adapter

Here are common customer network configuration on VIOS.

§ SEA is not used for LPM.

§ Dedicated network path for LPM because it is not shared with VIO client communication.

§ Best performance for LPM.

§ Least impact on VIO clients using VIOS.

4



Case 2: IP address on SEA

§ Use for untagged network.

5

Case 3: IP address on VLAN Adapter on top of SEA

§ Use for tagged network.



6

Case 4: IP address on Virtual Ethernet Adapter (Not recommended for LPM)

§ The network path for LPM is shared with VIO client communication.

§ Slowest performance.

§ Impacts network bandwidth of VIO client.



7

iperf throughput comparison between case 4 and 1:

§ Source and Destination MSP Configuration:Model: 9119-MME (Power8)ent_capacity: 2 (Shared-SMT-4)Memory: 12 GBSystem Firmware: SC860_180Type of 10 Gb Ethernet Adapter: 10GbE SFP+ SR PCIe3 Converged Network Adapter (Feature Code: EN0H)Adapter firmware level: 10.2.252.1927

§ iperf syntax:iperf syntax on source MSP: iperf -c <IP address of Destination MSP> -P10 -w 64M -t 20iperf syntax on destination MSP: iperf -s -w 64M

§ Iperf throughput

§ Conclusion:The iperf test shows setup with IP address physical adapter gives higher throughput as compare to setup with IP address on Virtual Ethernet Adapter.

8.92 Gbits/sec5.31 Gbits/sec

Case 4IP address on Virtual Ethernet Adapter

Case 1IP address on Physical Adapter

8

This document describes the networking best practices to implement and manage an LPM environment.

§ 10 Gb or higher link speed between Source MSP (Mover Service Partition) and destination MSP is preferred, but this is not required. Make sure all switches, routers between two MSPs are setup for the same link speed.

§ Highly recommend to use IP address on the physical adapter as shown in case 1.

§ Enable large_send and large_receive on the physical adapter.

§ Enable jumbo_frame on physical adapter to use packet size up to 9000 bytes. To use this option, it needs to be enabled on all switches in between.

§ It is highly recommended to migrate between same speed networks (e.g. 10 Gb to 10 Gb or 1 Gb to 1 Gb). For example, if 10 Gb ethernet adapter is used on source MSP and 1 Gb ethernet adapter on destination MSP then 10 Gb adapter can overwhelm the 1 Gb adapter andthat leads to retransmissions, poor LPM performance and LPM stalling. For situations like this, improve the LPM performance by either using same speed adapter on both MSPs or enabling flow control on the switch connected to 1 Gb ethernet adapter. The flow control is enabled by default on 10 Gb and 1 Gb adapters.

§ Redundant MSPs are used by default for better performance and reliability wherever the configuration allows. Redundant or dual MSP setups are recommended for all the cases except case 4 (IP address on Virtual Ethernet Adapter).

9

Recommended Setup for LPM: IP address on Physical Adapter

§ SEA is not used for LPM.

§ Dedicated network path for LPM because it is not shared with VIO client communication.

§ Best performance for LPM.

§ Least impact on VIO clients using VIOS.

§ Enable large_send and large_receive on PEA.e.g. chdev –dev ent4 –attr large_send=yes large_receive=yes



PEA = Physical Ethernet Adapter

✓

live partition mobility: networking best practices

Documents