powervm processor virtualization concepts and configuration

53
© 2009 IBM Corporation IBM Power Systems Technical Conference PowerVM Processor Virtualization: Concepts and Configuration Charlie Cler Executive IT Specialist [email protected]

Upload: vishaltyaggi1981

Post on 24-Oct-2014

186 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation

IBM Power Systems Technical Conference

PowerVM Processor Virtualization: Concepts and Configuration

Charlie Cler Executive IT [email protected]

Page 2: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation2

Power Systems processor partitioning capabilities by platform

The shared processor pool

The difference between physical, virtual, and logical processors

SPLPAR processor minimum, desired, and maximum settings

Recommendations for SPLPAR processor settings

Multiple Shared Processor Pools

Suggestions for shared processor pool settings

Agenda

Page 3: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation3

Introduction

St. Louis, MO, USA

Charlie ClerExecutive I/T SpecialistSystems & Technology [email protected]

1985-1987 Manufacturing engineer, specialized in robotic assembly lines

1988-1990 Manufacturing software specialist

1990-2005 Unix systems specialist: RS/6000, eServer pSeries, System p, Power Systems

2006-present IBM Systems Technology Group (STG) hardware systems architect

IBM

Page 4: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation4

Power Systems software

Simplify management

Reduce energy costs

Keep your data secure

A roadmap for continuous availability

Maximize your choice of solutions

Exploit the cost savings of virtualization

Software to help maximize the return on IT investments for UNIX, Linux and i5/OS clients

Page 5: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation5

Power Systems hardware virtualization = PowerVM

Power virtualization from the company who invented VM!

Page 6: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation6

PowerVM components

The scalable virtualization platform for your mission-critical UNIX, Linux and i5/OS® applications!

FeatureExpress Edition

Standard Edition

Enterprise Edition

Shared Processor Pool ■ ■ ■

Virtual I/O Server ■ ■ ■

Lx86 ■ ■ ■

Shared Dedicated Capacity ■ ■ ■

Multiple Shared Processor Pools

■ ■

Live Partition Mobility(AIX & Linux only)

Page 7: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation7

Virtualization Options for Power Systems

PowerVMAIX, IBM i, Linux

WorkloadPartitions

AIX 6.1

Dynamic LPARs

AIX, IBM I, Linux

- Multiple partitions per server- Whole CPU Increments- Dedicated I/O

- Multiple partitions per server- SPLPARS using Micro-Partitioning

technology - Virtual and/or dedicated I/O- Includes Dynamic LPARs

- Multiple workspaces per AIX image- Runs inside an LPAR or SPLPAR

(Micro-Partition)

Hardware Partitioning OS Based Partitioning

Page 8: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation8

POWER6

POWER5

POWER4

AIX 6.1AIX 5.3AIX 5.2

Power Systems Processor Partitioning capabilities

Multiple Shared Processor Pools

Shared Processor Pool(micropartitions)

Dynamic LPAR

• The shared processor pool was introduced with POWER5 and AIX 5.3

• POWER6 adds Multiple Shared Processor Pools which can be used with

both AIX 5.3 and 6.1

Page 9: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation9

Basic premise for processor sharingC

PU

s

Time

0

1

2

3

4

Guarantee

CP

Us

Time0

1

2

3

4

Using extra CPU cycles from the shared processor pool

CP

Us

Time

0

1

2

3

4

Donating extra CPU cycles to the shared processor pool

Each LPAR is operating in one of these modes

Page 10: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation10

Today’s objective: For you to be able to explain this chart

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weight = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

V

Weight = 30PU = 1.5

V

L

V

L

Weight = 10PU = 0.1

V

L L

Weight = 100PU = 0.8

V V

Weight = 100PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Logical

Physical

Physical

Page 11: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation11

Shared Processor Pool

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool 0

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Physical

Learning points: (1) All activated, non-dedicated cores are automatically used by the shared processor pool.

(2) The shared processor pool size can change as dedicated LPARs are started/stopped.

Shared Processor Pool (SPLPARs) are based on Micro-Partitioning technology

Page 12: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation12

Virtual Processors

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

V V V V V V V V V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Physical

Learning points: (1) Each virtual processor can represent 0.1 to 1 of a physical processor.(2) The number of virtual processors specified for an LPAR represents the maximum

number of physical processors that the LPAR can access.(3) You will not be sharing pooled processors until the number of virtual processors

exceeds the size of the shared pool.

Physical processing cycles are presented to AIX through Virtual Processors

Page 13: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation13

Processing Units

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

PU = 1.2

V V

PU = 0.5

V

PU = 1.5

V V

PU = 0.1

V

PU = 0.8

V V

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Physical

Physical

Processing Units allow physical processors to be allocated in fractional increments

Learning points: (1) One processing unit is equivalent to one core’s worth of compute cycles.

(2) The specified “Processing Units” is guaranteed to each LPAR no matter how busy the shared pool is.

(3) The sum total of assigned processing units cannot exceed the size of the shared pool.

Page 14: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation14

Virtual Processor and Processing Unit relationship

Virtual ProcessorsAssigned to LPAR

Range Of Processing Units that the SPLPAR

can utilize

1 0.1 - 1

2 0.2 - 2

3 0.3 - 3

4 0.4 - 4

x 0.1x - x

Example: An SPLPAR has two virtual processors. This means that the assigned processing units must be somewhere between 0.2 and 2. The maximum processing units that the SPLPAR can utilize is two.

If we want this SPLPAR to be able to use more than two processing units worth of cycles, we need to add more virtual processors, perhaps 2 more. Assigned processing units must now be at least 0.4 and the maximum utilization will be 4.

Learning point: The number of virtual processors establishes the maximum number of processing units that an SPLPAR can access.

Page 15: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation15

Distribution of extra processing cycles

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

Weight = 255PU = 1.2

V V

Uncap = NoPU = 0.5

V

Weight = 30PU = 1.5

V V

Weight = 10PU = 0.1

V

Weight = 100PU = 0.8

V V

Weight = 100PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Physical

Physical

Excess processing cycles are distributed based upon a weighting factor

Learning points: (1) Capped LPARs are limited to their PU setting and cannot access extra cycles

(2) Uncapped LPARs have a weight factor which is a share based mechanism for the distribution of excess processor cycles.

Page 16: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation16

Checkpoint - Desired Processing Units and Desired Virtual ProcessorsP

roce

ssin

g U

nits

(C

PU

s)

Time

0

1

2

3

4

Desired proc. unitsP

roce

ssin

g U

nits

(C

PU

s)

Time

0

1

2

3

4

User of extra proc. units

Pro

cess

ing

Uni

ts

(CP

Us)

Time

0

1

2

3

4

Donor of extra proc. Units

Desired Virtual Processors

Proc. Units Proc. Units Proc. Units

Desired Virtual Processors• Establishes an upper limit for possible processor consumption by an

LPAR (when uncapped =yes)

Desired Virtual Processors Desired Virtual Processors

Desired Processing Units • Establishes a guaranteed amount of processor cycles for each LPAR• Uncapped = yes ….. LPAR can utilize excess cycles• Uncapped = no …… LPAR is limited to the Desired Processing Units

Page 17: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation17

Different number of virtual processors

Same amount of processing units

Learning point: You need to consider peak processing requirements and the job stream (single or multi-threaded) when setting the desired number of virtual processors.

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

AIX 5.3

LPAR

V V

1.6 Proc Units

If all four virtual processors have work to be done, each will receive 0.4 processing units.

The maximum processing units possible to handle peak workload is 4.

Individual processes/threads may run slower

Workloads with a lot of processes/threads may run faster

If both virtual processors have work to be done, each will receive 0.8 processing units.

The maximum processing units possible to handle peak workload is 2.

Individual processes/threads may run faster

Workloads with a lot of processes/threads may run slower

Desired

Desired

Desired

Desired

Virtual Processors and Processing Unit relationship

Page 18: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation18

5.8 Avail. Units5.8 Proc. Units

Learning point: In the presence of excess processing units, SPLPARs with a higher desired virtual processor count will be able to access more excess processing units.

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

AIX 5.3

LPAR

V V

1.6 Proc. Units

Each virtual processor will receive 1.0 processing units from the 5.8 available.

Max processing units that can be consumed is 4 because we have 4 virtual processors.

Each virtual processor will receive 1.0 processing units from the 5.8 available.

Max processing units that can be consumed is 2 because we only have 2 virtual processors.

Different number of virtual processors

Excess processing Unit Capacity

AvailableDesired

AvailableDesired

Available

Desired Desired

Virtual Processors and Processing Unit relationship

Page 19: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation19

“Processor Folding” feature of AIX 5.3 TL3 and higher

Learning point: Size the number of desired virtual processors for the peak workload. The hypervisor will automatically allocate resources to the virtual processors with work to be done

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

If all four virtual processors have work to be done, each will receive 0.4 processing units

If only two virtual processors have work to be done, the hypervisor will temporarily direct all processing units to the two busy virtual processors. Each will receive 0.8 processing units.

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

ProcessorFolding

Reduced number of active threads

(Checked once each second by the

hypervisor)

Processor folding can be disabled: #schedo –o vpm_xvcpus=-1See section 5.8.3 Processor Folding of IBM Redbook AIX 5L Differences Guide Version 5.3 Addendum SG24-7414

Desired

Desired

Desired

Desired

Page 20: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation20

3

2

1

0

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

Prev very busy, receives full allocation Reduced, did not use prev allocation

Reduced, did not use prev allocation Waiting on I/O, cedes cycles

Running steady workload Prev busy, receives excess cycles

SPLPAR1

SPLPAR2

SPLPAR3

core

s

10 millisecond dispatch cycle

2

2

3

1 0.5

10 millisecond dispatch cycle

Units per Virtual Proc.

0.6

0.3

1.2

0.9

yes

yes

Virtual Processors

Processing Units Uncapped?

no

Hypervisor dispatch model

Learning point: The hypervisor automatically adjusts allocations based on each SPLPAR’s use of cycles during the previous dispatch cycle.

Page 21: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation21

SPLPAR Utilization – Greater than 100% (?)

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

PU = 1.2

V V

PU = 0.5

V

PU = 1.5

V V

PU = 0.1

V

PU = 0.8

V V

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

PU Consumption PU Utilization VP Utilization

0.50 33% 16.7%

1.50 100% 50.0%

2.25 150% 66.0%

3.00 200% 100.0%

Page 22: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation22

Processing Units & Virtual Processors - Minimum & Maximum Settings

SPLPAR#5

PU = 1.5

V VV

HMC Setting

Processing Units

(PU)

Virtual Processors

(VP)

Maximum 4.2 6

Desired 1.5 3

Minimum* 0.5 2

Learning point: The min/max settings have nothing to do with resource allocation during normal operation. Min/max are limits applied only when making a dynamic change to PU or VP via the HMC.

* Min also allows an LPAR to start with less than the “desired” resource allocations.

1

2

3

4

5

6

7

8

9

10

11

12

6

7

8

9

10

11

12

1

2

3

4

5

Changes requires LPAR

stop and reactivation

Dynamic changes via the HMC

Normal Operation

DesiredProcessing

Units

DesiredVirtual

Processors

Page 23: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation23

Simultaneous Multi-threading (SMT)

POWER4 (Single Threaded)

CRL

FX0

FX1

LSO

LS1

FP0

FP1

BRZ

Thread0 activeNo thread active

Sys

tem

th

rou

gh

pu

t

ST

Clock ticks

Exe

cutio

n U

nits

POWER5/6 (simultaneous multithreading)

One processor (dedicated or virtual) appears as two logical processors to the

operating system (AIX 5L V5.3 and Linux)

Utilizes unused execution unit cycles Dispatch two threads per processor: “It’s like doubling the number of processors.”

Thread1 active

SMT

Learning point: SMT = On Logical processors presentSMT = Off No logical processors

Page 24: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation24

POWER6, SMT = On

~ 5

0%

Simultaneous Multi-threading (cont.)

SMT=Off

Users

Thr

ough

put

POWER5, SMT = On

~ 3

0%

1

2

2’

1

2

2’

SMT = On/Off (no effect)

SMT = Off (less throughput)

SMT = On (more throughput)

Page 25: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation25

Simultaneous Multi-threading (cont.)

SMT = Off SMT = On

Low CPU Utilization

• SMT does not improve system throughput on a lightly loaded system

• SMT does not make a single thread run faster

SPEEDLIMIT

5.0 GHz

SMT = Off SMT = On

High CPU Utilization

• SMT does improve system throughput on a heavily loaded system

• SMT does not make a single thread run faster (unless it is waiting in the queue)

SPEEDLIMIT

5.0 GHz

Page 26: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation26

Logical Processors

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Weight = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

V

Weight = 30PU = 1.5

V

L

V

L

Weight = 10PU = 0.1

V

L L

Weight = 100PU = 0.8

V V

Weight = 100PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

Think “PVL“

Virtual

Logical

Physical

Simultaneous Multi-Threading (SMT) threads are represented by logical processors

Learning point: SMT requires a minimum of POWER5 hardware and AIX 5.3 (or supported Linux ver.) SMT can be dynamically enable/disable via an AIX command.

Page 27: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation27

Hypervisor dispatch model …. with SMT

Learning point: For SPLPARs with SMT enabled, each allocated processor presents two threads (logical processors) to the operating system.

Thread0

Thread1

Thread0

Thread1

Thread0

Thread1

Thread0

Thread1

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

Prev very busy, receives full allocation Reduced, did not use prev allocation

Reduced, did not use prev allocation Waiting on I/O, cedes cycles

Running steady workload Prev busy, receives excess cycles

SMT

no off

onyes

SPLPAR3 on

0.6

0.4 yes

1.2

0.93

2

Virtual Processors

SPLPAR1

SPLPAR2 2

3

10 millisecond dispatch cycle

core

s

10 millisecond dispatch cycle

2

1

0

Processing Units

Proc Units Per VP Uncapped?

0.51

Note that SMT is off for SPLPAR1, therefore it only runs thread0 during its dispatch window.

Page 28: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation28

Summary: Minimum, desired, and maximum settings

Virtual Processors• Minimum = 2• Desired = 3• Maximum = 5

Processing Units• Minimum = 0.2• Desired = 1.2• Maximum = 16

SPLPAR

SMT=on

V V V

L

1.2 Proc Units

16 core Shared Proc. Pool

Normal Operations (Desired)

• SPLPAR is guaranteed 1.2 processing units at all times.

• If the SPLPAR is not busy, it will cede unused processing units to the shared pool.

• If the SPLPAR is busy:• capped: Limited to 1.2 processing units• uncapped: Use up to 3 processing units

Changes to Desired (DLPAR Operation)

• To change the range of spare processing units that can be utilized, use the HMC to change Desired Virtual Processors to a new value between the min and max settings.

• To change guaranteed processing units, use HMC to change Desired Processing Units to a new value between the min and max settings.

HMC

Use DLPAR

To changeL L L L L

Page 29: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation29

Create LPAR – Shared or Dedicated?

Check Shared to have this LPAR’s processor resources come from the Shared Processor Pool.

Page 30: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation30

Create SPLPAR – Shared, detailed configuration

Specify minimum, desired, and maximum Processing Units, in 0.1 increments.

Specify minimum, desired, and maximum Virtual Processors, in whole processor increments.

Select Uncapped if you want this SPLPAR to utilize spare processor cycles.

Called “Entitled Capacity”

in some IBM perf. tools

Called “Online Virtual CPUs”

in some IBM perf. tools

Page 31: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation31

HMC help text - Desired Processing Units

Page 32: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation32

HMC help text - Minimum Processing Units

Page 33: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation33

HMC help text - Maximum Processing Units

Page 34: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation34

LPARs are defined as dedicated or shared Dedicated partitions use whole number of CPUs Shared partitions use whole and/or fractions of CPUs (smallest increment is 0.1, can be greater than 1.0)

Shared processor pool - subset (or all) of the physical CPUs in a system Physical processors shared among all of the SPLPARs within the shared processor pool

Entitled processing capacity expressed in 0.1 CPU increments Desired: Size of partition at boot time Minimum: Partition will start will less than desired, but won’t start if minimum capacity not available

DLPAR changes to desired cannot be below the minimum. Maximum:DLPAR changes to desired cannot exceed this capacity

Shared Pool LPARs run with ‘virtual’ processors Dedicated partitions use whole number of CPUs Shared partitions use whole and/or

Uncapped: No Processing unit usage is limited to desired setting. Yes Processing unit usage is allowed to exceed the desired processing unit setting.

Use weighting to determine preference for spare cyclesAutomatic Load Balancing (default is 128, 0 implies no use of spare cycles, 255 is max Weight)

Summary - Creating Shared Processor LPARs

Page 35: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation35

Memory configuration

0

Maximum - 8

0

Maximum – 32*

0

32

Hypervisor Page Table (HPT) Real Memory

Desired - 6

Desired - 5

Maximum – 16*

Maximum - 8*

512 MB

256 MB

128 MB

128 MB

HM

C S

ettin

gs

Learning Points: (1) The HPT presents a contiguous range of memory, starting at address 0, to each LPAR(2) Larger maximum memory settings increase the physical memory

consumed by the HPT

* Only one maximum memory setting is permitted per LPAR. Multiple maximums are shown here to demonstrate the corresponding HPT physical memory size.

Physical memory

consumed by the HPT

LPAR #1

LPAR #2

0 GB

0 GB

6 GB

5 GB

LPARs “see” a contiguous block

of memory starting at address 0

Page 36: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation36

Memory configuration

Maximum Memory Size Consideration

The logical memory map for each LPAR is contained in a Hypervisor Page Table (HPT). The size of the HPT is equal to Maximum / 64 (then rounded up to the next power of 2). Over sizing maximum memory can result in wasted memory.

Examples: Maximum memory setting = 64 GB HPT = 1 GBMaximum memory setting = 16 GB HPT = 256 MB

Learning Points: (1) Set maximum memory to 15%-30% greater than desired memory to allow for some

DLPAR increase, with minimal waste in the HPT. (2) Use powers of 2 for Max Memory setting (2GB, 4GB, 8GB, 16GB, 32GB, 64GB,etc)

Memory Configuration

Desired Amount of memory normally allocated to the LPAR

Minimum Minimal amount of memory that must be available for the LPAR to start. Also sets a low water mark for DLPAR changes to desired memory setting.

Maximum Sets the high water mark for DLPAR changes to the desired memory setting

Page 37: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation37

Memory configuration

Specify minimum, desired, and maximum Memory,

Page 38: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation38

New POWER6 Features

Page 39: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation39

Shared Dedicated Capacity P6 Req’d

When this Dedicated CPU LPAR is deactivated, allow unallocated CPUs to be used by the Shared Processor Pool?

Allow excess processor cycles from this Dedicated CPU LPAR to be donated to the Shared Processor Pool?

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool 0

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Page 40: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation40

Multiple Shared Processor Pools

SPLPAR

Appl Server

SPLPAR

Web Server

SPLAR

DB.

SPLPAR

Appl Server

SPLPAR

Web Server

Shared Processor Pool

SPLPAR

Appl

Server

SPLPAR

DB

Server

SPLPAR

DB.

SPLPAR

Appl

Server

SPLPAR

DB

Server

Pool-1 Pool-2Pool-0

Sets an upper limit on processor resources accessible to a group of SPLPARs This is not a hard division of the shared processor pool into smaller sub-pools Up to 64 pools can be configured per server Can help with software licensing Can help balance Prod/Dev on the same server

P6 Req’d

Page 41: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation41

Multiple Shared Processor Pools

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weight = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

V

Weight = 30PU = 1.5

V

L

V

L

Weight = 10PU = 0.1

V

L L

Weight = 100PU = 0.8

V V

Weight = 100PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Logical

Physical

Physical

MaxPU … A whole number which specifies maximum processing units that can be consumed by all of the SPLPARs running in this pool,

ReservedPU = Additional, guaranteed Processing Units for each pool (could be 0)

Default Pool ID = 0 (cannot specify MaxPU or ReservedPU for the Default Pool)

P6 Req’d

Page 42: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation42

Multiple Shared Processor Pool configuration

Pool IDs are fixed and numbered 0...63

SPLPARs can dynamically be moved to a different pool

Disable a pool by setting its Maximum processing units to zero

Default Pool ID = 0

You cannot set reserved processing units or Maximum processing units for the Default Pool

P6 Req’d

Page 43: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation43

Suggestions for Shared Processor Pool

LPAR Settings

Page 44: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation44

Operating within the Shared Processor Pool

Pro

cess

ing

Uni

ts

(CP

Us)

Time

0

1

2

3

4

Desired proc. unitsP

roce

ssin

g U

nits

(C

PU

s)

Time

0

1

2

3

4

User of extra proc. units

Pro

cess

ing

Uni

ts

(CP

Us)

Time

0

1

2

3

4

Donor of extra proc. Units

Virt. Procs Virt. Procs Virt. Procs

Proc. Units Proc. Units Proc. Units

Desired Virtual Processors• Find the peak, move up to the next whole number

Desired Processing Units • More subjective, no best answer• Need a mix of users and donors to have processor sharing• High priority applications: Set processing units higher• Low priority applications: Set processing units lower

Page 45: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation45

Setting desired Processing Units and Virtual Processors

Example: Normal requirement is 0.9 processing units Peak requirement is 3.8 processing units

LPAR Settings:

Virtual Processor Sizing: Set desired virtual processors large enough to handle peak load

Desired = 4 (round 3.8, peak requirement up to next whole number)

Minimum* = Starting point might be 2, or ½ of desired.

Maximum* = Number of CPUs in the shared pool

Processing Units: Set desired processing units to address non-peak, normal workloadDesired = 0.9 (set to match 0.9 processing units, normal requirement)Minimum* = Good starting point might be 0.5, or approx. ½ of Desired.Maximum* = Number of CPUs in the shared pool

* These only come into play if we make dynamic changes using the HMC

Pro

cess

ing

Un

its

(CP

Us)

Time

0

1

2

3

4

Page 46: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation46

Think P-V-L (physical, virtual, logical)

Virtual processors can represent 0.1 to 1 physical processing units

Set Desired Processing Units to cover major portion of workload

Set Desired Virtual Processors to match peak workload

LPAR CPU utilization > 100% is a good thing (using spare cycles!)

Plan to measure utilization at the server level

Consolidate like software onto the same server for improved software utilization and reduced software license costs.

CPU virtualization summary

Page 47: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation47

Virtualization FunctionsSummary

Page 48: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation48

Historical processor sharing/virtualization technologies

Learning point: Ability to deploy tools is dependent upon the OS version and processor model.

ToolProcessor

FamilyDescription

AIX 4.3

(out of service)

AIX 5.1

(out of service)

AIX 5.2 AIX 5.3 AIX 6.1

Application Stacking

(WLM)

AllApplication stacking on a single OS image

Yes Yes Yes Yes Yes

Static LPAR

POWER4

POWER5

POWER6

Static allocation of whole CPUs

No Yes Yes Yes Yes

Dynamic LPAR (PLM)

POWER4

POWER5

POWER6

Dynamic allocation of whole CPUs

No No Yes Yes Yes

Shared Processor Pool

POWER5

POWER6

Dynamic allocation of fractional CPUs

No No No Yes Yes

Application Stacking (WPAR)

POWER4

POWER5

POWER6

Improved application stacking on a single OS image

No No No No Yes

Page 49: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation49

YesNoYesNoYesNoWorkload Partitions (WPARs)

& WPAR Mobility

YesYesNoNoNoNoIntegrated Virtualized Ethernet

YesYesNoNoNoNoShared Dedicated Capacity

YesYesNoNoNoNoPartition Mobility

YesYesNoNoNoNoMultiple Shared Processor Pools

AIX 6.1AIX 5.3AIX 6.1AIX 5.3AIX 6.1AIX 5.3OS Level

POWER6POWER5POWER4Processor

AIX 6.1

Req’d

POWER6

Req’dLegend

Summary of new features for POWER6 and AIX 6.1

Learning point: Most of the new features introduced with POWER6 are supported with AIX 5.3

Page 50: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation50

Additional Information

http://publib.boulder.ibm.com/infocenter/systems/scope/hw• IBM Power Systems Logical Partitioning Guide (search on “Logical Partitioning Guide”)• PowerVM Editions Operations Guide (search on “PowerVM”)

www.ibm.com/redbooks• PowerVM Virtualization on IBM System p Introduction and Configuration (SG24-7940)• PowerVM Virtualization on IBM System p Managing and Monitoring (SG24-7590)

IBM Systems Hardware Information Center

Page 51: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation51

This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area.

Information in this document concerning non-IBM products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA.

All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied.

All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions.

IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice.

IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies.

All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary.

IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.

Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment.

Revised September 26, 2006

Special Notices

Page 52: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation52

The following terms are registered trademarks of International Business Machines Corporation in the United States and/or other countries: AIX, AIX/L, AIX/L (logo), AIX 6 (logo), alphaWorks, AS/400, BladeCenter, Blue Gene, Blue Lightning, C Set++, CICS, CICS/6000, ClusterProven, CT/2, DataHub, DataJoiner, DB2, DEEP BLUE, developerWorks, DirectTalk, Domino, DYNIX, DYNIX/ptx, e business (logo), e(logo)business, e(logo)server, Enterprise Storage Server, ESCON, FlashCopy, GDDM, i5/OS, i5/OS (logo), IBM, IBM (logo), ibm.com, IBM Business Partner (logo), Informix, IntelliStation, IQ-Link, LANStreamer, LoadLeveler, Lotus, Lotus Notes, Lotusphere, Magstar, MediaStreamer, Micro Channel, MQSeries, Net.Data, Netfinity, NetView, Network Station, Notes, NUMA-Q, OpenPower, Operating System/2, Operating System/400, OS/2, OS/390, OS/400, Parallel Sysplex, PartnerLink, PartnerWorld, Passport Advantage, POWERparallel, Power PC 603, Power PC 604, PowerPC, PowerPC (logo), Predictive Failure Analysis, pSeries, PTX, ptx/ADMIN, Quick Place, Rational, RETAIN, RISC System/6000, RS/6000, RT Personal Computer, S/390, Sametime, Scalable POWERparallel Systems, SecureWay, Sequent, ServerProven, SpaceBall, System/390, The Engines of e-business, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, Tivoli Ready (logo), TME, TotalStorage, TURBOWAYS, VisualAge, WebSphere, xSeries, z/OS, zSeries.

The following terms are trademarks of International Business Machines Corporation in the United States and/or other countries: Advanced Micro-Partitioning, AIX 5L, AIX PVMe, AS/400e, Calibrated Vectored Cooling, Chiphopper, Chipkill, Cloudscape, DataPower, DB2 OLAP Server, DB2 Universal Database, DFDSM, DFSORT, DS4000, DS6000, DS8000, e-business (logo), e-business on demand, EnergyScale, Enterprise Workload Manager, eServer, Express Middleware, Express Portfolio, Express Servers, Express Servers and Storage, General Purpose File System, GigaProcessor, GPFS, HACMP, HACMP/6000, IBM Systems Director Active Energy Manager, IBM TotalStorage Proven, IBMLink, IMS, Intelligent Miner, iSeries, Micro-Partitioning, NUMACenter, On Demand Business logo, POWER, PowerExecutive, PowerVM, PowerVM (logo), Power Architecture, Power Everywhere, Power Family, POWER Hypervisor, Power PC, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), PowerPC Architecture, PowerPC 603, PowerPC 603e, PowerPC 604, PowerPC 750, POWER2, POWER2 Architecture, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, pure XML, Quickr, Redbooks, Sequent (logo), SequentLINK, Server Advantage, ServeRAID, Service Director, SmoothStart, SP, System i, System i5, System p, System p5, System Storage, System z, System z9, S/390 Parallel Enterprise Server, Tivoli Enterprise, TME 10, TotalStorage Proven, Ultramedia, VideoCharger, Virtualization Engine, Visualization Data Explorer, Workload Partitions Manager, X-Architecture, z/Architecture, z/9.

A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml.The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org.UNIX is a registered trademark of The Open Group in the United States, other countries or both. Linux is a trademark of Linus Torvalds in the United States, other countries or both.Microsoft, Windows, Windows NT and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both.Intel, Itanium, Pentium are registered trademarks and Xeon is a trademark of Intel Corporation or its subsidiaries in the United States, other countries or both.AMD Opteron is a trademark of Advanced Micro Devices, Inc.Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries or both. TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC).SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC).NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both.AltiVec is a trademark of Freescale Semiconductor, Inc.Cell Broadband Engine is a trademark of Sony Computer Entertainment Inc.InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association. Other company, product and service names may be trademarks or service marks of others. Revised January 15, 2008

Special Notices (Cont.)

Page 53: PowerVM Processor Virtualization Concepts and Configuration

© 2009 IBM Corporation53

Trademarks

The following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.

The following are trademarks or registered trademarks of other companies.

* All other products may be trademarks or registered trademarks of their respective companies.

Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.

For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA, WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, BladeCenter®

Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market.

Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.