cmg processor virtualization

98
© 2009 IBM Corporation A discussion on Processor Virtualization using AIX as an example Charlie Cler Executive IT Specialist [email protected]

Upload: danielnicol

Post on 02-May-2017

275 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: CMG Processor Virtualization

© 2009 IBM Corporation

A discussion on Processor Virtualizationusing AIX as an example

Charlie Cler Executive IT [email protected]

Page 2: CMG Processor Virtualization

© 2009 IBM Corporation2

Power Systems processor partitioning capabilities by platform

The shared processor pool

The difference between physical, virtual, and logical processors

SPLPAR processor minimum, desired, and maximum settings

Recommendations for SPLPAR processor settings

Multiple Shared Processor Pools

Suggestions for shared processor pool settings

Agenda

Page 3: CMG Processor Virtualization

© 2009 IBM Corporation3

Introduction

St. Louis, MO, USA

Charlie ClerExecutive I/T SpecialistSystems & Technology [email protected]

1985-1987 Manufacturing engineer, specialized in robotic assembly lines

1988-1990 Manufacturing software specialist

1990-2005 Unix systems specialist: RS/6000, eServer pSeries, System p, Power Systems

2006-present IBM Systems Technology Group (STG) hardware systems architect

IBM

Page 4: CMG Processor Virtualization

© 2009 IBM Corporation4

Systems software Categorization

Simplify management

Reduce energy costs

Keep your data secure

A roadmap for continuous availability

Maximize your choice of solutions

Exploit the cost savings of virtualization

Software to help maximize the return on IT investments for UNIX, Linux and i5/OS clients

Page 5: CMG Processor Virtualization

© 2009 IBM Corporation5

Virtualization Options for Power Systems

PowerVMAIX, IBM i, Linux

WorkloadPartitions / Containers

Dynamic LPARs

AIX, IBM I, Linux

- Multiple partitions per server- Whole CPU Increments- Dedicated I/O

- Multiple partitions per server- SPLPARS using Micro-Partitioning

technology - Virtual and/or dedicated I/O- Includes Dynamic LPARs

- Multiple workspaces per AIX image- Runs inside an LPAR or SPLPAR

(Micro-Partition)

Hardware Partitioning OS Based Partitioning

Page 6: CMG Processor Virtualization

© 2009 IBM Corporation6

POWER6

POWER5

POWER4

AIX 6.1AIX 5.3AIX 5.2

Power Systems Processor Partitioning capabilities

Multiple Shared Processor Pools

Shared Processor Pool(micropartitions)

Dynamic LPAR

• The shared processor pool was introduced with POWER5 and AIX 5.3

• POWER6 adds Multiple Shared Processor Pools which can be used with both AIX 5.3 and 6.1

Page 7: CMG Processor Virtualization

© 2009 IBM Corporation7

Basic premise for processor sharingC

PU

s

Time

0

1

2

3

4

Guarantee

CP

Us

Time0

1

2

3

4

Using extra CPU cycles from the shared processor pool

CP

Us

Time

0

1

2

3

4

Donating extra CPU cycles to the shared processor pool

Each LPAR is operating in one of these modes

Page 8: CMG Processor Virtualization

© 2009 IBM Corporation8

Today’s objective: For you to be able to explain this chart

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weight = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

VWeight = 30

PU = 1.5

V

L

V

L

Weight = 10PU = 0.1

V

L L

Weight = 100PU = 0.8

V VWeight = 100

PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Logical

Physical

Physical

Page 9: CMG Processor Virtualization

© 2009 IBM Corporation9

Shared Processor Pool

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool 0

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Physical

Learning points: (1) All activated, non-dedicated cores are automatically used by the shared processor pool.

(2) The shared processor pool size can change as dedicated LPARs are started/stopped.

Shared Processor Pool (SPLPARs) are based on Micro-Partitioning technology

Page 10: CMG Processor Virtualization

© 2009 IBM Corporation10

Virtual Processors

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

V V V V V V V V V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Physical

Learning points: (1) Each virtual processor can represent 0.1 to 1 of a physical processor.(2) The number of virtual processors specified for an LPAR represents the maximum

number of physical processors that the LPAR can access.(3) You will not be sharing pooled processors until the number of virtual processors

exceeds the size of the shared pool.

Physical processing cycles are presented to AIX through Virtual Processors

Page 11: CMG Processor Virtualization

© 2009 IBM Corporation11

Processing Units

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

PU = 1.2

V V

PU = 0.5

V

PU = 1.5

V V

PU = 0.1

V

PU = 0.8

V V

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Physical

Physical

Processing Units allow physical processors to be allocated in fractional increments

Learning points: (1) One processing unit is equivalent to one core’s worth of compute cycles.(2) The specified “Processing Units” is guaranteed to each LPAR no matter how busy the shared pool is.(3) The sum total of assigned processing units cannot exceed the size of the shared pool.

Page 12: CMG Processor Virtualization

© 2009 IBM Corporation12

Virtual Processor and Processing Unit relationship

Virtual ProcessorsAssigned to LPAR

Range Of Processing Units that the SPLPAR

can utilize

1 0.1 - 1

2 0.2 - 2

3 0.3 - 3

4 0.4 - 4

x 0.1x - x

Example: An SPLPAR has two virtual processors. This means that the assigned processing units must be somewhere between 0.2 and 2. The maximum processing units that the SPLPAR can utilize is two.

If we want this SPLPAR to be able to use more than two processing units worth of cycles, we need to add more virtual processors, perhaps 2 more. Assigned processing units must now be at least 0.4 and the maximum utilization will be 4.

Learning point: The number of virtual processors establishes the maximum number of processing units that an SPLPAR can access.

Page 13: CMG Processor Virtualization

© 2009 IBM Corporation13

Distribution of extra processing cycles

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

Weight = 255PU = 1.2

V VUncap = No

PU = 0.5

VWeight = 30

PU = 1.5

V VWeight = 10

PU = 0.1

VWeight = 100

PU = 0.8

V VWeight = 100

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Physical

Physical

Excess processing cycles are distributed based upon a weighting factor

Learning points: (1) Capped LPARs are limited to their PU setting and cannot access extra cycles

(2) Uncapped LPARs have a weight factor which is a share based mechanism for the distribution of excess processor cycles.

Page 14: CMG Processor Virtualization

© 2009 IBM Corporation14

Checkpoint - Desired Processing Units and Desired Virtual ProcessorsP

roce

ssin

g U

nits

(C

PU

s)

Time

0

1

2

3

4

Desired proc. unitsP

roce

ssin

g U

nits

(C

PU

s)

Time

0

1

2

3

4

User of extra proc. units

Pro

cess

ing

Uni

ts

(CP

Us)

Time

0

1

2

3

4

Donor of extra proc. Units

Desired Virtual Processors

Proc. Units Proc. Units Proc. Units

Desired Virtual Processors• Establishes an upper limit for possible processor consumption by an

LPAR (when uncapped =yes)

Desired Virtual Processors Desired Virtual Processors

Desired Processing Units • Establishes a guaranteed amount of processor cycles for each LPAR• Uncapped = yes ….. LPAR can utilize excess cycles• Uncapped = no …… LPAR is limited to the Desired Processing Units

Page 15: CMG Processor Virtualization

© 2009 IBM Corporation15

Different number of virtual processors

Same amount of processing units

Learning point: You need to consider peak processing requirements and the job stream (single or multi-threaded) when setting the desired number of virtual processors.

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

AIX 5.3

LPAR

V V

1.6 Proc Units

If all four virtual processors have work to be done, each will receive 0.4 processing units.

The maximum processing units possible to handle peak workload is 4.

Individual processes/threads may run slower

Workloads with a lot of processes/threads may run faster

If both virtual processors have work to be done, each will receive 0.8 processing units.

The maximum processing units possible to handle peak workload is 2.

Individual processes/threads may run faster

Workloads with a lot of processes/threads may run slower

Desired

Desired

Desired

Desired

Virtual Processors and Processing Unit relationship

Page 16: CMG Processor Virtualization

© 2009 IBM Corporation16

5.8 Avail. Units5.8 Proc. Units

Learning point: In the presence of excess processing units, SPLPARs with a higher desired virtual processor count will be able to access more excess processing units.

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

AIX 5.3

LPAR

V V

1.6 Proc. Units

Each virtual processor will receive 1.0 processing units from the 5.8 available.

Max processing units that can be consumed is 4 because we have 4 virtual processors.

Each virtual processor will receive 1.0 processing units from the 5.8 available.

Max processing units that can be consumed is 2 because we only have 2 virtual processors.

Different number of virtual processors

Excess processing Unit Capacity

AvailableDesired

AvailableDesired

Available

Desired Desired

Virtual Processors and Processing Unit relationship

Page 17: CMG Processor Virtualization

© 2009 IBM Corporation17

“Processor Folding” feature of AIX 5.3 TL3 and higher

Learning point: Size the number of desired virtual processors for the peak workload. The hypervisor will automatically allocate resources to the virtual processors with work to be done

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

If all four virtual processors have work to be done, each will receive 0.4 processing units

If only two virtual processors have work to be done, the hypervisor will temporarily direct all processing units to the two busy virtual processors. Each will receive 0.8 processing units.

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

ProcessorFolding

Reduced number of active threads

(Checked once each second by the

hypervisor)

Processor folding can be disabled: #schedo –o vpm_xvcpus=-1See section 5.8.3 Processor Folding of IBM Redbook AIX 5L Differences Guide Version 5.3 Addendum SG24-7414

Desired

Desired

Desired

Desired

Page 18: CMG Processor Virtualization

© 2009 IBM Corporation18

3

2

1

0

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

Prev very busy, receives full allocation Reduced, did not use prev allocation

Reduced, did not use prev allocation Waiting on I/O, cedes cycles

Running steady workload Prev busy, receives excess cycles

SPLPAR1

SPLPAR2

SPLPAR3

core

s

10 millisecond dispatch cycle

2

2

3

1 0.5

10 millisecond dispatch cycle

Units per Virtual Proc.

0.6

0.3

1.2

0.9

yes

yes

Virtual Processors

Processing Units Uncapped?

no

Hypervisor dispatch model

Learning point: The hypervisor automatically adjusts allocations based on each SPLPAR’s use of cycles during the previous dispatch cycle.

Page 19: CMG Processor Virtualization

© 2009 IBM Corporation19

SPLPAR Utilization – Greater than 100% (?)

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

PU = 1.2

V V

PU = 0.5

V

PU = 1.5

V V

PU = 0.1

V

PU = 0.8

V V

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

PU Consumption PU Utilization VP Utilization0.50 33% 16.7%1.50 100% 50.0%2.25 150% 66.0%3.00 200% 100.0%

Page 20: CMG Processor Virtualization

© 2009 IBM Corporation20

Processing Units & Virtual Processors - Minimum & Maximum Settings

SPLPAR#5

PU = 1.5

V VV

HMC Setting

Processing Units

(PU)

Virtual Processors

(VP)

Maximum 4.2 6

Desired 1.5 3

Minimum* 0.5 2

Learning point: The min/max settings have nothing to do with resource allocation during normal operation. Min/max are limits applied only when making a dynamic change to PU or VP via the HMC.

* Min also allows an LPAR to start with less than the “desired” resource allocations.

1

2

3

4

5

6

7

8

9

10

11

12

6

7

8

9

10

11

12

1

2

3

4

5

Changes requires LPAR

stop and reactivation

Dynamic changes via the HMC

Normal Operation

DesiredProcessing

Units

DesiredVirtual

Processors

Page 21: CMG Processor Virtualization

© 2009 IBM Corporation21

Simultaneous Multi-threading (SMT)

POWER4 (Single Threaded)

CRL

FX0FX1LSOLS1FP0FP1BRZ

Thread0 activeNo thread active

Syst

em th

roug

hput

ST

Clock ticks

Exe

cutio

n U

nits

POWER5/6 (simultaneous multithreading)

One processor (dedicated or virtual) appears as two logical processors to the

operating system (AIX 5L V5.3 and Linux)

Utilizes unused execution unit cycles Dispatch two threads per processor: “It’s like doubling the number of processors.”

Thread1 active

SMT

Learning point: SMT = On Logical processors presentSMT = Off No logical processors

Page 22: CMG Processor Virtualization

© 2009 IBM Corporation22

POWER6, SMT = On

~ 50

%

Simultaneous Multi-threading (cont.)

SMT=Off

Users

Thro

ughp

ut

POWER5, SMT = On

~ 30

%

1

22’

1

2

2’

SMT = On/Off (no effect)

SMT = Off (less throughput)

SMT = On (more throughput)

Page 23: CMG Processor Virtualization

© 2009 IBM Corporation23

Simultaneous Multi-threading (cont.)

SMT = Off SMT = On

Low CPU Utilization

• SMT does not improve system throughput on a lightly loaded system

• SMT does not make a single thread run faster

SPEEDLIMIT

5.0 GHz

SMT = Off SMT = On

High CPU Utilization

• SMT does improve system throughput on a heavily loaded system

• SMT does not make a single thread run faster (unless it is waiting in the queue)

SPEEDLIMIT

5.0 GHz

Page 24: CMG Processor Virtualization

© 2009 IBM Corporation24

Logical Processors

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Weight = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

VWeight = 30

PU = 1.5

V

L

V

L

Weight = 10PU = 0.1

V

L L

Weight = 100PU = 0.8

V VWeight = 100

PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

Think “PVL“

Virtual

Logical

Physical

Simultaneous Multi-Threading (SMT) threads are represented by logical processors

Learning point: SMT requires a minimum of POWER5 hardware and AIX 5.3 (or supported Linux ver.) SMT can be dynamically enable/disable via an AIX command.

Page 25: CMG Processor Virtualization

© 2009 IBM Corporation25

Hypervisor dispatch model …. with SMT

Learning point: For SPLPARs with SMT enabled, each allocated processor presents two threads (logical processors) to the operating system.

Thread0

Thread1

Thread0

Thread1 Thread0

Thread1

Thread0

Thread1

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

Prev very busy, receives full allocation Reduced, did not use prev allocation

Reduced, did not use prev allocation Waiting on I/O, cedes cycles

Running steady workload Prev busy, receives excess cycles

SMTno off

onyes

SPLPAR3 on

0.6

0.4 yes

1.2

0.93

2

Virtual Processors

SPLPAR1

SPLPAR2 2

3

10 millisecond dispatch cycle

core

s

10 millisecond dispatch cycle

2

1

0

Processing Units

Proc Units Per VP Uncapped?

0.51

Note that SMT is off for SPLPAR1, therefore it only runs thread0 during its dispatch window.

Page 26: CMG Processor Virtualization

© 2009 IBM Corporation26

Summary: Minimum, desired, and maximum settings

Virtual Processors• Minimum = 2• Desired = 3• Maximum = 5

Processing Units• Minimum = 0.2• Desired = 1.2• Maximum = 16

SPLPAR

SMT=on

V V V

L

1.2 Proc Units

16 core Shared Proc. Pool

Normal Operations (Desired)

• SPLPAR is guaranteed 1.2 processing units at all times.

• If the SPLPAR is not busy, it will cede unused processing units to the shared pool.

• If the SPLPAR is busy:• capped: Limited to 1.2 processing units• uncapped: Use up to 3 processing units

Changes to Desired (DLPAR Operation)

• To change the range of spare processing units that can be utilized, use the HMC to change Desired Virtual Processors to a new value between the min and max settings.

• To change guaranteed processing units, use HMC to change Desired Processing Units to a new value between the min and max settings.

HMC

Use DLPARTo changeL L L L L

Page 27: CMG Processor Virtualization

© 2009 IBM Corporation27

Create LPAR – Shared or Dedicated?

Check Shared to have this LPAR’s processor resources come from the Shared Processor Pool.

Page 28: CMG Processor Virtualization

© 2009 IBM Corporation28

Create SPLPAR – Shared, detailed configuration

Specify minimum, desired, and maximum Processing Units, in 0.1 increments.

Specify minimum, desired, and maximum Virtual Processors, in whole processor increments.

Select Uncapped if you want this SPLPAR to utilize spare processor cycles.

Called “Entitled Capacity”

in some IBM perf. tools

Called “Online Virtual CPUs”

in some IBM perf. tools

Page 29: CMG Processor Virtualization

© 2009 IBM Corporation29

HMC help text - Desired Processing Units

Page 30: CMG Processor Virtualization

© 2009 IBM Corporation30

HMC help text - Minimum Processing Units

Page 31: CMG Processor Virtualization

© 2009 IBM Corporation31

HMC help text - Maximum Processing Units

Page 32: CMG Processor Virtualization

© 2009 IBM Corporation32

LPARs are defined as dedicated or shared Dedicated partitions use whole number of CPUs Shared partitions use whole and/or fractions of CPUs (smallest increment is 0.1, can be greater than 1.0)

Shared processor pool - subset (or all) of the physical CPUs in a system Physical processors shared among all of the SPLPARs within the shared processor pool

Entitled processing capacity expressed in 0.1 CPU increments Desired: Size of partition at boot time Minimum: Partition will start will less than desired, but won’t start if minimum capacity not available

DLPAR changes to desired cannot be below the minimum. Maximum:DLPAR changes to desired cannot exceed this capacity

Shared Pool LPARs run with ‘virtual’ processors Dedicated partitions use whole number of CPUs Shared partitions use whole and/or

Uncapped: No Processing unit usage is limited to desired setting. Yes Processing unit usage is allowed to exceed the desired processing unit setting.

Use weighting to determine preference for spare cyclesAutomatic Load Balancing (default is 128, 0 implies no use of spare cycles, 255 is max Weight)

Summary - Creating Shared Processor LPARs

Page 33: CMG Processor Virtualization

© 2009 IBM Corporation33

Memory configuration

0

Maximum - 8

0

Maximum – 32*

0

32

Hypervisor Page Table (HPT) Real Memory

Desired - 6

Desired - 5

Maximum – 16*

Maximum - 8*

512 MB

256 MB

128 MB

128 MB

HM

C S

ettin

gs

Learning Points: (1) The HPT presents a contiguous range of memory, starting at address 0, to each LPAR(2) Larger maximum memory settings increase the physical memory

consumed by the HPT

* Only one maximum memory setting is permitted per LPAR. Multiple maximums are shown here to demonstrate the corresponding HPT physical memory size.

Physical memory

consumed by the HPT

LPAR #1

LPAR #2

0 GB

0 GB

6 GB

5 GB

LPARs “see” a contiguous block

of memory starting at address 0

Page 34: CMG Processor Virtualization

© 2009 IBM Corporation34

Memory configuration

Maximum Memory Size ConsiderationThe logical memory map for each LPAR is contained in a Hypervisor Page Table (HPT). The size of the HPT is equal to Maximum / 64 (then rounded up to the next power of 2). Over sizing maximum memory can result in wasted memory.

Examples: Maximum memory setting = 64 GB HPT = 1 GBMaximum memory setting = 16 GB HPT = 256 MB

Learning Points: (1) Set maximum memory to 15%-30% greater than desired memory to allow for some

DLPAR increase, with minimal waste in the HPT. (2) Use powers of 2 for Max Memory setting (2GB, 4GB, 8GB, 16GB, 32GB, 64GB,etc)

Memory ConfigurationDesired Amount of memory normally allocated to the LPARMinimum Minimal amount of memory that must be available for the LPAR to start.

Also sets a low water mark for DLPAR changes to desired memory setting.Maximum Sets the high water mark for DLPAR changes to the desired memory setting

Page 35: CMG Processor Virtualization

© 2009 IBM Corporation35

Memory configuration

Specify minimum, desired, and maximum Memory,

Page 36: CMG Processor Virtualization

© 2009 IBM Corporation36

New POWER6 Features

Page 37: CMG Processor Virtualization

© 2009 IBM Corporation37

Shared Dedicated Capacity P6 Req’d

When this Dedicated CPU LPAR is deactivated, allow unallocated CPUs to be used by the Shared Processor Pool?

Allow excess processor cycles from this Dedicated CPU LPAR to be donated to the Shared Processor Pool?

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool 0

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Page 38: CMG Processor Virtualization

© 2009 IBM Corporation38

Multiple Shared Processor Pools

SPLPAR

Appl Server

SPLPAR

Web Server

SPLAR

DB.

SPLPAR

Appl Server

SPLPAR

Web Server

Shared Processor Pool

SPLPAR

ApplServer

SPLPAR

DBServer

SPLPAR

DB.

SPLPAR

ApplServer

SPLPAR

DBServer

Pool-1 Pool-2Pool-0

Sets an upper limit on processor resources accessible to a group of SPLPARs This is not a hard division of the shared processor pool into smaller sub-pools Up to 64 pools can be configured per server Can help with software licensing Can help balance Prod/Dev on the same server

P6 Req’d

Page 39: CMG Processor Virtualization

© 2009 IBM Corporation39

Multiple Shared Processor Pools

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weight = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

VWeight = 30

PU = 1.5

V

L

V

L

Weight = 10PU = 0.1

V

L L

Weight = 100PU = 0.8

V VWeight = 100

PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

Virtual

Logical

Physical

Physical

MaxPU … A whole number which specifies maximum processing units that can be consumed by all of the SPLPARs running in this pool,

ReservedPU = Additional, guaranteed Processing Units for each pool (could be 0)

Default Pool ID = 0 (cannot specify MaxPU or ReservedPU for the Default Pool)

P6 Req’d

Page 40: CMG Processor Virtualization

© 2009 IBM Corporation40

Multiple Shared Processor Pool configuration

Pool IDs are fixed and numbered 0...63

SPLPARs can dynamically be moved to a different pool

Disable a pool by setting its Maximum processing units to zero

Default Pool ID = 0 You cannot set reserved

processing units or Maximum processing units for the Default Pool

P6 Req’d

Page 41: CMG Processor Virtualization

© 2009 IBM Corporation41

Suggestions for Shared Processor Pool

LPAR Settings

Page 42: CMG Processor Virtualization

© 2009 IBM Corporation42

Operating within the Shared Processor Pool

Pro

cess

ing

Uni

ts

(CP

Us)

Time

0

1

2

3

4

Desired proc. unitsP

roce

ssin

g U

nits

(C

PU

s)

Time

0

1

2

3

4

User of extra proc. units

Pro

cess

ing

Uni

ts

(CP

Us)

Time

0

1

2

3

4

Donor of extra proc. Units

Virt. Procs Virt. Procs Virt. Procs

Proc. Units Proc. Units Proc. Units

Desired Virtual Processors• Find the peak, move up to the next whole number

Desired Processing Units • More subjective, no best answer• Need a mix of users and donors to have processor sharing• High priority applications: Set processing units higher• Low priority applications: Set processing units lower

Page 43: CMG Processor Virtualization

© 2009 IBM Corporation43

Setting desired Processing Units and Virtual Processors

Example: Normal requirement is 0.9 processing units Peak requirement is 3.8 processing units

LPAR Settings:

Virtual Processor Sizing: Set desired virtual processors large enough to handle peak load Desired = 4 (round 3.8, peak requirement up to next whole number)

Minimum* = Starting point might be 2, or ½ of desired.Maximum* = Number of CPUs in the shared pool

Processing Units: Set desired processing units to address non-peak, normal workloadDesired = 0.9 (set to match 0.9 processing units, normal requirement)Minimum* = Good starting point might be 0.5, or approx. ½ of Desired.Maximum* = Number of CPUs in the shared pool

* These only come into play if we make dynamic changes using the HMC

Proc

essi

ng U

nits

(CPU

s)

Time

0

1

2

3

4

Page 44: CMG Processor Virtualization

© 2009 IBM Corporation44

Think P-V-L (physical, virtual, logical)

Virtual processors can represent 0.1 to 1 physical processing units

Set Desired Processing Units to cover major portion of workload

Set Desired Virtual Processors to match peak workload

LPAR CPU utilization > 100% is a good thing (using spare cycles!)

Plan to measure utilization at the server level

Consolidate like software onto the same server for improved software utilization and reduced software license costs.

CPU virtualization summary

Page 45: CMG Processor Virtualization

© 2009 IBM Corporation45

Virtualization FunctionsSummary

Page 46: CMG Processor Virtualization

© 2009 IBM Corporation46

Historical processor sharing/virtualization technologies

Learning point: Ability to deploy tools is dependent upon the OS version and processor model.

Tool Processor Family Description

AIX 4.3

(out of service)

AIX 5.1

(out of service)

AIX 5.2 AIX 5.3 AIX 6.1

Application Stacking

(WLM)All Application stacking on a

single OS image Yes Yes Yes Yes Yes

Static LPAR

POWER4

POWER5

POWER6

Static allocation of whole CPUs No Yes Yes Yes Yes

Dynamic LPAR (PLM)

POWER4

POWER5

POWER6

Dynamic allocation of whole CPUs No No Yes Yes Yes

Shared Processor Pool

POWER5

POWER6Dynamic allocation of fractional CPUs No No No Yes Yes

Application Stacking (WPAR)

POWER4

POWER5

POWER6

Improved application stacking on a single OS image

No No No No Yes

Page 47: CMG Processor Virtualization

© 2009 IBM Corporation47

YesNoYesNoYesNoWorkload Partitions (WPARs)

& WPAR Mobility

YesYesNoNoNoNoIntegrated Virtualized Ethernet

YesYesNoNoNoNoShared Dedicated Capacity

YesYesNoNoNoNoPartition Mobility

YesYesNoNoNoNoMultiple Shared Processor Pools

AIX 6.1AIX 5.3AIX 6.1AIX 5.3AIX 6.1AIX 5.3OS Level

POWER6POWER5POWER4Processor

AIX 6.1

Req’d

POWER6

Req’dLegend

Summary of new features for POWER6 and AIX 6.1

Learning point: Most of the new features introduced with POWER6 are supported with AIX 5.3

Page 48: CMG Processor Virtualization

© 2009 IBM Corporation48

Additional Information

http://publib.boulder.ibm.com/infocenter/systems/scope/hw• IBM Power Systems Logical Partitioning Guide (search on “Logical Partitioning Guide”)• PowerVM Editions Operations Guide (search on “PowerVM”)

www.ibm.com/redbooks• PowerVM Virtualization on IBM System p Introduction and Configuration (SG24-7940)• PowerVM Virtualization on IBM System p Managing and Monitoring (SG24-7590)

IBM Systems Hardware Information Center

Page 49: CMG Processor Virtualization

© 2008 IBM Corporation

PowerVM Processor Virtualization:Capacity Planning &Software Licensing Models

Session: pVM02

Charlie Cler Executive IT [email protected]

Page 50: CMG Processor Virtualization

© 2009 IBM Corporation50

Power Systems Software positioning

PowerVM processor virtualization terminology review

Knobs and dials that affect Shared Processor Pool LPARs

General thoughts about CPU utilization/consumption

System capacity basics

System tuning considerations

Software licensing concepts

Agenda

Page 51: CMG Processor Virtualization

© 2009 IBM Corporation51

Terminology Review

Page 52: CMG Processor Virtualization

© 2009 IBM Corporation52

Processor virtualization …the whole picture

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weighting = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

V

Weighting = 30PU = 1.5

V

L

V

L

Weighting = 10PU = 0.1

V

L L

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

P V L

Virtual

Logical (SMT)

Physical

Physical

HMC AIX Performance ToolsDesired Processing Units Entitled Capacity (EC)

Desired Virtual Processors Online Virtual CPUs

Terminology translation…

Page 53: CMG Processor Virtualization

© 2009 IBM Corporation53

Knobs and Dials – Dedicated CPU LPARs

Feature Description

Logical Processors Present if SMT = ON (SMT is turned on/off via AIX)

Minimum Processors Only comes into play if you want to change the Desired Processors setting via the HMC –or- if the system is low on resources and you are willing to let your LPAR start with less than the Desired Processors setting.

Desired Processors Used for normal operation

Maximum Processors Only comes into play if you want to change the Desired Processors setting via the HMC

Processor Sharing:

Allow when partition is inactive

Checking this box will cause the number of processors in the Shared Processor Pool to increase when this LPAR is de-activated and then decrease when it is activated.

Processor Sharing:

Allow when partition is active

Allows unused, dedicated CPU processor cycles to be donated to the Shared Processor Pool.

Dedicated CPU LPAR

V V V3 cores

Shared Proc Pool

= Most commonly used

Page 54: CMG Processor Virtualization

© 2009 IBM Corporation54

Knobs and Dials – Shared Processor Pool LPARsFeature Description

Logical Processors Present if SMT = ON. SMT is turned on/off via AIX.

Minimum Virtual Processors

Only comes into play if you want to change the Desired Processing Units setting via the HMC –or- if the system is low on resources and you are willing to let your LPAR start with less than the Desired Processing Units setting.

Desired Virtual Processors

Used for normal operation.

Maximum Virtual Processors

Only comes into play if you want to change the Desired Virtual Processor setting via the HMC.

Minimum Processing Units

Only comes into play if you want to change the Desired Processing Units setting via the HMC –or- if the system is low on resources and you are willing to let your LPAR start with less than the Desired Processing Units setting.

Desired Processing Units

Used for normal operation.

Maximum Processing Units

Only comes into play if you want to change the Desired Processing Units setting via the HMC.

Uncapped Allow this SPLPAR to access extra processing cycles? (yes/no)

Uncapped Weighting Number between 0 and 255.

Pool ID Which pool to run this LPAR in (default =0).

SPLPAR

V V V

1.2 Proc Units

Shared Proc Pool

Pool = 0

Page 55: CMG Processor Virtualization

© 2009 IBM Corporation55

Knobs and Dials – Multiple Shared Processor Pools

Feature Description

Pool Name Descriptive text name for pool. User specified.

Pool ID Number between 0 and 64. Not changeable. DefaultPool = 0.

Reserved Processing Units

Additional Processing Units guaranteed to be available to a specific pool. This is in addition to the Desired Processing Units specified for each SPLPAR in this pool.

Maximum Processing Units

Whole number, which represents the maximum, total number of Processing Units that can be consumed the by the SPLPARs running in this pool.

SPLPAR

ApplServer

SPLPAR

WebServer

SPLPAR

DB.

SPLPAR

ApplServer

SPLPAR

WebServer

Pool-1 Pool-2Pool-0

Page 56: CMG Processor Virtualization

© 2009 IBM Corporation56

System Capacity Basics

Page 57: CMG Processor Virtualization

© 2009 IBM Corporation57

Need to focus on consumption & timing, not utilization

Virtualized CPU Utilization (old way) Measurement focused on percentage of entitlement used

• You may have been given to much (low utilization)• You may have been given to little (high utilization)

Does not indicate (by itself) if application is running OK Changes in utilization can be used for trending application health

CPU Consumption (new way) Measurement of how much is being used, plotted against time Entitlement does not affect consumption (unless capped -or- pool is at 100%) Can be used for application chargeback Helpful for capacity planning

Page 58: CMG Processor Virtualization

© 2009 IBM Corporation58

SPLPAR Utilization

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

PU = 1.2

V

L

V

L L L

PU = 0.5

V

PU = 1.5

V

L

V

L

PU = 0.1

V

L L

PU = 0.8

V V

PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

PU Consumption PU Utilization VP Utilization

0.50 33% 16.7%

1.50 100% 50.0%

2.25 150% 66.0%

3.00 200% 100.0%

Desired VP utilization can be used to easily measure CPU consumption

We can create any utilization level that we want by changing the Desired PU setting :-)

Page 59: CMG Processor Virtualization

© 2009 IBM Corporation59

lpar2rrd tool

http://www.ibm.com/developerworks/wikis/display/virtualization/lpar2rrd+tool

Intended for Micro-Partitioned systems (it will show 100% utilization for dedicated CPU LPARs) Creates charts based on utilization data collected on HMC's LPAR2RRD shows up CPU donation of partitions with dedicated CPUs in "allowed processor sharing“ Supports all types of SPLPAR (AIX/Linux/IBM i) Creates physical (HW) and logical configuration inventory of all managed systems and their lpars

(once a day) The tool automatically recognizes any change/adding/remove/rename of LPAR or managed system Creates 4 kind of graphs for each LPAR, CPU shared pool and memory usage. Last day, week and

month are based on 60sec averages. Yearly chart is based on daily averages provided by the HMC. Supports more Utilization data collection sample rates for HMC 7.3.3+. Will work with original 1h

sample rate for previous HMC versions. No clients (agents) on SPLPARs (it uses ssh-keys based access to HMC servers under role hmcviewer

to get data automatically) Automatically generates a menu based WWW front end for viewing charts and configuration Easy to install, configure and use. Install & configuration should not take more than an hour. Tool can be hosted on any platform that provides: Web server, SSH, Perl and RRDTool installation.

Page 60: CMG Processor Virtualization

© 2009 IBM Corporation60

lpar2rrd tool – Shared pool data

Page 61: CMG Processor Virtualization

© 2009 IBM Corporation61

lpar2rrd tool – Memory & Multiple shared pools

Page 62: CMG Processor Virtualization

© 2009 IBM Corporation62

lpar2rrd tool – Specific SPLPAR data

Page 63: CMG Processor Virtualization

© 2009 IBM Corporation63

“Processor Folding” feature of AIX 5.3 TL3 and higher

Learning point: Size the number of desired virtual processors for the peak workload. The hypervisor will automatically allocate resources to the virtual processors with work to be done

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

If all four virtual processors have work to be done, each will receive 0.4 processing units

If only two virtual processors have work to be done, the hypervisor will temporarily direct all processing units to the two busy virtual processors. Each will receive 0.8 processing units.

AIX 5.3

LPAR

V V V V

1.6 Proc. Units

ProcessorFolding

Reduced number of active threads

(Checked once each second by the

hypervisor)

Processor folding can be disabled: #schedo –o vpm_xvcpus=-1See section 5.8.3 Processor Folding of IBM Redbook AIX 5L Differences Guide Version 5.3 Addendum SG24-7414

Desired

Desired

Desired

Desired

Page 64: CMG Processor Virtualization

© 2009 IBM Corporation64

Example of Virtual Processor Folding

Desired Virtual Processors = 20 Desired Processing Units = 10

Note that this LPAR only used its 20 VPs for a short period of time. It might be possible to reduce desired VPs to 16 to reduce software costs.

nmon data

Learning point: Review “processor folding” stats to see if you need to increase or decrease the Desired Virtual Processor setting.

Page 65: CMG Processor Virtualization

© 2009 IBM Corporation65

Can I add additional LPARs or SPLPARs to my system?

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

PU = 1.2

V V

PU = 0.5

V

PU = 1.5

V V

PU = 0.1

V

PU = 0.8

V V

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Activated Cores = 12

Total PU = 4.9Reserved PU = 0.8

Dedicated Cores = 3.0

8.7

Cores available to allocate = Total # of cores – Allocations

= Total # of cores – Total Desired PU – Reserved PU – Dedicated Cores

= 12 – 8.7 = 3.3 cores available

Page 66: CMG Processor Virtualization

© 2009 IBM Corporation66

Is my system positioned for processor sharing?

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Shared Processor Pool

PU = 1.2

V V

PU = 0.5

V

PU = 1.5

V V

PU = 0.1

V

PU = 0.8

V V

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Total VP = 11

Pool Size = 9

If Total Desired Virtual Processors > Pool Size Potential for Processor Sharing

Single Shared Processor Pool

11 > 9 All virtual processors cannot peak at the same time

Page 67: CMG Processor Virtualization

© 2009 IBM Corporation67

Is my system positioned for processor sharing?

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Shared Processor Pool

PU = 1.2

V V

PU = 0.5

V

PU = 1.5

V V

PU = 0.1

V

PU = 0.8

V V

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Total VP = 7

Pool Size = 9

Desired Virtual Processors < Pool Size No Potential for Processor Sharing

Single Shared Processor Pool

Spare cores

Learning point: All of the active virtual processors have access to a full core …. No sharing required.

Page 68: CMG Processor Virtualization

© 2009 IBM Corporation68

Is my system positioned for processor sharing?

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

PU = 1.2

V V

PU = 0.5

V

PU = 1.5

V V

PU = 0.1

V

PU = 0.8

V V

PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Multiple Shared Processor Pools

Pool #0 – All VP’s could peak

Pool #1 – Not all VP’s can peak at the same time. 4 VPs configured but MaxPU = 3

Pool #2 – Not all VP’s can peak at the same time. 4 VPs configured but MaxPU = 2

Pool #0Total VP = 3Total PU = 1.7MaxPU = n/a

Pool #1Total VP = 4Total PU = 2.1 MaxPU = 3

Pool #2Total VP = 4Total PU = 1.9MaxPU = 2

Page 69: CMG Processor Virtualization

© 2009 IBM Corporation69

Multiple Shared Processor Pools – Spare CPU cycle sharing

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weighting = 255PU = 1.2

V V

Uncap = NoPU = 0.5

V

Weighting = 30PU = 1.5

V V

Weighting = 10PU = 0.1

V

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

1. Unused SPLPAR CPU cycles are redistributed to other SPLPARs within the same Shared Processor Pool (SPP)

2. If there is no demand for spare CPU cycles within the SPP, the hypervisor redistributes the spare CPU cycles to other SPLPARs in the system using uncapped priorities (system-wide)

Page 70: CMG Processor Virtualization

© 2009 IBM Corporation70

# lparstat -iNode Name : DB_serverPartition Name : DB_server_migratePartition Number : 3Type : Shared-SMT SMT = OnMode : Uncapped Can consume extra processor cyclesEntitled Capacity : 1.00 Desired Processing UnitsPartition Group-ID : 32771Shared Pool ID : 1 Running in shared pool #1Online Virtual CPUs : 2 Desired Virtual ProcessorsMaximum Virtual CPUs : 4 Maximum Virtual ProcessorsMinimum Virtual CPUs : 2 Minimum Virtual ProcessorsOnline Memory : 4608 MB Desired MemoryMaximum Memory : 6144 MBMinimum Memory : 4096 MBVariable Capacity Weight : 128Minimum Capacity : 0.50Maximum Capacity : 4.00Capacity Increment : 0.01Maximum Physical CPUs in system : 16Active Physical CPUs in system : 4Active CPUs in Pool : 2 Shared pool #1 has 2 CPUsShared Physical CPUs in system : 4 4 CPUs available for use by shared poolsMaximum Capacity of Pool : 200 2 CPUs x 100 10millisecond dispatch windowsEntitled Capacity of Pool : 140Unallocated Capacity : 0.00Physical CPU Percentage : 50.00% Entitled Capacity / Online Virtual CPUsUnallocated Weight : 0

lparstat –I provides SPLPAR configuration information

Good source of SPLPAR configuration without HMC access

Page 71: CMG Processor Virtualization

© 2009 IBM Corporation71

Many SPLPAR commands report consumption, not utilization

# sar -P ALL 3 2AIX Server1 3 5 00C0G3B04C0 11/28/07System configuration: lcpu=2 ent=0.30 mode=Uncapped11:33:28 cpu %usr %sys %wio %idle physc %entc11:33:31 0 3 85 0 12 0.01 2.8 1 0 36 0 63 0.00 0.4 U - - 0 96 0.29 96.8 - 0 3 0 97 0.01 3.211:33:34 0 4 63 0 33 0.00 1.1 1 0 35 0 65 0.00 0.4 U - - 0 98 0.30 98.5 - 0 1 0 99 0.00 1.5Average 0 3 79 0 18 0.01 1.9 1 0 36 0 64 0.00 0.4 U - - 0 97 0.29 97.6 - 0 2 0 98 0.01 2.4

%entc = % of entitled capacity used = (2.8%) (0.30) = 0.0084 PU85 %sys is of the 0.0084PU consumed = 0.007 PU Notes: lcpu = 2 => SMT = On –and- 1 Virtual Processor

U = Unused capacity

Be sure to read the documentation to understand what data is being reported!

1 CPU

0.3 CPU

Page 72: CMG Processor Virtualization

© 2009 IBM Corporation72

Utilization is high … average consumption is only 40% of allocation

The hypervisor is very efficient at allocating just the right amount of CPU to each LPAR.

User% + Sys% is constantly running above 90% …. The system appears to be very busy.

Page 73: CMG Processor Virtualization

© 2009 IBM Corporation73

System Tuning Considerations

Page 74: CMG Processor Virtualization

© 2009 IBM Corporation74

Shared Processor Pool Performance Tuning Considerations

• First, you need to understand if the application is performing O.K. • It is possible to have 150% CPU utilization at the LPAR level and still have good performance • CPU utilization above 100% just means that you are consuming spare cycles from the pool, above

your desired processing units guarantee.

My CPU utilization is

95%!

My CPU utilization is

150%!!!

End-UserI/T OperationsApplication Monitoring

Page 75: CMG Processor Virtualization

© 2009 IBM Corporation75

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weighting = 255PU = 1.2

V V

Uncap = NoPU = 0.5

V

Weighting = 30PU = 1.5

V V

Weighting = 10PU = 0.1

V

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Shared Processor Pool Performance Tuning Considerations

If the Shared Processor Pool utilization is normal (not peaking), you can:• For Capped SPLPARs … Increase the desired processing units setting -or- uncap the

SPLPAR so that it can gain access to extra share processor pool cycles.• For Uncapped SPLPARs … You might need to increase the number of desired virtual

processors assigned to the SPLPAR to allow it to consume more processing units.

Page 76: CMG Processor Virtualization

© 2009 IBM Corporation76

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

SPLPAR#5

SPLPAR#6

SPLPAR#7

SPLPAR#8

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weighting = 255PU = 1.2

V V

Uncap = NoPU = 0.5

V

Weighting = 30PU = 1.5

V V

Weighting = 10PU = 0.1

V

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V VV

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

Shared Processor Pool Performance Tuning Considerations

If the Shared Processor pool utilization is high, you can:• Enable processor sharing on Dedicated CPU LPARs.• Increase MaxPU and/or ReservedPU for Multiple Shared Processor Pools.• Reduce uncapped Weighting for other SPLPARs.• Dial back some SPLPARs ability to consume extra cycles by reducing their desired virtual processors

setting.• Cap some SPLPARs.• As a last resort, you may need to reduce Desired Processing Units for lower priority SPLPARs.

Page 77: CMG Processor Virtualization

© 2009 IBM Corporation77

General thoughts about CPU Utilization/Consumption

Page 78: CMG Processor Virtualization

© 2009 IBM Corporation78

Why do we care about CPU utilization/consumption ?

Deployment Costing One-time (purchase)

• Initial application deployment• Increasingly more difficult in a virtualized world

Ongoing chargeback (rental)• Desirable, but does not fit most business environments

Poor application performance Historical trends Application problem Server problem / resource allocation changes

Capacity planning for application growth

Proc

essi

ng

Uni

ts (C

PUs)

Time

0

1

2

3

4

Page 79: CMG Processor Virtualization

© 2009 IBM Corporation79

Server utilization –and- LPAR consumption

Server Level Focus on CPU utilization & consumption for the

entire server Less concern about individual LPARs Am I maximizing the return on my asset? Do I need to purchase additional servers? When? Minimize software license costs Most common limits: Available memory & “How

many eggs in one basket”

LPAR Level Focus on CPU consumption Less concern about other LPARs Is my application performing OK? Budgeting for future growth Minimize software license costs

SPLPAR#5

SMT=On

PU = 1.5

V

L

V

L

V

L L L L

Page 80: CMG Processor Virtualization

© 2009 IBM Corporation80

Operational strategy shift …. Something to think about

1. Dedicated Admin/Server assignments High level of accountability Focus on increasing Admin to server ration 10:1, 20:1, 30:1 … Admins must know how to do everything from A-Z Employee with 6 months experience expected to perform the

same duties as a 20 year veteran Move towards virtualization can blur boundaries of responsibility

2. Pooled Resources Work dispatched via ticketing system Provides the ability to master a skills at an individualized pace Easier to balance skills & staffing levels Encourages automation

2.5 “Server management” skill group Manage HMCs Create/modify LPARs Track, manage, and install firmware Monitor server utilization

Page 81: CMG Processor Virtualization

© 2009 IBM Corporation81

Software Licensing

Page 82: CMG Processor Virtualization

© 2009 IBM Corporation82

Software Licensing Options – Dedicated CPU LPAR

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

PU = 1.2

V

L

V

L L L

PU = 0.5

V

PU = 1.5

V

L

V

L

PU = 0.1

V

L L

PU = 0.8

V V

PU = 0.8

V

L

V

L L L

V

L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

- Desired Processors123

- Number of logical processors (rare)- Number of cores in the server (unfortunate)

Some Dedicated CPU LPAR Licensing Techniques

Less

Des

irabl

e

3

1

2L

Page 83: CMG Processor Virtualization

© 2009 IBM Corporation83

Software Licensing Options – SPLPAR

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Shared Processor Pool (9 cores)

Weighting = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

V

Weighting = 30PU = 1.5

V

L

V

L

Weighting = 10PU = 0.1

V

L L

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

- Desired Virtual Processors12345

- Number of logical processors (rare)Some SPLPAR Licensing Techniques Le

ss D

esira

ble

- Total of all Shared Processor cores- Number of cores in the server

3

1

2

45

- Desired Processing Units (if uncapped=no)

Page 84: CMG Processor Virtualization

© 2009 IBM Corporation84

Software Licensing – SPLPAR, multiple processor pools

LPAR#1

SMT=On

LPAR#2

SMT=Off

SPLPAR#3

SMT=On

SPLPAR#4

SMT=Off

SPLPAR#5

SMT=On

SPLPAR#6

SMT=On

SPLPAR#7

SMT=On

SPLPAR#8

SMT=On

Hypervisor

Pool # 0

Pool MaxPU = 3 #1 ReservedPU = 0.5

Pool MaxPU = 2 #2 ReservedPU = 0.3

Weighting = 255PU = 1.2

V

L

V

L L L

Uncap = NoPU = 0.5

V

Weighting = 30PU = 1.5

V

L

V

L

Weighting = 10PU = 0.1

V

L L

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V

L

V

L L L

V

L L L L

L L

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

L L L L

1 Core (dedicated)

2 Cores(dedicated)

- Desired Virtual Processors (uncapped=yes)12345

- Number of logical processors (rare)- Maximum Processing Units for the local pool

Some SPLPAR licensing techniques Less

Des

irabl

e

- Total of all shared processor pool cores- Number of cores in the server

3

1

2

4

5

- Desired Processing Units (uncapped=no)

6

6

Page 85: CMG Processor Virtualization

© 2009 IBM Corporation85

LPAR#1

IBM Software Example: Server with no partitions

WAS MQ

Scenario #1 - Full capacity software license

Hypervisor

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

12 Cores (dedicated)

WAS MQDedicated Cores 12 12License Count 12 12

Page 86: CMG Processor Virtualization

© 2009 IBM Corporation86

LPAR#2

LPAR#1

IBM SW Example: Server with Simple Virtualization (LPARs)

WAS

MQ

Hypervisor

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

8 Cores (dedicated)

4 Cores(dedicated)

WAS

Scenario #2 – With Sub-Capacity software licensing option

Notes: Logical partitioning can increase processor capacity

utilization TCO improves as cores to be licensed decreases

WAS MQDedicated Cores 8 + 4 = 12 4License Count 12 4

Page 87: CMG Processor Virtualization

© 2009 IBM Corporation87

IBM SW Example: Server with shared pool (SPLPARs)

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

WAS

SPLPAR#5

WAS

SPLPAR#6

WAS

SPLPAR#7

MQ

SPLPAR#8

Hypervisor

Pool (9 cores)

Weighting = 255PU = 1.2

V V

Weighting = 30PU = 1.5

Weighting = 30PU = 1.5

Weighting = 10PU = 1.5

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V V

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

V V V V V V V V V V V V

Scenario #3a – With Sub-Capacity software licensing option

WAS MQVirtual Processors 4 + 4 + 4 = 12 2Pool Size 9 9License Count 9 2

Notes: Shared pool can further increase processor

utilization TCO improves as cores to be licensed decreases

Page 88: CMG Processor Virtualization

© 2009 IBM Corporation88

IBM SW Example: Effect of shared dedicated capacity

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

WAS

SPLPAR#5

WAS

SPLPAR#6

WAS

SPLPAR#7

MQ

SPLPAR#8

Hypervisor

Pool (9 cores)

Weighting = 255PU = 1.2

V V

Weighting = 30PU = 1.5

Weighting = 30PU = 1.5

Weighting = 10PU = 1.5

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V V

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

V V V V V V V V V V V V

Scenario #3b – With Sub-Capacity software licensing option

WAS MQVirtual Processors 4 + 4 + 4 = 12 2Pool Size 11 11License Count 11 2

Note: Shared dedicated capacity increases the size of

the shared pool from 9 to 11 in this example.

Shared dedicated capacity enabled

Page 89: CMG Processor Virtualization

© 2009 IBM Corporation89

IBM SW Example: Server with multiple shared pools (SPLPARs)

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

WAS

SPLPAR#5

WAS

SPLPAR#6

WAS

SPLPAR#7

MQ

SPLPAR#8

Hypervisor

Pool# 0

Weighting = 255PU = 1.2

V V

Weighting = 30PU = 1.5

Weighting = 30PU = 1.5

Weighting = 10PU = 1.5

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V V

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

V V V V V V V V V V V V

Scenario #4 – With Sub-Capacity software licensing option

WAS MQVirtual Processors 4 + 4 + 4 = 12 2Sub Pool Size 6 4License Count 6 2

Same server, fewer software licenses Multiple shared pools can further increase

processor utilization TCO improves as cores to be licensed decreases Shared dedicated capacity does not affect license

Pool #1 MaxPU = 6 Pool #2 MaxPU = 4

Shared dedicated capacity enabled

Page 90: CMG Processor Virtualization

© 2009 IBM Corporation90

Checkpoint - IBM License Rules for Partitions & Shared Pools

For a Dedicated LPAR, the highest* number of cores allocated to the LPAR

For a Capped Partition, the highest* amount of Entitled Capacity (EC)

For a Uncapped Partitions, the highest* amount of VP (Virtual Processor Cores)

For a Shared Processor Pool, the lower of the sum of each partition for a product or the

processor core capacity of the shared pool

Aggregate fractional processor cores, apply shared pool capping rules, and round up at

the server level to the next whole processor core

Lower of the sum of the virtual cores or the server capacity)

* The greater of what the partition starts with or the result of a DLPAR operation.

ftp://ftp.software.ibm.com/software/passportadvantage/SubCapacity/Scenarios_System_p_System_i.pdf

Page 91: CMG Processor Virtualization

© 2009 IBM Corporation91

IBM SW Example: Server with WPARs* (shared based allocations)

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

WAS

SPLPAR#5

WAS

SPLPAR#6

WAS

SPLPAR#7DB2

30 shares

MQ10 shares

SPLPAR#8

Hypervisor

Pool# 0

Weighting = 255PU = 1.2

V V

Weighting = 30PU = 1.5

Weighting = 30PU = 1.5

Weighting = 10PU = 1.5

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V V

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

V V V V V V V V V V V V

Scenario #5a – With Sub-Capacity software licensing option

DB2 MQVirtual Processors 2 2Allocation 30 Shares 10 sharesLicense Count 2 2

Notes: When using share based CPU allocations with

WPARs, licenses are based on the virtual processors (or cores for LPAR) assigned to the partition

Pool #1 MaxPU = 6 Pool #2 MaxPU = 4

Shared dedicated capacity enabled

* Planned future support

Page 92: CMG Processor Virtualization

© 2009 IBM Corporation92

IBM SW Example: Server with WPARs* (% based allocations)

LPAR#1

LPAR#2

SPLPAR#3

SPLPAR#4

WAS

SPLPAR#5

WAS

SPLPAR#6

WAS

SPLPAR#7DB2

75% CPU

MQ25% CPU

SPLPAR#8

Hypervisor

Pool# 0

Weighting = 255PU = 1.2

V V

Weighting = 30PU = 1.5

Weighting = 30PU = 1.5

Weighting = 10PU = 1.5

Weighting = 100PU = 0.8

V V

Weighting = 100PU = 0.8

V V

Core Core Core CoreCoreCoreCoreCoreCoreCore CoreCore

1 Core (dedicated)

2 Cores(dedicated)

V V V V V V V V V V V V

Scenario #5b – With Sub-Capacity software licensing option

DB2 MQVirtual Processors 2 2Allocation 75% 25%License Count 2 1

Notes: When using fixed percent CPU allocations with

WPARs, licenses are based on the fixed percent multiplied by the number of virtual processors (or cores for LPAR) assigned to the partition

Pool #1 MaxPU = 6 Pool #2 MaxPU = 4

Shared dedicated capacity enabled

* Planned future support

Page 93: CMG Processor Virtualization

© 2009 IBM Corporation93

Consideration for Live Partition and WPAR mobility

Note: Consult an IBM software sales specialist for official terms & conditions

Need to confirm with IBM Software sales representative

IBM SW License Rules for Live Partition or Application (WPAR) Mobility:(Planned future support) For Qualified Mobility Events, license the larger peak capacity between the ‘to’ and

‘from’ systems Partition or WPAR moved is not counted twice

Qualified Mobility Events include: A WPAR or Partition moved between 2 systems for server maintenance. A WPAR or Partition moved between 2 servers for a permanent migration.

All other WPAR or Partition movements are not ‘qualified mobility events’ Standard capacity licensing applies - peak processor capacity available for each

system

Page 94: CMG Processor Virtualization

© 2009 IBM Corporation94

Remember P-V-L (physical, virtual, logical)

Focus on consumption and timing, not utilization

LPAR CPU utilization > 100% is a good thing (making use of spare cycles!)

Understand what your performance tuning commands are reporting

For poor application performance: Pool Utilization low -> Focus on allocations fo this SPLPAR Pool Utilization high -> Look at all LPARs on your system for balancing opportunities.

Software licensing can take many forms. Understand your configuration so that you can negotiate with your software vendor.

Summary

Page 95: CMG Processor Virtualization

© 2009 IBM Corporation95

Additional Information

http://publib.boulder.ibm.com/infocenter/systems/scope/hw• IBM Power Systems Logical Partitioning Guide (search on “Logical Partitioning Guide”)• PowerVM Editions Operations Guide (search on “PowerVM”)

www.ibm.com/redbooks• PowerVM Virtualization on IBM System p Introduction and Configuration (SG24-7940)• PowerVM Virtualization on IBM System p Managing and Monitoring (SG24-7590)

IBM Systems Hardware Information Center

Page 96: CMG Processor Virtualization

© 2009 IBM Corporation96

This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area.Information in this document concerning non-IBM products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied.All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions.IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice.IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies.All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary.IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment.

Revised September 26, 2006

Special Notices

Page 97: CMG Processor Virtualization

© 2009 IBM Corporation97

The following terms are registered trademarks of International Business Machines Corporation in the United States and/or other countries: AIX, AIX/L, AIX/L (logo), AIX 6 (logo), alphaWorks, AS/400, BladeCenter, Blue Gene, Blue Lightning, C Set++, CICS, CICS/6000, ClusterProven, CT/2, DataHub, DataJoiner, DB2, DEEP BLUE, developerWorks, DirectTalk, Domino, DYNIX, DYNIX/ptx, e business (logo), e(logo)business, e(logo)server, Enterprise Storage Server, ESCON, FlashCopy, GDDM, i5/OS, i5/OS (logo), IBM, IBM (logo), ibm.com, IBM Business Partner (logo), Informix, IntelliStation, IQ-Link, LANStreamer, LoadLeveler, Lotus, Lotus Notes, Lotusphere, Magstar, MediaStreamer, Micro Channel, MQSeries, Net.Data, Netfinity, NetView, Network Station, Notes, NUMA-Q, OpenPower, Operating System/2, Operating System/400, OS/2, OS/390, OS/400, Parallel Sysplex, PartnerLink, PartnerWorld, Passport Advantage, POWERparallel, Power PC 603, Power PC 604, PowerPC, PowerPC (logo), Predictive Failure Analysis, pSeries, PTX, ptx/ADMIN, Quick Place, Rational, RETAIN, RISC System/6000, RS/6000, RT Personal Computer, S/390, Sametime, Scalable POWERparallel Systems, SecureWay, Sequent, ServerProven, SpaceBall, System/390, The Engines of e-business, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, Tivoli Ready (logo), TME, TotalStorage, TURBOWAYS, VisualAge, WebSphere, xSeries, z/OS, zSeries.

The following terms are trademarks of International Business Machines Corporation in the United States and/or other countries: Advanced Micro-Partitioning, AIX 5L, AIX PVMe, AS/400e, Calibrated Vectored Cooling, Chiphopper, Chipkill, Cloudscape, DataPower, DB2 OLAP Server, DB2 Universal Database, DFDSM, DFSORT, DS4000, DS6000, DS8000, e-business (logo), e-business on demand, EnergyScale, Enterprise Workload Manager, eServer, Express Middleware, Express Portfolio, Express Servers, Express Servers and Storage, General Purpose File System, GigaProcessor, GPFS, HACMP, HACMP/6000, IBM Systems Director Active Energy Manager, IBM TotalStorage Proven, IBMLink, IMS, Intelligent Miner, iSeries, Micro-Partitioning, NUMACenter, On Demand Business logo, POWER, PowerExecutive, PowerVM, PowerVM (logo), Power Architecture, Power Everywhere, Power Family, POWER Hypervisor, Power PC, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), PowerPC Architecture, PowerPC 603, PowerPC 603e, PowerPC 604, PowerPC 750, POWER2, POWER2 Architecture, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, pure XML, Quickr, Redbooks, Sequent (logo), SequentLINK, Server Advantage, ServeRAID, Service Director, SmoothStart, SP, System i, System i5, System p, System p5, System Storage, System z, System z9, S/390 Parallel Enterprise Server, Tivoli Enterprise, TME 10, TotalStorage Proven, Ultramedia, VideoCharger, Virtualization Engine, Visualization Data Explorer, Workload Partitions Manager, X-Architecture, z/Architecture, z/9.

A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml.The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org.UNIX is a registered trademark of The Open Group in the United States, other countries or both. Linux is a trademark of Linus Torvalds in the United States, other countries or both.Microsoft, Windows, Windows NT and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both.Intel, Itanium, Pentium are registered trademarks and Xeon is a trademark of Intel Corporation or its subsidiaries in the United States, other countries or both.AMD Opteron is a trademark of Advanced Micro Devices, Inc.Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries or both. TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC).SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC).NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both.AltiVec is a trademark of Freescale Semiconductor, Inc.Cell Broadband Engine is a trademark of Sony Computer Entertainment Inc.InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association. Other company, product and service names may be trademarks or service marks of others. Revised January 15, 2008

Special Notices (Cont.)

Page 98: CMG Processor Virtualization

© 2009 IBM Corporation98

TrademarksThe following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.

The following are trademarks or registered trademarks of other companies.

* All other products may be trademarks or registered trademarks of their respective companies.

Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.

For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA, WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, BladeCenter®

Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market.

Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.