© 2012 ibm corporation zcp3000 sysplex demo & lab gretchen frye ibm ats system z masters series...
Post on 04-Jan-2016
225 Views
Preview:
TRANSCRIPT
© 2012 IBM Corporation
zCP3000 Sysplex Demo & Lab
Gretchen Frye
IBM ATS System z Masters Series – Sept 10-13, 2012IBM ATS System z Masters Series – Sept 10-13, 2012
© 2012 IBM Corporation
TrademarksThe following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.
The following are trademarks or registered trademarks of other companies.
* All other products may be trademarks or registered trademarks of their respective companies.
Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.
Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market.
Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.
For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:
*BladeCenter®, DB2®, e business(logo)®, DataPower®, ESCON, eServer, FICON, IBM®, IBM (logo)®, MVS, OS/390®, POWER6®, POWER6+, POWER7®, Power Architecture®, PowerVM®, S/390®, Sysplex Timer®, System p®, System p5, System x®, System z®, System z9®, System z10®, Tivoli®, WebSphere®, X-Architecture®, zEnterprise®, z9®, z10, z/Architecture®, z/OS®, z/VM®, z/VSE®, zSeries®
© 2012 IBM Corporation
Looking at Sysplex using zCP3000
• Not covering sysplex concepts
• Not covering zCP3000 methodology
Focus on• Newer topics
• Things people seem to get stuck on
• Things that could make a big difference
© 2012 IBM Corporation
Model data should include …
All systems participating in all sysplexes.
• Combine production and test/dev traffic on Infiniband
• Eligibility for Sysplex Aggregation Pricing
• Sysplex level graphs (the blue logical view) enabled only when all sysplex members are present.
• Better chance of understanding “standalone” CFs
© 2012 IBM Corporation
Doing a Capacity Planning Study
• Any current performance concerns?
• Which CFs matter, and which don’t?
• Should the study include overview information?
• Are any partitions currently sharing channel paths?
• Are *all* of the mainframes (z/OS and CF) co-located? Is the distance changing?
• If there is a standalone CF, what type of machine is it on, what type of engines is the CF using and how many, and is it sharing?
© 2012 IBM Corporation
Sysplex Graph Recommendations
•CF Link Summary
•Service Time for Synchronous Structures
• Request Rate by System over Time
• Request Rate by Request Type and System
•CF Link Summary
•Service Time for Synchronous Structures
• Request Rate by System over Time
• Request Rate by Request Type and System
• Lock Contention % of Requests
• Synchronous Intensity
• Structures with the Highest Queue Time
• Busiest Structures with Queue Time
Probably in every study• zCP3000 main view : Topology
• Sysplex Logical window : Sysplex Topology
• For each CF : CF Summary, Health Check Analysis
Look for areas of interest• Sysplex Aggregation
• CF Structures Table
• CF Link Topology
Drill down in areas of interest• CF Logical Utilization over Time
• Advanced CF (shared ICF)
•Subchannel Contention
• Synchronous Service Time (Link)
PA Mode
PA and CP Mode
© 2012 IBM Corporation
Physical View vs Sysplex View
CF23 and its connections to PLEXIO sysplex members are highlighted at left.CF23 and its connections to PLEXIO sysplex members are highlighted at left.
View->PhysicalView->Physical View->Sysplex LogicalView->Sysplex Logical
© 2012 IBM Corporation
Sysplex Aggregation Pricing
Pay for software as if it was executing on a single large CPC.
© 2012 IBM Corporation
Sysplex Aggregation Pricing
Not currently eligible, but what if COP2 moved to CPCE7C5?
© 2012 IBM Corporation
Incomplete Configuration Information
• Red Triangles mean something is not fully understood.
• All InfiniBand CFs will come up this way the first time.
• Standalone CFs will often need further definition.
© 2012 IBM Corporation
The Mysterious CF25
This is all we know about CF25:
– CFTYPE="2097-E12"– CFNCPS="01" – SR="59644"
Plus some creative guessing.
If the last 4 digits of the SR match another CEC, this might be the empty ICF partition there.
© 2012 IBM Corporation
zCP3000 CF Analysis
CF Window Summary:
CF InformationCF Information
Link InformationLink Information
Structure InformationStructure Information
Error Messages Summary
© 2012 IBM Corporation
Effective number of engines
CP3KEXTR Calculated as: CP3KEXTR Calculated as:
(CF Busy + CF wait)
Interval time
When EFFCP < Defined CPsWhen EFFCP < Defined CPs
Shared EnginesShared Engines
Something else is sharing that ICFOr is it a GCP?
© 2012 IBM Corporation
Dynamic ICF Expansion Option
Allowed a CF partition to have *both*
shared and dedicated ICF engines.
Unsupported on z196 and above
The Lab Testcase Used
• 1 dedicated and one share ICF in production, one shared ICF in non production on z10
• 2 shared ICFs in for production, one shared ICF for non-production on z196
© 2012 IBM Corporation
The effect of sharing ICF engines
z10 + ICB4 + Dyn_ICF_Expansion z196 + IFB12x + shared engines
© 2012 IBM Corporation
CF Health Check
•Lock Contention
•False Lock Contention
CF Utilization < SDP
Effective Engines >= 1
Is Current Technology
Subchannel Link% ok
Subchannel Sync->Async
Subchannels online
© 2012 IBM Corporation
CF Health Check - CPU
CF Utilization
should always be < SDP
Effective CPs
>1 engine, and the farther from # defined CPs, the worse it is.
Hardware Technology
Current generation Z?
* Also, System Health Check Sysplex rule 2
CF21
© 2012 IBM Corporation
CF Health Check - Structures
Lock Contention
Lock Requests that had to wait
Fix in the Application.
False Lock Contention
2 locks hash to the same index
Increase the Structure size.
© 2012 IBM Corporation
CF Health Check - Subchannels
Subchannel Utilization should be very low (<10%)
Sync -> Async Conversion to avoid path busy. Normally very low, this is an indicator that partitions may be sharing channel paths.
Subchannels Online
unusual; found when there is a sustained pattern of path busy.
© 2012 IBM Corporation
CF Structures table
• Use to identify the busiest structures.
• Use to identify the key CF links for the busiest structures.
In your customer deliverable
• focus on critical applications
• focus on identified problems
• otherwise, use this to identify important structures and links.
CF21
© 2012 IBM Corporation
Service Time
• Synchronous is more important than asynchronous
• Service Time graphs available from the CF window, the Link window, and the Structure detail window.
• CF window averages service time for all z/OS members.
• Use this more as a guide to analysis.
© 2012 IBM Corporation
Structure Detail
• Request Rate and Service Time
• Synchronous and Asynchronous
• By sysplex member system
• Average, max rate, and for the study interval.
• Graphs:
•Synchronous Service Time
•Asynchronous Service Time
© 2012 IBM Corporation
Coupling Links
• An important part of Sysplex performance.
• Often, there is incomplete configuration information:– Link types being used – distance from the CF to the z/OS image– Whether any channel paths are being shared– CF model and relative speed
• Estimated Service Time is both calculated and calibrated to the current service time.
• Heuristic process on z/OS decides how to launch a request (synchronously or asynchronously)
© 2012 IBM Corporation
Coupling Links
Service Time is dependent on – Type of Request– Amount of data being carried*– How request will be executed (sync or async)– Path busy or anticipated path busy (queued request)– Link latency– Link speed– Link distance– CF speed and responsiveness– z/OS speed
Sysplex Overhead is dependent on– How request will be executed (sync or async)– Service Time, for synchronous requests
* (not measured)
© 2012 IBM Corporation
12x InfiniBand
12x InfiniBand
HCA2-O*
HCA2-C I/O Drawer or I/0 Cage
ISC-3
ISC-3
ISC-3
ISC-3
z196, z114, z10 EC, z10 BC
IFB-MP
z196/z114
ISC-3
z196, z114, z10 EC, z10 BC, z9 EC, z9 BC
HCA2-O LR*
Up to 150 meters
Up to 150 meters
Up to 10/100 km
.... ....
HCA2-O**
.... ....
1x InfiniBand
Up to 10/100KM
z9 EC and z9 BC
Fanouts
HCA3-O HCA3-O LR
.... ....
12x InfiniBand
Up to 150 meters
1x InfiniBand
Up to 10/100KM
** Not supported on the z114 ** Carry forward only** Not supported on the z114 ** Carry forward only
zEC12/z196/z114 Coupling Links• Fanout, not I/O slot, used for InfiniBand
• ICB-4 – No longer supported
• ETR – No longer supported
• All coupling links support STP
• Sysplex Coexistence – z10 EC and BC; z9 EC and BC only with z196,z114
© 2012 IBM Corporation
CF Coupling – Relative Link Speed in zCP3000
* ISC3 is supported as carryover only on zEC-12;
* Relative speed does not include latency or any other component of service time other than data transfer time.
zCP3000 Name
Latency microseconds
Relative Speed* Supported on
ISC3 ISC3* 12 (base) z800, z900, z890, z990, z9 , z10 , z114, z196, zEC-12*
IFB-1x IFB 1X 10 2 z9 , z10 , z114, z196, zEC12
IFB-12x IFB 12X – z9 8 3 z9
IFB-12x IFB 12X 8 5 z10 , z114, z196, zEC12
ICB4 ICB4 4 8 z890, z990, z9 , z10
IFB3-12xIFB3 12X – z114 3.5 13 z114
IC IC-z10 BC 1 16 z10-BC
IC IC-2084 1 18 z990
IC IC-2086 1 18 z890
IC IC-z9 BC 1 20 z9-BC
IFB3-12x IFB3 12X 3.5 25 z196, zEC12
IC IC-z9 EC 1 25 z9-EC
IC IC-z114 1 33 z114
IC IC-z10 EC 1 38 z10-EC
IC IC-z196 1 45 z196
IC IC-zEC 12 1 47 zEC-12
© 2012 IBM Corporation
Coupling Links
To model a link type change:(estimate new service time)
• Must be in Capacity Planning Mode
• One CF-SYS set of links at a time
• If CEC model change also, do that first.
• Generate the Before/After link change report *before* applying.
© 2012 IBM Corporation
Changing the Link Type
• Traffic for each structure will be re-calculated.
• A change in the service time can also mean a change in how the request is issued (sync vs async).
• In this example:– IFB-1X is faster– Because it’s faster, 7k requests will
be run as SYNC instead of ASYNC
© 2012 IBM Corporation
CF Link Summary
• Lock structure was already mostly synchronous, and the service time will go from 48.12 to 40.05.
• DB2 cache structures will go from mostly asynchronous to mostly synchronous, so service time will go from 93.73 to 24.89 microseconds.
© 2012 IBM Corporation
CF Link Consolidation
Prior to InfiniBand• One link = one chpid = one feature
• A link could be shared because the chpid could be shared, between members of the same sysplex on the same mainframe, going to the same CF
With InfiniBand• The physical link can carry multiple chpids• Chpids can be from the same sysplex or a different one• Each chpid can still be shared by partitions on the same mainframe
going to the same destination CF
zCP3000 Terminology• Link = channel path (chpid)
• Adapter = the physical path
IFB-1x : 8-16 chpids on 4 ports on HCA2-O LR
IFB-12x : 8-16 chpids across 2 ports on HCA2-O
IFB3 : max 4 chpids per port(2) on HCA3-O
© 2012 IBM Corporation
CF Link Consolidation
• Minimum two HCA2-O or HCA3-O adapters
• Meaning that each machine will have 16-32 channels paths
• IFB3 is a protocol, not an adapter, requiring– HCA3-O adapter types on both ends– Maximum 4 chpids per port
• Typically multiple physical ICB4 or ISC3 links/adapters/chpids
• Often chpids are shared between 2-3 partitions
• Meaning you really have (chpids/partitions) number of ICB4 links.
Consolidate this
• One for one replacement of non-IFB chpids with IFB chpids
• Many to one replacement of non-IFB adapters with IFB adapters
© 2012 IBM Corporation
Incomplete Configuration Information
Red Triangles mean something is not fully understood.
• Currently, all InfiniBand chpids come in with channel type “CIB”.
• Correct link definitions before proceeding.
• With CFCC 18, RMF will be able to differentiate various IFB link types and Detect if CIB link running "degraded“
* Not yet supported in zCP3000 *
© 2012 IBM Corporation
CF Link Consolidation – Shared CHPID
•2 or more partitions on the same CEC
• talking to the same CF on another CEC
• same sysplex
• using the same chpid
• same type (ISC3, ICB4, IFB)
CHPID PATH=(CSS(0),F4),SHARED, PARTITION=((DEVJ,DEV1,DEV2),(=)), PCHID=101,TYPE=CFP
© 2012 IBM Corporation
Defining Shared Channel Paths
• CP3000 does not know if a link is shared.
• Shared Links table lists the partitions that could possibly share these 6 chpids.
• CF linktype must be fully defined and must match exactly.
You have to do this yourself
© 2012 IBM Corporation
Defining Shared Adapters
• 2 or more partitions on the same *pair* of CECs
• talking to any CF on either CEC
• same sysplex, or not
• using different chpids
• which run through the same IFB pipeline.
© 2012 IBM Corporation
Defining Shared Adapters
• InfiniBand only • Shared Links table lists the
sysplex members and CFs on these 2 CECs that could run on the same physical InfiniBand feature.
• *NO* performance implications.
• P2DF could be *both* logical share and physical share.
© 2012 IBM Corporation
Defining Shared Adapters
• 54 chpids running on 54 ISC3 links…
Replace with
• 54 chpids running on 7 IFB adapters.
7
© 2012 IBM Corporation
CHPID Sharing
The P1DF system on CEC7B92 shares 6 chpids to the CF05 coupling facility on CEC7BA2 with 0 other partitions.
54 chpids share 14 ports on 7 InfiniBand HCA2-O adapters.
Adapter Sharing
Defining Shared Adapters
CF Link Topology Report PLEX01:CF05:P1DF
© 2012 IBM Corporation
Index to recommended Graphs
• zCP3000 main view– INV1012 Topology (7)
• zCP3000 sysplex view– PLEX1007 Sysplex Topology (7)– PLEX1005 Sysplex Aggregation by Sysid (8,9)
• CF Window– CF1014 CF Summary (12)– CF1000 Health Check Analysis (16-19)– CF1001 Logical Utilization over Time– CF1008 Subchannel Contention– CF1020 Advanced CF Utilization– CF1010 Structures Table (20)– CF1012 Service Time for Synchronous Structures (15,21)– CF1009 Request Rate by System over Time– CF1004 Request Rate by Request Type and System– CF1015 Structures with the Highest Queue Time– CF1016 Busiest Structures with Queue Time
• Structure Window
• STR1015 Lock Contention % of Requests
• STR1017 Synchronous Intensity
• CF Link Window
• CFL001 CF Link Summary (29,34,36)
•CFL005 CF Link Topology
• CFL012 Synchronous Service Time
* See it on Slide (n)
© 2012 IBM Corporation
zCP3000 Online Education Other Online Courses available at:
IBM - http://w3.ibm.com/support/americas/wsc/cpsproducts.htmlBPs - http://partners.boulder.ibm.com/src/atsmastr.nsf/WebIndex/PRS1762
Contact: cpstools@us.ibm.com
© 2012 IBM Corporation
top related