emc modeling tools data collection guide v1.17

EMC Global Services Technology Implementation Services

EMC Modeling Tools Data Collection Guide v1.17

Date: November 24, 2013

Abstract

This document defines the data collection methodology for the EMC modeling tools.

The target audience is composed of EMC GS/TS Technology Consultants and TS Solutions Architects.

EMC Modeling Tools Data Collection Guide

EMC GS/TIS P2/68

Technology Implementation Services

Copyright 2007, 2013 EMC Corporation. All rights reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

EMC2, EMC, EMC ControlCenter, AlphaStor, ApplicationXtender, Captiva, Catalog Solution, Celerra, CentraStar, CLARalert, CLARiiON, ClientPak, Connectrix, Co-StandbyServer, Dantz, Direct Matrix Architecture, DiskXtender, DiskXtender 2000, Documentum, EmailXaminer, EmailXtender, EmailXtract, eRoom, FLARE, HighRoad, Input Accel, Navisphere, NetWorker, nLayers, OpenScale, Powerlink, PowerPath, Rainfinity, RepliStor, ResourcePak, Retrospect, Smarts, SnapShotServer, SnapView/IP, SRDF, Symmetrix, TimeFinder, VisualSAN, VSAM-Assist, WebXtender, where information lives, Xtender, and Xtender Solutions are registered trademarks and EMC Developers Program, EMC OnCourse, EMC Proven, EMC Snap, EMC Storage Administrator, Acartus, Access Logix, ArchiveXtender, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, C-Clip, Celerra Replicator, Centera, CLARevent, Codebook Correlation Technology, Common Information Model, CopyCross, CopyPoint, DatabaseXtender, Direct Matrix, EDM, E-Lab, Enginuity, FarPoint, Global File Virtualization, Graphic Visualization, InfoMover, Infoscape, Invista, Max Retriever, MediaStor, MirrorView, NetWin, OnAlert, PowerSnap, RepliCare, SafeLine, SAN Advisor, SAN Copy, SAN Manager, SDMS, SnapImage, SnapSure, SnapView, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix DMX, UltraPoint, UltraScale, Viewlets, and VisualSRM are trademarks of EMC Corporation.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com

All other trademarks used herein are the property of their respective owners.


EMC GS/TIS P3/68


TABLE OF CONTENTS

Document Control ................................................................................................................................................ 7

Contributors ...................................................................................................................................................................... 7

Build Status ...................................................................................................................................................................... 7

1 Introduction to Data Collection ..................................................................................................................... 9

1.1 Recommended 3 Day Minimum at 10 Minute Intervals ............................................................................................ 9

1.2 Minimum Data Collection Interval is 1 Minute......................................................................................................... 9

1.3 Data Collection Interval Must Be the Same............................................................................................................. 9

1.4 Standardized Statistics ......................................................................................................................................... 9

1.4.1 Required Fields ........................................................................................................................................ 9

1.4.2 Optional Fields ........................................................................................................................................ 9

1.5 Supported Array and Host Statistics ..................................................................................................................... 10

1.6 General Assumptions ........................................................................................................................................... 10

1.7 Sample Data ........................................................................................................................................................ 11

1.7.1 EMC Internal Users .................................................................................................................................. 11

1.7.2 External Users......................................................................................................................................... 11

2 AIX iostat .................................................................................................................................................... 12

2.1 Data Collection Considerations ............................................................................................................................ 12

2.2 Data Collection Procedure .................................................................................................................................... 12

2.3 Command Syntax ................................................................................................................................................. 12

2.4 Record Layout ...................................................................................................................................................... 13

2.5 Sample Output .................................................................................................................................................... 13

3 BCSD Export ................................................................................................................................................ 14


3.2 Record Layout ...................................................................................................................................................... 14

3.3 Sample Output .................................................................................................................................................... 14

4 BTP ............................................................................................................................................................. 15

4.1 Collection Considerations .................................................................................................................................... 15

4.1.1 RAID-10 ................................................................................................................................................. 15


EMC GS/TIS P4/68


4.1.2 Time Zone .............................................................................................................................................. 15


4.2.1 Performance Manager ............................................................................................................................ 15

4.2.2 STP ........................................................................................................................................................ 16

5 HP-UX sar .................................................................................................................................................... 17



5.2.1 Command Syntax ................................................................................................................................... 18

5.3 Record Layout ...................................................................................................................................................... 19

5.4 Sample Output ................................................................................................................................................... 20

6 iSeries (System i, IBM i) WRKDSKSTS .......................................................................................................... 21



6.3 Command Syntax ................................................................................................................................................. 21

6.4 Record Layout ..................................................................................................................................................... 22

6.5 Sample Output ................................................................................................................................................... 24

7 Linux iostat ................................................................................................................................................ 25



7.3 Command Syntax ................................................................................................................................................. 25

7.4 Record Layout ..................................................................................................................................................... 26

7.5 Sample Output ................................................................................................................................................... 28

8 Mainframe (SMF/CMF) ................................................................................................................................ 29

8.1 Data Collection Considerations ........................................................................................................................... 29

8.2 Mainframe Assembler Kit .................................................................................................................................... 30

8.2.1 Additional notes .................................................................................................................................... 30

8.3 Onsite Processing ................................................................................................................................................ 31

8.4 Offsite Processing ................................................................................................................................................ 31

8.5 Record Layout ...................................................................................................................................................... 33

8.6 Sample Output .................................................................................................................................................... 35


EMC GS/TIS P5/68


8.7 Accessing EMCs E01 LPAR .................................................................................................................................. 36

9 MirrorView/A LOR (locality of reference)..................................................................................................... 39


9.2 Acquiring the MirrorView/A LOR Tool ................................................................................................................... 39

9.3 Data Collection Procedure ................................................................................................................................... 39

9.4 Command Syntax ................................................................................................................................................ 39

9.5 Sample Output ................................................................................................................................................... 40

10 Navisphere Analyzer (NAR) .......................................................................................................................... 41

10.1 Data Collection Considerations for metaLUNs and Thin LUNs ................................................................................. 41

10.1.1 RAW files ............................................................................................................................................... 41

10.1.2 CSV files ................................................................................................................................................ 41


10.2.2 Sample batch file to convert all NAR files in a directory to CSV files ......................................................... 42

10.3 Command Syntax ................................................................................................................................................ 42

10.4 Record Layout ......................................................................................................................................................44

10.5 Sample Output .................................................................................................................................................... 45

11 Solaris iostat ............................................................................................................................................. 46



11.3 Command Syntax ................................................................................................................................................ 46

11.4 Record Layout ...................................................................................................................................................... 47

11.5 Sample Output .................................................................................................................................................... 47

12 VMware esxtop/resxtop ............................................................................................................................. 48



12.3 Command Syntax ................................................................................................................................................ 49

12.4 Record Layout ..................................................................................................................................................... 49

12.5 Sample Output ....................................................................................................................................................50

13 VPLEX Stats ................................................................................................................................................. 51



EMC GS/TIS P6/68



13.3 Command Syntax ................................................................................................................................................. 51

13.4 Record Layout ...................................................................................................................................................... 52

13.5 Sample Output .................................................................................................................................................... 53

14 Windows Perfmon ...................................................................................................................................... 54



14.3 Conversion from BLG to CSV ................................................................................................................................ 62

14.4 Record Layout ..................................................................................................................................................... 62

14.5 Sample Filtered Output ....................................................................................................................................... 64

Appendix A TroubleShooting and Support ...................................................................................................... 65

Appendix B Feature Enhancements and Feedback ........................................................................................... 66

B.1 Feature Enhancements .............................................................................................................................................. 66

B.2 Feedback .................................................................................................................................................................. 68


EMC GS/TIS P7/68


DOCUMENT CONTROL

Contributors

Role Name

Author(s) Mike Lawrence

Build Status

Version Date of issue Author Reason

1.10 10-Mar-2011 Mike Lawrence 1. Rewrite data collection procedure for VMware esxtop.

2. FLARE 30 requires a change to the collection procedure for NAR CSV.

1.11 13-Dec-2011 Mike Lawrence 1. Clarify how to use metaLUNs when using NAR files and include/exclude lists.

2. Removed NaviCLI commands for pre-FLARE 24.

3. Add VPLEX stats support 4. As the SVC no longer exists, removed

the section on the SVC.

1.12 30-May-2012 Mike Lawrence Correct some typos.

1.13 25-Nov-2012 Mike Lawrence 1. Change NAR section to include raw. 2. Change - archivedump to-

archivedump (removed the space). 3. For all data imported into the same

application, the data collection interval must be the same.

4. Change WRKDSKSTS total read and write IOs to IOPS.

1.14 29-Jan-2013 Mike Lawrence Dropped CLARiiON/Unified NAR CSV and Simulated (Workload Generator) performance data.

1.15 29-May-2013 Mike Lawrence 1. Updated Windows Perfmon data collection for Windows 2008 and above.

2. Added Linux iostat considerations for partitioned devices and emcpower devices.

3. Removed references to the now retired ET tool.

4. Added T option as default parameter for AIX iostat as newer versions of the OS support it.

5. Added the collection consideration, the read/write ratio is more of a "best guess" based on all of the read/write activity for all of the volumes in any given interval. In other words, it is not accurate, for HP-UX sar.

6. Added CLARiiON/Unified NAR CSV as it is temporarily back in BCSD.

7. Added to HP-UX sar Data Collection Considerations.

8. Added to Minimum Data Collection


EMC GS/TIS P8/68


Interval is 1 Minute.

1.16 03-Jul-2013 Mike Lawrence 1. Correct Windows Perfmon relog syntax. 2. Added consideration for AIX iostat

partitioned records.

1.17 14-Nov-2013 Mike Lawrence Removed comments about iSeries RAID-1 volumes as the parser now correctly handles them.


EMC GS/TIS P9/68


1 INTRODUCTION TO DATA COLLECTION EMCs remote replication modeling tools require host or array based statistical data to determine data change activity. This enables modeling based on real world data rather than a best guess based on highly averaged data.

1.1 Recommended 3 Day Minimum at 10 Minute Intervals

The minimum data capture interval should be 3 days, while a week is quite common as it will include the various daily peaks and valleys. The actual times for the peaks should be the responsibility of the customer as they know when their peak processing occurs. Ideally, the data capture will include the peak workload period such as month-end or year-end processing. By capturing at least 24 hours, a distinct online and batch processing profile is observed. This is important because the average IO size of write I/Os tends to be smaller for online workloads and larger for batch, which has a direct impact on bandwidth requirements.

Data capture should be in 10 minute intervals or less, but no less than 1 minute. If longer intervals are used you may be impacted by the flattening out of the true peaks due to longer averages, thus skewing the results. Even at 10 minute intervals your true peaks could be twice as much as the average peak recorded in the interval. This can cause a significantly under-configured solution. If a collection interval of less than 10 minutes is desired, note the tools require a minimum of 1 minute.

1.2 Minimum Data Collection Interval is 1 Minute

The minimum data collection interval is 1 minute. Using a data collection interval of less than 1 minute will result in unpredictable results such as corrupted data, invalid error messages, and invalid modeling results. This is due to the fact that the time rounding routine rounds to the nearest data collection interval in minutes.

1.3 Data Collection Interval Must Be the Same

For all data imported into the same application, the data collection interval must be the same.

1.4 Standardized Statistics

Performance data is imported into the tool where all statistics are normalized into a single format.

1.4.1 Required Fields

Field Description

Timestamp Date and Time of the performance data (converted to GMT based on user specified offset). The timestamp is rounded to the nearest collection interval.

Read I/Os Total read I/Os

Read I/O Size Average read I/O size (bytes)

Write I/Os Total write I/Os

Sequential WriteIOs Total sequential write I/Os (If not contained in the data, set to zero)

Write IO Size Average write I/O size (bytes)

Volume ID LUN, device name, volume ID or volser

1.4.2 Optional Fields

These fields are used exclusively for BCSDs VPLEX modeling.

Field Description

RRH I/Os Random read hit IO type


EMC GS/TIS P10/68


Field Description

RRM I/Os Random read miss IO type

SR I/Os Sequential read IO type

RW I/Os Random write IO type

SW I/Os Sequential write IO type

1.5 Supported Array and Host Statistics

Array Description

Symmetrix EMC Symmetrix

CLARiiON EMC CLARiiON

Statistics Description

AIX iostat IBM AIX iostat command output

Symmetrix BTP EMC Symmetrix data in binary format

HP-UX sar HP-UX sar command output

iSeries WRKDSKSTATS IBM iSeries WRKDSKSTATS

Linux iostat Linux iostat command output

Mainframe Assembler Kit v2 IBM mainframe SMF data processed by EMCs Mainframe Assembler Kit v2 and output in CSV format

MirrorView/A LOR Tool EMCs MirrorView/A LOR tool output in CSV format

CLARiiON/Unified Block NAR CSV CLARiiON and Celerra Navisphere Analyzer data in CSV format

CLARiiON/Unified Block NAR (raw) CLARiiON and Celerra Navisphere Analyzer data in raw format

Solaris iostat Sun Solaris iostat command output

Windows Perfmon CSV Microsoft Windows Perfmon data in CSV format (raw BLG data is not supported)

VMware esxtop VMware esxtop command output (BCSD only)

VPLEX stats VPLEX statistics output (BCSD only)

1.6 General Assumptions

All headings are in English, so non-English headings need to be translated to English.

Missing data such as timestamps and collection intervals will be supplied by the user during importing.

For BCSD, after determining the IO rates and/or counts:

o All records with zero I/O's for reads, writes and sequential writes will be skipped and not imported.

o All reads and writes with more than 250,000 IOPS will be flagged and excluded from modeling.

o Any write IO size larger than 20MB will be flagged and excluded from modeling.


EMC GS/TIS P11/68


1.7 Sample Data

1.7.1 EMC Internal Users

Sample data can be found in the EMC GS Tools eRoom in the GS Tools Common Components folder at http://ctseroom02.corp.emc.com/eRoom/tsfuncprac/TSTools/0_47cc6. Internal EMC users who do not have access may request it by emailing [email protected]. but all other users must request copies of the data.

1.7.2 External Users

You may request copies of the sample data by emailing [email protected].


EMC GS/TIS P12/68


2 AIX IOSTAT

2.1 Data Collection Considerations

AIX iostat formats may vary from version to version with some versions providing timestamps while others do not. If timestamps are not present then you will need to know when the data was collected and what collection interval was used and enter it on import.

Partitioned records are redundant data and will be ignored.

sd* device driver names (as an example, sddj and then sddj1) may possibly be multiple entries for the same basic host device. In the example previously, sddj will be totally encompassing for all subcomponent partitions (i.e. sddj will show total IOPs for sddj1 and sddj2 and any other partitions). Because of this, you will need an include list to strip out all numbers after the primary device driver name.

The command can be scheduled in cron or other scheduling products for the desired capture period.

2.2 Data Collection Procedure

Follow these steps to collect iostat data:

1. Identify the servers and the specific LUNS (disks) that will be replicated.

2. Run iostat for a short duration and verify the output matches the required format (see Record Layout and Sample Output below).

3. On each host, run iostat from the command line or optionally create a cron entry.

4. Collect the iostat output file(s).

2.3 Command Syntax

The command syntax should be similar to the following and be issued on each host:

iostat d T interval count >> hostname.iostat.date.txt

Options Description

-d Displays only the device utilization report

-T Adds a column containing the time, but no date (AIX 5.2 and above)

interval Collects data every specified seconds

Suggested setting is 600 (10 minutes)

count Collects statistics for specified intervals

Suggested setting is 432 (3 days of 10 minute intervals)

CRON Syntax:

* * * * * iostat d T interval count >> hostname.iostat.date.txt - - - - - | | | | | | | | | + day of week (1 - 7) (Monday = 1) (Sunday = 0 or 7) | | | +------- month (1 - 12) | | +--------- day of month (1 - 31) | +----------- hour (0 - 23) +------------- min (0 - 59)

Sample CRON Entry: 0 0 * * 5-7 iostat d T 600 144 >> zeus.iostat.04142009.txt


EMC GS/TIS P13/68


This entry schedules iostat to start Friday at 00:00 and end Sunday at 00:00. The iostat command pulls statistics every 600 seconds (10 minute) for 24 hours creating 144 intervals. The UNIX redirect >> appends the output to the same file.

2.4 Record Layout

Heading Description

Disks: Disk name

% tm_act Percentage of time device was busy servicing a request

Kbps Amount of data transferred (both reads and writes) to the drive in KB per second

tps Total I/Os per second

Kb_read Total amount of data read from the drive in KB

Kb_wrtn Total amount of data written to the drive in KB

time Time the statistics were written (5.2+)

2.5 Sample Output

Without Time

System configuration: lcpu=32 drives=976 paths=2 vdisks=0 Disks: % tm_act Kbps tps Kb_read Kb_wrtn hdisk0 1.1 27.3 2.2 0 16430 hdisk1 1.1 27.3 2.2 0 16430 hdisk263 0.8 30.0 3.5 17632 408 hdisk259 0.9 29.9 3.4 17704 288 hdisk256 0.8 30.8 3.5 17504 1027 hdisk261 0.8 29.6 3.4 17008 752 Disks: % tm_act Kbps tps Kb_read Kb_wrtn hdisk0 1.3 24.2 2.4 8 14532 hdisk1 1.2 24.2 2.4 0 14532 hdisk263 0.5 15.2 1.9 9040 80 hdisk259 0.6 18.1 2.2 10832 48 hdisk256 0.7 17.2 2.1 9912 400 hdisk261 0.7 17.7 2.1 10184 424

With Time

System configuration: lcpu=4 drives=161 paths=233 vdisks=0 Disks: % tm_act Kbps tps Kb_read Kb_wrtn time hdisk0 0.2 4.0 0.5 0 2408 09:50:39 hdisk1 0.0 0.0 0.0 0 0 09:50:39 hdisk5 0.0 0.0 0.0 0 0 09:50:39 hdisk3 0.0 0.0 0.0 0 0 09:50:39 hdisk0 0.2 4.0 0.5 0 2392 10:00:39 hdisk1 0.0 0.0 0.0 0 0 10:00:39 hdisk5 0.0 0.0 0.0 0 0 10:00:39 hdisk3 0.0 0.0 0.0 0 0 10:00:39


EMC GS/TIS P14/68


3 BCSD EXPORT


Data can be exported from BCSD by going to the replication/consistency/volume group and using the export performance data to CSV feature. This will export only the mapped data for that group.

3.2 Record Layout

Field Description

Performance Data Tag Project level user tag used to identify a group of performance data that will be accessed by one or more BCSD projects

Load Tag Performance data level user tag used to identify specific performance data files

File Name Name of the performance data file for these specific records

Timestamp Date and Time of the loaded data after being normalized to GMT based on the user selected offset and rounded to the nearest collection interval

Data Type Type of performance data

Seconds per Interval Calculated time interval in seconds between two statistic records

Volume ID LUN, device name, volume ID or volser

Read IOs Total read IOs

Read IO Size Average read IO size in bytes

Write IOs Total write I/Os

Sequential Write IOs Total sequential write I/Os (If not contained in the data, set to zero)

Write IO Size Average write IO size in bytes

Response Time Host IO response time in ms (If not contained in the data, set to zero)

Random Read Hit Total random read hit IOPS

Random Read Miss Total random read miss IOPS

Sequential Read Total sequential read IOPS

Random Write Total random write IOPS

Sequential Write Total sequential write IOPS

3.3 Sample Output

Performance Data Tag,Load Tag,File Name,Timestamp,Data Type,Seconds per Interval,Volume ID,Read IOs,Read IO Size,Write IOs,Sequential Write IOs,Write IO Size,Response Time,Random Read Hit,Random Read Miss,Sequential Read,Random Write,Sequential Write, Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 18:25:00,BTP,300,027A,124.584716796875,999.4239501953125,2.9900331497192383,0.9966776967048645,5802.66650390625,0.0,0.37541529536247253,0.039867103099823,0.0,0.006644518114626408,0.003322259057313204 Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 19:45:00,BTP,300,027A,64.21404266357422,880.0,0.0,0.0,0.0,0.0,0.1872909665107727,0.02675585448741913,0.0,0.0,0.0 Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 20:05:00,BTP,300,027A,67.60562896728516,944.0,0.0,0.0,0.0,0.0,0.19718310236930847,0.02816900610923767,0.0,0.0,0.0 Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 21:05:00,BTP,300,027A,64.21404266357422,816.0,0.0,0.0,0.0,0.0,0.18394649028778076,0.030100330710411072,0.0,0.0,0.0


EMC GS/TIS P15/68


Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 21:25:00,BTP,300,027A,64.0,1024.0,0.0,0.0,0.0,0.0,0.19333332777023315,0.02000001072883606,0.0,0.0,0.0 Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 21:45:00,BTP,300,027A,64.21404266357422,720.0,0.0,0.0,0.0,0.0,0.20066890120506287,0.013377919793128967,0.0,0.0,0.0 Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 18:05:00,BTP,300,027B,0.0,0.0,64.21404266357422,0.0,512.0,0.0,0.0,0.0,0.0,0.21404682099819183,0.0 Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 18:10:00,BTP,300,027B,0.0,0.0,128.0,0.0,512.0,0.0,0.0,0.0,0.0,0.4266666769981384,0.0 Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 18:15:00,BTP,300,027B,0.0,0.0,64.0,0.0,512.0,0.0,0.0,0.0,0.0,0.2133333384990692,0.0 Test,PeakFiles,T1_20120414_180004_000292600925.btp,2012-04-14 18:20:00,BTP,300,027B,0.0,0.0,128.0,0.0,680.0,0.0,0.0,0.0,0.0,0.4266666769981384,0.0

4 BTP

4.1 Collection Considerations

If your data will include metas, then you will need to specify both the meta-heads and meta-members for any include or exclude list used during modeling.

Access to the Symmetrix BIN file or symapi_db.bin file is needed to accurately identify the meta-members. Use the Symmetrix Meta Member Expansion & Validation tool to expand your list of meta-heads for use as an include or exclude list.

4.1.1 RAID-10

A fix has been implemented in 5671, 5772, 5773 and 5874 for the issue where the RAID-10 volume statistics are written in total to all four meta-members, thus if processed as is would result in a 4x increase in the actual number of IOs. For additional information, see Primus EMC174047.

When importing data containing RAID-10 volumes the EMC remote replication modeling tools scan through all of the imported records (during modeling) and for each RAID-10 group with the same read and write statistics, will divide those read and write statistics by 4. This is similar to the fix provided in SymmMerge.

4.1.2 Time Zone

BCSD does not request a GMT offset and will always reflect GMT with zero offset, unless you specify a different offset at modeling time.

There are several factors that can impact the accuracy of the BTP time-zone information when viewed by any of the EMC tools:

The actual time-zone set on the Symmetrix/DMX service processor. The actual time-zone of the Agent used to collect the BTP data. The actual time-zone of where the data is being viewed.


Follow these steps to collect BTP data:

1. Identify the arrays and the specific LUNS (disks) that will be replicated

2. For each array, use Performance Manager or STP to start BTP data collection

3. Collect the BTP or TTP output file(s)

4. For TTP files, convert them to BTP files using STPNavigator

4.2.1 Performance Manager

To create BTP files, follow the same collection process as defined in ECC. As a recommendation, use the Analyst Method as it creates a single BTP file for any interval and duration specified by the user.


EMC GS/TIS P16/68


4.2.2 STP

Coordinate STP data collection with your Customer Engineer (CE) as they will need to setup a script to run on the Service Processor.


EMC GS/TIS P17/68


5 HP-UX SAR


The read/write ratio is a "best guess" based on all of the read/write activity for all of the volumes in any given interval. In other words, it is not accurate. If you can collect and use data from another source, that will be more accurate.



Follow these steps to collect sar data:

1. Identify the servers and the specific LUNS (disks) that will be replicated.

2. Run sar for a short duration and verify the output matches the required format (see


EMC GS/TIS P18/68


3. Record Layout and Sample Output below).

4. On each host, run sar from the command line or optionally create a cron entry.

5. Collect the sar output file(s).

5.2.1 Command Syntax


sar d b interval count >> hostname.sar.date.txt

-d Displays the device activity

-b Displays buffer I/O statistics





CRON Syntax:

* * * * * sar d b interval count >> hostname.sar.date.txt - - - - - | | | | | | | | | + day of week (1 - 7) (Monday = 1) (Sunday = 0 or 7) | | | +------- month (1 - 12) | | +--------- day of month (1 - 31) | +----------- hour (0 - 23) +------------- min (0 - 59)

Sample CRON Entry: 0 0 * * 5-7 sar d b 600 144 >> zeus.sar.04142009.txt

This entry schedules sar to start Friday at 00:00 and end Sunday at 00:00. The sar command pulls statistics every 600 seconds (10 minute) for 24 hours creating 144 intervals. The UNIX redirect >> appends the output to the same file.


EMC GS/TIS P19/68


5.3 Record Layout

First Header

Field Description

Day of Week Day of week truncated to first three characters

Month Month truncated to first thre characters (MMM)

Day Day (dd)

Time Time (hh:mm:ss)

? Unknown

Year Year (YYYY)

Second Header

Field Description

OS Constant HP-UX

Server Name Name of the server where sar was run

OS Version Version of the OS where sar was run

? Unknown

Hardware Family

Hardware family where sar was run

Date Date (MM/DD/YY)

Statistics

Heading Description

Time (HH:MM:SS). Only on first record in set of statistics.

device Device name

%busy Percentage of time device was busy servicing a request

avgque Average number of I/Os queued

r+w/s Total I/Os per second

blks/s Total blocks/sec in 512 byte blocks

avwait Average I/O queue time (ms)

avserv Average I/O service time (ms). AKA response time.

Buffer statistics

bread/s Total physical reads per second from disk

lread/s Total reads per second from buffer cache

%rcache Buffer cache hit ratio for reads

bwrit/s Total physical writes per second from disk

lwrit/s Total writes per second from buffer cache

%wcache Buffer cache hit ratio for writes

pread/s Total physical reads per second from character device using raw I/O


EMC GS/TIS P20/68


pwrit/s Total physical writes per second from character device using raw I/O

5.4 Sample Output

HP-UX pshqu1 B.11.00 U 9000/800 02/15/06 07:51:01 device %busy avque r+w/s blks/s avwait avserv bread/s lread/s %rcache bwrit/s lwrit/s %wcache pread/s pwrit/s 08:01:01 c1t6d0 16.33 0.52 21 167 4.79 10.12 c2t6d0 9.56 0.52 16 146 4.87 7.56 c8t0d1 0.05 0.50 0 1 4.37 3.87 c8t0d2 7.05 0.50 16 66 3.47 6.59 0 5927 100 14 63 77 169 21 08:11:01 c1t6d0 12.64 0.50 16 134 4.70 10.29 c2t6d0 7.90 0.50 13 115 4.82 7.78 c8t0d1 0.03 0.50 0 0 4.08 4.02 c8t0d2 3.99 0.50 13 59 3.11 4.16


EMC GS/TIS P21/68


6 ISERIES (SYSTEM I, IBM I) WRKDSKSTS


Overflow fields indicated by +++++ must manually be converted to 999.9 in order to successfully import. There is no way to specify the actual value so 999.9 will be lower than the actual value, but it is the largest value you can specify.


Follow these steps to collect WRKDSKSTS data:

1. Identify the servers and the specific disks that will be replicated.

2. FTP the source code to the iSeries (System i, IBM i) that would be the SRDF source host.

3. Compile the CL (Control Language) program.

4. Add the library containing the compiled program to the library list (if not already there)

5. Run the program.

6. If the program is run multiple times, rename or copy the resulting file (QGPL/DSKSFOREMC) before each time it is run.

7. Collect the QGPL/DSKSFOREMC output file(s).

6.3 Command Syntax


SBMJOB CMD(CALL PGM(WRKDSKSTS) PARM('Count' 'Interval')) Options Description



interval Collects data every specified minutes

Suggested setting is 10

WRKDSKSTS program to write output to spool and then to physical file

WRKDSKSTS: PGM PARM(&PARM1 &PARM2) /******************************************************************************/ /* Author: Russ Lewis, EMC Corporation */ /* Date: 3/9/01 */ /* Desc: This program is provided as an example of writing output from the */ /* WRKDSKSTS command to a spool file. The spool file(s) output will */ /* then be copied & combined into physical file QGPL/DSKSFOREMC. */ /* The purpose is to allow capturing of KB per I/O, separated by */ /* reads operations versus write operations, which is available */ /* from the WRKDSKSTS cmd in OS/400. The statistics for WRKDSKSTS */ /* are reset between each iteration. 2 character format parameters */ /* must be passed to this program, 4 bytes for the number of */ /* iterations to run, and 4 bytes for the number of minutes to wait */ /* between each iteration. If blanks or zeroes are passed as */ /* parameter 1, the default is to run for 24 iterations. If blanks */ /* or zeroes are passed for parameter 2, the delay time between each */ /* iteration will be 60 minutes. */ /* */ /* Example - SBMJOB CMD(CALL PGM(WRKDSKSTS) PARM('0012' '0005')) */ /* will run 12 iterations of the WRKDSKSTS command with a 5 minute */ /* delay between each interval. */ /* */


EMC GS/TIS P22/68


/* This program is not part of any EMC product & may be used and/or */ /* modified as desired. */ /* */ /* Modification History: */ /* */ /* 11/01/04 RJL - Changed &PARM1, &ITERATIONS, &COUNT and &SPOOLCOUNT from */ /* 2 to 4 or 5 byte variables to allow > 99 iterations to run. */ /* Also changed file size of QGPL/DSKSFORMEMC to *NOMAX to */ /* allow for more iterations with large configurations. */ /******************************************************************************/ DCL VAR(&PARM1) TYPE(*CHAR) LEN(4) DCL VAR(&PARM2) TYPE(*CHAR) LEN(4) DCL VAR(&ITERATIONS) TYPE(*DEC) LEN(4 0) VALUE(24) DCL VAR(&ITVMINUTES) TYPE(*DEC) LEN(4 0) VALUE(60) DCL VAR(&DELAYSECS) TYPE(*DEC) LEN(6 0) VALUE(0) DCL VAR(&COUNT) TYPE(*DEC) LEN(4 0) VALUE(0) DCL VAR(&SPOOLCOUNT) TYPE(*DEC) LEN(5 0) VALUE(0) IF COND(&PARM1 *GT '0000') + THEN(CHGVAR VAR(&ITERATIONS) VALUE(&PARM1)) IF COND(&PARM2 *GT '0000') + THEN(CHGVAR VAR(&ITVMINUTES) VALUE(&PARM2)) CHGVAR VAR(&DELAYSECS) VALUE(&ITVMINUTES * 60) LOOP1: IF COND(&COUNT *GE &ITERATIONS) + THEN(GOTO CMDLBL(WRITEFILE)) WRKDSKSTS RESET(*YES) /**** The reset creates a spool file that we don't use so delete it. ****/ DLTSPLF FILE(QPWCDSKS) SPLNBR(*LAST) DLYJOB DLY(&DELAYSECS) WRKDSKSTS OUTPUT(*PRINT) CHGVAR VAR(&COUNT) VALUE(&COUNT + 1) GOTO CMDLBL(LOOP1) WRITEFILE: DLTF FILE(QGPL/DSKSFOREMC) MONMSG MSGID(CPF2105) CRTPF FILE(QGPL/DSKSFOREMC) RCDLEN(132) + TEXT('Temporary Physical File for + WRKDSKSTS output') SIZE(*NOMAX) CHGVAR VAR(&SPOOLCOUNT) VALUE(&ITERATIONS * 2) CHGVAR VAR(&COUNT) VALUE(2) LOOP2: IF COND(&COUNT *GT &SPOOLCOUNT) + THEN(GOTO CMDLBL(ENDPGM)) CPYSPLF FILE(QPWCDSKS) TOFILE(QGPL/DSKSFOREMC) + SPLNBR(&COUNT) MBROPT(*ADD) /**** Add 2 to counter each time through to skip over the "extra" spool ****/ /**** file created by the statistics reset during each pass. ****/ CHGVAR VAR(&COUNT) VALUE(&COUNT + 2) GOTO CMDLBL(LOOP2) ENDPGM: ENDPGM

6.4 Record Layout

First Header

Field Description

Licensed program name for the OS

OS version


EMC GS/TIS P23/68


Unknown

Title Constant Work with Disk Status

Date Date (MM/DD/YY)

Time Time (HH:MM:SS)

Statistic pre-header

Field Description

Constant Elapsed time

Interval Elapsed time (HH:MM:SS)

Constant System name

Server Name Server name

Statistics

Heading Description

Unit System assigned number identifying a specific disk

Type The disk type

Size (M)

Disk space (MB)

% Used

Percentage of disk space allocated

I/O Rqs

Total I/Os

Request Size (K)

Average I/O size (KB)

Read Rqs

Total read IOPS

Write Rqs

Total write IOPS

Read (K)

Average Read I/O size (KB)

Write (K)

Average Write I/O size (KB)

% Busy

Percentage of time device was busy servicing a request

ASP Auxiliary Storage Pool ID

Protection Type

Protection Status

Compression


EMC GS/TIS P24/68


6.5 Sample Output

5722SS1 V5R2M0 020719 Work with Disk Status 10/13/04 11:07:28 Page 1 Elapsed time . . . . . . . : 01:00:00 System name . . . . . . . : A005 Size % I/O Request Read Write Read Write % --Protection-- Unit Type (M) Used Rqs Size (K) Rqs Rqs (K) (K) Busy ASP Type Status Compression 1 6717 6442 68.9 22.6 6.9 3.1 19.4 7.6 6.7 3 1 DPY ACTIVE 2 4326 30769 88.8 33.6 8.1 3.0 30.6 9.6 8.0 1 1 DPY ACTIVE 3 4326 30769 91.9 40.9 7.0 2.1 38.7 7.9 7.0 1 1 DPY ACTIVE 4 4326 30769 92.4 31.1 8.1 2.0 29.0 10.0 7.9 1 1 DPY ACTIVE 5 4326 30769 92.2 35.6 7.9 2.6 32.9 9.1 7.8 2 1 DPY ACTIVE 34 6717 6442 69.0 41.3 8.3 4.5 36.8 8.5 8.2 5 1 DPY ACTIVE 35 6717 6442 68.9 38.1 8.0 4.2 33.8 7.3 8.1 5 1 DPY ACTIVE 36 6717 6442 68.9 41.4 6.8 4.4 36.9 7.5 6.8 5 1 DPY ACTIVE 38 4326 30769 92.4 32.1 8.1 2.3 29.8 9.9 7.9 2 1 DPY ACTIVE


EMC GS/TIS P25/68


7 LINUX IOSTAT


Linux iostat formats may vary from version to version with some versions providing additional fields while others do not.

There are generally three types of UNIX OS device driver name types internal type drives (the cciss/* types), the sd* types, and the emcpower* types.

If PowerPath is installed on the host, then only the emcpower* (PowerPath) device driver names have valid iostat perf data and all of the lower level device driver metrics are not reliable and/or not even valid. (This is because of how the Linux/UNIX kernel works with the PowerPath kernel device driver.)

Multiple emcpower devices

For the sd* device driver names, then just like with Solaris you will see (as an example, sddj and then sddj1) possibly multiple entries for the same basic host device. In this example, sddj is totally encompassing for all subcomponent partitions (i.e. sddj shows total IOPs for sddj1 and sddj2 and any other partitions). Because of this, we need to strip out all numbers after the primary device driver name (supported with BCSD v1.6.2 parsers and above) as they are overlaps of the primary and if you dont, then the total sum of metrics like IOPs will be incorrect (doubled/tripled/etc.) from what is really being requested. (In other words, only the primary sd entry will be inclusive and if subsequent partition entries are also included in any totals, then the totals will be wrong.)



Follow these steps to collect iostat data:

1. Identify the oservers and the specific LUNS (disks) that will be replicated.

2. Run iostat for a short duration and verify the output matches the required format (see Record Layout and Sample Output below).

3. On each host, run iostat from the command line or optionally create a cron entry.

4. Collect the iostat output file(s).

7.3 Command Syntax


#iostat d x t interval count >> hostname.iostat.date.txt

Options Description

-d Displays only the device utilization report

-x Displays extended format

-t Displays the time of each report interval






EMC GS/TIS P26/68


CRON Syntax:

* * * * * iostat d x t interval count >> hostname.iostat.date.txt - - - - - | | | | | | | | | + day of week (1 - 7) (Monday = 1) (Sunday = 0 or 7) | | | +------- month (1 - 12) | | +--------- day of month (1 - 31) | +----------- hour (0 - 23) +------------- min (0 - 59)

Sample CRON Entry: 0 0 * * 5-7 iostat d x t 600 144 >> zeus.iostat.04142009.txt

This entry schedules iostat to start Friday at 00:00 and end Sunday at 00:00. The iostat command pulls statistics every 600 seconds (10 minute) for 24 hours creating 144 intervals. The UNIX redirect >> appends the output to the same file.

7.4 Record Layout

Header1 (may not exist)

Field Description

Date Date statistics were started

Time Time statistics were started

Year Year statistics were started

Header2

Field Description

OS Constant Linux

OS Version Version of the OS where iostat was run

Server Name Name of the server, in parenthesis, where iostat was run

Date Date preceded by a tab (MM/DD/YYYY)

CPU Info CPU type and quantity

Time

Field Description

Date Date (MM/DD/YYYY) May not exist

Time Time (HH:MM:SS AM/PM) or (HH:MM:SS)

Extended Statistics (Old format)

Heading Description

Device: Device name

rrqm/s Total read requests merged per second

wrqm/s Total write requests merged per second

r/s Total reads per second


EMC GS/TIS P27/68


w/s Total writes per second

rsec/s Total sectors read per second

wsec/s Total sectors written per second

avgrq-sz Average I/O size (KB)

avgqu-sz Average queue length

await Average I/O queue and service time (ms). AKA response time.

svctm Average I/O service time (ms)

%util Percentage of CPU time I/O requests were issued to the device

Extended Statistics (New format)

Heading Description

Device: Device name

rrqm/s Total read requests merged per second

wrqm/s Total write requests merged per second

r/s Total reads per second

w/s Total writes per second

rsec/s Total sectors read per second

wsec/s Total sectors written per second

rkB/s Total kilobytes read per second

wkB/s Total kilobytes written per second

avgrq-sz Average I/O size (KB)

avgqu-sz Average queue length

await Average I/O queue and service time (ms). AKA response time.

svctm Average I/O service time (ms)

%util Percentage of CPU time I/O requests were issued to the device


EMC GS/TIS P28/68


7.5 Sample Output

Old Format

Mon Aug 4 09:06:09 EDT 2008 Linux 2.6.18-53.1.4.el5 (bio201.agencourt.com) 08/04/2008 Time: 09:06:09 AM Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 2.51 215.32 6.42 5.97 1209.95 1770.33 240.60 0.27 21.57 3.09 3.83 sdb 0.02 6.47 0.05 0.58 3.60 56.39 94.95 0.03 42.96 1.21 0.08 sdc 29.51 191.91 89.37 5.69 5460.28 1585.99 74.12 0.28 13.58 1.52 14.42 dm-0 0.00 0.00 8.88 228.01 1212.62 1824.05 12.82 0.51 2.13 0.16 3.88 dm-1 0.00 0.00 0.12 0.33 0.92 2.66 8.00 0.04 99.84 0.40 0.02 Time: 09:16:09 AM Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 0.00 0.58 0.00 1.11 0.00 13.49 12.16 0.01 4.53 0.49 0.05 sdb 0.00 1.88 0.00 0.50 0.00 19.09 37.93 0.00 0.40 0.28 0.01 sdc 0.00 0.00 0.06 0.01 0.44 0.07 8.22 0.00 4.46 4.46 0.03 dm-0 0.00 0.00 0.00 4.07 0.00 32.59 8.00 0.01 2.83 0.16 0.07 dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

New Format

Mon Aug 4 09:07:09 EDT 2008 Linux 2.6.9-67.ELsmp (bio202.agencourt.com) 08/04/2008 Time: 09:07:09 AM Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util sda 0.10 112.46 2.82 4.57 229.87 162.64 114.94 81.32 53.10 0.31 41.26 3.54 2.62 dm-0 0.00 0.00 2.90 116.91 229.66 161.66 114.83 80.83 3.27 0.19 1.58 0.22 2.62 dm-1 0.00 0.00 0.02 0.12 0.18 0.98 0.09 0.49 8.00 0.04 290.93 0.61 0.01 Time: 09:17:09 AM Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util sda 0.00 1.31 0.00 1.02 0.00 18.60 0.00 9.30 18.30 0.00 0.63 0.43 0.04 dm-0 0.00 0.00 0.00 2.32 0.00 18.60 0.00 9.30 8.00 0.00 0.61 0.19 0.04 dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00


EMC GS/TIS


8 MAINFRAME (SMF/CMF)


Mainframes write SMF and/or CMF data. This data contains binary information and much more data than needed for modeling. A program to extract and analyze this data is available and called the Mainframe Assembler Kit. This program is written in assembler and is dependant on specific SMF record types being recorded on each LPAR.

The use of Mainframe SMF requires the following SMF records:

70 subtype 1, 73 74 subtype 1 and 5 78 subtype 3

The shortest SMF interval recording time the assembler program will accept is five minutes. In a multi-frame, multi-LPAR environment, it may not be possible for all systems to complete the writing of SMF interval data within the confines of a smaller recording interval, so five minutes is the smallest practical interval allowed by the program.

The assembler program, and the EMC remote replication modeling tools, do not support mixed SMF recording intervals. If there are systems that use different SMF recording intervals, the SMF data with like intervals are processed together, creating a mainframe CSV file for each different recording interval. As these CSV files have different collection intervals, the EMC remote replication modeling tools require separate projects for each of these as the data can not be combined into composite views.

The assembler program must have SMF (RMF record 74 subtype 5) data for ALL DEVICES. The cache counters contained in the RMF 74 subtype 5 records are maintained in the control unit and are universal to all systems. Users therefore often only record the RMF 74 subtype five records on one LPAR to avoid recording redundant data. The LPAR that records the RMF 74 subtype 5 records must have all disk devices varied online or cache statistics for the offline devices will not be recorded. This will result in incomplete input data and inaccurate results.

The CMF records written to the SMF dataset are a superset that includes the CMF-specific records and the RMF records used by the assembler program. Users often however, suppress the recording of the RMF records because they use CMF reports in their place and do not want to record data they are not using. The assembler program does not support the native CMF record formats. If the required RMF record types are not being recorded, the user must temporarily enable RMF recording during the data collection period.


EMC GS/TIS


Figure 1. Mainframe SMF and CMF Data Collection Process

Process SMF/CMF Onsite or Offsite?

If possible, run the Mainframe Assembler kit at the customer site to create the CSV file necessary for the EMC remote replication modeling tools. This is usually the quickest and easiest way to process the data and minimizes the amount of data being transferred across the network. Alternatively, the required SMF/CMF records can be sent to EMC corporate via FTP or 3490 tape, processed on the Hopkinton mainframe, thus creating the CSV file for the EMC remote replication modeling tools.

8.2 Mainframe Assembler Kit

The Mainframe Assembler Kit is available in one of several ways:

5. Packaged in the BCSD install zip file as Mainframe Assembler Kit vX.zip.

6. For EMC internal users only, in the EMC GS Tools eRoom in the GS Tools Common Components folder. It is packaged in the file called Mainframe Assembler Kit vX.zip which can be downloaded from http://ctseroom02.corp.emc.com/eRoom/tsfuncprac/TSTools/0_53238. If you do not have access, please email [email protected] to request access.

The kit contains all necessary programs, sample JCL and a Users Guide.

8.2.1 Additional notes

Inputs

Control Unit FICON Channel Paths Data File, user parameters

Are there FICON channels?

Customer Onsite

Approach

EMC Offsite Approach

Get Mainframe Assembler Kit

Install Assembler Kit Onsite

Customize and run RMFCUGEN

Customize and run ET000CSV

Retrieve flat-file from Mainframe

(as CSV file)

TRSMAIN SMF/CMF data

FTP data to EMC

SMFDUMP SMF/CMF data

Dump to Tape and post to EMC

Obtain TSOid for Remote MF

access

Restore FTP or Tape data to E01

Yes

No


EMC GS/TIS


Sorted RMF (and CMF) record types 70 subtype 1, 73, 74 subtype 1, 74 subtype 5, and 78 subtype 3 (74 subtype 1 and 74 subtype 5 are the only record types needed if all channels are ESCON)

The CSV file is sorted by DATE, TIME, and descending WRITES.

Volume Inclusion/Exclusion

If not replicating all volumes you can include or exclude using SYSIN control cards in ETALLCSV or during data import into the EMC remote replication modeling tools. The difference is:

If done in ETALLCSV, the CSV will strip out the volsers you specified and cannot be re-included in the remote replication tool unless ETALLCSV is rerun. The advantage is a smaller output file and quicker processing by the EMC remote replication modeling tools.

If done in the remote replication tool, the import include/exclude filters can be used against the same input file multiple times depending on the date to be modeled. The disadvantage is a larger output file and slower processing by the remote replication tool.

8.3 Onsite Processing

Follow these steps to collect mainframe data:

1. Identify the LPARs and the specific volumes that will be replicated

2. Install the Mainframe Assembler Kit

3. Modify and run the appropriate JCL

4. Collect the output file(s)

Optionally Upload Output to EMC FTP Site

Follow these steps to upload the CSV file(s) to EMCs FTP site:

1. Optionally zip the CSV file to reduce transfer times

2. Use FTP via the ISPF Command Shell

3. Connect to FTP.EMC.COM

4. Logon as user anonymous, password should be your email address

5. Remain in ASCII Transfer Mode

6. cd incoming/customer 7. put CSV-OUTPUT-FILE CSV FILENAME ON FTP SITE

8. Confirm that the CSV file was transmitted and QUIT

8.4 Offsite Processing

Step 1 Extract the Required SMF/CMF Records

The TRSMAIN program does not support tape, move the dataset to disk first!!! Edit and run the below SMFDUMP JCL to dump the Type 70, 73, 74 and 78 records to an output dataset.

//SMFDUMP JOB (EMC),,CLASS=S,MSGCLASS=X,MSGLEVEL=(1,1),

// NOTIFY=&SYSUID

//**************************************************

//* This job will collect SMF - Type 70, 73, 74 and 78 (dataset

//* level statistics) for I/O performance analysis

//**************************************************


EMC GS/TIS


//*

//SMFDUMP EXEC PGM=IFASMFDP

//DUMPIN DD DISP=SHR,DSN=INPUT SMF DATASET

// DD DISP=SHR,DSN=EXTRA INPUT SMF DSN IF NEEDED

//OUTDD1 DD DISP=(,CATLG),DSN=TEMP OUTPUT DATASET,

// UNIT=SYSDA,SPACE=(CYL,(25,5),RLSE),

// VOL=(,,,5),DCB=(RECFM=VBS,LRECL=32760)

//SYSPRINT DD SYSOUT=*

//SYSIN DD *

INDD(DUMPIN,OPTIONS(DUMP))

OUTDD(OUTDD1,TYPE(70,73,74,78))

Step 2 Terse the SMFDUMP output

Documentation on the TRSMAIN program can be found at:

http://techsupport.services.ibm.com/390/trsmain.html

Use the following JCL to Terse the output dataset prior to FTP transmission:

//TRSMAIN JOB (EMC),,CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),

// NOTIFY=&SYSUID,REGION=4M

//************************************

//* TRSMAIN with PACK option *

//*************************************

//TRSMAIN EXEC PGM=TRSMAIN,PARM=PACK

//SYSPRINT DD SYSOUT=A

//INFILE DD DISP=SHR,DSN=TEMP OUTPUT DATASET

//OUTFILE DD DISP=(NEW,CATLG,DELETE),SPACE=(CYL,(50,10),RLSE),

// DSN=TERSED DATASET NAME,

// DCB=(RECFM=FB,LRECL=1024,DSORG=PS)

Step 3 FTP the Tersed File to EMC

Use the following instructions to transmit the tersed file to EMCs FTP site.

1. Use FTP via the ISPF Command Shell. Connect to FTP.EMC.COM

2. Logon as user anonymous, password should be your email address.

3. BIN to change to BINARY Transfer Mode.

4. cd incoming/customer

5. put TERSED DATASET NAME TERSED FILENAME ON FTP SITE

6. Confirm that the tersed file was transmitted and QUIT.


EMC GS/TIS


Step 4 Login to the Hopkinton Mainframe

See Accessing EMCs E01 LPAR for details.

Step 5 Retrieve the Tersed SMFDUMP file

Transmit the data from our FTP site to our analytics LPAR (E01).

1. Email the TS_Tools_Support mailbox for a TSO userid and password.

2. Logon to the E01 LPAR (IP ADDR=10.243.142.41)

3. Ensure that the target dataset name that you are going to use to receive the tersed file from the FTP site has been pre-allocated with the following DCB:

a. LRECL =1024 b. DSORG = PS c. RECFM = FB d. The dataset name should match TERSED INPUT FILE as per below.

4. Use FTP via the ISPF Command Shell. Connect to FTP.EMC.COM

5. Logon as user anonymous, password should be your email address.

6. BIN to change to BINARY Transfer Mode.

7. cd incoming/customer

8. get TERSED FILENAME ON FTP SITE TERSED INPUT FILE (REPLACE)

9. Confirm that the tersed file was received from the FTP site and QUIT.

Step 6 Unterse the SMFDUMP file

Use the following JCL to unpack the tersed file:

//TRSMAIN JOB (EMC),,CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),

// NOTIFY=&SYSUID,REGION=4M

//****************************************

//* TRSMAIN with UNPACK option *

//****************************************

//TRSMAIN EXEC PGM=TRSMAIN,PARM=UNPACK

//SYSPRINT DD SYSOUT=A

//INFILE DD DISP=SHR,DSN=TERSED INPUT FILE

//OUTFILE DD DISP=(NEW,CATLG,DELETE),SPACE=(CYL,(50,10),RLSE),

// DSN=TARGET UNPACKED FILE ON EMC E01

NOTE: The TARGET UNPACKED FILE should have the same attributes as the original TEMP OUTPUT DATASET.

8.5 Record Layout

Field Description SITE Site/Customer Identifier

IDATE Interval Start Date (MM/DD/YYYY)

ITIME Interval Start Time (HH:MM)

ISEC Interval Elapsed Seconds (SSSS)


EMC GS/TIS


VOLSER Volser

DEVNAME Device Type Name

SGNAME SMS Storage Group Name

UNIT CCUU

SSID SSI

OEM Vendor

CUSER# Control Unit Serial #

IREADS All Device Reads (random and sequential)

IWRITES All Device Writes (random and sequential)

ISEQREADS Sequential Device Reads

ISEQWRTS Sequential Device Writes

IWRTBLKL Average Write Block Length (Same as IREADBLKL if ESCON)

IREADBLKL Average Read Block Length (Same as IWRTBLKL if ESCON)

IDEVB True Device Busy All LPARs (PP.PP)

IOPS Device I/O/Sec All LPARs ((NNNN.N)N)

IRESP Device Response All LPARs (NNNN.N)

IQUE Device Avg Q Time All LPARs (NNNN.N)

ISERV Device Svc Time All LPARs (NNNN.N)

ICNN Device Connect All LPARs (NNNN.N)

IDIS Disconnect Time All LPARs (NNNN.N)

IPEND Device Pend Time All LPARs (NNNN.N)

CARHR Cache Read Hit % X 10 (PPP.P)

CARWR Read Percentage X 10 (PPP.P)

CAWHR Cache Write Hit % X 10 (PPP.P)

CASRR Cache Seq Read % X 10 (PPP.P)

CASWR Cache Seq Write % X 10 (PPP.P)

CASRH Cache Seq Read Hit % X 10 (PPP.P)

CASWH Cache Seq Write Hit % X 10 (PPP.P)

CADSR Cache Destage Count


EMC GS/TIS P35/68


8.6 Sample Output

VERIZON STE FICON ,12/25/2006,23:59,0900,E0M7B1,33909 , ,4262,4200,HTC,000000022352, 4065, 26, 20, 0, 24886, 61863, .60, 4.54, 1.9, 0.0, 1.9, 1.1, 0.3, 0.5, 98.0, 99.3,100.0, 0.5, 0.0, 81.0, 0.0, 8 VERIZON STE FICON ,12/25/2006,23:59,0900,ADBPJT,33903 ,SGDB2PTS,4200,4200,HTC,000000022352, 496, 162, 0, 0, 24886, 61863, .31, .73, 4.9, 0.0, 4.9, 1.4, 2.9, 0.6, 50.9, 75.3,100.0, 0.2, 0.0,100.0, 0.0, 272 VERIZON STE FICON ,12/25/2006,23:59,0900,EDI1F1,33903 ,SGDB2PIX,4220,4200,HTC,000000022352, 679, 17, 1, 0, 24886, 61863, .37, .77, 5.4, 0.0, 5.4, 0.7, 4.2, 0.5, 49.1, 97.4,100.0, 0.2, 0.0,100.0, 0.0, 40 VERIZON STE FICON ,12/25/2006,23:59,0900,ENH1FD,33909 ,SGEVNHA ,4260,4200,HTC,000000022352, 98, 4, 2, 0, 24886, 61863, .06, .11, 6.0, 0.0, 6.0, 5.3, 0.5, 0.3, 95.6, 95.7,100.0, 2.2, 0.0,100.0, 0.0, 0


EMC GS/TIS P36/68


8.7 Accessing EMCs E01 LPAR

Before you attempt to use the z/OS TSO LPAR you need to be familiar with TSO, ISPF and JCL. Additionally, you need to email TS_Tools_Support in order to obtain access to a TSOid and Password for temporary use. Keep in mind that you must be behind EMCs Firewall in order to access this LPAR.

The first step in getting access to the LPAR, is obtaining and installing a 3270 emulator. EMC uses TN3270 which can be downloaded from http://mfe01.lss.emc.com. The site will look something like this:

Now click on Get Vista TN3270. This will download the install program to your PC. Once downloaded, install and then start it.


EMC GS/TIS P37/68


This will start a new terminal session:

When prompted, type the host IP name mfe01.lss.emc.com, then click Connect.


EMC GS/TIS P38/68


Now you can login to TSO:

Just type in the supplied TSOid and password to bring up ISPF. As there is only a small pool of TSOids, your assigned TSOid could be shared, so please do not acquire a session if it is already in use.


EMC GS/TIS P39/68


9 MIRRORVIEW/A LOR (LOCALITY OF REFERENCE) MirrorView/A LOR (locality of reference) provides host-based (Windows only) I/O statistics for modeling MirrorView/A replication solutions with increased accuracy. This tool attempts to mimic what MirrorView/A will actually do if this same workload was run on a CLARiiON array with MirrorView/A active. This is done by determining the locality of reference and how that impacts COFW for snapshots and data movement (SnapView and Incremental SAN Copy).


There are two steps that need to be run. The first step creates a trace file and the second step uses that trace file as input and creates an analysis file for import.

Collection will cease when one of three conditions occurs: when a non-zero duration is complete, when the output file reaches a size of max-output-size, or when the user executes locality with the stop operation.

When importing LOR files into the EMC remote replication modeling tools, the Time Zone must be set to GMT.

Warning! This tool can produce many GBs of trace data depending on the number of volumes attached to the server and, the sub-period and duration specified.

9.2 Acquiring the MirrorView/A LOR Tool

The Mainframe Assembler Kit is available in one of two ways:

1. Packaged with the EMC remote replication modeling tools in the tools install subfolder called MV/A LOR Tool.

2. In the EMC GS Tools eRoom in the GS Tools Common Components folder. It is packaged in the file called MirrorView-A LOR Tool.zip which can be downloaded from:

http://ctseroom02.corp.emc.com/eRoom/tsfuncprac/TSTools/0_5499c

If you do not have access, please email [email protected] to request access. If you are a non-EMC user, email the support mailbox above and provide the contact details of your EMC account team after which the kit will be made available to you.

The kit contains the program and a Users Guide.


Follow these steps to collect the LOR tool data:

1. Identify the servers and the specific LUNS (disks) that will be replicated 2. On each host, run locality trace from the command line

3. Once the locality trace command has completed, run locality analyze from the command line

4. Collect the LOR output file(s)

9.4 Command Syntax

Create Trace File


locality trace file max-output-size duration

Options Description

file File name for trace data (should end with the file extension .elt)

max-output- Specifies a number [modifier] where modifier may be k or K (kilobytes), or m or M (megabytes),


EMC GS/TIS P40/68


size and must be at least 1 MB.

duration

The duration is specified as number [modifier] where modifier can be s or S (seconds, default) m or M (minutes), h or H (hours), or d or D (days)

Suggested setting is 3d (3 days)

Analyze Trace File

The analyze must run on same server where .elt file was created. The command syntax should be similar to the following and be issued on each host:

locality analyze file period sub-period chunk-size grain-size output-prefix

Options Description

file File name for trace data (should end with the file extension .elt)

period MirrorView/A update period (must match what is specified in the EMC remote replication modeling tools or you may get erroneous results)

sub-period

Collection interval (1 minute default and must be a divisor of the MirrorView/A update period specified)

Suggested setting is 5 minutes to minimize file size and allow period changes in increments of 5 minutes

chunk-size Should always be 64K (used to determine reserved LUN pool size and max COFW operations/sec)

grain-size FLARE 14/16 should be 16KB and FLARE 19+ should always be 2KB (used to determine network throughput)

output-prefix Prefix applied to CSV output files

Example

A typical use of the locality program:

locality trace q:\logs\JetStress.elt 200 24h

locality analyze q:\logs\JetStress.elt 1h 5m 64k 2k 1hour

The first command begins a trace into q:\logs\JetStress.elt. The trace will run for 24 hours, and will stop collecting data once 24 hours is reached or q:\logs\JetStress.elt reaches 200 MB.

The second command will analyze the data for a 1 hour MirrorView/A update cycle (which would result in a 2 hour RPO), reporting incremental CoFW data every 5 minutes, using the default 64KB SnapCopy chunk size and default 2KB MirrorView/A transfer size and will create the files 1hour.csv and 1hour_log.csv.

9.5 Sample Output

LoR version,0.7 Timestamp,7/20/2007 16:26:05 PM disk,seconds,period,# reads,# writes,bytes read,bytes written,chunk size,modified chunks,overwritten chunks,cofw chunks,chunk size,modified chunks,overwritten chunks,cofw chunks 0,60,1,0,33,0,107008,65536,5,28,0,2048,50,25,0 0,120,0,0,9,0,32256,65536,3,7,2,2048,18,6,12 0,120,1,0,9,0,32256,65536,3,7,2,2048,18,6,12 0,180,1,0,24,0,93696,65536,5,20,2,2048,50,19,12 0,240,0,0,34,0,125440,65536,4,31,2,2048,43,50,12 0,240,1,0,34,0,125440,65536,4,31,2,2048,43,50,12 0,300,1,0,9,0,36864,65536,2,8,1,2048,19,8,1 0,360,0,0,28,0,100864,65536,3,26,0,2048,32,43,0 0,360,1,0,28,0,100864,65536,3,26,0,2048,32,43,0 0,420,1,0,31,0,108032,65536,4,27,2,2048,47,34,12 0,480,0,0,16,0,56320,65536,3,14,2,2048,20,22,12 0,480,1,0,16,0,56320,65536,3,14,2,2048,20,22,12


EMC GS/TIS P41/68


10 NAVISPHERE ANALYZER (NAR)

10.1 Data Collection Considerations for metaLUNs and Thin LUNs

10.1.1 RAW files

With raw NAR file support, conversion to CSV is no longer necessary. Note that loading of FLARE 28 and older raw NAR files may fail due to a bug in FLARE which causes an incorrect earliest polltime or Error parsing MetaLUN Stats. Converting these files to CSV and loading the NAR CSV files into BCSD solves this problem.

10.1.2 CSV files

For FLARE 28 and above and you can specify metaLUNs and thin LUNs in the NAR to CSV conversion allowing you to create an include or exclude list based on meta-heads, otherwise you must specify all of the meta-members within a metaLUN for the meta to be properly included in the modeling.


Step 1 Start Analyzer Performance Logging

As the data is shared between BOTH SPs, you only need to collect and process a single SPs files, but it may be desired to retrieve NAR files from both SPs in case one file is corrupt. For a MirrorView/A analysis, the collection period should be specified as a divisor of the desired cycle time which is half of the desired RPO time. If you expect an RPO of 2 hours, then the data should be collected in 1 hour intervals or less, such as 30 minutes or even 10 minutes. For any other analysis, the time interval should be specified as 10 minutes or less.

Changing the narinterval, which affects both SPs, while logging is active will result in a file with multiple collection intervals. To prevent this, you can first determine if logging is active and what the current collection interval is. If archiving is enabled, then you must stop Analyzer, change the Archive Interval, and then start Analyzer. When you stop analyzer, the current archive data will be saved in a new archive. Use the analyzer STOP command, then the START command to start Analyzer running with the new narinterval.

1. Check if logging is active naviseccli -address -user -password -scope analyzer -status

2. Display the archive interval naviseccli -address -user -password -scope analyzer -get

3. If necessary, change the narinterval (remember to enable Periodic Archiving) and restart logging naviseccli -address -user -password -scope analyzer stop naviseccli -address -user -password -scope analyzer -set -narinterval -periodicarchiving 1 naviseccli -address -user -password -scope analyzer start

Step 2 Copy NAR archive file(s) to your local hard drive

List all available NAR files stored on the array

naviseccli -address -user -password -scope analyzer archive -list

Retrieve a specific NAR file


EMC GS/TIS P42/68


naviseccli -address -user -password -scope analyzer -file -location -overwrite -retry

Retrieve all Files

naviseccli -address -user -password -scope analyzer all -location -overwrite -retry

For raw NAR files, no further processing of the files is required.

Step 3 Convert NAR file(s) to CSV file(s)

FLARE 30 and Above CLI Host LUNs (allows you to specify meta-heads only)

You can use the latest version of Navisphere CLI unless your NAR file is pre-FLARE 26.

For include/exclude lists, meta-heads must be specified and not the meta-members (meta-heads and not meta-members are presented to the host)

naviseccli analyzer archivedump data -out -object hl -format on,pt,rb,rs,rio,wb,ws,wio

FLARE 28 and Above NAR FLARE LUNs, Meta-Members and Thin LUNs

For include/exclude lists, meta-members must be specified and not the meta-heads

naviseccli analyzer archivedump data -out -object l tl -format on,pt,rb,rs,rio,wb,ws,wio

FLARE 28 and Above NAR Meta-Heads and Thin LUNs

For include/exclude lists, meta-heads must be specified and not the meta-members

naviseccli analyzer archivedump data -out -object ml tl -format on,pt,rb,rs,rio,wb,ws,wio

FLARE 24 and 26 NAR

For include/exclude lists, meta-members must be specified and not the meta-heads

naviseccli analyzer archivedump data -out -object l -format on,pt,rb,rs,rio,wb,ws,wio

10.2.2 Sample batch file to convert all NAR files in a directory to CSV files

Replace the naviseccli command with any of the Step 3 Convert NAR file(s) to CSV file(s) commands. The example is using FLARE 30 and Above CLI Host LUNs (allows you to specify meta-heads only)

path "C:\Program Files\EMC\Navisphere CLI";%path% for %%f in (*.nar) do naviseccli analyzer archivedump data -out -object hl -format on,pt,rb,rs,rio,wb,ws,wio

10.3 Command Syntax

Options Description

-address Specifies the IP address or network name of the targeted SP on the desired storage system

-user Specifies the username on the storage system you want to login to

-password Specifies the password on the storage system you want to login to

-scope Specifies whether the user account on the storage system you want to log in to is local, global, or lightweight directory access protocol (LDAP). 0 (default) indicates


EMC GS/TIS P43/68


global, 1 indicates local, and 2 indicates LDAP.

analyzer Issue an Analyzer command

archiveretrieve Creates and Retrieves a single archive file

archivedump Dumps Analyzer archive files to a CSV file

-all Retrieves all files

-data Dumps data from archive files

-file Specifies the name of the archive file stored on the array to be retrieved

-format

Specifies which performance characteristics to output and the order in which they appear. The -format switch requires one or more of the codes listed below (not all are listed):

ml MetaLUN

l LUN

tl Thin LUN

-get Returns the current values of the performance logging properties

-list Lists all the archive files

-location Specifies t

emc modeling tools data collection guide v1.17

Documents