© 2008 ibm corporation ibm system storage ® san storage solutions for system i jana jamsek, ats...

© 2008 IBM Corporation

IBM System Storage®

SAN Storage Solutions for System i

Jana Jamsek, ATS Europe



Agenda

I5/OS Specific Architecture

DS Copy Services

Scenarios and functions

Management tools

Comparing and Positioning



Main Memory

Single-Level StorageI5/OS Partition

Main Memory

Single-Level StorageI5/OS Partition



Auxiliary Storage Pools (Disk Pools)

Sysbas

User ASPs (Disk Pools 2-255)

Traditional User ASPs(Basic Pools 2-32)

System ASP(Disk pool 1)

Independent ASPs - IASPs( Independent Pools 33-255)



IASP Attributes

IASP’s are “like” other mountable file systems on other platforms.

Must be part of an System i Cluster to be able to be switched/moved to a second system

Always requires work management changes to the operational environment to use this solution

May require some application changes, a few customers have found this. In house testing or an IASP feasibility study should be performed to check this.



Cluster Node

Recovery Domain

Cluster Resource Group

CRG C

CRG B

CRG B

CRG C

CRG A

CRG A

Device Domain

Elements of a Cluster

Cluster Resources(e.g., IASP)

Cluster



Your Progra

m

File A

PutFile A Journa

l

Jrn Rcvr

Journal Receiver's content

File A: PT

New Row

PT:New Row

21

New Row

The DB is the last to know

PTNew Row

New Row

System i Journaling



FlashCopy



FlashCopy

Backuppartition

ProductionpartitionBfS

BfS

FlashCopy

IASP

IASP

Backuppartition

Productionpartition

•Cluster

Scenarios with FlashCopy

BfS = Boot from SAN

Full System IASP



Flashcopy - Benefits to the Customer

The production application downtime is only is as long as it takes to vary off the IASP, perform a FlashCopy of the volumes in the IASP, and vary on IASP. Or, Power-down system, perform a FlashCopy and IPL

On V6R1 we can quiesce data to disk without power-down production system i

The performance impact on the production application during the save to tape operation is minimal since it is only influenced by the FlashCopy activity which is mostly confined within the DS system. .

This solution can be implemented together with Backup, Recovery and Media Services for i5/OS (BRMS)



Production

Backup

B f S

B f SFlashCopy SE

Space Efficient FlashCopy

Test at the residency: Workload CPW with 20 000 users, Production capacity 1125 GB, repository 280 GB

Duration 4 h, Repository occupied about 25%



Journalling

– Journal is first written to disk, then database record is written

• If DB makes a change that crosses segment boundary it results i 3 writes:• first at the bottom of segment • second at the top of segment, and 3 write to update record was successful. • 3. write must follow writes 1 and 2.

Consistency group and dependant writes in System i

Without CG With CGFlashcopy

1. Journal

2. DB record

Flashed

Flashcopy

Flashed

1. Journal

2. DB record



FlashCopy integration with BRMS

CALL PGM(QBRM/Q1AOLD) PARM('FLASHSYS ' '*YES')

Enable FlashCopy function for BRMS

Set BRMS to FlashCopy mode

CALL QBRM/Q1AOLD PARM(‘FLSSYSSTS’ ‘*BEGIN’)

Production System Backup System

Set BRMS to backup FlashCopy state

CALL QBRM/Q1AOLD

PARM(‘FLSSYSSTS’ ‘*BACKUPSYS’)

QSYS/CALL QBRM/Q1AOLD

PARM(‘FLSSYSSTS’ ‘*ENDBACKUP’)

Backup

Set BRMS to End backup mode

Send QUSRBRM to

Production system



Quiesce data in an IASP or Sysbas

To quiesce data in an IASP to disk use command CHGASPACT with the following parameters:

– ASP Device (ASPDEV)

specify the name of the IASP being quiesced, or *SYSBAS.

– Option (OPTION)

specify option *SUSPEND to quiesce the data in the IASP.

– Suspend timeout (SSPTIMO)

specify duration of the timeout during which the system is quescing the data to disk.

– Suspend timeout action (SSPTIMOACN)

specify the desired action at the end of timeout if the system was not able to quesce all data during the timeout.

Requires V6R1

Memory

IASP,sysbas



Metro Mirror

2

3

1

4Server write

Write to secondary

Write acknowledge

Write completeacknowledgment

LUN orvolume

Primary(source)

LUN orvolume

Primary(source)

LUN orvolume

Secondary(target)

Suspend

Bitmap

Bitmap

Failover

Faiback

or



Metro MirrorBfS BfS

System i Cluster

Metro MirrorIASP IASP

Local site Remote site

Scenarios with Metro Mirror

Full

System

IASP



Metro Mirror – Benefits to the Customer

When using IASP: Basically, recovery takes as long as is needed to vary on an IASP, Therefore recovery time is significantly shorter than with solutions which use Boot from SAN, or external Loadsource, where IPL is needed for recovery.

This solution has only impact on write operations performances. It doesn’t use any of the System i5 CPU resources.

When using IASP: With this solution we can run a non production workload, like test or development tasks in the remote partition. This is again different from the implementation using Boot from SAN where the recovery partition has to be in stand-by, typically without any workload.

The System i Copy Services Toolkit which is used for this solution provides full automation of steps performed at planned or unplanned outages. All the needed steps for failover and failback are initiated by one command in a partition of System i.

Once this solution is established it requires very little maintenance and administration.Therefore some customers may prefer it to other high availability solutions available in System i.



B

CA

Global Copy

Data transmission

1

0

0

0

0

1

0

1

0

0

0

0

1

0

Consistency group save

Consistency group formation

Create consistency group

Drain consistency group and sendto remote disk subsystem using Global Copy.

Transmit updates in Global Copy mode Consistency group interval–0 to18hrs

Start next consistency group

FlashCopy issued with revertible

option

FlashCopy committed

Global Mirror



Global Mirror Recovery

•Global Mirror

B

CA

10

0

0

0

1

0

10

0

00

1

0

2

1 3

4•Failover Global copy

•Revert or commit Flashcopy if needed

•Reverse

•Flashcopy to B

•IPL or vary-on IASP from B

Action Required

The action to be taken depends on where in the cycle of creating a consistency group we were at the time of the disaster event

•Action is based on observing Flashcopy status and sequence numbers

•Create CG •Drain CG

•Issue FC •Commit FC

Some important steps:



GMBfS

BfS

System i Cluster

GMIASP

Local site Remote site

IASP

Scenarios with Global Mirror

Full

System

IASP



Benefits to the Customer

Very small impact to production performances

Suitable solution for long distances

Customer has the possibility to balance between RPO and bandwidth

Due to consistency group on production volumes i5/OS dependant writes are preserved

Once this solution is established it requires very little maintenance and administration

The System i Copy Services Toolkit is designed to do all neccesary steps at recovery autimatically. TPC-R can be used to do necessary steps at recovery on System Storage.



GM Recovery Point Objective and Recovery Time with System i

Recovery Point Objective (RPO)– Data on C volumes are 1 to 2 CG intervals behind data on A, depending at which

point in CG interval the failure happened

– Average RPO = 1.5 CG interval

Recovery Time

– GM of IASP: up to 30 minutes

– GM with Boot from SAN: as long as it takes to IPL

• Some measured durations of abnormal IPL: • I570 V5R3 – 44 min• I570 V5R4 – 1h:22min



Customers with Global Mirror and System i

AAA Cooper Transportation, US

– DS8100, System i, Global Mirror, distance 2000 miles

Crete Carrier Corporation, US

– DS8000, system i, Global Mirror

Estes Express Lines, Inc., US

– DS8100, ESS, System i, Global Mirror, distance 2000 miles

A Retail bank in UK

– DS8100, System i, distance 200 km

A customer in Italy

– DS6000, System i, Global Mirror, distance 30 km

A customer in Italy

– DS8000, System i, Global Mirror, distance 400 km

A customer in Czech republic

– DS8000, System i, Global Mirror, distance to US



Combined scenarios

Metro Mirror and FlashCopy - Supported with System i

– Supported by the Copy Services for System i

Global Mirror and FlashCopy – Supported with System i

– Supported by the Copy Services for System i

Metro Global Mirror – Supported with System i

– On the roadmap for Copy Services for System i



What is Copy Services for System i

Services and program code for high availability and disaster recovery of System i.

The Copy Services for System i (Toolkit) contains code which gets installed on all partitions in a FlashCopy, Metro Mirror or Global Mirror clustered environment; or management partition.

Employing System i clustering function with DS8000 Copy Services.

Works with IASPs.

Works with full disk space.

Requires approval from Rochester to sell the solution.

Sold as service through the Lab Services - Client Technology Center. First year maintenance is required as part of the sale.

Works on V6R1 and previous i5/OS releases.



System StorageMirroring

Production iSeries

Backup iSeries Partition

(powered off)

DS8000

FlashCopy

LPAR

LPAR

Copy Services for System i - Full System Copy Services

Power down the production partition

FlashCopy the disks

Power up the production partition, return to operations

Power up the FlashCopy partition

Perform backups (or any other operation) on the FlashCopy partition (normal backup time)

Full System FlashCopy Toolkit automates all of this for you !

Planned: Quiesce data in *sysbas with CHGASPACT

Planned: Support for Metro Mirror and Global Mirror

FlashCopy



Example: Full System FlashCopy



Copy Services for System i - IASP

Flash Copy, Metro or Global Mirror generates a second IASP copy identical to first Changes to production IASP replicated to second copy of IASP Cluster and SAN combined replication solution Switch to DR site or activate Flashcopy is done with one command IBM only HA solution

OS and other system data

OS and other system data

DS8000 DS8000

Metro MirrorFlashCopy

IASP Copy

IASP Copy

IASP Copy

I5/OS cluster services



Release 6.1 of Copy Services for System i IASP Toolkit Enhancements

Requires Power 5/6 and i5/OS V5R4

Updated commands

WRKCSE enahncements

– Scripts may be run intercatively from WRKCSE command

– Volume count validation

Option in MM and GM to show out-of-synch sectors

Configuration and switch-over enhancements

– No longer communication with HMC

– Faster switch for around 20-30 sec per IOA

Enhanced detection of primary DS HMC failure - we will now detect a DSCLI code freeze on the primary HMC

Toolkit supports FlashCopy SE



Example: Start FlashCopy of IASP

STRFLASH ENV(<name of IASP>)

Steps performed:

• Check that the server jobs are active

• Check that clustering is correctly configured

• Autostart any nodes in the cluster that are required

• Send an information message to operatior on the Production system

• Vary off the IASP on production system

• Run the mkflash script using the DSCLI

• Vary-on the production copy of the IASP

• Release/Reset the IOP/IOA resources on the backup partition

• Make the IASP available to the backup system

ENDFLASH

Steps performed:

Vary off all the FlashCopy IASPs

Modify clustering to enable a clean up.

IPL the backup system to clean up its virtual addresses.



Example: Switch Metro Mirror

Steps performed:

Send *INQ message to QSYSOPR on the Production node

Vary off the IASP on the current Production node.

Run Metro Mirror Failover tasks for the IASP device using DSCLI.

Release/Reset the IOP/IOA resources on the backup partition and get the disks to register correctly as the IASP.

Vary on the IASP on the current Backup node (which then becomes the Production node).



Example: Display Metro Mirror Environment



IBM PowerHA for i

The IBM i5/OS HA/DR clustering solution

– Simple web-based management

– Provides Cross Site Mirroring (XSM) function via

• i5/OS Geographic Mirroring• DS8000 Metro Mirror • DS8000 Global Mirror

Product Number

– 5761-HAS

Supported Release

– i5/OS V6R1

Prerequisites

– HA Switchable Resources

(option 41)



i5OS

DSCLI

LP 5761-HASLP 5761-HAS

Option 41 - HASR

Metro Mirror

Global Mirror

Flash CopySwitchable Devices & Disk

Source/Target Side Tracking

IASP infrastructure CRG infrastructure

Admin Domain infrastructure

Geographic Mirroring

Solution Based Task Based Commands

XSM

Cluster mgmt,

User inetrfaces

SAN Solutions

Switchable

resources

Cluster

technology

PowerHA Structure



FlashCopy and Metro Mirror of IASP, Global Mirror will follow

GUI for creating cluster and IASP

GUI for FlashCopy and Metro Mirror of IASP

DS/ ESS

System i Cluster, Device domain

Metro Mirror

IASP IASPPowerHAPowerHA

IOA IOA

Online

DS CLI

Online

DS CLI

PowerHA with DS8000 Copy Services



Metro Mirror Scenario

Setup:

Define and connect LUNs on DS8000

Create IASP on Production system with DS8000 LUNs

Create cluster

Create Device domain

Create Device CRG

Install DS CLI on System i

Create Metro Mirror session

Create Copy description for the IASP on primary and backup node

Start CRG



Metro Mirror scenario - continue

Scenario for planned outages :

• Make IASP unavailable on primary node

• Detach Metro Mirror session on primary node at primary site

• Stop device CRG

• Change roles in recovery domain

• Restart device CRG

• Reattach Metro Mirror ASP session on backup node at primary site

• Make IASP available on primary node at secondary site



FlashCopy Scenario

Setup:

Define and connect LUNs on DS8000

Create IASP on Production system with DS8000 LUNs

Create cluster

Create Device domain

Install DS CLI on System i

Create Copy description for the IASP on primary and backup node



FlashCopy Scenario - continue

Scenario for daily backups:

Day 1

CHGASPACT suspend

STRASPSSN

CHGASPACP resume

Perform backup of application from Backup node

CHGASPSSN *DETACH



FlashCopy Scenario - continue

Every other day

CHGASPACT suspend

CHGASPSSN *REATTACH

CHGASPACT resume

Perform backup of application from Backup node

CHGASPSSN *DETACH



Creating Metro Mirror session



Creating Copy Description



Detach MM session



Introducing IBM TotalStorage Productivity Center

IBM TotalStorage Productivity Center

Standard Edition

Limited Edition

Productivity Center For Data

Productivity Center For Fabric

Productivity Center For Disk

Productivity Center For

Replication



What is TPC for Replication?

Volume level Copy Service Management Coordinates Copy Service Functionalities

–Flash Copy–Metro Mirror–Global Mirror–Metro Global Mirror

Ease of Use–Single common point of control–Web browser based GUI –Wizard based configuration



Simple Setup

1. Add Storage Device

2. Setup Paths via Wizard

3. Create Session

4. Add Copy Sets

–Box to Box

–LSS to LSS

–Volume to Volume

–CSV Import

5. Issue Start!

NEW

NEW



TPC for Replication with System i

FlashCopy, Metro Mirror, Global Mirror of entire disk space



TPC for Replication – Global Mirror Recovery



Managing Tools for Solutions with External Storage

Copy Services for System i (Toolkit)

HASM TPC-R

FlashCopy of IASP Yes Yes No

Full System FlashCopy

Yes No Storage part

Metro Mirror of IASP

Yes Yes No

Full System Metro Mirror

Yes No Storage part

Global Mirror of IASP

Yes Yes No

Fully System Global Mirror

Yes No Storage part



How Does it Work? - iCluster Architecture Primary Node/LPAR

DB2

ERP

Journals

iC Read

Remote Journal

TCP/IP

Backup Node/LPAR

DB2

iC Apply

MemoryCache

MatchMerge

• Remote journal based• High speed send and apply• Configure once on source

iC ReadiC Read

iC ApplyiC Apply



iCluster Switching Primary Node/LPAR

DB2

ERP

Journals

iC Read

Remote Journal

Backup Node/LPAR

DB2

iC Apply

High SpeedMemoryCache

MatchMerge

iC ReadiC Read

iC ApplyiC Apply

TCP/IP

iC Read

Remote Journal

DB2

iC Apply

MemoryCache

MatchMerge

iC ReadiC Read

iC ApplyiC Apply

Primary Node/LPARBackup Node/LPAR

• User exits before and after switch for automation•Journaling automatically started on new source• Typical switch time in minutes

TCP/IP

DB2

ERP

Journals



Storage management based page level replication.

Geographic mirroring runs over TCP/IP

Up to four communication interfaces on multiple adapters can be configured

Disk

– Can be internal or external

– Type, size and number of disk units may be different

Detach / Reattach

Overview of Geographic Mirroring



IBM i5/OS Multi System Data Resiliency

Switcheable IASP Clusters

Strategic building block for IBM i5/OS High Availability and Disaster Recovery solutions

Utilizes i5/OS® Cluster Resources for automation and application interface

Geographic Mirroring Storage agnostic HA and DR

Cross Site Mirroring

Metro Mirror

Metro Mirror Global Mirror Flash Copy HA and DR FlashCopy


Switched Disk

Storage agnostic

Single copy of data

HA (Local only)

iCluster

Switched Disk Cluster

iCluster Storage agnostic HA and DR Concurrent access Supports i5/OS Cluster

Logical Replication

Geographic Mirroring Cluster Metro Mirror/Global Mirror Cluster

Basic San Copy Services

Global Mirror Boot From SAN DR/Tape Backup

FlashCopy Global Mirror



Comparing Different Business Continuity Solutions iCluster Geo Mirroring Metro Mirror

of IASPGlobal Mirror of IASP

Metro Mirror full space

Gloabl Mirror full space

Recovery time Measured in min

Less than 30 min (?)

Less than 30 min

Less than 30 min

IPL IPL

Recovery point If remote journaling, zero

Zero (?) If journaling, zero

Depends on GM interval

If journaling, zero

Depends on GM interval

Production system resources

CPU , disk memmory usage

CPU usage - - - -

Usage of recovrey system

Can be used for other workload

Sysbas can be used for other workload



Must be stand-by

Must be stand-by

Performance impact

Some impact Some impact Minimal Near zero Some impact at certain workloads

Near zero

Resynchronization

Incremental Incremental Incremental Incremental Incremental Incremental

Administartion and maintenance

Significant Need to miantain *sysbas

Need to miantain *sysbas

Need to miantain *sysbas

- -

Automation Autmated Can be done Automated Automated Automated Automated

Support Ca be concern in some countires ?

Fully Supported Fully Supported

Fully supported Fully supported Fully supported



Positioning: Questions to consider

How big is System i workload ?

How much can my data be behind at recovery ?

How quickly must I be up and runnung ?

Do I need DR solution for System i and other servers ?

Do I need DR solution for multiple System i ?

Which distance is suitable for my DR ?

What bandwidth can I afford ?

How reliable are the links ?

Do I need DR system for testing/developing ?

Is my system heavily used by my applications ?



Feature Benefit Products Considerations

Storage Based

Replication of all data written to disk

Virtually unlimited geographic distance when using asynchronous

Generally implemented for DR* and HA* (synchronous)

Can be cluster controlled

Synchronous (Metro Mirror only) or asynchronous network transmission (Global Mirror only)

No manual management of replicated objects required

FlashCopy means second copy of data can be used for fast backups, development, testing, etc.

Order of changes is preserved via DS8000 and Metro Mirror

Resolution of out of synch conditions (objects or data) is under control of DS8000

IBM Metro Mirror

IBM Global Mirror

IBM FlashCopy

Can be used alone or with IASPs & Toolkit

Note: used with IBM TotalStorage DS8000

Network Bandwidth (all data and objects are replicated to second system)

Only supported via external SAN storage

No concurrent access to second copy of data

IF not using IASPs, system IPL required for switching to backup copy

Additional disk for Consistency Groups (Global Mirror only)

Operating System Based

Page replication controlled by operating system

Ordering of changes is preserved by the operating system

No manual management of replicated objects required

Synchronous network data transmission

Is cluster controlled

Supports external or internal storage

Only vary on of IASP is required for switching to backup system

Resolution of out of synch conditions (objects or data) is under control of i5/OS

Generally implemented for HA*

Independent Auxiliary Storage Pools (IASP)

Cross Site Mirroring (XSM)

High Availability Solutions Manager (HASM) (implemented with XSM and IASP) (V6R1 only)

Network Bandwidth (all data and objects in an IASP are replicated to the second system)

Distance limitations

No concurrent access to second copy of data

Prior to V5R4 no source side

tracking

Requires IASP implementation

Logical Replication

Selective data/object replication may reduce network overhead

Virtually unlimited geographic distance

Concurrent access to second data copy (for backups, read only access, etc.)

No special hardware configuration requirements

Choice of synchronous or asynchronous network transmission of data (synchronous only with remote journaling)

Generally implemented for DR* and HA*

Can be cluster controlled

Can be used alone or with other HA technology choices

Supports external or internal storage

No IPLs required for switching. When used in conjunction with IASP—vary on of IASP is required

Ordering of changes controlled by HA ISV software interfaces

IBM:

IBM DataMirror iClusterTM

Maximum Availability:

www.maximumavailability.com

Trader’s:

http://www.quick-edd.com/uk/index_uk.htm

Vision Solutions:

http://www.visionsolutions.com

Possible apply lags on target system for changes in high volume environments

Management to ensure all new objects are replicated

Bandwidth has to be sufficient for replicated objects and data , and to ensure no data send lag

Resolution of out of synch conditions (data or objects) is not under control of i5/OS



DS4000 with System i – A Practical Example

VIOS IBM i client

POWER6 570

DS4800

Controller A

Disk enclosure

Controller B

Disk enclosure

Virtual SCSI Virtual SCSI



Create hot spares in DS4800, each in one enclosure



Create Raid-1 array



Created Raid-1 arrays and logical drives



Assign logical drives to the host ports in VIOS



Assign logical drives to Virtual SCSi adapters in VIOS



Assign VIOS Virtual Adapters to virtual adapters in IBM i client partition



Load Source tagging



DS4000 LUNs in System i

© 2008 ibm corporation ibm system storage ® san storage solutions for system i jana jamsek, ats...

Documents