survey on procedures for migrating data between storage arrays

EMC Proven Professional Knowledge Sharing 2010

Survey on procedures for migrating data between Storage Arrays Alvaro Clemente

Alvaro ClementeImplementation [email protected]

2010 EMC Proven Professional Knowledge Sharing 2

1. Introduction ............................................................................................................. 3 2. Data migration between IBM DS through PPRC for AIX HACMP Servers. ............ 4

2.1. AIX Server analysis ............................................................................... 4 2.1.1. IBM Array side ................................................................................ 4

2.2. Previous Tasks ...................................................................................... 4 2.2.1. Zoning between IBM Arrays. .......................................................... 4 2.2.2. Zoning between IBM Array and server. .......................................... 5 2.2.3. Heartbeat and volgrps creation and configuration. ........................ 5 2.2.4. HACMP Heartbeat re-configuration. ............................................... 7 2.2.5. PPRC creation tasks and copy establishment. .............................. 8

2.3. Migrating process ................................................................................... 9 2.3.1. Actions over servers and SAN. .................................................... 10 2.3.2. Actions over IBM arrays. .............................................................. 11 2.3.3. Actions over SAN, disks and cluster. ........................................... 13

3. Data migration between IBM DS and DMX through OpenReplicator for AIX HACMP-VCS Servers. .................................................................................................. 15

3.1. AIX Server analysis ............................................................................. 15 3.1.1. IBM Array side .............................................................................. 15 3.1.2. DMX Array side ............................................................................ 15

3.2. Previous Tasks .................................................................................... 16 3.2.1. Volume pairs between IBM and DMX. ......................................... 16 3.2.1.1. IBM ............................................................................................ 16 3.2.1.2. SAN ........................................................................................... 16 3.2.1.3. DMX .......................................................................................... 16

3.3. Migration Process ................................................................................ 17 3.3.1. Actions over servers and SAN. .................................................... 17 3.3.1.1. Stop resources groups and applications. ................................. 17 3.3.1.2. Delete disks and uninstall previous drivers. ............................. 17 3.3.1.3. Actions over the SAN. ............................................................... 18 3.3.1.4. Install and configure EMC software .......................................... 18 3.3.2. Actions over arrays. ...................................................................... 19 3.3.3. Actions over servers. .................................................................... 19 3.3.3.1. Heartbeat re-configuration. ....................................................... 19 3.3.3.2. Boot from SAN configuration .................................................... 21 3.3.3.2.1. Prepare Boot from SAN disks. .................................................. 21 3.3.3.2.2. System copy and mirror configuration. ..................................... 22

4. Considerations about Oracle ASM. ....................................................................... 25 4.1. Migration with Oracle ASM and AIX LVM. ........................................... 25

Disclaimer: The views, processes, or methodologies published in this article are those of the author. They do not necessarily reflect EMC Corporation’s views, processes, or methodologies.


Introduction This paper presents a survey on different ways of moving data between DS Arrays

through PPRC, migrating with EMC Open Replicator between IBM Storage Array and

Symmetrix® DMX-4 and intelligent control of the SAN and the hosts involved. Hosts

used are AIX inside cluster (HACMP, Veritas Cluster) and stand-alone frameworks.

Different types of operating systems are presented, such as LVM and VxVM.

The data migration process is explained as a step-by-step procedure in order to identify

how, when, and why these actions are relevant to implementation success. This

document covers the use of tools such as PPRC between IBM’s, Open Replicator

between DS8000 IBM Storage, and EMC Symmetrix.

This survey deals with the growing compatibility between operating systems, volume

management, and cluster software. Additionally, AIX operating with LVM, HAMCP,

Veritas Volume Manager, and Veritas Cluster is analyzed before, during, and after

migration.

These methods have been successfully tested over more than one hundred migrations.

The procedure is methodical, secure, and functional for data movement between

Storage Arrays and the document constitutes a complete guide for Specialists.


1. Data migration between IBM DS through PPRC

for AIX HACMP Servers

1.1. AIX Server analysis

1.1.1. IBM Array side

It’s necessary to discover those IBM devices managed by AIX Operating System and

requiring migration to other IBM arrays. To make an accurate determination, it’s

possible to run ./inq –shark_wwn (path: /emcgrab/tools/bin) and filter the data like:

/dev/rhdisXX 75 6005076305ffc65e0000000000001c01

• 6005076305ffc65e corresponds to IBM Serial Number Array.

• 1c01 corresponds to Device Identification Number.

To check if the Device Identification Number is correct or find the number of servers by

which it is managed, we need DSCLI interface to query the volumes inside the volgrps

that contain both servers and devices:

# dscli –cfg IBM_Configuration_File

dscli> lsfbvol –volgrp VX (VX is the volgrp assigned to one host of the cluster)

dscli> lsfbvol –volgrp VY (VY is the volgrp assigned to the other host of the cluster)

1.2. Previous Tasks

1.2.1. Zoning between IBM Arrays

IBM Arrays work in pairs, similar to EMC. While EMC uses SRDF to replicate devices

between them, IBM needs PPRC. Some customers link two sides by a dedicated path.

To minimize migration downtime, it’s desirable to concatenate four arrays this way:


IBM DS Array send and receive data streaming among all its IO ports, so zoning could

be established between the same or different IO ports. It’s recommended to create

zones between two IO ports from different fabrics.

1.2.2. Zoning between IBM Array and server

Before migrating the data, cluster heartbeat needs to be moved from IBM source to

IBM destination so both servers must have active zones between one of their HBAs

and Production-IBM destination and also between the other HBA and Production-IBM

source. Both servers would have access to both arrays but it’s important to note that

the rest of the devices would be out of the volgrps in the destination array.

Connectivity can be checked by querying the IBM hostconnect database through:

dscli> lshostconnect -unknown

1.2.3. Heartbeat and volgrps creation and configuration

We have to create:

• Volgrp, hostconnect, and devices in Production-IBM destination.

• Volgrp, hostconnect, and devices in Recovery-IBM destination.


The cluster heartbeat is the only device included in the volgrp. The rest of the volumes

would be created but not mapped to a specific volgrp.

Heartbeat accessibility by the two nodes of the cluster

//Production-IBM destination should contain two volgrps: one for each node. Each volgrp must contain hostconnect of minimum one server’s HBA and heartbeat devices. It’s not necessary to create the rest of the devices at this moment.

//Recovery-IBM destination should contain two volgrps: one for each node. Each volgrp should contain hostconnect of server’s HBAs and all the devices (depends on Customer Recovery Policy). There are no active zones from the hosts to this array so the devices wouldn’t be available.

//Heartbeat configuration in Production-IBM destination array. The option extpool depends on the pool capacity (lsextpool list available capacity for each pool). XXXX stands for the IBM Serial Number Device (must be available). Capacity: 1GB.

dscli> mkfbvol -extpool PX -cap 1 -type ds -name HEARBEAT XXXX

dscli> mkvolgrp -hosttype pSeries NODE1_VG

Date/Time: mm/dd/yyyy hh:mm:ss CEST IBM DSCLI Version: V DS: IBM.Serial Number

CMUC00030I mkvolgrp: Volume group V0 successfully created.

dscli> mkvolgrp -hosttype pSeries NODE2_VG


CMUC00030I mkvolgrp: Volume group V1 successfully created.

dscli> chvolgrp -action add -volume XXXX V0

dscli> chvolgrp -action add -volume XXXX V1

//Server’s HBAs configuration in Production-IBM destination array. Check with lshostconnect –volgrp VX.

dscli> mkhostconnect -wwname WWNN -hosttype pSeries -volgrp V0 -ioport all NODE1_HBA0_V0

dscli> mkhostconnect -wwname WWNN -hosttype pSeries -volgrp V1 -ioport all NODE2_HBA1_V1


1.2.4. HACMP Heartbeat re-configuration

Once the server accesses old and new heartbeats simultaneously, migrating the

“quorum” process starts:

Heartbeat migrating process:

1. Once heartbeat masking is done, rescan both nodes:

node1-node2# cfgmgr

node1-node2# lspv //Check there’s a new hdisk in the list

node1-node2# bootinfo –s hdiskX //Check device size: 1GB.

node1-node2# lscfg -vl hdiskX //Check if in the Serial Number field appears IBM serial number (7 digits) and device identification number (4 digits).

2. Stop cluster service from one of the nodes:

node1 (or from node2)# smit hacmp

Smitty HACMP “System Management (C-SPOC)” “Manage HACMP Services” “Stop Cluster Services”

Stop Cluster Services

Type or select values in entry fields.

Press Enter AFTER making all desired changes.

[Entry Fields]

* Stop now, on system restart or both both

Stop Cluster Services on these nodes [node2,node1]

BROADCAST cluster shutdown? true

* Select an Action on Resource Groups Bring Resource Groups

3. Check if the resources are offline in both nodes:.

node1-node2:root:/usr/sbin/cluster/utilities# ./cldump

4. Run exportvg of quorum volume group in order to delete the configuration.

node1# exportvg quorum_vg

node2# exportvg quorum_vg

5. Quorum_vg recreation. Check major number availability in both nodes.

node1-node2# lvlstmajor //Choose the common number in both nodes.

node1# mkvg -V ZZ -y quorum_vg hdiskX

node1# chvg -c quorum_vg //Mode Enahnced-concurrent configuration.

node1# chvg -a n quorum_vg //Not to be activated automatically.

node1# lsvg quorum_vg

node1# varyoffvg quorum_vg

node2# chdev -a pv=yes -l hdiskX //PVID activation over new heartbeat in the second node.

node2# importvg -V 44 -y quorum_vg hdiskX //Volume group import in node2. This warning will appear:

synclvodm: No logical volumes in volume group quorum_vg.

quorum_vg

0516-783 importvg: This imported volume group is concurrent capable. Therefore, the volume group must be varied on manually.


6. Configure the new heartbeat network in node:


Smit hacmp “Extended Configuration” “Discover HACMP-related Information from Configured Nodes”

Smit hacmp “Extended Configuration” “Extended Topology Configuration” “Configure HACMP Networks” “Remove a Network from the HACMP Cluster” <Device Network Name>

Smit hacmp “Extended Configuration” “Extended Topology Configuration” “Configure HACMP Networks” ” Add a Network to the HACMP Cluster” “# Discovered Serial Device Types (diskhb)” <Device Network Name>

Smit hacmp “Extended Configuration” “Extended Topology Configuration” “Configure HACMP Communication Interfaces/Devices” “Add Communication Interfaces/Devices” (Add Pre-defined Communication Interfaces and Devices) and (Communication Devices) and (<Device Network Name>) /dev/<new device>

7. Synchronize to the other node and start cluster service: Smit hacmp “Extended Configuration” “Extended Verification and Synchronization”

Smitty HACMP “System Management (C-SPOC)” “Manage HACMP Services” “Start Cluster Services”

8. Check if the resources are online:

node1-node2:root:/usr/sbin/cluster/utilities# ./cldump

9. Delete previous heartbeat from the server. This device corresponds with Production-IBM source array:

node1-node2# rmdev –dl hdiskX

1.2.5. PPRC creation tasks and copy establishment

It’s necessary to configure the link between Source Production-IBM and Recovery-IBM

as mmir –cascade:

1. Pause the link for each volume involved in the data migration:

dscli> pausepprc -remotedev IBM.Source.Recovery AAAA:AAAA

2. Resume the link (mmir –cascade):

dscli> resumepprc -remotedev IBM.Source.Recovery -type mmir -cascade AAAA:AAAA

3. Check the link:

dscli> lspprc -l AAAA

Configure the link between Destination Production-IBM and Recovery-IBM as gcp –

cascade. This link should be previously created.

1. Create the data volume in Destination Production-IBM:

dscli> mkfbvol -extpool PX -cap CAP -type DS -name DATA AAAA

2. Create the data volume in Destination Recovery-IBM:

dscli> mkfbvol -extpool PX -cap CAP -type ds -name DATA AAAA

3. Establish the synchronization in global copy mode:



CMUC00234I lspprc: No Remote Mirror and Copy found.

dscli> mkpprc -remotedev IBM.Destination.Recovery -type gcp -cascade AAAA


4. Check the link:



ID State Reason Type Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs) Critical Mode First Pass Status

========================================================================================================

AAAA: AAAA Copy Pending - Global Copy 0 Disabled Enabled Invalid - FE 300 Disabled True

The final step is to create the synchronization gcp –cascade mode between Source

Recovery-IBM and Destination Production-IBM. In this case, volume AAAA would be

replicated to volume AAAA in the destination array. The communication between them

needs to be established through the SAN so zones are necessary.

o SOURCE_RECOVERY_I0F1__DESTINATION_PRODUCTION_I0F1

o SOURCE_RECOVERY_I0F2__DESTINATION_PRODUCTION_I0F2

1. Check previous paths between them. The first two digits of the volume AAAA corresponds with the pprcpath between the arrays:

dscli> lspprcpath -l AA

2. Configure new paths without deleting previous ones:

dscli> mkpprcpath -remotedev IBM.Destination.Production -remotewwnn ARRAYWWNN –srclss AA -tgtlss AA

I0F1prev:I0F1prev I0F2prev:I0F2prev I0F1new:I0F1new I0F2new:I0F2new

3. Check the paths:

dscli> lspprcpath -l AA


Src Tgt State SS Port Attached Port Tgt WWNN

=========================================================

AA AA Success ... I0F1prev I0F1prev ARRAYWWNN

AA AA Success ... I0F2prev I0F2prev ARRAYWWNN

AA AA Success ... I0F1new I0F1new ARRAYWWNN

AA AA Success ... I0F1new I0F1new ARRAYWWNN

4. Create the synchronization volume-pairs:

dscli> mkpprc -remotedev IBM.Destination.Production -type gcp -cascade AAAA:AAAA

5. Check the synchronization:


1.3. Migrating process

Configuration before migrating process:


At this stage, it is necessary to make a backup of the operating system:

1. HACMP Configuration backup.

2. Backup of the rootvg.

To filter the steps taken to investigate if something goes wrong, it’s desirable to reboot

both servers before starting the migration process.

1.3.1. Actions over servers and SAN

Actions to be taken on the server side:

1. Stop cluster service from one of the nodes:


Smitty HACMP “System Management (C-SPOC)” “Manage HACMP Services” “Stop Cluster Services”

Stop Cluster Services

Type or select values in entry fields.


[Entry Fields]

* Stop now, on system restart or both both

Stop Cluster Services on these nodes [node2,node1]

BROADCAST cluster shutdown? true

* Select an Action on Resource Groups Bring Resource Groups

2. Check if the resources are online:

node1-node2:root:/usr/sbin/cluster/utilities# ./clRGinfo

3. Varyoff, umount the resources in both nodes in order to prevent access to disks:

node1-node2# lsvg –o //Check what are the vgs that are online.

node1-node2# lsvg –l vg //Check the filesystems that are mounted for EACH VOLUME GROUP that is online.

node1-node2# umount /fs //If we aren’t able to umount the file systems, run fuser –k /fs/ or fuser –cku /fs/

node1-node2# varyoff vg

4. Identify hardware paths that are going to be migrated in both nodes:

node1-node2# lsdev -Cc disk //Check the hardware paths that are Available and the source (Symmetrix, IBM,...)


.....

hdisk8 Available 03-00-01 IBM MPIO FC 2107

hdisk9 Available 03-00-01 IBM MPIO FC 2107

.....

5. Once identified, create a script (or manually) for deleting all the devices managed by LVM:

node1-node2# lsdev -Cc disk |grep “IBM MPIO FC 2107” | awk '{print "rmdev -dl", $1}'|sh

After completing the previous section, we have to deactivate:

o NODE1_A__SOURCE_PRODUCTION_I0F1

o NODE1_B__SOURCE_PRODUCTION_I0F2

o NODE2_A__SOURCE_PRODUCTION_I0F1

o NODE2_B__SOURCE_PRODUCTION_I0F2

Prior to rebooting both nodes, it’s important to note in the inittab file if all the processes

that access disks are commented.

1.3.2. Actions over IBM arrays

From the array side:

1. Check synchronization between Source Production-IBM and Source Recovery-IBM:

dscli-Source.Production> lspprc -l AAAA //Check there are no invalid tracks.

2. Split previous synchronization and check again:

dscli-Source.Production> rmpprc –remotedev IBM.Source.Recovery AAAA:AAAA


CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair relationship AAAA:AAAA:? [y/n]:y

CMUC00155I rmpprc: Remote Mirror and Copy volume pair AAAA:AAAA relationship successfully withdrawn.

dscli-Source.Production> lspprc -l AAAA


Modify the synchronization between Source Recovery-IBM and Destination Production-

IBM from gcp –cascade to mmir -cascade:

3. Resume synchronization between Source Recovery-IBM and Destination Production-IBM and set mode synchronous:

dscli-Source.Recovery> resumepprc –remotedev IBM.Destination.Production –tpye mmir –cascade AAAA:AAAA


CMUC00158I resumepprc: Remote Mirror and Copy volume pair AAAA:AAAA relationship successfully resumed. This message is being returned before the copy completes.

4. Wait some time and check if there are no invalid tracks:

dscli-Source.Recovery> lspprc -l AAAA //Check there are no invalid tracks.

Split the synchronization between Source Recovery-IBM and Destination Production-

IBM:

5. Split previous synchronization and check again:

dscli-Source.Recovery> rmpprc –remotedev IBM.Destination.Production AAAA:AAAA


CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair relationship AAAA:AAAA:? [y/n]:y

CMUC00155I rmpprc: Remote Mirror and Copy volume pair AAAA:AAAA relationship successfully withdrawn.

dscli-Source.Recovery> lspprc -l AAAA

Modify the synchronization between Destination Production-IBM and Destination

Recovery-IBM from gcp –cascade to mmir. After that, assign devices to destination

volgrps:

6. Check synchronization between Destination Production-IBM and Destination Recovery-IBM:

dscli-Destination.Production> lspprc -l AAAA

7. Modify from gcp to mmir:

dscli-Destination.Production> resumepprc –remotedev IBM.Destination.Recovery –tpye mmir –cascade AAAA:AAAA



CMUC00158I resumepprc: Remote Mirror and Copy volume pair AAAA:AAAA relationship successfully resumed. This message is being returned before the copy completes.

8. Add volume to volgrp:

dscli-Destination.Production> chvolgrp –action add –volume AAAA V0 //NODE1_VG

dscli-Destination.Production> chvolgrp –action add –volume AAAA V1 //NODE2_VG

dscli-Destination.Production> lsfbvolgrp –volgrp V0

dscli-Destination.Production> lsfbvolgrp –volgrp V1

1.3.3. Actions over SAN, disks, and cluster

After this moment, start both nodes. Because the zones aren’t yet activated, servers

couldn’t manage disks. After checking the stability of the operating system, activate:

o NODE1_A__DESTINATION_PRODUCTION_I0F1

o NODE1_B__DESTINATION_PRODUCTION_I0F2

o NODE2_A__DESTINATION_PRODUCTION_I0F1

o NODE2_B__DESTINATION_PRODUCTION_I0F2

Back to the server side:

1. Rescan disks and varyon the vg:

node1-node2# cfgmgr

node1-node2# lsdev –Cc disk

node1-node2# lspv

node1-node2# varyonvg vg

2. It’s possible to find an error when trying to mount the filesystems. In order to fix this, run a fsck and then varyoff the vg:

node1-node2# fsck –p /fs

node1-node2# varyoffvg vg


Start the cluster and resources through smit or manually, balance the resources

between the two nodes, and uncomment the lines in the inittab file.


2. Data migration between IBM DS and DMX through OpenReplicator for AIX HACMP-VCS Servers

2.1. AIX Server analysis

2.1.1. IBM Array side

It’s necessary to discover those IBM devices managed by AIX Operating System and

requiring migration to DMX arrays. To make an accurate determination, it’s possible to

run ./inq –shark_wwn (path: /emcgrab/tools/bin) and filter the data like:

/dev/rhdisXX 75 6005076305ffc65e0000000000001c01

• 6005076305ffc65e corresponds to IBM Serial Number Array.

• 1c01 corresponds to Device Identification Number.

To check if the Device Identification Number is correct or find the number of servers by

which it is managed, we need DSCLI interface to query the volumes inside the volgrps

that contain both servers and devices:

# dscli –cfg IBM_Configuration_File

dscli> lsfbvol –volgrp VX (VX is the volgrp assigned to one host of the cluster)

dscli> lsfbvol –volgrp VY (VY is the volgrp assigned to the other host of the cluster)

2.1.2. DMX Array side

After determining the size of the volumes in the source, choose the Symmetrix Device

Volumes that fit all the data. Destination volumes must be larger than source volumes.

It’s important to get an emcgrab of the servers. Check the interoperability matrix for

HBA’s firmware, PowerPath versions, etc. It’s possible to check the firmware version

running the command lsmcode –d fcsX.


2.2. Previous Tasks

2.2.1. Volume pairs between IBM and DMX

2.2.1.1. IBM Configure a volgrp in the IBM array that would contain the volumes that are going to

be migrated. This volgrp would also contain the hostconnects that correspond with

DMX FAs:

1. Create migration volgrp:

dscli> mkvolgrp -hosttype pSeries OPENREP_MIGRATION_VG


CMUC00030I mkvolgrp: Volume group Vor successfully created.

2. Add volumes (it’s possible to have the same volume in two different volgrps):

dscli> chvolgrp -action add -volume XXXX Vor

3. Create hostconnect (DMX FA’s). Realize that the hosttype is Win2003:

//Server’s HBAs configuration in Production-IBM destination array. Check with lshostconnect –volgrp VX.

dscli> mkhostconnect –wwname 5006048... -hosttype Win2003 -volgrp Vor -ioport all DMX_PROD_FAF1

dscli> mkhostconnect –wwname 5006048... -hosttype Win2003 -volgrp Vor -ioport all DMX_PROD_FAF2

2.2.1.2. SAN

Create and activate the zones between both arrays:

o IBM_I0F1__DMX_PROD_FAF1

o IBM_I0F1__DMX_PROD_FAF1

Create but not activate the zones between both nodes of the cluster and the DMX:

o NODE1_A__ DMX_PROD_FAF1

o NODE1_B__ DMX_PROD_FAF2

o NODE2_A__ DMX_PROD_FAF1

o NODE2_B__ DMX_PROD_FAF2

2.2.1.3. DMX Activate SCSI3_persist_reserv flag for every device, form metas, and map devices to

the same FAs that handle the copy (hot pull). Create the masking, keeping in mind that

it’s important to notice that masking and zoning must not be activated together. Apply

masking at the end of the migration process.


To check if the volume pairs are properly configured and to log DMX FAs into IBM, run

symsan –sid ID list –sanports –dir all –p all.

Create the Open Replicator session after the pair file:

1. Create Open Replicator Volume Pair:

symdev=ID:SymDevID wwn=LUNWWNN

2. Create Open Replicator session:

symcli-server# symrcopy –sid ID –file OR.txt create –copy –hot –pull –name OR

2.3. Migration Process

2.3.1. Actions over servers and SAN

2.3.1.1. Stop resources groups and applications Stop all the resources groups, stop HAMCP (see 2.3.1) or Veritas Cluster, and

comment the veritas cluster boot file:

1. Stop VCS through GUI or command line:

node1-node2# hagrp –state

node1-node2# hagrp –offline < resources_group> -sys <node>

2. Change veritas cluster boot file and if I/O fencing if it’s necessary:

node1-node2# cd /etc/rc.d/rc2.d

node1-node2 root:/etc/rc.d/rc2.d# mv S99vcs _S99vcs

node1-node2 root:/etc/rc.d/rc2.d # mv S97vxfen _S97vxfen

3. Delete line “UseFence=SCSI3” in file main.cf for both nodes of the cluster.

If there are processes that start out of the cluster, stop them with the scripts included in

/etc/rc.shoutdown and comment them in the /etc/inittab file.

2.3.1.2. Delete disks and uninstall previous drivers Unmount file systems, deactivate all devices groups:

//HAMCP (see 2.3.1)

4. Delete disks in the Veritas configuration previous deleting hdisks:

node1-node2# vxdisk rm <device>

//HAMCP or VCS once deleted hdisks corresponding to IBM devices:

//Previous driver: SDD

5. Uninstall SDD driver:


node1-node2# rmdev –dl fcsX –R

node1-node2# lssrc –s sddsrv

node1-node2# stopsrc –s sddsrv

node1-node2# lsdev –C | grep dpo

node1-node2# rmdev –dl dpo

node1-node2# /usr/lib/instl/sm_inst installp_cmd -u -f'devices.sdd.53.rte'

node1-node2# /usr/lib/instl/sm_inst installp_cmd -u -f'devices.fcp.disk.ibm.rte’

//Previous driver: SDD-PCM

6. Uninstall SDD-PCM driver:

node1-node2# rmdev –dl fcsX –R

node1-node2# lssrc –s pcmsrv

node1-node2# stopsrc –s pcmsrv

node1-node2# /usr/lib/instl/sm_inst installp_cmd -u -f'devices.sdd.53.rte'

node1-node2# /usr/lib/instl/sm_inst installp_cmd -u -f'devices.fcp.disk.ibm.rte’

2.3.1.3. Actions over the SAN Deactivate zones from both servers to IBM array and reboot both servers. It’s desirable

to start the operating system without any disk or multipathing software.

2.3.1.4. Install and configure EMC software Deactivate zones from both servers to IBM array and reboot both servers. It’s desirable

to start the operating system without any disk or multipathing software.

//Update FIRMWARE

1. Check the version:

node1-node2# lsmcode –d fcsX

2. Update:

node1-node2# diag

Task Selection (Diagnostics, Advanced Diagnostics, Service Aids, etc.) --> Microcode Tasks --> Download Latest Available Microcode --> Enter another directory --> /SOFTWARE/EMC/... --> Select the hbas that need the update.

//Install ODM, PowerPath and change the HBA parameters.

3. Update parameters of the HBAs:

node1-node2# chdev -l fcsX -a init_link=pt2pt –P

node1-node2# chdev -l fscsiX -a dyntrk=yes -a fc_err_recov=fast_fail –P

node1-node2# shutdown -Fr

//Once installed EMC software and reboot, check there are no disks managed by both servers.


2.3.2. Actions over arrays

Activate Open Replicator session and zones from both servers to DMX array.

Check the status of the session. It’s desirable to run the masking when the copy

reaches more than 1% or 2% for each device. If performance problems appear, change

the speed of the copy.

2.3.3. Actions over servers

Finally, as it is a hot pull session, we can activate volume groups and mount file

systems without having the copy completed.

//HAMCP (see 2.3.3)

//VCS

1. Rescan disks through Veritas Cluster and run powerpath:

node1-node2# cfgmgr

node1-node2# lspv

node1-node2# powermt config

node1-node2# powermt display

node1-node2# vxdisk scandisks

node1-node2# vxdisk list

node1-node2# varyonvg vg

node1-node2# chvg -g vg //If the size of the destination volumes is bigger than sources ones.

2.3.3.1. Heartbeat re-configuration If the heartbeat isn’t replicated by Open Replicator and a new disk of 1GB (or less) is

given to both servers, it’s important to create a new quorum volume group:

//New quorum disk (not replicated)

1. Quorum_vg recreation. Check major number availability in both nodes.

node1-node2# lvlstmajor //Choose the common number in both nodes.

node1# mkvg -V ZZ -y quorum_vg hdiskX

node1# chvg -c quorum_vg //Mode Enahnced-concurrent configuration.

node1# chvg -a n quorum_vg //Not to be activated automatically.

node1# lsvg quorum_vg

node1# varyoffvg quorum_vg

node2# chdev -a pv=yes -l hdiskX //PVID activation over new heartbeat in the second node.


node2# importvg -V 44 -y quorum_vg hdiskX //Volume group import in node2. This warning will appear:

synclvodm: No logical volumes in volume group quorum_vg.

quorum_vg

0516-783 importvg: This imported volume group is concurrent capable. Therefore, the volume group must be varied on manually.

After that, it’s necessary to reconfigure the heartbeat:

2. Delete only one previous heartbeat:


Smitty HACMP Extended Configuration --> Extended Topology Configuration --> Configure HACMP Communication Interfaces/Devices --> Remove Communication Interfaces/Devices:

Node / Network

Interface/Device IP Label/Device Path IP Address

NODE2 / Network_Administration

en7 NODE2-adm X.X.X.X

NODE2 / Network_Service

en6 NODE2 X.X.X.X

NODE2 / Network_disk (none) disk-NODE2 /dev/vpath

NODE1 / Network_Administration

en7 NODE1-adm X.X.X.X

NODE1 / Network_Service

en6 NODE1 X.X.X.X

NODE1 / Network_disk (none) disk-NODE1 /dev/vpath

3. Add new disk or migrated disk instead of disk-NODE2:


Smitty HACMP Extended Configuration --> Extended Topology Configuration --> Configure HACMP Communication Interfaces/Devices --> Add Communication Interfaces/Devices --> Add Pre-defined Communication Interfaces and Devices --> Communication Devices --> Network_disk --> Type or select values in entry fields.:


* Device Name [disk-NODE2]

* Network Type diskhb

* Network Name Network_disk

* Device Path [/dev/hdiskpowerX]

* Node Name [NODE2] +

4. Delete disk-NODE1 and add new disk-NODE1 that should correspond with /dev/hdiskpowerY

5. Synchronize both devices:


Smitty HACMP Extended Configuration --> Extended Verification and Synchronization

* Verify, Synchronize or Both [Both] +

* Automatically correct errors found during [Yes] +

verification?

* Force synchronization if verification fails? [No] +

* Verify changes only? [No] +

* Logging [Standard] +


6. Start cluster services:


Smitty HACMP “System Management (C-SPOC)” “Manage HACMP Services” “Start Cluster Services”

7. Check if the resources are online.

Check the OR session status. Once completed, finish the session and create the

replica for the volumes if needed.

2.3.3.2. Boot from SAN configuration Due to customer’s previous architecture, we have to adapt to configure boot from SAN

for AIX servers. As shown in the figure below, we have created a mirror with two

hdiskpowers (one of each DMX) and the local disk will behave as a backup. This

configuration faces two failures: CPD, array or disk failure and fabric failure.

2.3.3.2.1. Prepare Boot from SAN disks

The server should manage one disk from each array and, at the first time, manage one

disk (hdiskpowerX) through Fabric1 (F1) and the other disk (hdiskpowerY) through

Fabric2 (F2).

1. Check if they are bootable:

# bootinfo -B hdiskpowerX

# bootinfo -B hdiskpowerY

// If the command returns 1, the device will be bootable. If not, check if the HBA firmware is installed and configured correctly (it’s possible to check if this disks is visible from the HBA by entering SMS application).

2. Check if the server manage only one path per disk:

# powermt display dev=hdiskpowerX

Pseudo name=hdiskpowerX


Symmetrix ID=Production

Logical device ID=XXXX

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---

### HW Path I/O Paths Interf. Mode State Q-IOs Errors

==============================================================================

0 fscsi0 hdiskX FA 9aA active alive 0 0

# powermt display dev=hdiskpowerY

Pseudo name=hdiskpowerY

Symmetrix ID=Recovery



==============================================================================



==============================================================================

1 fscsi1 hdiskY FA 9aA active alive 0 0

3. Change the status of the disks to “Defined”:

//Veritas Volume Manager

# vxdisk list

# vxdisk rm hdiskpower(s)

//LVM

# lsdev –Cc disk

# rmdev –l hdiskpower(s)

# rmdev –l powerpath0

# chdev -a pv=clear -l hdiskX //If we reuse a previous data disk with this purpose.

# chdev -a pv=clear -l hdiskY

2.3.3.2.2. System copy and mirror configuration

If we have a previous altinst_rootvg, we should make an export so that the operating

system could generate another altinst_rootvg after the completion of the system copy.

After rebooting the system, we must let the rootvg stay over hdiskpowers.

4. Copy the system to one of the SAN disks:

# lspv

# exportvg altinst_rootvg

# alt_disk_copy -nd hdiskX

# shutdown –Fr

5. Let hdiskpowers contain rootvgs:

# pprootdev on

# shutdown –Fr


Once the system is copied to a SAN disk, make the mirror copy to the other disk (disk

that belongs to the recovery array).

//VxVM: after rebooting the server, Veritas will handle and manage the disks again so we should delete only the disk that doesn’t contain the rootvg, in this case, hdiskpowerY.

6. Delete Veritas managing:

# vxdisk rm hdiskpowerY

# extendvg rootvg hdiskpowerY

7. Execute the mirror copy:

# mirrorvg -S rootvg

8. Check the status of the background-copy:

# lsvg –M rootvg | grep stale | wc //Until it reaches 0, it’s desirable not to continue with the process.

9. Once we have copied everything, add hdisks to bootlist:

# bootlist -m normal hdiskX hdiskY

10. Check if they’re properly added:

# bootlist -om normal

11. Fix and bosboot in order to start from SAN in the next reboot:

# pprootdev fix

# bosboot -ad /dev/ipldevice

# ipl_varyon -i

# shutdown –Fr

The final step consists of adding the other path for each disk (masking and zoning):

12. Rescan for new disks:

# cfgmgr

# powermt display dev=hdiskpowerX

Pseudo name=hdiskpowerX

Symmetrix ID=Production



==============================================================================



==============================================================================

0 fscsi0 hdiskX FA 9aA active alive 0 0

1 fscsi1 hdiskW FA 8aA active alive 0 0

# powermt display dev=hdiskpowerY

Pseudo name=hdiskpowerY

Symmetrix ID=Recovery



==============================================================================




==============================================================================

1 fscsi1 hdiskY FA 8aA active alive 0 0

0 fscsi0 hdiskZ FA 9aA active alive 0 0

13. Add new hardware paths alternatively to bootlist:

# bootlist -m normal hdiskX hdiskY hdiskW hdiskZ


3. Considerations about Oracle ASM

3.1. Migration with Oracle ASM and AIX LVM

When migrating to an Oracle ASM architecture, it’s important to notice that Oracle uses

disks from both arrays (IBM Production and IBM Recovery). To avoid data

inconsistency, data should be moved from IBM Production to DMX Production and

from IBM Recovery to DMX Recovery separately.

But, there are several disks that need special care: OCR disks and VOTE disks. To

make Oracle work properly, it needs three 1GB disks (VOTE) from three different

arrays and four 1GB disks (OCR), two from each array that contains data disks. When

migrating OCR and VOTE disks, it’s important to create a relationship for Oracle’s

administrator: which destination devices correspond with which source devices. Once

migrated, it’s necessary to set reserve_lock=none and configure the permissions as in

the original configuration for OCR and VOTE disks. This parameter could be checked

through the command lsattr –EHl hdiskX.

survey on procedures for migrating data between storage arrays

Documents