Unix Profession Webcast October 2007

HP-UX LVM, OnlineJFS and Oracle ASM Basics*Dusan BaljevicSydney, Australia

20091My humble attempt to summarise some best-known features.I used many resources and valuable comments by others.I cannot claim big credits for this presentation.

Disk Partitioning Concepts HP-UXPartitions can be configured using:

The Whole Disk Approach (no volume manager).Logical Volume Manager (LVM).Veritas Volume Manager (VxVM).22Whole Disk Partitioning ConceptsSwapFile SystemFile SystemSwap SpaceRaw SpaceSwapFile SystemBoot AreaThe whole disk approach supports partitioning a disk in five different ways:33Whole Disk Partitioning - Pros Simple to use, almost no Unix knowledge required. No licensing. Supports any type of physical volume.

44Whole Disk Partitioning - Cons Partitions cannot span multiple disks. Each disk can contain at most one file system partition. Partitions cannot be easily extended.55Logical Volume Manager ConceptsHP-UX LVM is much more flexible than the whole disk approach. Introduced in HP-UX 9.0:Partitions/volumes can span multiple disksMultiple partitions/volumes may be configured on a single diskPartitions/volumes can be easily extended and reduced as needs changeLVM is included in all current versions of HP-UXBaseLVM is included with the operating systemLVM MirrorDisk/UX is available for an extra charge or in higher OE releasesPhysicalVolumePhysicalVolumeVolume GroupLogical Volumes667HP-UX File Systems and Volume ManagersLogical VolumeManager (LVM)MirrorDiskUX (MDUX)Veritas VolumeManager (VxVM)Cluster VolumeManager (CVM) Volume Management Solutions Base File System(VxFS-lite)MirrorDiskUX (MDUX)Cluster File System (CFS)File System SolutionsIncludedin VSE, HA, and DCOEIncluded in VSE, HA, and DCOEIncluded in Base OE8HP-UX with Symantec ReleasesHP-UX OS VersionHP-UX 11.23HP-UX 11.31Symantec Release Version3.54.15.04.15.05.0.1 Sep 2009Base ProductsBase-VxFSIn BaseOSBase-VxFSBase-VxFS-50Base-VxFSBase-VxFS-5.0Base-VxFS-5.0

Base-VxVMBase-VxVMBase-VxVMBase-VxVM-50Base-VxVMBase-VxVM-5.0Base-VxVM-5.0Standalone ProductsOnline-JFSB3929DAB3929EAB3929FAB3929EAB3929FBB3929FBVxVM-FULLB9116AAB9116BAB9116CAB9116BAB9116CBB9116CB8Scalability: LVMLVMCurrent L1 LimitsOn Disk L2 LimitsCode L2 LimitsArchImplArchImplTestedMax LV size(64TB) 16TB28428416EB264256TBMax PV size241 (2TB)26426426426416TBMax Volume Groups (VG)256N/AN/A224-1211 2304*Max Logical Volumes (LV)255 232-1232-1211-1 2047Max Physical Volumes (PV)255232-1232 -1232-1232-1 2047Max number of mirrors3254254766Max Nodes for mirroring2N/AN/AN/A1616Max number of Extents/VG216 (64K)264264264264225Max extent size (range)1MB 256MB1MB 4TB1MB4TB1MB4GB1MB4GB1MB 256MBMax stripe width28 way (255)232 -1 way232-1232-1

232-1

511Supported maximumsHP-UX 11i v1/v2HP-UX 11i v3 (L2 Layout Limits)Update 3enhancementMax Volume Groups (VG): Additive of L1 and L2 limits i.e 256 + 2048 = 2304LVM and VxVM FeaturesFeatureHP-UX 11.31LVM, MD/UXVxVM 4.1/5.0 1SAN Boot SupportYY3RAID0YYRAID1MD/UXFULL license4Mirrored Stripes (RAID1+0)MD/UXFULL licenseStriped Mirrors (RAID 0+1)MD/UXFULL licenseMax Volume Size16EB (256TB)2 32TBMax Vol/Volume Group2048 (511) 2UnlimitedMax Volume Groups 2304 (768) 2UnlimitedRootabilityYYOnline Volume ReconfigurationSLVMFULL license5 + CVM5Fast ResyncMD/UXFULL license + VVRCluster AwareY(SLVM)Y(CVM)Static Path Failover (A/P)YYDynamic Path Load balancing (A/A)YFULL license# Mirrors632Dynamic Root Disk (DRD)YY3HP Storage Device SupportALLLimited63rd Party Storage SupportLimited6Limited61 VxVM 4.1 available with 11.31, VxVM 5.0 on 11.31 3 - VxVM 5.0 only 6 Validate device in question with HP or Symantec qualification list: available H2CY074 - Root Mirroring part of BASE http://www.hp.com/products1/serverconnectivity/mass_storage_devices.html2 supported limit 5 For all featureshttp://ftp.support.veritas.com/pub/support/products/VolumeManager_UNIX/277390.pdfLVM and VxVM Features - continuedFeatureHP-UX 11.31LVM, MD/UXVxVM 4.1/5.0Instant SnapshotsNFULL licensePortable Data ContainersNFULL license1Multi-volume File System supportNFULL licenseDynamic Storage Tiering supportNFULL licenseDynamic LUN expansionYYExtended Campus, Metrocluster, and Continental cluster integrationYYDisk group configuration from any nodeYNNumber Of Nodes164 VxVM & CVM Base16/8 FULL License VxVM/CVMStorage Distance Support (standard/Metrocluster)100 km / 300 km100 km / 300 kmRolling UpgradeYY3RAID 5NFULL LicenseCluster lock supportY (disk, lun)NHot RelocationNYHeterogeneous platformsSame commands on AIX, LinuxAIX, Solaris, Linux, Windows1 VxVM 5.0 Only3 As long as no functional or protocol changes (y within 3.x, n between 3.x = 4.x) 2 AIX and Solaris have similar cluster solution4 Not supported for all configurations

Scalability: Expanded file systemN/A = Not architecturally limitedFile systemsHP-UX 11i v1HP-UX 11i v2HP-UX 11i v3VxFS 5.0 - Number of files N/AVxFS5.0 - File size 16TBVxFS 5.0 - File system size 50TBVxFS 5.0 - # Access Control List entries 1024VxFS 4.1 - Number of files N/AN/AVxFS 4.1 - File size 2TB16TBVxFS 4.1 - File system size 32TB32TBVxFS 3.5 - number of filesN/AN/AVxFS 3.5 - File size2TB2TBVxFS 3.5 - File system size2TB32TBVxVM volume32TB32TB32 TBVxVM volumes2^232^232^23VxVM mirrors323232Supported maximums12Disk Space Management Tool ComparisonBoot disk support?Available on other platforms?Can partitions span disks?Online resizing supported?Striping (RAID 0) supported?Mirroring (RAID 1) supported?RAID 0+1 and 1+0 supported?RAID 5 supported?GUI configuration tool available?HP-UX versions supportedOnline backups supported?Dynamic relayout support?* Features indicated by an asterisk may require an extra licenseDynamic multi-pathing support?Whole DiskLVMVxVM10.x,11.x,11i10.x,11.x,11i11iYesYesYessam, smhsam, smhveaSimilarSimilarYesNoYesYesNoYesYesNoYesYesNoNoYes*NoYes*Yes*NoNoYes*No Yes*Yes*NoYes*Yes*NoActive/PassiveActive/Active*1313Benefits of HP-UX Native MultipathingZero-Config. No initial setup or configuration needed. No license or add-on software installation needed.Free. Comes bundled with Core HP-UX. Is not an add-on product. No additional license fee for Native MP.Transport Aware Failover/Failback.Fibre Channel RSCN based failover/failback.Detection and failover of all paths leading to affected component (hba, target port, switch, inter-switch link). Fully parallel path failover/failback and path monitoring.Built into Storage Stack. More responsive and better performing.Exploits both server platform and storage characteristics.1414Benefits of HP-UX Native Multipathing - continuedTested as a core part of HP-UX. Engineered for scalability, reliability, performance, serviceability.Time to Market support for new technologies. Being a core part of HP-UX, new technologies are supported when released (ex : HPVM, VPARs, VSE, new transports like SAS, storage management products, etc).Simplifies Storage Management. Eliminates an entire layer in the IO Stack (add-on MP product) and reduces amount of management overhead for sysadmin. No more need to install, tune, update, monitor add-on MP product.Ideal for Oracle ASM. Oracle ASM provides its own volume management capability and Native MP is a good fit in that environment since the customer does not need to install VxVM solely for DMP capabilities.1515HP-UX Native Multipathing vs. Classic MPClassic MP Product (DMP)Add-on product layered above IO stack.Agnostic of low level SAN and SCSI events.Unaware of transport specific failures (HBA, switch, switch port offline).Discovers failures on IO errors.Needs to issue test IO on IO error.Path Failover initiated following IO error and Test IO failure.Each such path failure detected individually on IO error on each path.Need to ping each failed path to detect recovery.HP-UX 11iv3 Native MPNot an add-on product. Built into the IO stack and HBA drivers are multi-path aware.Aware of SAN and SCSI events and takes advantage of those.Transport Aware. (Knows scope of outage such as HBA, switch, switch port, tgt port).Can failover at the scope of outage. (All paths under HBA, All paths thru the switch, all paths thru the switch port, all paths to the tgt port).SAN Fabric Events signal recovery. Pinging is avoided.Fully parallel failover/failback.1616Product-use application17Volume ManagementLogical Volume Manager (LVM)If you need:VxVMIf you need:Advanced Volume Management solutionInstant SnapshotsCross-platform data sharingQoSS supportRaid 5Hot Relocate/UnrelocateCFS supportNo-cost solutionBasic Volume Management-default Operating Environment installSimple operationSAN bootDisk-mirroring capability with MirrorDisk/UX18Product-use applicationBasic file system functionalityA no cost solutionSimple operationFast file system recoveryDirect I/O performance*Online defragmentationOnline log and file system resizingOracle Database ManagementConcurrent I/O performance*File change logStorage CheckpointsFunctionality for aclustered environmentCluster Volume ManagerDynamic Storage TiersFlashsnap technologyFile SystemsVxFS LiteIf you need:VxFS Full (OnlineJFS)If you need:Cluster File System (CFS)If you need:* Available 10/2009 5.0.1

Oracle ASM vs. Classical Storage Layers

1919Bill Bridge the original architect of Automatic Storage Management. In the Oracle Press title, Oracle ASM, Bill provides a forward where he discusses the problems with using vendor specific OS file systems to manage Oracle datafile placement:for arch logs & backups, OS vendors don't provide shared disk file system2. logical volume managers hide the location of files making it difficult to manage disk I/O and provide good stats3. existing LVMs don't work well with over 100 disks4. OS's and Oracle don't handle databases well that have 1000's of datafiles5. naming becomes difficult with large number of datafiles6. features, and filesystem limitations vary across different OSs7. users at the OS level can touch Oracle files with standard utilities, without Oracle knowingSo, he set out to solve these problems by building Oracle's own filesystem. His intention was to provide these features:1. tightly integrated with Oracle, and work with clusters (then parallel server)2. new storage automatically used, managed as disk unit or disk group3. thousands of disks supported4. files won't have names, and will be hidden from the OSOracle Traditional Options - Raw versus Cooked and With or Without an LVMJuly 17, 2015

Slide courtesy of http://www.toadworld.com/Default.aspx?tabid=67&EntryID=108

20Disk Groups in the ASM Architecture

2121Oracle ASM BasicsAn ASM file system layer is implicitly created within a disk group. ASM provides a vertical integration of the file system and volume manager for Oracle database files.This file system is transparent to users and only accessible through ASM instance, interfacing databases, and ASMs utilities. For example, database backups of ASM files can be performed only with RMAN.Does not completely bypass the O/S I/O stack. It uses the asyncdsk driver in order to perform asynchronous I/O. Without it, the logwriter and dbwriters would be doing direct, synchronous I/O and would not be able to perform acceptably in high IO workloads. The I/O requests also have to pass though the SCSI and FC layers.2222Oracle ASM Basics - continuedThe functionality of an ASM instance can be summarized as follows:Manages groups of disks, called disk groups.Protects the data within a disk group. *Provides near-optimal I/O balancing without any manual tuning.Enables the user to manage database objects such as tablespaces without needing to specify and track filenames.Supports large files.2323* Choices for disk group redundancy include:External: defers to hardware protectionNormal: 2-way mirroringHigh: 3-way mirroring

In addition, ASM failure groups can be created, providing a set of disks sharing a commonresource whose failure needs to be tolerated. Redundant copies of an extent are stored in separate failure groups. ASM failure groups can be assigned by DBAs or automatically by ASM.Oracle ASM Basics - continuedOne can use Oracle Enterprise Manager (EM) or the Database Configuration Assistant (DBCA) for a GUI interface to Automatic Storage Management that replaces the use of SQL or SQL*Plus for configuring and altering disk groups and their metadata.DBCA eases the configuring and creation of the database.EM provides an integrated approach for managing both the ASM instance and database instance.2424Oracle ASM Limits63 disk groups in a storage system.10,000 ASM disks in a storage system.4 Petabyte maximum storage for each ASM disk.40 Exabyte maximum storage for each storage system.140 Petabyte in external redundancy (no ASM mirroring). 42 Petabyte in normal redundancy (2-way ASM mirroring).15 Petabyte in high-redundancy (3-way ASM mirroring).2525NOTE: Oracle database limits file sizes to 128 TB with big file tablespacesand 32k block size, which is considerably lower than the ASM limits.

Oracle ASM LimitsASM disk group that is implemented with External Redundancyhas a maximum file size of 35 TB.ASM Disk Group that is implemented with Normal Redundancy has a maximum file size of 5.8 TB.ASM Disk Group that is implemented with High Redundancy has a maximum file size of 3.9 TB.1 million files for each disk group.2.4 Terabyte maximum storage for each file.2626NOTE: Oracle database limits file sizes to 128 TB with big file tablespacesand 32k block size, which is considerably lower than the ASM limits.

File Types Supported by Automatic Storage ManagementFile Supported Default Templates Control files yes CONTROLFILEDatafiles yes DATAFILERedo log files yes ONLINELOGArchive log files yes ARCHIVELOGTrace files no N/ATemporary files yes TEMPFILEDatafile backup pieces yes BACKUPSETDatafile incremental backup pieces yes BACKUPSETArchive log backup piece yes BACKUPSETDatafile copy yes DATAFILEPersistent initialization parameter file (SPFILE) yes PARAMETERFILEDisaster recovery configurations yes DATAGUARDCONFIGFlashback logs yes FLASHBACKChange tracking file yes CHANGETRACKINGData Pump dumpset yes DUMPSETAutomatically generated control file backup yes AUTOBACKUPCross-platform transportable datafiles yes XTRANSPORTO/S files no N/A

July 17, 201527Oracle ASM - ProsASM provides some file system and volume management capabilities for Oracle database files only. These include DB control files, redo logs, archived redo logs, data files, spfiles and Oracle Recovery Manager (RMAN) backup files (see previous slide).File-level striping/mirroring.Ease of manageability. Instead of running LVM software, run an ASM instance, a new type of "instance" that largely consists of processes and memory and stores its information in the ASM disks it is managing.Attempts to identify the configuration errors. *

2828* 1. A single disk with different mount points is presented to an ASM instance. This can be caused by multiple paths to a single disk. In this case, if the disk in question is part of a disk group, disk group mount fails. If the disk is being added to a disk group with ADD DISK or CREATE DISKGROUP, the command fails. To correct the error, restrict the disk string so that it does not include multiple paths to the same disk.2. Multiple ASM disks, with the same ASM label, passed to separate ASM instances as the same disk. In this case, disk group mount fails.3. Disks that were not intended to be ASM disks are passed to an ASM instance by the discovery function. ASM does not overwrite a disk if it recognizes the header as that of an Oracle object.Oracle ASM - ProsGives Oracle Corporation control over the storage system, which makes them happy, so they promote it heavily. *No large Unix-level administration needed.Provides a single point of support (Oracle) so there is no finger-pointing.Provides easy management of block devices (raw partitions).Automatically moves hot blocks to the outside of the disk.Vendor and operating system neutral.2929* Oracle Automatic Storage Management (ASM) was introduced with Oracle Database 10g.Free, and widely adopted.Oracle ASM - ProsIncluded in the Oracle license so no additional cost for the software or its support.Supports very large disk groups and datafiles.Database File System with performance of RAW I/O.Supports clustering (RAC) and single instance.Automatic data distribution.Memory requirements for an ASM instance are small. 100 MB of RAM is typically all that is required to run an ASM instance on a production server.

3030Oracle ASM - ProsOn-line add/drop/resize disk with minimum data relocation.Automatic file management. *Flexible mirror protection.Inode locks not applicable to ASM.Ability to grow diskgroup capacity on the fly.Supports multiple database instances running on a single host, and does not have its own data dictionary.

3131* ASM uses system generated filenames, otherwise referred to as fully qualified filenames for each file that is under ASM control. This filename gets generated automatically at file creation time. These are also referred to as Oracle Managed Files and will be automaticallydeleted when it is no longer needed.Oracle ASM Pros (Oracle 11gr)Fast mirror resynchronization. *Preferred mirror read in a cluster. **Support for large ALU. ***Variable size extents. ****Rolling upgrade and patching.Table level migration wizard in EM.New ASMCMD commands.New SYSADM privilege separate from the SYSDNA privilege.More flexible FORCE options to MOUNT or DROP disk groups.

3232* Benefits: Fast recovery from transient failures Enables pro-active maintenance** Allows local mirror read operations and eliminates network latencies in extended clusters*** ALUs can be set to 1/2/4/8/16/32/64 MB at disk group creation time. Higher performance for large seq I/O (DW) Better leverage of Hardware RAID read-ahead Set Oracle MAXIO = AU size**** Extent size = AU size up to 20,000 extents Extent size = 8*AU up to 40,000 extents Extent size = 64*AU beyond 40,001 extents Striping Coarse Stripe size always = one AU Fine Stripe size always = 128 KBOracle ASM - ConsASM cannot be used for Oracle executables and non-database files.ASM files can only be managed through an Oracle application such as RMAN. This can be a weakness if you prefer third-party backup software or simple backup scripts. Cannot store CRS files or database software. Cannot manage ASM through standard Unix tools.Potentially disrupts the balance of power between the Unix Systems Administration groups, and the Database/DBA groups. Traditionally the former group manages disks, hardware, and the operating system level, leaving the DBAs to coordinate with them for new resources. This would change that balance, which could cause some resistance. *

3333* ASM enable a DBA to manage storage directly DBA now has to consider:How do I deal with disk failures? Data resilienceHow do I back up storage? Data protectionHow do I make the best use of the raw storage? Storage UtilizationHow do I make it run fast enough? Performance

Oracle ASM - ConsASM does not have multi-pathing capability. It assumes the underlying O/S will provide this functionality. In HP-UX, multi-pathing is provided by a Volume Manager feature such as PVLinks in the HP-UX Logical Volume Manager (LVM), native multipathing in HP-UX 11.31, or DMP in Veritas Volume Manager from Symantec (VxVM), or by other third-party software such as Securepath or Powerpath.ASM is still in the enterprise computing, relatively new. There are a number of vendors whose core business has been in the logical volume manager/file system space for years. Often, maturity matters a lot when it comes to software systems, reliability, and proven success rates.New technology to get familiar with.

3434Oracle ASM - ConsAutomatic Storage Management load balances file activity by uniformly distributing file extents across all disks in a disk group. For this technique to be effective it is important that the disks used in a disk group be of similar performance characteristics. There may be situations where it is acceptable to temporarily have disks of different sizes and performance co-existing in a disk group (for example, when migrating from an old set of disks to a new set of disks). The new disks would be added and the old disks dropped. As the old disks are dropped, their storage is migrated to the new disks while the disk group in online.

3535Oracle ASM - ConsThere is no shared awareness of LUN use between ASM and LVM or VxVM. It means that the system administrator must be careful not to accidentally allocate a LUN already allocated for LVM or VxVM use to ASM use (or vice-versa).ASM is not an enterprise-class file system.ASM is a proprietary solution.The customer is dependant on the reliability of the new ASM code stack. *Does not offer network monitoring.Be careful about ASM hidden parameters.3636* An example of a long-running bug:Oracle ASM holds disk after removing it (Metalink 403526.1).Workaround is to restart instances, or if using Oracle version 10.2.0.4apply patches 7225720 and 7330611 they are required for ServiceGuardsupport too.Oracle ASM - ConsNot for high I/O environments (that is what some tests claim).Everything is single threaded through one process at a very low level. *If one uses Oracle ASM and CRS, they will still require a 3rd party clustering solution to support the non-Oracle data. They will then have to manage multiple clustering solutions.3737* I found this claim somewhere but cannot find its reference now. This needs to be verified!Oracle ASM - ConsDBAs must still watch and then perform the task of adding and removing disks to an ASM disk group when needed. This leads back to the problem of DBAs under allocating, and worse yet, over allocating disk storage, just to be safe, which recreates the problem of wasted space and leads to higher than needed storage costs. This is where thin provisioning comes into play. Thin provisioning will automatically allocate on a just-enough and just-in-time basis which relieves the DBA from both having to watch and then add or remove disk to a disk group. Oracle's ASM and thin provisioning could be combined to offer a complete, end-to-end, storage solution. Oracle's ASM feature would create, allocate, place, and rebalance data files for performance and thin provisioning would dedicate disk space on the fly and only when needed.3838Oracle ASM - ConsConfiguration details and performance metrics are exposed via V$ views. Other possibilities are the command line interface, asmcmd and the graphical interface of OEM.Metadata are however partially hidden to the end user. That is the mapping between physical storage, ASM allocation units, and database files is not completely exposed via V$ views. It was found that is possible to query such information via undocumented X$ tables. For example, it is possible to determine the exact physical location on disk of each extent (or mirror copies of extents) for each file allocated on ASM (and if needed access the data directly via the O/S). This can be used by Oracle DBAs wanting to extend their knowledge of the inner workings of the ASM or wanting to diagnose hotspots and ASM rebalancing issues.3939What about Non-Database Files?Since ASM supports only database data files and log files, the following storage management methods are required for non-database files if ASM is used:A) A local file system or Cluster File System for Oracle Clusterware binaries and configuration files and RAC binaries and configuration files.B) Shared storage for Oracle Clusterware data: voting disk and OCR files. This storage has to be configured either as shared raw devices, shared raw volumes, or files in the Cluster File System. Oracle Clusterware needs to be up and running before the ASM daemon can start. Therefore, ASM cannot be used for Oracle Clusterware data.

4040Some of the Oracle Plans and Efforts *Elimination of Symantec by destroying Veritas Cluster as a viable product (replaced with CRS), and the same for Veritas File System (replaced by ASM).Elimination of Sun by adopting Linux as a low-cost alternative to Solaris (Sun has now been acquired by Oracle who has been primarily responsible for its downfall).Attempted elimination of Red Hat by migrating Red Hat Linux under Oracle Enterprise Linux (limited success so far).Challenge to EMC and NetApp in the area of storage using the Oracle Exadata Storage (version 1 was with HP and just-announced version 2 on Intel X86 and Linux).Changes in licensing structure.Acquisition of Sun Microsystems.

4141This slide simply points out the facts that Oracle istrying to run the show alone as much as possible.

Capability compareBusiness BenefitVendorProduct(s)Oracle executable and non-database filesMulti-pathing capabilityStrong I/O fencing Single-instance Database object storage RAC Database object storage Near raw database performanceHP OnlineJFS+LVM/MirrorDisk/UX (Single Instance only) YesYesYesYesnoYesHP OnlineJFS+VxVM/ODM Storage Management for Oracle (Single Instance only) YesYesYesYesnoYesHP Cluster file system(both RAC & Single Instance) Yes YesYesy Yesy HP No file system raw volumesNoNoYesNoYesYES it is raw volume (RAC ONLY) Oracle Automatic Storage Management (RAC & SI) NoNoNo*NoYesYes(limited) 4242Capability compare - continuedBusiness BenefitVendorProduct(s)Ability to use standard UNIX commands to move or copy dataProven, time-tested quality and reliabilityAll disaster-recovery needs can be metEase of management (e.g., backup and database maintenance) HP OnlineJFS+LVM/MirrorDisk/UX (Single Instance only) YesYesYesy HP OnlineJFS+VxVM/ODM (Storage Management for Oracle)(Single Instance only) YesYesYesy HP Cluster file system(both RAC & Single Instance) y y HP No file system raw volumesn y y n (RAC ONLY) Oracle Automatic Storage Management (RAC & SI) NoNoNoy 4343Oracle ASM versus HP-UX SLVM ** ASM lacks Multipathing: The ASM-over-SLVM configuration provides multipathing for ASM disk groups (using LVM PV Links or storage based multipathing). This is not an issue on HP-UX 11i v3 (native multipathing).ASM-over-SLVM enables the HP-UX devices used for disk group members to have the same names on all nodes, easing ASM configuration. Generally, when using raw disk devices with ASM, the device files are created under /dev/oracle, so they can be the same on all nodes.ASM-over-SLVM protects ASM data against inadvertent overwrites from nodes inside/outside the cluster. If the ASM disk group members are raw disks, there is no protection currently against these disks being setup in LVM or VxVM volume/disk groups. *There are some disadvantages too. **

4444* LVM pvcreate(1M) recognises the ASM disks and does not destroy themunless the -f (force) flag is used.** Disadvantages:- Additional configuration and management tasks introduced by the extra layer of volume management (volume groups, logical volumes, and physical volumes);- Performance impact of extra layer of volume management.Oracle Discontinues Raw/Block Device SupportIn release 11.2, the Oracle Installer and DBCA (Database Configuration Assistant) will no longer support raw/block devices for database files. In addition, there will no longer be raw/block device support for storing the OCR and Voting Disks for newinstalls. Customers who create a new 11.2 database will need to store their database files in either ASM, a file system, or on an NFS filer. RAC database files must be stored on ASM, a certified clustered file system, or a certifed NFS filer.

4545As stated in Metalink note 578455.1, Oracle plans to fully desupport RAW/Block device storage effective with the next major release following 11.2. At this time, customers will need to migrate any data files stored on RAW/Block devices to ASM, a cluster file system, or NFS. Thus, we recommend new databases not be deployed on RAW/Block devices. For more information on migrating from RAW to ASM, please see :

http://www.oracle.com/technology/deploy/availability/pdf/maa_wp_10gr2_asmmigrationwithdg.pdfOracle ASM on HP-UX Some ExperiencesIt was found Oracle ASM was a good choice to simplify Oracle related storage and volume managementHowever do not use ASM redundancy features

The HP-UX 11v3 MSS (Mass Storage Stack) and Oracle ASM were a killer combo.

ASM works very well for Oracle data management, but a severe performance penalty gets incurred if the ASM redundancy features are used on top of the storage array redundancy features.

4646One of HP-UX Solutions* Currently this is offered through the purchase of a Storage Management Suite bundleHP-UX file systems can provide read/write performance 95-98% asfast as a raw-device setup.*OJFS 5.0.1: performance impact virtually eliminated.The manageability and reduced administration from afile system, with near-raw performance.47*Currently, HP offers advanced file system performance from its Storage Management Suite bundleswhich include Oracles Database Manager (ODM)

After October, 2009 Direct I/O capability will be enabled in the base file system and Concurrent I/O will be enable in OnlineJFS (which is included in all but the Base OE).47HP-UX 11iv3 Mass Storage StackTransparent native multipathing to LUNs is useful and necessary decreases work and rework.Load balancing increased performanceleast command load policy provided the best performance for this BI workloadround robin policy, the default, provided slightly less performancePersistent LUN bindings and LUN DSFs reduce design and maintenance efforts significantly for mid-range and high-end BI implementations containing large storage subsystems.4848Storage management is extremely important fro BI customer due to the large volume of storage consumed by the BI workload. MSS greatly simplifies this task.Large Enterprise Example: Due to the work involved with migrating FC HBA to new cabinet in the SD they declined to do so and missed a good opportunity to balance the IO across cabinets and achieve better IO distribution and performance. MSS and v3 would have made the task much easier to do, and more likely for the customer to do.

Some Best Practices for Oracle ASM with HP StorageWorksAs a result of testing, a set of best practices for using Oracle ASM with HP StorageWorks on HP-UX servers is presented: Configure ASM disk groups to use external redundancy. When building a disk group or adding to an existing one, use disks of similar capacity and performance characteristics in the same disk group. To leverage I/O distribution across as many resources as possible, it is best to present more than one LUN to a disk group (allowing ASM to do the striping). Use HP Secure Path with ASM on HP-UX 11.23 or MSS (Mass Storage Stack) on HP-UX 11.31 because it complements the high availability and performance of the entire stack.4949Some Best Practices for Oracle ASM with HP StorageWorks - continued Each device (LUN) should be managed either by Oracle ASM or by HP-UX LVM, but not both. Care should be taken not to attempt inadvertently to manage an ASM disk by a traditional volume manager or vice versa. Configure async I/O (Please consult the Oracle Administration Guide documentation). Use "insf" instead of "insf -e" to create the devices associated with new hardware because "insf -e" will reset the ownership to user "bin.

5050