11gr2 deployment guide

8/10/2019 11gr2 Deployment Guide

1/51

Oracle Solutions Engineering

Wes Vaske

Deploying Oracle RAC 11g R2 Database

on RHEL 6.x / Oracle Linux 6.x

This document providesinstructions for deploying Oracle Database on

Red Hat Enterprise Linux 6 or Oracle Linux 6


2/51

Deploying Oracle RAC 11g R2 Database on RHEL 6.x / Oracle Linux 6.x

ii

This document is for informational purposes only and may contain typographical errors and technical

inaccuracies. The content is provided as is, without express or implied warranties of any kind.

2012 Dell Inc. All rights reserved. Dell and its affiliates cannot be responsible for errors or omissions in

typography or photography. Dell, the Dell logo, and PowerEdge are trademarks of Dell Inc. Intel and Xeon areregistered trademarks of Intel Corporation in the U.S. and other countries. Microsoft, Windows, and Windows

Server are either trademarks or registered trademarks of Microsoft Corporation in the United States and/or other

countries. Other trademarks and trade names may be used in this document to refer to either the entities claiming

the marks and names or their products. Dell disclaims proprietary interest in the marks and names of others.

January 2012| Version 1.0

April 2013| Version 1.1


3/51


iii

Contents

Overview ................................................................................................................. 5Hardware and software requirements .............................................................................. 5

Hardware requirements .............................................................................................. 5

Memory requirements ............................................................................................... 5

Disk space requirements ............................................................................................ 5

Network Requirements .............................................................................................. 6

Operating System Requirements .................................................................................... 6

Preparing Nodes for Oracle Installation ............................................................................. 6

Attaching to RHN/ULN Repository .................................................................................. 6

Installing the Dell Oracle Preinstall RPM .......................................................................... 8

Installing the Dell Oracle Utilities RPM ............................................................................ 8

Oracle Software Binary Location .................................................................................... 9

Setting up the Network ............................................................................................... 9

Public Network ....................................................................................................... 9

Private Network .................................................................................................... 10

IP Address and Name Resolution Requirements ................................................................ 12

For a Cluster Using DNS ........................................................................................... 12

Preparing Shared Storage for Oracle RAC Installation ........................................................ 13

Setting up Device Mapper Multipath ............................................................................ 14

Partitioning the Shared Disk ..................................................................................... 16

Identifying ASM Disks to Setup Udev Rules **Applies to RHEL 6 and Oracle Linux 6 (if not using

ASMlib)** ............................................................................................................. 17

Installing and Configuring ASMLib **Applies ONLY to Oracle Linux 6 and if not setting udev rules**

........................................................................................................................ 18

Using ASMLib to Mark the Shared Disks as Candidate Disks **Applies ONLY to Oracle Linux 6 and if

not setting udev rules** ........................................................................................... 18

Installing Oracle 11g R2 Grid Infrastructure ..................................................................... 19Before You Begin .................................................................................................... 19

Configure the System Clock Settings for All Nodes ............................................................ 19

Configuring Node One ............................................................................................... 20

Configuring Shared Oracle Home using ACFS..................................................................... 28

Installing Oracle 11g R2 Database (RDBMS) Software .......................................................... 31


4/51


iv

Creating Diskgroup Using ASM Configuration Assistant (ASMCA) ............................................ 37

Creating Database Using DBCA .................................................................................... 39

Tables

Table 1. Swap Space Requirements ................................................................................ 5

Table 2. Disk Space Requirements ................................................................................. 5

Table 3. Cluster Requirements for DNS ......................................................................... 12


5/51


5

Overview

This document applies to Oracle Database 11g R2 running on Red Hat Enterprise Linux 6.x AS x86_64 or

Oracle Enterprise Linux 6.x AS x86_64.

Hardware and software requirementsHardware requirements

For Dell supported hardware configurations, see the Software Deliverable List (SDL) for each Dell

Validated Component at the Current Release Page:en.community.dell.com/techcenter/enterprise-

solutions/w/oracle_solutions/1429.current-release.aspx.

Memory requirements

1. Oracle requires 1.5 gigabytes (GB) of physical memory.

2. Swap space is proportional to the amount of RAM allocated to the system as indicated in the table

below.

Swap Space RequirementsTable 1.

Ram Swap Space

Up to 2GB 1.5 times the size of RAM

Up to 16GB Equal to the size of RAM

16GB or more 16GB

3. Oracle's temporary space (/tmp) must be at least 1 GB in size.

4. A monitor that supports resolution of 1024 x 768 to correctly display the Oracle Universal Installer

(OUI)

Disk space requirements

The following table describes the disk space required for an Oracle Database installation:

Disk Space RequirementsTable 2.

Software Installation

Location

Size Required

Grid Infrastructure Home 4.5GB

Oracle Database Home 4 GB

Shared Storage DiskDatabase size + FlashbackRecovery Area (FRA)
http://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspx


6/51


6

Network Requirements

1. It is recommended that each node contain at least three network interface cards (NICs). One NIC

for public network and two NICs for private network to ensure high availability of the Oracle RAC

cluster.

2. Public and private interface names must be the same on all nodes. For example, if em1 is used as

the public interface on node one, then all other nodes require em1 as the public interface.

3. All public interfaces for each node should be able to communicate with all nodes within the

cluster.

4. All private interfaces for each node should be able to communicate with all nodes within the

cluster.

5. The hostname of each node must follow the RFC 952 standard (ietf.org/rfc/rfc952.txt). Hostnames

that include an underscore ("_") are not permitted.

6. Each node in the cluster requires the following IP address:

o One public IP address

o

Two private IP address

o One virtual IP address

o Three Single Client Access Name (SCAN) addresses for the cluster

Operating System Requirements

Red Hat Enterprise Linux 6.x AS x86_64 (Kernel 2.6.32-71 or higher)

Oracle Linux 6.x AS x86_64 (Kernel 2.6.32-71 or higher)

Preparing Nodes for Oracle Installation

Attaching to RHN/ULN Repository

NOTE: The documentation provided below discusses how to setup a local yum repository using your

operating system installation media. If you would like to connect to the RHN/ULN channels, see the

appropriate documentation. For Red Hat see,redhat.com/red_hat_network.For information relating

to ULN network see,linux.oracle.com.

The recommended configuration is to serve the files over http using an Apache server (package name:

httpd). This section discusses hosting the repository files from a local filesystem storage. While other

options to host repository files exist, they are outside of the scope of this document. It is highly

recommended to use local filesystem storage for speed and simplicity of maintenance.

1. One of the requirements is to have the DVD image mounted either by physical media or by ISO

image.

a. To mount the DVD, insert the DVD into the server and it should auto-mount into the / medi a

directory.

b. To mount an ISO image we will need to run the following command as root, substituting the

path name of your ISO image for the field myI SO. i so:
http://www.ietf.org/rfc/rfc952.txthttp://www.ietf.org/rfc/rfc952.txthttp://www.ietf.org/rfc/rfc952.txthttp://www.redhat.com/red_hat_networkhttp://www.redhat.com/red_hat_networkhttp://www.redhat.com/red_hat_networkhttp://linux.oracle.com/http://linux.oracle.com/http://linux.oracle.com/http://linux.oracle.com/http://www.redhat.com/red_hat_networkhttp://www.ietf.org/rfc/rfc952.txt


7/51


7

[ r oot @l ocal host ~] # mkdi r / medi a/ myI SO

[ r oot @l ocal host ~] # mount - o l oop myI SO. i so / medi a/ myI SO

2. To install and configure the http daemon, configure the machine that will host the repository for

all other machines to use the DVD image locally. Create the file

/ et c/ yum. r epos. d/ l ocal . r epoand enter the following:

[ l ocal ] name=Local Reposi t ory

baseur l =f i l e: / / / medi a/ myI SO/ Ser ver

gpgcheck=0

enabl ed=0

3. Now we will install the Apache service daemon with the following command which will also

temporarily enable the local repository for dependency resolution:

[ r oot@l ocal host ~] # yum - y i nst al l ht t pd - - enabl er epo=l ocal

4.

After the Apache service daemon is installed, start the service and set it to start up for us next

time we reboot. Run the following commands as root:

[ r oot @l ocal host ~] # ser vi ce ht t pd st ar t

[ r oot @l ocal host ~] # chkconf i g ht t pd on

5. To use Apache to serve out the repository, copy the contents of the DVD into a published web

directory. Run the following commands as root (make sure to switch myISO with the name of your

ISO)commands:

[ r oot@l ocal host ~] # mkdi r / var/ www/ html / myI SO

[ r oot@l ocal host ~] # cp - R / medi a/ myI SO/ * / var / www/ html / myI SO

a. This step is only necessary if you are running SELinux on the server that hosts the repository.

The following command should be run as root and will restore the appropriate SELinux

context to the copied files:[ r oot @l ocal host ~] # r estorecon - Rvv / var / www/ ht ml / .

6. The final step is to gather the DNS name or IP of the server that is hosting the repository. The DNS

name or IP of the hosting server will be used to configure your yum repository repo file on the

client server.

The following is the listing of an example configuration using the RHEL 6.x Server media and is held

in the configuration file: / et c/ yum. r epos. d/ myRepo. r epo

[ myRepo]

name=RHEL6. 2 DVD

baseurl = ht t p: / / r eposer ver . mydomai n. com/ RHEL6_2/ Server

enabl ed=1

gpgcheck=0

NOTE: Replace reposerver.mydomain.com with your server's DNS name or IP address.

NOTE: You can also place the configuration file on the server hosting the repository for all other

servers such that it can also use the repository as a more permanent solution to what was done in

step 2 .


8/51


8

Installing the Dell Oracle Preinstall RPM

Once your nodes have attached to the appropriate yum repository, we will need to install the Dell-

Oracle-RDBMS-Server-11gR2-Preinstall RPM package. The Dell-Oracle-RDBMS-Server-11gR2-

Preinstall RPM package automates certain pieces of the installation process required for the installation

of Oracle RAC or Oracle single instance.

The process to install the Dell Oracle Preinstall RPM package is as follows:

1. Download the latest Dell Oracle Deployment tar file fromen.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspx

NOTE: The filename will follow the convention: Dell-Oracle-Deployment-OS-version-year-

month.tar, for example: Dell-Oracle-Deployment-Lin-EL6-2012-08.tar

2. Copy the Dell Oracle Deployment tar file to a working directory of all your cluster nodes.

3.

To go to your working directory, enter the following command:

[ r oot @l ocal host ~] # cd

4.

Untar the Dell-Oracle-Deployment release using the command:

[ r oot@l ocal host ~] # t ar zxvf Del l - Or acl e- Depl oyment - o- v- y- m. t ar . gz

NOTE: Where, o is the operating system, v is the operating system version, y is the year, and m is

the month of the tar release.

5. Change directory to Dell-Oracle-Deployment-o-v-y-m

6. Install the Dell-Oracle-RDBMS-Server-11gR2-Preinstall RPM package on all your cluster nodes using

the following command:

[ r oot @l ocal host ~] # yum l ocal i nst al l del l - or acl e- r dbms- ser ver - 11gR2-

pr ei nst al l * - - nogpgcheck

Installing the Dell Oracle Utilities RPM

The Dell Oracle utilities RPM is designed to do the following Dell and Oracle recommended settings:

Create Grid Infrastructure directories, set ownership, and permissions.

Create grid user.

Create Oracle Database (RDBMS) directories, set ownership, and permissions.

Create the Oracle base directories, set ownership, and permissions.

Set pam limits within /etc/pam.d/login

Setup /etc/profile.

Set SELinux to Disabled.

Install the Dell PowerEdge system component drivers if applicable.

Set kernel parameters.

Set nproc for grid user within /etc/security/limits.conf

To install the RPM:
http://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspx


9/51


9

1. Download the latest Dell Oracle Deployment tar file fromen.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspx

NOTE: The filename will follow the convention: Dell-Oracle-Deployment-OS version-year-

month.tar.gz, for example: Dell-Oracle-Deployment-Lin-2011-07.tar.gz

2. Copy the Dell Oracle Deployment tar file to a working directory of all your cluster nodes.

3.

Go to your working directory via the command:

[ r oot @l ocal host ~] # cd

4.

Untar the Dell-Oracle-Deployment release using the command:

[ r oot@l ocal host ~] # t ar zxvf Del l - Or acl e- Depl oyment - o- y- m. t ar. gz

NOTE: Where, o is the operating system version, y is the year, and m is the month of the tar

release.

5. Change directory to Dell-Oracle-Deployment-o-y-m

6.

Install the Dell oracle utilities RPM package on all your cluster nodes the by typing:

[ r oot @l ocal host ~] # yum l ocal i nst al l del l - or acl e- ut i l i t i es* - - nogpgcheck

7. Once the rpm is installed, run the dodeploy script to setup the environment as follows:

[ r oot @l ocal host ~] # dodepl oy - g - r 11gR2

For more information about the Dell oracle utilities RPM and its options, check the man pages using

the command:

[ r oot @l ocal host ~] # man 8 dodepl oy

NOTE: The Dell-Oracle-Deplyoment tar contains the latest supported drivers provided from our

Software Deliverable List (SDL). Consult the README file found within the Dell-Oracle-Deployment

tar for installation instructions of the latest drivers.

Oracle Software Binary Location

The Oracle software binaries should be located on node 1 of your cluster. It is important to note that

starting with Oracle 11g R2 (11.2.0.3), Oracle Database patch sets are full installation of the Oracle

software. For more information on how this impacts future Oracle deployments, see My Oracle Support

note: 1189783.1 Important Changes to Oracle Database Patch Sets Starting with 11.2.0.2.

Setting up the Network

Public Network

NOTE: Ensure that the public IP address is a valid and routable IP address.

NOTE: Ensure to disable NetworkManager via commands:

[ r oot @l ocal host ~] # ser vi ce NetworkManager s t op

[ r oot @l ocal host ~] # chkconf i g NetworkManager of f

To configure the public network on each node:
http://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspxhttp://en.community.dell.com/techcenter/enterprise-solutions/w/oracle_solutions/1429.current-release.aspx


10/51


10

1. Log in as root.

2. Edit the network device file / et c/ sysconf i g/ net wor k- scr i pt s/ i f cf g- em#

where # is the number of the network device:

NOTE: Ensure that the Gateway address is configured for the public network interface. If the

Gateway address is not configured, the Oracle Grid installation may fail.

DEVI CE=em1

ONBOOT=yes

NM_CONTROLLED=no

I PADDR=

NETMASK=

BOOTPROTO=st at i c

HWADDR=

SLAVE=no

GATEWAY=

3. Edit the / et c/ sysconf i g/ net wor kfile, and, if necessary, replace l ocal host . l ocal domai n

with the qualified public node name. For example, the command for node 1:host name=node1. domai n. com

4. Restart the network service.

[ r oot @l ocal host ~] # servi ce net work r estart

5. Type i f conf i gto verify that the IP addresses are set correctly.

6. To check your network configuration, ping each public IP address from a

client on the LAN that is not a part of the cluster.

7. Connect to each node via SSH to verify that the public network is functioning

[ r oot @l ocal host ~] # ssh

Private Network

NOTE: Each of the two NIC ports for the private network must be on separate PCIe buses.

The grid infrastructure of Oracle 11gR2 (11.2.0.3) supports IP failover natively using a new feature

known as Redundant Interconnect. Oracle uses its ora.cluster_interconnect.haip resource to

communicate with Oracle RAC, Oracle ASM, and other related services. The Highly Available Internet

Protocol (HAIP) has the ability to activate a maximum of four private interconnect connections. These

private network adapters can be configured during the initial install process of Oracle Grid or after the

installation process using the oi f cf gutility.

Oracle Grid currently creates an alias IP (also known as virtual private IP) on your private network

adapters using the 169.254.*.* subnet for the HAIP. If subnet range is already in use, Oracle Grid will

not attempt to use it. The purpose of HAIP is to load balance across all active interconnect interfaces,

and failover to other available interfaces if one of the existing private adapters becomes unresponsive.

NOTE: Configure a different subnet for each of the private networks you want to configure as a part

of HAIP. When adding additional HAIP addresses after the installation of Oracle Grid (maximum of

four), restart your Oracle Grid environment to make these new HAIP addresses active.


11/51


12/51


12

i net addr : 192. 168. 11. 53 Bcast : 192. 168. 11. 255 Mask: 255. 255. 255. 0

i net 6 addr : f e80: : 224: e8f f : f e6b: e95c/ 64 Scope: Li nk

UP BROADCAST RUNNI NG MULTI CAST MTU: 1500 Met r i c: 1

RX packet s: 12768 err ors: 0 dr opped: 0 overr uns: 0 f r ame: 0

TX packet s: 2211 er r or s: 0 dropped: 0 over r uns: 0 car r i er : 0

col l i si ons: 0 t xqueuel en: 1000

RX byt es: 2163856 ( 2. 0 Mi B) TX byt es: 144187 ( 140. 8 Ki B)I nter r upt : 32 Memory: da000000- da012800

em3: 1 Li nk encap: Ether net HWaddr 00: 24: E8: 6B: E9: 5C

i net addr : 169. 254. 167. 163 Bcast : 169. 254. 255. 255 Mask: 255. 255. 0. 0

UP BROADCAST RUNNI NG MULTI CAST MTU: 1500 Met r i c: 1

l o Li nk encap: Local Loopback

i net addr : 127. 0. 0. 1 Mask: 255. 0. 0. 0

i net 6 addr : : : 1/ 128 Scope: Host

UP LOOPBACK RUNNI NG MTU: 16436 Met r i c: 1

RX packet s: 28662 err ors: 0 dr opped: 0 overr uns: 0 f r ame: 0

TX packet s: 28662 er r or s: 0 dropped: 0 over r uns: 0 car r i er : 0

col l i si ons: 0 t xqueuel en: 0

RX byt es: 1604049 ( 1. 5 Mi B) TX byt es: 1604049 ( 1. 5 Mi B)

For more information on Redundant Interconnect and ora.cluster_interconnect.haip, see metalink

note: 1210883.1.

IP Address and Name Resolution Requirements

The steps below show how to setup your cluster nodes for using Domain Name System (DNS). For

information on how to setup cluster nodes using GNS, see the wiki article

en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1416.aspx

For a Cluster Using DNS

To set up an Oracle 11g R2 RAC using Oracle DNS (without GNS):

1.

At least two interfaces must be configured on each node, one for the private IP address and one for

the public IP address.

2. A SCAN NAME configured on the DNS for Round Robin resolution to three addresses (recommended)

or at least one address. The SCAN addresses must be on the same subnet as virtual IP addresses and

public IP addresses.

NOTE: For high availability and scalability, it is recommended that you configure the SCAN to use

Round Robin resolution to three IP addresses. The name for the SCAN cannot begin with a

numeral. For installation to succeed, the SCAN must resolve to at least one address.

The table below describes the different interfaces, IP address settings and the resolutions in a

cluster.

Cluster Requirements for DNSTable 3.

Interface Type Resolution
http://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1416.aspxhttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1416.aspxhttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1416.aspx


13/51


13

Interface Type Resolution

Public Static DNS

Private Static Not Required

Node virtual IP Static DNS

SCAN virtual IP Static DNS

Configuring a DNS Server

To configure changes on a DNS server for an Oracle 11g R2 cluster using a DNS (without GNS):

1. Configure SCAN NAME resolution on DNS server. A SCAN NAME configured on the DNS server using

the Round Robin policy should resolve to three public IP addresses (recommended), however the

minimum requirement is one public IP address.

For example:scancl ust er I N A 192. 0. 2. 1

I N A 192. 0. 2. 2

I N A 192. 0. 2. 3

Where scancluster is the SCAN NAME provided during Oracle Grid

installation.

NOTE: The SCAN IP address must be routable and must be in public range.

Configuring a DNS Client

1. Configure the resolv.conf on all the nodes in the cluster to contain name

server entries that are resolvable to the appropriate DNS server. Provide anentry similar to the following:

/ et c/ resol v. conf :

sear ch ns1. domai nserver . com

nameserver 192. 0. 2. 100

Where, 192.0.2.100 is a valid DNS server address in your network and ns1.domainserver.com is the domain

server in your network.

2. Verify the order configuration. / et c/ nsswi t ch. confcontrols the name service order. In some

configurations, the NIS can cause issues with Oracle SCAN address resolution. It is recommended

that you place the NIS entry at the end of the search list and place the dns entry first. For

example,

hosts: dns f i l es ni s

3. Once you have modified the / et c/ nsswi t ch. conf, restart the nscd service by issuing the

command:

[ r oot@l ocal host ~] # / sbi n/ ser vi ce nscd r est ar t

Preparing Shared Storage for Oracle RAC Installation


14/51


14

NOTE: In this section, the terms disk(s), volume(s), virtual disk(s), LUN(s) mean the same and are used

interchangeably, unless specified otherwise. Similarly, the terms Stripe Element Size and Segment

Size both can be used interchangeably.

Oracle RAC requires shared LUNs for storing your Oracle Cluster Registry (OCR), voting disks, Oracle

Home using ACFS, Oracle Database files, and Flash Recovery Area (FRA). To ensure high availability for

Oracle RAC it is recommended that you have:

Three shared volumes/LUNs each of 1 GB in size for normal redundancy or five volumes/LUNs

for high redundancy for the Oracle clusterware.

At least two shared disks to store your database. Each shared disk should be the same disk

speed and size.

At least two shared volumes/LUNs to store your Automatic Storage Management Cluster File

System (ACFS). Each shared disk must be at least 10 GB, for a total size of 20 GB.

At least two shared volumes/LUNs or volumes to store your FRA. Ideally, the FRA space should

be large enough to copy all of your Oracle datafiles and incremental backups. For more

information on optimally sizing your FRA, see My Oracle Support ID 305648.1 section "Whatshould be the size of Flash Recovery Area?"

NOTE: The use of device mapper multipath is recommended for optimal performance and

persistent name binding across nodes within the cluster.

NOTE: For more information on attaching shared LUNs/volumes, see the Wiki documentation

found at:en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/3-

storage.aspx

Setting up Device Mapper Multipath

The purpose of Device Mapper Multipath is to enable multiple I/O paths to improve performance andprovide consistent naming. Multipathing accomplishes this by combining your I/O paths into one device

mapper path and properly load balancing the I/O. This section will provide the best practices on how to

setup your device mapper multipathing within your Dell PowerEdge server.

1. Verify that your device-mapper and multipath driver are at least the version shown below or

higher:

[ r oot @l ocal host ~] # r pm - qa | gr ep devi ce- mapper - mul t i path

devi ce- mapper - mul t i pat h- 0. 4. 9- 46. el 6

2. Identify your local disks i.e. /dev/sda. Once your local disk is determined run the

following command to obtain its scsi_id:

[ r oot@l ocal host ~] # scsi _i d - - page=0x83 - - whi t el i st ed - - devi ce=/ dev/ sda

360026b900061855e000007a54ea53534

3.

Open the /etc/multipath.conf file and locate and comment out the section below:

#bl ackl i st {

# devnode *

#}
http://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/3-storage.aspxhttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/3-storage.aspxhttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/3-storage.aspxhttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/3-storage.aspxhttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/3-storage.aspx


15/51


15

4. Once the scsi_id of your local disk has been retrieved, you must blacklist this scsi_id from being

used as a multipath device within /etc/multipath.conf file and locate,uncomment,and modify the

section below:

bl ackl i s t {

wwi d

devnode " ( r am| r aw| l oop| f d| md| dm- | sr | | scd| st ) [ 0- 9] *"devnode " hd[ a- z] "

}

5. Uncomment your defaults section within your / et c/ mul t i pat h. conf:

def aul t s {

udev_di r / dev

pol l i ng_i nt er val 10

sel ect or "r ound- r obi n 0"

path_groupi ng_pol i cy mul t i bus

get ui d_cal l out " / sbi n/ scsi _i d - g - u - s/ bl ock/ %n"

pri o_cal l out / bi n/ t rue

pat h_checker r eadsect or0r r _mi n_i o 100

max_f ds 8192

r r_wei ght pr i or i t i es

f ai l back i mmedi ate

no_pat h_r et r y f ai l

user_ f r i endl y_names yes

}

6.

Locate the multipaths section within your / et c/ mul t i pat h. conffile. In this section you will

provide the scsi_id of each LUN/volume and provide an alias in order to keep a consistent naming

convention across all of your nodes. An example is shown below:

mul t i paths {

mul t i path {

wwi d

al i as al i as_of _vol ume1

}

mul t i path {

wwi d

al i as al i as_of _vol ume2

}

}

7.

Restart your multipath daemon service using:

[ r oot @l ocal host ~] # ser vi ce mul t i pat hd rest ar t

8. Verify that your multipath volumes alias are displayed properly:

[ r oot @l ocal host ~] # mul t i path - l l

9.

Make sure iSCSI service starts upon boot using the command:

[ r oot @l ocal host ~] # chkconf i g mul t i pat hd on


16/51


16

10.Repeat steps 1-9 for all nodes.

Partitioning the Shared Disk

This section describes how to use Linuxs native partition utility fdisk to create a single partition on a

volume/virtual disk that spans the entire disk.

To use the fdisk utility to create a partition:

1. At the command prompt, type one of the following:

[ r oot @l ocal host ~] # f di sk cu / dev/

[ r oot @l ocal host ~] # f di sk cu / dev/ mapper /

Where, is the name of the block device that you are

creating a partition on. For example, if the block device is / dev/ sdb, type:

[ r oot @l ocal host ~] # f di sk cu / dev/ sdb

Devi ce cont ai ns nei t her a val i d DOS part i t i on t abl e, nor Sun, SGI or OSF

di skl abelBui l di ng a new DOS di skl abel wi t h di sk i dent i f i er 0x89058e03.

Changes wi l l r emai n i n memory onl y, unt i l you deci de to wr i t e t hem.

Af t er t hat , of cour se, t he pr evi ous cont ent won' t be r ecover abl e.

War ni ng: i nval i d f l ag 0x0000 of par t i t i on t abl e 4 wi l l be cor r ect ed by

w( r i t e)

The devi ce present s a l ogi cal sect or si ze t hat i s smal l er t han

t he physi cal sect or si ze. Al i gni ng t o a physi cal sect or ( or opt i mal

I / O) si ze boundary i s r ecommended, or per f ormance may be i mpact ed.

2. Enter the following commands:

Command (m f or hel p) : n #n to create new partition

Command act i one ext ended

p pr i mar y par t i t i on ( 1- 4) :

p #p for primary parition

Par t i t i on number ( 1- 4) : 1 #1 for first partition

Fi r st sect or ( 4096- xxxxxx, def aul t 4096) :

Usi ng def aul t val ue 1

Last cyl i nder or +si zeM or +si zeK ( 1- ###, def aul t ###) :

3. Repeat steps 1-2 for all the disks.

4. Type the following to re-read the partition table to see the newly created partition(s)

[ root @l ocal host ~] # par t pr obe

or

[ r oot@l ocal host ~] # ser vi ce mul t i pat hd r est ar t

or

[ r oot @l ocal host ~] # kpart x a / dev/ mapper /

5. Reboot the system if your newly created partition is not displayed properly.


17/51


17

Identifying ASM Disks to Setup Udev Rules **Applies to RHEL 6 and Oracle Linux 6 (if not using

ASMlib)**

Red Hat Enterprise Linux 6/Oracle Linux 6 has the ability to use udev rules to ensure that the system

properly manages permissions of device nodes. In this case, we are referring to properly setting

permissions for our LUNs/volumes discovered by the OS. It is important to note that udev rules are

executed in enumerated order. When creating udev rules for setting permissions, please include theprefix 20- and append . rul esto the end of the filename. An example file name is 20-

del l _oracl e. r ul es

In order to set udev rules, you need capture the WWIDs of each disk to be used within your ASM device

using the scsi _ i dcommand:

[ r oot @l ocal host ~] # scsi _i d - - page=0x83 - - whi t el i st ed - - devi ce=/ dev/ sdX

where sdX is the name of your block device.

If you need to run this command to capture multiple WWIDs you can use the following f or loop via the

shell:

[ r oot @l ocal host ~] # f or i i n sdb sdc sdd sde; do \ pr i nt f "%s %s\ n" "$i "

"$(scsi _i d - - page=0x83 - - whi t el i st ed - - devi ce=/ dev/ $i ) "; done

Out put :

sdb 360026b900061855e000008a54ea5356a

sdc 360026b9000618571000008b54ea5360b

sdd 360026b900061855e000008a54ea5356a

sde 360026b9000618571000008b54ea5360b

Once the WWIDs have been captured, create a file within the / et c/ udev/ r ul es. d/ directory and

name it 20- del l _or acl e. r ul es. A separate KERNEL entry must exist for each storage device and

will require adding the WWID to the "RESULT==" field.

An example of what needs to be placed in the / et c/ udev/ r ul es. d/ 20- del l _or acl e. r ul esfile

#- - - - - - - - - - - - - - - - - - - - - - - - s t ar t udev rul e cont ent s - - - - - - - - - - - - - - - - - - #

KERNEL=="dm- *" , PROGRAM="scs i _i d - - page=0x83 - - whi t el i st ed - -

devi ce=/ dev/ %k" , RESULT=="360026b9000618571000008b54ea5360b", OWNER: ="gr i d" ,

GROUP: ="asmadmi n"

KERNEL=="dm- *" , PROGRAM="scs i _i d - - page=0x83 - - whi t el i st ed - -

devi ce=/ dev/ %k" , RESULT=="360026b900061855e000008a54ea5356a" , OWNER: ="gr i d",

GROUP: ="asmadmi n"

#- - - - - - - - - - - - - - - - - - - - - - - - - - end udev rul e cont ent s - - - - - - - - - - - - - - - - - - #

As you can see from the above, the KERNEL command looks at all dm devices, executes the PROGRAM

which captures the WWID, if the RESULT of the WWID is matched, appropriately assign the grid user as

the OWNER and the asmadmin group as the GROUP.


18/51


18

Installing and Configuring ASMLib **Applies ONLY to Oracle Linux 6 and if not setting udev

rules**

1. Useoracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.htmlto download

the following files:

o

oracleasm-support

o

oracleasmlib

o

oracleasm

NOTE: If your current OS distribution is Oracle Linux, you can obtain the software from theUnbreakable Linux Network using ULN.

NOTE: Download the latest versions of oracleasm-support and oracleasmlib but the version oforacleasm must match the current kernel used in your system. Check this information issuing

the command uname - r .

2. Enter the following command as root:

[ r oot @l ocal host ~] # r pm - Uvh oracl easm- support - * \

or acl easml i b- * \ oracl easm- $( uname - r ) - *

NOTE: Replace * by the correct version numbers of the packages or you can leave them in place of

the command ensuring that there are no multiple versions of the packages in the shell's current

working directory.

Using ASMLib to Mark the Shared Disks as Candidate Disks **Applies ONLY to Oracle Linux 6

and if not setting udev rules**

1. To configure ASM use the init script that comes with the oracleasmsupport package. The

recommended method is to run the following

command as root:

[ r oot @l ocal host ~] # / usr / sbi n/ or acl easm conf i gur e i

NOTE: Oracle recommends using the oracleasm command found under /usr/sbin. The

/etc/init.d path has not been deprecated, but the oracleasm binary provided by Oracle in this

path is used for internal purposes.

Def aul t user t o own t he dr i ver i nt er f ace [ ] : gr i d

Def aul t gr oup t o own t he dr i ver i nt er f ace [ ] : asmadmi n

St ar t Or acl e ASM l i br ar y dr i ver on boot ( y/ n) [ n ] : y

Fi x per mi ssi ons of Or acl e ASM di sks on boot ( y/ n) [ y ] : y

NOTE: In this setup the default user is set to grid and the default group is set to asmadmin.

Ensure that the oracle user is part of the asmadmin group. You can do so by using the dell-

validated and dell-oracle-utilities rpms.

The boot time parameters of the Oracle ASM library are configured and a sequential text interface

configuration method is displayed.

2. Set the ORACLEASM_SCANORDERparameter in / et c/ sysconf i g/ or acl easm

NOTE: When setting the ORACELASM_SCANORDER to a value, specify a common string associated

with your device mapper pseudo device name. For example, if all the device mapper device had a
http://www.oracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.htmlhttp://www.oracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.htmlhttp://www.oracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.htmlhttp://www.oracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.html


19/51


19

prefix string of the word "asm", (/dev/mapper/asm-ocr1,/dev/mapper/asm-ocr2), populate

the ORACLEASM_SCANORDER parameter as:ORACLEASM_SCANORDER="dm". This would ensure that

oracleasm will scan these disks first.

3. Set the ORACLEASM_SCANEXCLUDEparameter in / et c/ sysconf i g/ or acl easmto exclude non-

multipath devices.

For example: ORACLEASM_SCANEXCLUDE=

NOTE: If we wanted to ensure to exclude our single path disks within /dev/ such as sda and sdb,

our ORACLEASM_SCANEXCLUDE string would look like: ORACLEASM_SCANEXCLUDE="sda sdb"

4.

To create ASM disks that can be managed and used for Oracle database installation, run the

following command as root:

[ r oot @l ocal host ~] # / usr / sbi n/ oracl easm cr eatedi sk DI SKNAME

/ dev/ mapper / di skpar t i t i on

NOTE: The fields DISKNAME and /dev/mapper/diskpartition should be substituted with the

appropriate names for your environment respectively.

NOTE: It is highly recommended to have all of your Oracle related disks to be within Oracle ASM.This includes your OCR disks, voting disks, database disks, and flashback recovery disks.

5. Verify the presence of the disks in the ASM library by running the following command as root:

[ r oot @l ocal host ~] # / usr/ sbi n/ or acl easm l i st di sks

All the instances of DISKNAME from the previous command(s) are displayed.

6.

To delete an ASM disk,run the following command:

[ r oot @l ocal host ~] # / usr / sbi n/ oracl easm del et edi sk DI SKNAME

7. To discover the Oracle ASM disks on other nodes in the cluster, run the following command on the

remaining cluster nodes:

[ r oot@l ocal host ~] # / usr / sbi n/ oracl easm scandi sks.

Installing Oracle 11g R2 Grid Infrastructure

This section gives you the installation instructions for Oracle 11g R2 in a cluster.

Before You Begin

Before you install the Oracle 11g R2 RAC software on your system:

Ensure that you have already configured your operating system, network, and storage based on thesteps from the previous sections within this document.

Locate your Oracle 11g R2 media kit.

Configure the System Clock Settings for All Nodes

To prevent failures during the installation procedure, configure all the nodes with identical system

clock settings. Synchronize your node system clock with the Cluster Time Synchronization Service

(CTSS) which is built in Oracle 11g R2. To enable CTSS, disable the operating system network time

protocol daemon (ntpd) service using the following commands in this order:


20/51


20

[ r oot @l ocal host ~] # servi ce nt pd st op

[ r oot @l ocal host ~] # chkconf i g nt pd of f

[ r oot @l ocal host ~] # mv / et c/ nt p. conf / et c/ nt p. conf . or i g

[ r oot @l ocal host ~] # r m / var / r un/ nt pd. pi d

Configuring Node One

The following steps are for node oneof your cluster environment, unless otherwise specified.

1. Log in as root.

2. If you are not in a graphical environment, start the X Window System by typing: s tartx

3. Open a terminal window and type: xhost +

4. Mount the Oracle Grid Infrastructure media.

5. Log in as grid user, for example: su - gr i d.

6. Type the following command to start the Oracle Universal Installer:

/ r unI nst al l er 7. In the Download Software Updates window, enter your My Oracle Support credentials to download

the latest patch updates. If you choose not to download the latest patches, select Skip software

updates.

8. In the Select Installation Option window, select Install and Configure Grid Infrastructure for a

Cluster and click Next.

Select Installation OptionsFigure 1.

9. In the Select Installation Type window, select Advanced Installation option, and click Next.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/8156.installation_5F00_option_5F00_screen.jpg


21/51


21

Select Installation TypeFigure 2.

10. In the Select Product Languages window, select English, and click Next.

11. In the Grid Plug and Play Information window, enter the following information:

a. Cluster NameEnter a name for your cluster.

b. SCAN NameEnter the named registered in the DNS server which is unique for the entire

cluster. For more details on setting up your SCAN name see, "IP Address and Name Resolution

Requirements". For details on enabling GNS, see the wiki article:en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1416.aspx.

c. SCAN PortRetain the default port of 1521.

d. Configure GNSUncheck this option.

e. Click Next.

Grid Plug and Play InformationFigure 3.

12.

In the Cluster Node Information window, click Add to add additional nodes that must be managed

by the Oracle Grid Infrastructure.

a. Enter the public Hostname information

b. Enter the Virtual IP name

c. Repeat step 12 for each node within your cluster
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3426.instllation_5F00_type_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3426.instllation_5F00_type_5F00_screen.jpghttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1416.aspxhttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1416.aspxhttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/1614.grid_5F00_plug_5F00_n_5F00_play_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3426.instllation_5F00_type_5F00_screen.jpghttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1416.aspx


22/51


22

Cluster Node InformationFigure 4.

13. Click SSH Connectivity and configure your passwordless SSH connectivity by entering the OS

Password for the grid user and click Setup.

NOTE: The default password set by the dell-validated and dell-oracle-utilities rpms is 'oracle' for

both the grid user and oracle user.

SSH ConnectivityFigure 5.

14.Click Ok and then click Next to go to the next window.

15. In the Network Interface Usage window, make sure that the correct interface types are selected

for the interface names.

16. From the Interface Type drop-down list, select the required interface type. The available options

are Private, Public, and Do Not Use then click Next.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5670.cluster_5F00_node_5F00_info_5F00_screen1.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5670.cluster_5F00_node_5F00_info_5F00_screen1.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/6518.cluster_5F00_node_5F00_info_5F00_screen2.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5670.cluster_5F00_node_5F00_info_5F00_screen1.jpg


23/51


23

Specify Network Interface UsageFigure 6.

17.

In the Storage Option Information window, select Automatic Storage Management (ASM) and click

Next.

Storage Option InformationFigure 7.

18. In the Create ASM Disk Group window, enter the following information:

a. ASM diskgroup Enter a name, for example: OCR_VOTE

b. Redundancy For your OCR and voting disks, select High if five ASM disks are available, select

Normal if three ASM disks are available, or select External if one ASM disk is available (not

recommended).

NOTE: If no candidate disks are displayed, click Change Discovery Path and enter ORCL:*or

/dev/oracleasm/disks/*. Ensure that you have marked your Oracle ASM disks, for more

informations see,"Using ASMLib to Mark the Shared Disks as Candidate Disks".
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3438.network_5F00_interface_5F00_usage.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3438.network_5F00_interface_5F00_usage.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3386.storage_5F00_option_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3438.network_5F00_interface_5F00_usage.jpg


24/51


24

Create ASM Disk GroupFigure 8.

19. In the Specify ASM Password window, choose the relevant option under Specify the passwords for

these accounts and enter the relevant values for the password. Click Next.

Specify ASM PasswordFigure 9.

20. In the Failure Isolation Support window, select Do Not use Intelligent Platform Management

Interface (IPMI). For information on enabling IPMI, see wiki article,

en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1414.aspx
http://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1414.aspxhttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1414.aspxhttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/0513.asm_5F00_pw_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3022.create_5F00_asm_5F00_dg_5F00_screen.jpghttp://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/1414.aspx


25/51


25

Failure Isolation SupportFigure 10.

21.

In the Privileged Operating Systems Groups window, select:

a.

asmdba for Oracle ASM DBA (OSDBA for ASM) Groupb.

asmoper for Oracle ASM Operator (OAOPER for ASM) Group

c. asmadmin for Oracle ASM Administrator (OSASM) Group

Privileged Operating System Groups

22.

In the Installation Location window, specify the values of your Oracle Base and Software Location

as configured within the Dell Oracle utilities RPM.

NOTE: The default locations used within the Dell Oracle utilites RPM are:

Oracle Base -/u01/app/grid

Software Location - /u01/app/11.2.0/grid
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/0535.ipmi_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/0535.ipmi_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/1234.os_5F00_groups_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/0535.ipmi_5F00_screen.jpg


26/51


26

Specify Installation LocationFigure 11.

23. In the Create Inventory window, specify the location for your Inventory Directory. Click Next.

Create InventoryFigure 12.

NOTE:The default location based on the Dell Oracle utilites RPM for Inventory Directory

is/u01/app/oraInventory

24.

In the Perform Prerequisite Checks window, check the overall status of all the prerequisites. If anyof the prerequisites fail and have the status as Fixable, click Fix & Check Again button and execute

the runfixup.sh script provided by the Oracle Universal Installer (OUI).

Perform Prerequisite ChecksFigure 13.

NOTE: If there are other prerequisites that contain the status Error, repeat step 23 else select

Ignore All, if proper requirements have been met and Error status still persists after all changes

have been fixed.

25.

In the Summary window, select Install.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7271.install_5F00_loc_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7271.install_5F00_loc_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/0647.prereq_5F00_checks_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7506.inventory_5F00_sceen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7271.install_5F00_loc_5F00_screen.jpg


27/51


27

SummaryFigure 14.

26. After the installation is complete, the Execute Configuration Scripts wizard is displayed. Complete

the instructions in the wizard and click Ok.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4532.summary_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4532.summary_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4532.summary_5F00_screen.jpg


28/51


28

Execute Configuration ScriptsFigure 15.

27. In the Finish window, click Close.

Configuring Shared Oracle Home using ACFS

NOTE: Please review the following My Oracle Support document ACFS Supported On OS Platforms. [ID

1369107.1] as depending on your kernel version, a patch might be required for ACFS support.

The following steps are for node 1 of your cluster environment unless otherwise specified:

1. Login as root and type: xhost +

2. Login as grid user and run the asmca utility by typing:

[ r oot @l ocal host ~] # su gr i d

[ gr i d@l ocal host ~] # $/ bi n/ asmca
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3531.exec_5F00_config_5F00_script_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3531.exec_5F00_config_5F00_script_5F00_screen.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3531.exec_5F00_config_5F00_script_5F00_screen.jpg


29/51


29

ASM Configuration AssistantFigure 16.

3.

In the ASM Configuration Assistant window, select the Disk Groups tab, click Create, and perform

the following steps:

a. Enter a name of the disk group. For example, ORAHOME.

b. Select the External Redundancy, and then select the ASM stamped disk that you want to use

for the shared database home.

NOT If no candidate disks are displayed, click Change Discovery Path and

enterORCL:* or /dev/oracleasm/disks/*

NOT Ensure that you have marked your Oracle ASM disks. For more information, see "Using

ASMLib to Mark the Shared Disks as Candidate Disks".
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5861.asmca1.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5861.asmca1.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5861.asmca1.jpg


30/51


30

Create Disk GroupFigure 17.

4.

Click Ok.

5. Right-click the disk group you have created for the shared Oracle home, and select Create ACFS for

Database Home.

6. The Create ACFS Hosted Database Home Volume option is displayed.

a. Enter name for the volume (for example, ORAHOME.)

b. Enter name for the mount point for Database Home (for example:

/u01/app/oracle/acfsorahome).

c. Enter the Database Home Size (must equal at least 20 GB).

d. Enter the name of the Database Home Owner. (for example: oracle).

e.

Enter the name of the Database Home Owner Group (for example: oinstall).

f. Click Ok.


31/51


31

Create Database HomeFigure 18.

7.

As root, run the acf s_scr i pt . shmentioned in the RUN ACFS Script window. This automounts the

new ACFS Home on all nodes.

Run ACFS ScriptFigure 19.

8. Click Close to exit ACFS script window.

Installing Oracle 11g R2 Database (RDBMS) Software

The following steps are for node 1 of your cluster environment, unless otherwise specified.

1.

Log in as root and type: xhost +.

2.

Mount the Oracle Database 11gR2 media.

3. Log out as root user and log in as Oracle user by typing:

su - or acl e
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4606.asmca3.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4606.asmca3.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/6114.asmca4.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4606.asmca3.jpg


32/51


32

4. Run the installer script from your Oracle database media:

/ r unI nst al l er

5. In the Configure Security Updates window, enter your My Oracle

Support credentials to receive security updates, else click Next.

Configure Security UpdatesFigure 20.

6.

In the Download Software Updates window, enter your My Oracle Support credentials to download

patch updates available after the initial release. If you choose not to update at this time, select

Skip software updates and click Next.

Download Software UpdatesFigure 21.

7. In the Select Installation Option window, select Install database software only.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5684.software_5F00_updates.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/8037.configure_5F00_security_5F00_updates.jpg


33/51


33

Installation OptionsFigure 22.

8.

In the Grid Installation Options window:

a. Select Oracle Real Application Clusters database installation and select all the nodes by

clicking the Select All button.

b. Click SSH Connectivity and configure your passwordless SSH connectivity by entering the OS

Password for the oracle user and selecting Setup. Click Ok and click Next to go the next

window.

NOTE: The default password set by the dell-validated and dell-oracle-utilities rpms is oracle

for both the grid user and oracle user.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7723.install_5F00_db_5F00_software.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7723.install_5F00_db_5F00_software.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7723.install_5F00_db_5F00_software.jpg


34/51


34

Grid Installation OptionsFigure 23.

SSH ConnectivityFigure 24.

9. In the Select Product Lanaguages window, select English as the Language Option and click Next.

10. In the Select Database Edition window, select Enterprise Edition and click Next.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/8308.db_5F00_grid_5F00_install.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/8308.db_5F00_grid_5F00_install.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/1067.db_5F00_grid_5F00_ssh.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/8308.db_5F00_grid_5F00_install.jpg


35/51


35

Select Database EditionFigure 25.

11. In the Installation Location window,

a. Specify the location of your Oracle Base configured within the Dell oracle utilities RPM.

b. Enter the ACFS shared Oracle home address for Software Location.

NOTE: The default locations used within the Dell Oracle utilites RPM are as follows:

i. Oracle Base/u01/app/oracle

ii. Software Location/u01/app/oracle/product/11.2.0/db_1

Specify Installation LocationFigure 26.

12. In the Privileged Operating System Groups window, select dba for Database Administrator

(OSDBA) Group and asmoper for Database Operator (OSOPER) Group and click Next.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7840.db_5F00_enterprise_5F00_db_5F00_edition.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7840.db_5F00_enterprise_5F00_db_5F00_edition.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/6318.db_5F00_installation_5F00_loc.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/7840.db_5F00_enterprise_5F00_db_5F00_edition.jpg


36/51


36

Privileged Operating System GroupsFigure 27.

13. In the Perform Prerequisite Checks window, check the overall status of all the prerequisites.

14. If any prerequisites fail and have the status as Fixable, click the Fix & Check Again button.

15. Execute the r unf i xup. shscript provided by the Oracle OUI.

NOTE: If there are other prerequisites that display status Error, repeat step 13, If the Error status

still persists after all changes have been fixed, select Ignore All.

Perform Prerequisite ChecksFigure 28.

16. In the Summary window, select Install.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/0020.db_5F00_os_5F00_sys_5F00_grps.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/0020.db_5F00_os_5F00_sys_5F00_grps.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4152.db_5F00_prereq_5F00_check.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/0020.db_5F00_os_5F00_sys_5F00_grps.jpg


37/51


37

Summary WindowFigure 29.

17. On completion of the installation process, the Execute Configuration scripts wizard is displayed.

Follow the instructions in the wizard and click Ok.

Execute Configuration ScriptsFigure 30.

NOTE: Root.sh should be run on one node at a time.

Creating Diskgroup Using ASM Configuration Assistant (ASMCA)

This section contains procedures to create the ASM disk group for the database files and Flashback

Recovery Area (FRA).
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3414.db_5F00_summary.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3414.db_5F00_summary.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/2744.db_5F00_exec_5F00_config_5F00_script.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/3414.db_5F00_summary.jpg


38/51


38

1. Log in as grid user.

2.

Start the ASMCA utility by typing:

[ gr i d@l ocal host ~] # $/ bi n/ asmca

3.

In the ASM Configuration Assistant window, select the Disk Groups tab.

4.

Click Create.

ASM Configuration Assistant: Configure ASM: Disk GroupsFigure 31.

5.

Enter the appropriate Disk Group Name, for example: DBDG.

6. Select External for Redundancy.

7.

Select the appropriate member disks to be used to store your database

files, for example: ORCL:DB1, ORCL:DB2
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4064.asmca1.jpg


39/51


39

Create Disk GroupFigure 32.

NOTE: If no candidate disks are displayed, click Change Discovery Path and

type: ORCL:* or/dev/oracleasm/disks/*

NOTE: Please ensure you have marked your Oracle ASM disks. For more information, see "Using

ASMLib to Mark the Shared Disks as Candidate Disks".

8.

Click Ok to create and mount the disks.

9.

Repeat step 4 to step 8 to create another disk group for your Flashback Recovery Area (FRA).

NOTE: Make sure that you label your FRA disk group differently than your database disk group

name. For labeling your Oracle ASM disks, see "Using ASMLib to Mark the Shared Disks as Candidate

Disks".

10.

Click Exit to exit the ASM Configuration Assistant.

Creating Database Using DBCA

The following steps are for node 1 of your cluster environment unless otherwise specified:

1. Login as oracle user.

2.

From $, run the DBCA utility by typing:

[ or acl e@l ocal host ~] # $/ bi n/ dbca &

3. In the Welcome window, select Oracle Real Application Cluster Database and click Next.


40/51


40

Database Configuration Assistant: WelcomeFigure 33.

4. In the Operations window, select Create Database, and click Next.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/6811.dbca_5F00_1.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/6811.dbca_5F00_1.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/6811.dbca_5F00_1.jpg


41/51


41

Database Configuration Assistant: OperationsFigure 34.

5. In the Database Templates window, select Custom Database, and click Next.


42/51


42

Database Configuration Assistant: Database TemplatesFigure 35.

6. In the Database Identification window:

a. Select Admin-Managed for Configuration Type.

b. Enter appropriate values for Global Database Name and SID Prefix.

c.

In the Node Selection list box, select All Nodes.

d. Click Next.


43/51


44/51


44

Database Configuration Assistant: Management OptionsFigure 37.

8. In the Database Credentials window, enter the appropriate credentials for your database.


45/51


45

Database Configuration Assistant: Database CredentialsFigure 38.

9. In the Database File Location window, select:

a. Automatic Storage Management (ASM) for Storage Type.

b. Use Oracle-Managed Files for Storage Location.

c.

Browse to select the ASM disk group that you created to store the database files (DBDG)for

Database Area.


46/51


46

Database Configuration Assistant: Database File LocationsFigure 39.

10. In the Recovery Configuration window:

a. Select Specify Flash Recovery Area.

b. Browse and select the ASM disk group that you created for Flash Recovery Area.

c. Enter a value for Flash Recovery Area Size.

d. Select Enable Archiving.

e. Click Next.
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5811.dbca_5F00_7.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5811.dbca_5F00_7.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5811.dbca_5F00_7.jpghttp://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/5811.dbca_5F00_7.jpg


47/51


47

Database Configuration Assistant: Recovery ConfigurationFigure 40.

11. In the Database Content window, click Next.

12.

In the Initialization Parameters window:

a. Select Custom.

b. For the Memory Management section, select Automatic Shared Memory Management.

c. Specify appropriate values for the SGA Size and PGA Size.

d. Click Next.


48/51


48

Database Configuration Assistant: Initialization ParametersFigure 41.

13. In the Database Storage window, click Next.


49/51


49

Database Configuration Assistant: Database StorageFigure 42.

14. In the Creation Options window, click Finish.


50/51


50

Database Configuration Assistant: Creation OptionsFigure 43.

15. In the Summary window, click Ok to create database.


51/51


Create Database: SummaryFigure 44.

NOTE: Database creation can take some time to complete.

16.Click Exit on the Database Configuration Assistant window after the database creation is complete.

Acknowledgements

The original version of this document was written by Roger Lopez and Adam Miller. Additional thanks to

Naveen Iyengar, Phani MV, Balamurugan B, Kai Yu, and Wendy Chen for their contributions
http://en.community.dell.com/cfs-file.ashx/__key/communityserver-wikis-components-files/00-00-00-01-18/4137.dbca_5F00_12.jpg

11gr2 deployment guide

Documents