emc vspex for virtualized oracle database 12c oltp - dell emc · oracle database 12c ... emc vspex...

DESIGN GUIDE

EMC VSPEX FOR VIRTUALIZED ORACLE DATABASE 12c OLTP Enabled by EMC VNXe and EMC Data Protection

VMware vSphere 5.5 Red Hat Enterprise Linux 6.4

EMC VSPEX

Abstract

This Design Guide describes how to design virtualized Oracle database resources in an EMC® VSPEX® Proven Infrastructure for VMware vSphere that uses EMC VNXe® and EMC Data Protection storage. This document also describes how to size Oracle Database 12c on VSPEX, allocate resources following best practices, and use all the benefits that VSPEX offers.

February 2015

2 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled By VNXe and EMC Data Protection Design Guide

Copyright © 2015 EMC Corporation. All rights reserved. Published in the USA.

Published February 2015

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

The information in this publication is provided “as is”. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

EMC VSPEX for Virtualized Oracle Database 12c Enabled by VNXe and EMC Data Protection Design Guide

Part Number: H12968.2

http://www.emc.com/legal/emc-corporation-trademarks.htm

http://www.emc.com/legal/emc-corporation-trademarks.htm

Contents

3 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection

Design Guide

Contents

Chapter 1 Introduction 7Purpose of this guide .................................................................................................. 8

Business value ........................................................................................................... 8

Scope ......................................................................................................................... 9

Audience .................................................................................................................... 9

Terminology................................................................................................................ 9

Chapter 2 Before You Start 11Deployment workflow ............................................................................................... 12

Essential reading ...................................................................................................... 13

VSPEX Implementation Guide .............................................................................. 13

VSPEX Proven Infrastructure Guide ...................................................................... 13

EMC Data Protection for VSPEX Guide ................................................................... 13

Chapter 3 Solution Overview 15Overview .................................................................................................................. 16

EMC VSPEX Proven Infrastructure .............................................................................. 16

Solution architecture ................................................................................................ 18

Key components ....................................................................................................... 19

Introduction ......................................................................................................... 19

Oracle Database 12c............................................................................................ 19

VMware vSphere 5.5 ............................................................................................ 20

EMC VNXe3200 .................................................................................................... 20

Red Hat Enterprise Linux 6.4 ................................................................................ 22

EMC Data Protection solutions ............................................................................. 22

Chapter 4 Choosing a VSPEX Proven Infrastructure 23Overview .................................................................................................................. 24

Step 1: Evaluate the customer use case.................................................................... 24

Step 2: Design the application architecture .............................................................. 25

Step 3: Select the right VSPEX Proven Infrastructure ................................................. 27

Considerations .................................................................................................... 27

Examples ............................................................................................................. 28

Chapter 5 Solution Design Considerations and Best Practices 31Overview .................................................................................................................. 32

Contents

4 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide

Designing the network .............................................................................................. 32

Overview .............................................................................................................. 32

SAN best practices ............................................................................................... 32

IP network best practices ..................................................................................... 32

vSphere network best practices ........................................................................... 32

Oracle settings for NFS-specific considerations .................................................... 33

Designing the storage layout .................................................................................... 33

Overview .............................................................................................................. 33

Storage layout and design considerations ........................................................... 34

Storage best practices ......................................................................................... 35

VSPEX storage layout example ............................................................................. 35

Configuring FAST Cache for Oracle ............................................................................ 36

Overview .............................................................................................................. 36

FAST Cache best practices ................................................................................... 37

Configuring FAST VP for Oracle .................................................................................. 37

Overview .............................................................................................................. 37

FAST VP best practices ......................................................................................... 38

Designing the virtualization layer .............................................................................. 38

Overview .............................................................................................................. 38

Virtualization best practices ................................................................................ 38

Designing the Oracle Database 12c implementation ................................................ 40

Overview .............................................................................................................. 40

Best practices for ASM and database deployment ............................................... 40

Best practices for dNFS configuration .................................................................. 40

Manage automatic shared memory ...................................................................... 41

Enable the HugePages setting ............................................................................. 41

Configure I/O operations for file system files ....................................................... 41

Configure the database data type layout .............................................................. 42

EMC Data Protection design considerations .............................................................. 43

Chapter 6 Solution Verification Methodologies 45

Overview .................................................................................................................. 46

Verifying the solution ............................................................................................... 46

Understand key metrics ....................................................................................... 46

Chapter 7 Reference Documentation 49

EMC documentation ................................................................................................. 50

Other documentation ............................................................................................... 51

Oracle documentation ......................................................................................... 51

Contents


Design Guide

Appendix A Qualification Worksheet 53

VSPEX for Virtualized Oracle OLTP qualification worksheet ....................................... 54

Gather the information from the customer’s Oracle databases example ................... 54

Printing the qualification worksheet ......................................................................... 57

Appendix B High-level Oracle Database Server Sizing Logic and Methodology 59

Overview .................................................................................................................. 60

Resource considerations .......................................................................................... 60

Sizing considerations ............................................................................................... 60

OS capacity resources and IOPS .......................................................................... 61

Determine the I/O workload and the disk type, number, and capacity ................. 61

Example 1: Homogenous pools without FAST suite .............................................. 62

Example 2: Sizing with FAST Cache ...................................................................... 64

Example 3: Sizing with FAST VP ............................................................................ 66

Figures Figure 1. VSPEX Proven Infrastructure ................................................................ 17

Figure 2. Architecture of the validated infrastructure .......................................... 18

Figure 3. Example: Storage layout of Oracle 12c Database on VMware ............... 34

Figure 4. Storage layout example: Oracle Server for VNX series with FAST VP/Cache enabled ............................................................................... 36

Figure 5. init.ora Parameters from the AWR Report ............................................. 55

Figure 6. IOStat by function summary from the AWR Report ................................ 56

Figure 7. Foreground Wait Event from the AWR report ......................................... 56

Figure 8. Transactions in a load profile from the AWR report ............................... 57

Figure 9. Printable qualification worksheet ........................................................ 57

Tables Table 1. Terminology........................................................................................... 9

Table 2. VSPEX workflow for virtualized Oracle Database deployment ............... 12

Table 3. VSPEX Proven Infrastructure selection steps ........................................ 24

Table 4. VSPEX for virtualized Oracle OLTP qualification worksheet guidelines . 24

Table 5. VSPEX Sizing Tool output ..................................................................... 26

Table 6. Oracle sizing model for virtual machines ............................................. 26

Table 7. Select the right VSPEX Proven Infrastructure ........................................ 28

Table 8. Qualification worksheet example: Multiple small-sized Oracle 12c servers ................................................................................................ 28

Table 9. Required resources example: Multiple small-sized Oracle 12c servers 29

Contents


Table 10. Recommendations for the storage layout example: Multiple small-sized Oracle 12c servers ............................................................................... 29

Table 11. Qualification worksheet example: Large-sized Oracle 12c server ......... 30

Table 12. Required resources example: Large-sized Oracle 12c server ................ 30

Table 13. Recommendations for the storage layout example: Large-sized Oracle 12c server ........................................................................................... 30

Table 14. Oracle Sever storage pools example .................................................... 34

Table 15. Database layout for a consolidated Oracle environment example ........ 42

Table 16. High-level steps for application verification ......................................... 46

Table 17. Key metrics .......................................................................................... 47

Table 18. Qualification worksheet for virtualized Oracle OLTP databases ............ 54

Table 19. Example of user input for multiple Oracle databases ........................... 61

Table 20. Required CPU and RAM for different size databases ............................ 61

Table 21. RAID type, write penalty, and capacity utilization ................................. 62

Table 22. Random disk IOPS and bandwidth by drive type .................................. 62

Table 23. Storage pool calculation example without FAST Cache ........................ 63

Table 24. FAST Cache hit rate and workload calculations .................................... 64

Table 25. Storage pool calculation example with FAST Cache ............................. 65

Table 26. Example workload for a two-tier FAST VP pool capacity ........................ 66

Table 27. Storage pool calculation Example with FAST VP ................................... 67

Chapter 1: Introduction


Design Guide

Chapter 1 Introduction

This chapter presents the following topics:

Purpose of this guide ................................................................................................. 8

Business value ........................................................................................................... 8

Scope ......................................................................................................................... 9

Audience .................................................................................................................... 9

Terminology ............................................................................................................... 9



Purpose of this guide

EMC® VSPEX® Proven Infrastructures are optimized for virtualizing business-critical applications. VSPEX provides modular solutions built with technologies that enable faster deployment, greater simplicity, greater choice, higher efficiency, and lower risk.

VSPEX provides partners with the ability to design and implement the virtual assets that are required to support a fully integrated virtualization solution for an Oracle relational database management system (RDBMS) on a VSPEX private cloud infrastructure.

The VSPEX for virtualized Oracle infrastructure provides customers with a modern system capable of hosting a virtualized database solution that is scalable and delivers a constant performance level. This solution uses VMware vSphere backed by the EMC VNXe® storage array. The compute and network components, while vendor-definable, are designed to provide redundancy and sufficient power to handle the processing and data needs of the virtual machine environment.

This Design Guide describes how to design the resources necessary to deploy Oracle Database 12c in a VSPEX Proven Infrastructure that uses EMC VNXe storage arrays. This guide also describes using the EMC VSPEX Sizing Tool to select the correct VSPEX Proven Infrastructure. This guide applies to Oracle online transaction processing (OLTP) workloads but does not cover data warehousing workloads.

Business value

Software for database management systems is used by many different types of businesses. Despite the increasing market share of other data management tools, growth in sales is expected to continue and will accelerate as customers continue to diversify their infrastructures and supporting technologies and use more hardware and software appliances and configurations.

This VSPEX Proven Infrastructure is focused on helping EMC partners understand the value that the VNXe series and Oracle bring to customers who often have growing, isolated IT environments running server-centric applications, facing increased Oracle backup and recovery issues.

This VSPEX solution is designed to meet Oracle database challenges and enable customers to increase performance, scalability, reliability, and automation. By consolidating database applications on EMC VNXe, customers can use a single centralized storage platform to manage exploding data growth more effectively. This solution has been sized and proven by EMC to:

Deploy systems faster, saving time and effort with EMC Proven Solutions

Increase performance and scalability

Minimize storage requirements and reduce costs



Design Guide

Scope

This Design Guide describes how to plan and design a VSPEX Proven Infrastructure for VMware vSphere virtualized Oracle 12c databases. The guide assumes that a VSPEX Private Cloud already exists in the customer environment.

This guide also shows how to use the VSPEX Sizing Tool to size Oracle 12c Release 1 (12.1) on the VSPEX Proven Infrastructures, how to use best practices to efficiently allocate resources, and how to use all the benefits that VSPEX offers.

The EMC Data Protection solutions for Oracle Server 12c data protection are described in a separate document, EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2 Design and Implementation Guide.

Audience

This guide is intended for internal EMC personnel and qualified EMC VSPEX partners. It assumes that VSPEX partners who intend to deploy this solution are:

Qualified by EMC to sell, install, and configure the EMC VNX family of storage systems

Qualified to sell, install, and configure the network and server products required for VSPEX Proven Infrastructures

Certified for selling VSPEX Proven Infrastructure

Partners who plan to deploy the solution must also have the necessary technical training and background to install and configure:

VMware vSphere 5.5

Redhat Enterprises Linux 6.4

Oracle Database 12c

This guide provides external references where applicable. EMC recommends that partners who implement this solution are familiar with these documents. See Essential reading and Chapter 7: Reference Documentation for details.

Terminology

Table 1 lists the terminology used in the guide.

Table 1. Terminology

Term Definition

AWR Automatic Workload Repository is a powerful monitoring utility bundled with Oracle Database 10g and later releases

dNFS Direct Network File System client can be used to provide access to Network File System (NFS) storage devices



Term Definition

FAST™ Cache Fully Automated Storage Tiering Cache is an EMC storage feature that uses flash drives as an extension to system cache

FAST VP Fully Automated Storage Tiering for Virtual Pools, is an EMC storage feature that provides automatic storage tiering at the sub-LUN level

OLTP Online transaction processing, a class of systems that manage applications for data entry and retrieval transaction processing

PowerCLI A Windows PowerShell interface to the VMware vSphere and vCloud APIs

SGA System Global Area is a group of shared memory structures that contains data and control information from one Oracle database instance

PGA Process Global Area is memory specific to an operating process or thread that is not shared by other processes or threads on the system

VMDK VMware Virtual Machine Disk is an open file format that is a container for virtual hard disk drives for virtual machines

VMFS VMware Virtual Machine File System is a cluster file system that uses storage virtualization for multiple installations of VMware ESX server

Chapter 2: Before You Start


Design Guide

Chapter 2 Before You Start


Deployment workflow .............................................................................................. 12

Essential reading ..................................................................................................... 13



Deployment workflow

Refer to the process flow in Table 2 to design and implement a VSPEX for virtualized Oracle Database 12c OLTP solution.

Table 2. VSPEX workflow for virtualized Oracle Database deployment

Step Action

1 Use the VSPEX for Oracle Database 12c OLTP qualification worksheet to collect user requirements. The one-page qualification worksheet is in Appendix A of this Design Guide.

2 Use the EMC VSPEX Sizing Tool to determine the recommended VSPEX Proven Infrastructure for your Oracle Database 12c OLTP solution, based on the user requirements collected in Step 1.

For more information about the Sizing Tool, refer to the VSPEX Sizing Tool on the EMC Business Value Portal.

Note: You need to register the first time you access the tool. If the VSPEX Sizing Tool is not available, you can manually size the application using the guidelines in Appendix B High-level Oracle Database Server Sizing Logic and Methodology.

3 Use this Design Guide to determine the final design for your VSPEX solution.

Note: Consider all application requirements and not only the requirements for the Oracle database.

4 Select and order the correct VSPEX Proven Infrastructure. Refer to the appropriate VSPEX Proven Infrastructure document in Essential reading for guidance.

5 Deploy and test your VSPEX solution. Refer to the appropriate VSPEX Implementation Guide in Essential reading for guidance.

http://express.salire.com/go/emc



Design Guide

Essential reading

EMC recommends that you read the following documents before implementing the solution described in this document. The documents are available from the VSPEX space in the EMC Community Network or from EMC.com and the VSPEX Partner Portal. If you do not have access to a document, contact your EMC representative.

Refer to the following VSPEX Implementation Guide:

EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide

Refer to the following VSPEX Proven Infrastructure Guide:

EMC VSPEX Private Cloud VMware vSphere 5.5 for up to 200 virtual machines Enabled by Microsoft Windows Server 2012 R2, EMC VNXe3200, and EMC Powered Backup Proven Infrastructure Guide

Refer to the following EMC Data Protection for VSPEX guide:

EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2 Design and Implementation Guide

VSPEX Implementation Guide

VSPEX Proven Infrastructure Guide

EMC Data Protection for VSPEX Guide

https://community.emc.com/community/partner/vspex?view=overview

http://www.emc.com/utilities/globalsiteselect.jhtml?checked=true

http://www.emc.com/platform/virtualizing-information-infrastructure/vspex.htm

Chapter 3: Solution Overview


Design Guide

Chapter 3 Solution Overview


Overview .................................................................................................................. 16

EMC VSPEX Proven Infrastructure ............................................................................. 16

Solution architecture ............................................................................................... 18

Key components ...................................................................................................... 19



Overview

This solution offers users a powerful approach to deploying Oracle databases on EMC VNXe systems. By using advanced VNXe storage functionality such as FAST Cache or FAST VP, users can achieve increased performance for their Oracle deployments. With these advanced data features, the VNXe series not only reduces the initial cost of the Oracle database deployment, but also reduces the complexity associated with day-to-day data management by automating the complex and time-consuming storage tiering process.

EMC VNXe with Oracle:

Automatically delivers the highest input/output operations per second (IOPS) and lowest response times at the lowest cost

Provides automated storage tiering, requiring no manual tuning

Supports separate workloads with different I/O profiles

Supports Network File System(NFS), Fibre Channel(FC), and other protocols depending on a customer’s specific needs

Provides greater virtualization integration and lower Oracle licensing costs through server and database consolidation

The solution enables customers to quickly and consistently deploy a virtualized Oracle Database 12c in the VSPEX Proven Infrastructure. This solution was validated using NFS and FC for data storage.

This Design Guide can help EMC personnel and qualified EMC VSPEX Partners to deploy a simple, effective, and flexible Oracle Database 12c solution on a VSPEX Proven Infrastructure for their customers.

EMC VSPEX Proven Infrastructure

EMC has joined forces with IT infrastructure providers to create a complete virtualization solution that accelerates the deployment of the private cloud. VSPEX enables faster deployment, greater simplicity and choice, greater efficiency, and lower risk. Validation by EMC ensures predictable performance and enables customers to select technology that uses their existing IT infrastructure while eliminating planning, sizing, and configuration burdens. VSPEX provides a virtual infrastructure for customers who want the simplicity that is characteristic of truly converged infrastructures, while simultaneously gaining more choice for individual stack components.

VSPEX solutions are proven by EMC and packaged and sold exclusively by EMC channel partners. VSPEX provides channel partners with more opportunity, a faster sales cycle, and end-to-end enablement. By working together, EMC and its channel partners can deliver proven infrastructure that accelerates the journey to the cloud for even more customers.

VSPEX Proven Infrastructure, as shown in Figure 1, is a modular, virtualized infrastructure that is validated by EMC and delivered by EMC’s VSPEX partners. VSPEX



Design Guide

includes a virtualization layer , server, network, and EMC storage and data protection, designed by EMC to deliver reliable and predictable performance.

Figure 1. VSPEX Proven Infrastructure

VSPEX provides the flexibility to select network, server, and virtualization technologies that fit a customer’s environment to create a complete virtualization solution. VSPEX delivers faster deployment for EMC partner customers, with greater simplicity and efficiency, more choice, and lower risk to a customer’s business.



For more information about VSPEX Proven Infrastructures, refer to EMC VSPEX Private Cloud VMware vSphere 5.5 for up to 200 virtual machines Enabled by Microsoft Windows Server 2012 R2, EMC VNXe3200, and EMC Powered Backup Proven Infrastructure Guide

Solution architecture

Figure 2 shows the architecture that characterizes the infrastructure validated for an Oracle database 12c overlay on a VSPEX infrastructure. The data protection components of the solution provide data protection to the Oracle databases.

Figure 2. Architecture of the validated infrastructure

To validate this solution, we1 completed the following tasks:

Deployed all Oracle Database 12c servers as virtual machines on VMware vSphere 5.5.

Used the VSPEX sizing tool for Oracle Database 12c to determine the number of detailed compute resources for each Oracle Database 12c database. We provided three Oracle sizing options (small, medium, and large) to help size the customers’ environment and select the options that best suit their needs.

1 In this guide, “we” refers to the EMC Solutions engineering team that validated the solution.



Design Guide

Determined the recommended storage layout for Oracle Database 12c and the virtual infrastructure pool in the VNXe series storage arrays.

Note: The minimum Oracle version for this solution is Release 1 (12.1). We refer to this as 12c throughout this document.

Key components

This section provides an overview of the key technologies used in this solution:

Oracle Database 12c

VMware vSphere 5.5

EMC VNXe3200™

Red Hat Enterprise Linux 6.4

EMC Data Protection solutions

Oracle Database 12c is the latest version of Oracle database technology. Oracle 12c is available in a variety of editions that are tailored to meet the business and IT needs of an organization. In this solution we will consider:

Oracle Database 12c Release 1 Standard Edition (SE)

Oracle Database 12c Release 1 Enterprise Edition (EE)

Oracle Standard Edition (Oracle SE) is an affordable, full-featured data management solution that is ideal for all companies. It is available on single or clustered servers and can be licensed on a maximum capacity of four processor sockets, regardless of core count. The SE license includes Oracle Real Application Clusters (RAC) as a standard feature with no additional cost.

Oracle Enterprise Edition (Oracle EE) delivers industry-leading performance, scalability, security, and reliability on a choice of clustered or single servers running Windows, Linux, or UNIX. Oracle Database 12c EE supports advanced features, either included or as extra-cost options, that are not available with Oracle Database 12c SE. For example, security features such as Virtual Private Database are included with Oracle Database 12c EE, as well as data warehousing options such as partitioning and advanced analytics.

Note: The Oracle Database edition can affect the licensing cost and the size and number of VMware ESXi clusters that you can configure. How to Find the Oracle Processor Core Factor Multipliers (Doc ID 1330016.1) on My Oracle Support provides more information about Oracle processor licensing.

Oracle Multitenant is a new feature for the Oracle Database 12c Enterprise Edition that helps to reduce IT costs by consolidation, provisioning, upgrades, and more. Oracle Multitenant is supported by a new architecture that enables a single super database to hold many sub-databases. Oracle Multitenant is fully interoperable with Oracle Real Application Cluster (RAC).

Introduction

Oracle Database 12c



VMware vSphere 5.5 transforms a computer’s physical resources by virtualizing the CPU, RAM, hard disk, and network controller. This transformation creates fully functional virtual machines that run isolated and encapsulated operating systems and applications in the same way as physical computers.

VMware High Availability (HA) provides easy-to-use and cost-effective high availability for applications running on virtual machines. The VMware vSphere vMotion and VMware vSphere Storage vMotion features of vSphere 5.5 enable the seamless migration of virtual machines and stored files from one vSphere server to another, with minimal or no performance impact. Coupled with VMware vSphere Distributed Resource Scheduler (DRS) and VMware vSphere Storage DRS, virtual machines have access to the appropriate resources at any point in time through load balancing of compute and storage resources.

The VMware Native Multipathing Plug-In (NMP) is the default module in vSphere that is used for multipathing. NMP provides a default path selection algorithm based on the array type and it associates the physical paths with a specific storage device or logical unit number (LUN). The specific details for handling path failover for a particular storage array are delegated to a Storage Array Type Plug-In (SATP). The specific details for determining which physical path is used to issue an I/O request to a storage device are handled by a Path Selection Plug-In (PSP). SATPs and PSPs are sub-plug-ins within the NMP module.

EMC VNXe3200 is an affordable, flash-optimized, unified storage platform that delivers innovation and enterprise capabilities for file and block storage in a single, scalable, and easy-to-use solution. Ideal for mixed workloads in physical or virtual environments, VNXe3200 combines powerful and flexible hardware with advanced efficiency, management, and protection software to meet the demanding needs of today’s virtualized application environments.

Features and enhancements

VNXe3200 includes many features and enhancements designed and built on the success of the midrange EMC VNX family. These features and enhancements include:

Greater efficiency with a flash-optimized hybrid array

More capacity with multicore optimization with EMC Multicore Cache, Multicore RAID, and Multicore FAST Cache (MCx™)

Easier administration and deployment with VNXe Base Software components including Monitoring, Reporting, and Unified Snapshots

VMware ecosystem integration

Unified multiprotocol support for FC, iSCSI, NFS, and CIFS

VSPEX is built with the next generation of VNXe to deliver even greater efficiency, performance, and scalability than ever before.

Flash-optimized hybrid array

VNXe3200 is a flash-optimized hybrid array that provides automated tiering to deliver the best performance to your critical data, while intelligently moving less frequently accessed data to lower-cost disks.

VMware vSphere 5.5

EMC VNXe3200



Design Guide

In this hybrid approach, a small percentage of flash drives in the overall system can provide a high percentage of the overall IOPS. VNXe3200 takes full advantage of the low latency of flash to deliver cost-saving optimization and high performance scalability. EMC Fully Automated Storage Tiering Suite (FAST Cache and FAST VP) tiers both block and file data across heterogeneous drives. The suite boosts the most active data to the flash drives, ensuring that customers never have to make concessions for cost or performance.

Data is generally accessed most frequently at the time it is created; therefore, new data should first be stored on flash drives to provide the best performance. As the data ages and becomes less active over time, FAST VP can tier the data from high-performance drives to high-capacity drives automatically, based on customer-defined policies. FAST Cache dynamically absorbs unpredicted spikes in system workloads. FAST Cache can provide immediate performance enhancement by promoting suddenly active data from slower high-capacity drives to speedier flash drives. All VSPEX use cases benefit from the increased efficiency.

VSPEX Proven Infrastructures deliver private cloud, end-user computing, and virtualized application solutions. With VNXe3200, customers can realize an even greater return on their investment.

VNX Intel MCx code path optimization

The advent of flash technology has been a catalyst for changing the requirements of midrange storage systems. EMC redesigned the midrange storage platform to efficiently optimize multicore CPUs to provide the most efficient storage system at the lowest cost in the market.

MCx distributes all VNXe data services across all cores, and can dramatically improve the file and block performance for transactional applications such as databases or virtual machines over network-attached storage (NAS) and Fibre Channel (FC).

VNXe base software

The enhanced VNXe base software extends the EMC Unisphere® easy-to-use interface to include VNX monitoring and reporting for validating performance and anticipating capacity requirements. The suite also includes EMC Unisphere Central® for centrally managing thousands of VNX and VNXe systems.

Virtualization and ecosystem management

You can use the following applications for virtualization and ecosystem management:

VMware vSphere Storage APIs for Storage Awareness VMware vStorage APIs for Storage Awareness (VASA) is a VMware-defined API that displays storage information through vCenter. Integration between VASA technology and VNX makes storage management a seamless experience in a virtualized environment.

VMware vSphere Storage APIs for Array Integration VMware vSphere Storage APIs for Array Integration (VAAI) offloads VMware storage-related functions from the server to the storage system, enabling more efficient use of server and network resources for increased performance and consolidation.



EMC Storage Analytics for VNXe EMC Storage Analytics (ESA) for VNXe delivers a storage-only version of VMware vCenter Operations with a built-in VNXe connector that provides detailed analytics, relationships, and unique icons for EMC arrays and components.

EMC Virtual Storage Integrator EMC Virtual Storage Integrator (VSI) is a free plug-in for VMware vCenter that is available to all VMware users with EMC storage. VSPEX customers can use VSI to simplify management of virtualized storage. VMware administrators can gain visibility into their VNXe storage using the same familiar vCenter interface to which they are accustomed.

With VSI, IT administrators can do more work in less time. VSI offers unmatched access control that enables you to efficiently manage and delegate storage tasks with confidence. With VSI, you can perform daily management tasks with up to 90 percent fewer clicks and up to 10 times higher productivity.

Red Hat Enterprise Linux is a versatile platform for x86 and x86-64 that can be deployed on physical systems, as a guest on the major hypervisors, or in the cloud. It supports all leading hardware architectures with compatibility across releases. Red Hat Enterprise Linux 6.4 includes enhancements and new capabilities that provide rich functionality, especially the developer tools, virtualization features, security, scalability, file systems, and storage.

EMC Data Protection solutions—EMC Avamar® and EMC Data Domain®—deliver the protection confidence needed to accelerate deployment of a virtualized Oracle server.

Optimized for virtualized application environments, EMC Data Protection reduces backup times by 90 percent and increases recovery speeds by 30 times for worry-free protection. EMC Data Protection appliances add another layer of assurance with end-to-end verification and self-healing for ensured recovery.

For full technical guidance, refer to EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2 Design and Implementation Guide. This guide describes how to design, size, and implement EMC backup and recovery solutions for VSPEX Proven Infrastructure for a virtualized Oracle server.

Red Hat Enterprise Linux 6.4

EMC Data Protection solutions

Chapter 4: Choosing a VSPEX Proven Infrastructure


Design Guide

Chapter 4 Choosing a VSPEX Proven Infrastructure


Overview .................................................................................................................. 24

Step 1: Evaluate the customer use case ................................................................... 24

Step 2: Design the application architecture ............................................................. 25

Step 3: Select the right VSPEX Proven Infrastructure ............................................... 27



Overview

This chapter describes how to design the VSPEX Proven Infrastructure with a virtualized Oracle server and how to select the right VSPEX Proven Infrastructure to meet your requirements. Table 3 outlines the main steps to complete when selecting a VSPEX Proven Infrastructure.

Table 3. VSPEX Proven Infrastructure selection steps

Step Action

1 Use the VSPEX for virtualized Oracle OLTP qualification worksheet to evaluate the customer’s Oracle OLTP workload. Step 1: Evaluate the customer use case provides details.

2 Use the VSPEX Sizing Tool to determine the required infrastructure, Oracle server resources, and architecture. Step 2: Design the application architecture provides details.

Note: If the Sizing Tool is not available on the EMC Support website, manually size the application using the guidelines in Appendix B: High-level Oracle Database Server Sizing Logic and Methodology.

3 Select the right VSPEX Proven Infrastructure, based on the recommendations from Step 2. Step 3: Select the right VSPEX Proven Infrastructure provides details.

Step 1: Evaluate the customer use case

Before you select a VSPEX infrastructure solution, you must understand the customer’s real workload and dataset, based on the business requirements.

To help you better understand the customer’s business requirements for the VSPEX infrastructure design, EMC strongly recommends that you use the VSPEX for Virtualized Oracle OLTP qualification worksheet to evaluate the workload requirements for the VSPEX solution. For more information about the qualification worksheet for this solution, see Appendix A: VSPEX for Virtualized Oracle OLTP qualification worksheet.

In the VSPEX for Virtualized Oracle OLTP qualification worksheet, we asked some simple questions to help understand and describe the Oracle OLTP workload requirements and usage characteristics for the customer.

Table 4. VSPEX for virtualized Oracle OLTP qualification worksheet guidelines

Question Description

Do you have an existing Oracle database that you would like to size for in the environment?

Select Yes if the customer already has an Oracle database and understands the characteristics that are going to migrate to VSPEX Private Cloud in the VSPEX environment.

How many databases do you want to deploy?

Type the database number that the customer expects to deploy in the VSPEX environment.



Design Guide

Question Description

What is the size of user database (GB)? Type the database size that the customer expects to have in the VSPEX environment.

What is the annual growth rate (%)? Future growth is a key characteristic of the VSPEX solution. This value is the expected annual growth rate of the user database in three years. Type a number that is appropriate for the customer’s environment.

Do you intend to use FAST Cache? FAST Cache is particularly suited to applications that randomly access storage with high frequency. Fast Cache boosts Oracle application performance by ensuring that the most active data is served from high-performing flash drives and this data can reside on this the faster medium for as long as it is needed. A small number of flash drives implemented as FAST Cache provide a greater performance increase than a large number of hard disk drives.

Do you intend to use FAST VP? FAST VP significantly reduces total cost of ownership (TCO) and increases performance. A target workload that requires many Performance Tier drives can be serviced with a mix of tiers and a lower drive count. Using FAST VP instead of a homogeneous drive deployment has proven highly effective for lowering capital expenditures, reducing power and cooling costs, and increasing performance for many applications, including OLTP tests with Oracle Database.

What is the maximum number of IOPS (read/write)?

Understanding the maximum number of IOPS of Oracle databases can help to prevent potential storage performance issues. Work with the customer to estimate the IOPS at peak loads.

(Optional) What is the expected number of concurrent users for peak loads?

The maximum number of concurrent users is a key characteristic of the user database. If the customer can estimate the user number at peak loads in the environment, type that number.

Step 2: Design the application architecture

In this VSPEX Proven infrastructure solution, we defined how a representative customer workload is to be sized.

After you gather the customer’s information and populate the VSPEX for the virtualized Oracle qualification worksheet, you can use that information to populate



the VSPEX sizing tool located on the EMC Business Value Portal, or you can use Appendix B: Sizing considerations to manually size the solution.

VSPEX Sizing Tool output: Requirements and recommendations

The VSPEX Sizing Tool enables you to put a database configuration from the customer’s answers into the qualification worksheet. Refer to the VSPEX Sizing Tool portal for more information about the Sizing Tool.

After you complete the inputs to the VSPEX Sizing Tool, the tool generates a series of recommendations, as listed in Table 5.

Table 5. VSPEX Sizing Tool output

Type Description Reference

vCPU The number of vCPUs to configure for each Oracle Server virtual machine

Oracle website

Memory The amount of memory recommended for each Oracle Server virtual machine

Storage layout for Oracle server databases

The suggested user database pool configuration on VNXe or VNX

Designing the storage layout

We also used a standard compute-size model for Oracle, which simplified and standardized the validation testing. The model enabled us to identify the configuration required to run a comparable TCP-C OLTP database workload with a 60:40 read/write ratio, yielding acceptable response time.

Table 6 shows how we mapped the Oracle sizing model to the virtual machine resources.

Table 6. Oracle sizing model for virtual machines

Oracle model Virtual machine resources

Small virtual machine for up to 200 users Compute requirements:

2 vCPU

8 GB memory

Storage requirements (OS and Oracle binaries):

100GB

25 IOPS

Storage requirements (Oracle datafiles):

275 IOPS

http://express.salire.com/go/emc

https://mainstayadvisor.com/Default.aspx?t=EMC&atid=224a9bcc-caa3-48f4-8ff8-33051513c410



Design Guide

Oracle model Virtual machine resources

Medium virtual machine for up to 1200 users

Compute requirements:

12 vCPU

38 GB memory


100GB

50 IOPS


1650 IOPS

Large virtual machine for more than 8000 users

Compute requirements:

32 vCPU

98 GB memory


100GB

100 IOPS


6600 IOPS

See the examples in Step 3: Select the right VSPEX Proven Infrastructure for more information.

If the sizing tool is not available on the EMC Support website, use the sizing instructions provided in Appendix B: High-level Oracle Database Server Sizing Logic and Methodology.

Step 3: Select the right VSPEX Proven Infrastructure

The VSPEX program provides many solutions designed to simplify the deployment of a consolidated virtual infrastructure using VMware vSphere and the EMC VNXe family of products and EMC Data Protection. After you confirm the application architecture, you can select the right VSPEX Proven infrastructure based on the calculated results.

Note: While this Design Guide is intended for Oracle Database OLTP requirements, the Oracle server may not be the only application deployed on the VSPEX Proven Infrastructure. You must carefully consider the requirements for each application that you plan to deploy. If you are uncertain about the best VSPEX Proven Infrastructure to deploy, consult an EMC representative before making the decision.

Considerations



Follow the steps in Table 7 when choosing a VSPEX Proven Infrastructure.

Table 7. Select the right VSPEX Proven Infrastructure

Step Action

1 Use the VSPEX Sizing Tool to determine the total number of required resources for virtual machines and any additional suggested storage layout requirements for the Oracle server.

2 Use the VSPEX Sizing Tool to design the resource requirements for other applications, based on business needs. The VSPEX Sizing Tool calculates the total number of required resources for virtual machines and recommended storage layout requirements for both the Oracle server and the other applications.

3 Discuss with your customers the maximum use of VSPEX Proven Infrastructure that meets their business requirements—this is the maximum use for both Oracle server and other applications. Put the maximum utilization percentage of the VSPEX Proven Infrastructure into the VSPEX Sizing Tool. The tool provides a minimum recommendation for the VSPEX Proven Infrastructure offering.

4 Select your network vendor and server vendor for the recommended VSPEX Proven Infrastructure offering. EMC VSPEX: Choose the Right Path to Your Cloud provides more information.

Overview

This section describes the following scenarios to help select the VSPEX Proven Infrastructure:

Multiple small-sized Oracle Database 12c servers, each with less than 200 concurrent users

A large-sized Oracle Database 12c server with up to 8,000 concurrent users

Example 1: Multiple small-sized Oracle Database 12c servers, each with less than 200 concurrent users

Use this scenario to deploy multiple small-sized Oracle Database 12c servers on a VSPEX Proven Infrastructure. The customer has multiple 500 GB databases. The expected R/W IOPS for each database is about 165/110.

After you type the answers from the qualification worksheet into the VSPEX Sizing Tool, the tool generates a series of recommendations for the resources needed from the VSPEX private cloud pool, as shown in Table 8.

Table 8. Qualification worksheet example: Multiple small-sized Oracle 12c servers

Question Answer

Do you have an existing Oracle Server database that you would like to size for in the environment?

Yes

How many databases do you want to deploy? 10

What is the size of the user database (GB)? 500

Examples





Design Guide

Question Answer

What is the annual growth rate (%)? 10

Do you intend to use FAST Cache? No

Do you intend to use FAST VP? No

What is the maximum number of IOPS (read/write)? 2,750

(Optional) What is the maximum number of concurrent users at peak loads?

N/A

After you type the answers from the qualification worksheet into the VSPEX Sizing Tool, the tool generates recommendations for the resources needed from the VSPEX private cloud pool, as shown in Table 9.

Table 9. Required resources example: Multiple small-sized Oracle 12c servers

Oracle Server vCPUs Memory OS volume capacity

OS volume

Resource requirement 20 80 GB 1,000 GB 250 IOPS

The VSPEX Sizing Tool also lists recommendations for the storage layout, as shown in Table 10.

Table 10. Recommendations for the storage layout example: Multiple small-sized Oracle 12c servers

Pool name RAID type Disk type Disk capacity No. of disks

VSPEX private cloud pool RAID 5 SAS disks 10,000 rpm 500 GB 5

Oracle database datafile pool

RAID 5 SAS disks 10,000 rpm 500 GB 30

Oracle database log file pool

RAID 1/0 SAS disks 10,000 rpm 500 GB 4

Example 2: A large-sized Oracle Database 12c server with up to 8,000 concurrent users

Use this scenario to deploy large-sized Oracle Database 12c servers on a VSPEX Proven Infrastructure. The customer has a 1 TB database. The expected number of concurrent users for the database is 8000 and the expected R/W IOPS is 3960/2640.

After you put the answers from the qualification worksheet into the VSPEX Sizing Tool, the tool generates a series of recommendations for the resources needed from the VSPEX private cloud pool, as shown in Table 11.



Table 11. Qualification worksheet example: Large-sized Oracle 12c server

Question Answer

Do you have an existing Oracle Server database that you would like to size for in the environment?

Yes

How many databases do you want to deploy? 1

What is the size of the user database (GB)? 1,000

What is the annual growth rate (%)? 10

Do you intend to use FAST Cache? No

Do you intend to use FAST VP? Yes

What is the maximum number of IOPS (read/write)? 6,600

(Optional) What is the maximum number of concurrent users at peak loads? N/A

After you type the answers from the qualification worksheet into the VSPEX Sizing Tool, the tool generates a series of recommendations for the resources needed from the VSPEX private cloud pool, as shown in Table 12.

Table 12. Required resources example: Large-sized Oracle 12c server

Oracle Server vCPUs Memory OS volume capacity OS volume

Resource requirement 32 98 GB 100 GB 100 IOPS

The VSPEX Sizing Tool also lists recommendations for the storage layout, as shown in Table 13.

Table 13. Recommendations for the storage layout example: Large-sized Oracle 12c server

Pool name RAID type Disk type Disk capacity No. of disks

VSPEX private cloud pool RAID 5 SAS disks 10,000 rpm 500 GB 5

Oracle database datafile pool RAID 5 SAS disks 10,000 rpm 500 GB 30

RAID 1 FAST VP SSD 200 GB 2

Oracle database log file pool RAID 10 SAS disks 10,000 rpm 500 GB 8

Chapter 5: Solution Design Considerations and Best Practices


Design Guide

Chapter 5 Solution Design Considerations and Best Practices


Overview .................................................................................................................. 32

Designing the network ............................................................................................. 32

Designing the storage layout ................................................................................... 33

Configuring FAST Cache for Oracle ........................................................................... 36

Configuring FAST VP for Oracle ................................................................................ 37

Designing the virtualization layer ............................................................................ 38

Designing the Oracle Database 12c implementation ................................................ 40

EMC Data Protection design considerations ............................................................. 43



Overview

This chapter describes the EMC VSPEX for Virtualized Database 12c OLTP solution design and describes the best practices for the network, storage, virtualization, and applications.

Designing the network

This section describes the network details for SAN and IP network configuration and an ESXi server network. In this VSPEX Proven Infrastructure for Virtualized Oracle Database 12c solution, EMC recommends that you consider the ESXi Server’s advanced settings and the network redundancy when designing the network.

The advanced networking features of the VNXe series provide protection against network connection failures at the array. Each virtual host also has multiple connections to storage Ethernet networks to guard against link failures. Spread the connections across multiple Ethernet switches to guard against component failure in the network.

EMC recommends that you use the following SAN best practices:

Use multiple host bus adapter (HBA) and FC switches for network redundancy.

Zone each FC port from the database servers to both storage processor ports for high availability.

Use path management and dynamic multipathing software, such as PowerPath, on the hosts to enable the failover process to alternate paths and to provide load balancing.

EMC recommends that you use the following IP network best practices:

Use multiple network cards and switches for network redundancy.

Use a 10 Gb Ethernet for network connection.

Use Virtual local area networks (VLANs) to logically group devices that are on different network segments or sub networks.

Enable and configure jumbo frames throughout the physical or virtual stack.

Note: Maximum Transfer Unit (MTU) sizes of greater than 1,500 bytes are referred to as jumbo frames. Jumbo frames require a Gigabit Ethernet across the entire network infrastructure, including servers, switches, and database servers.

Networking in the virtual world follows the same concepts as networking in the physical world, but some of these concepts are applied in the software instead of using physical cables and switches. Although many of the best practices that apply in the physical world continue to apply in the virtual world, there are additional considerations for traffic segmentation, availability, and throughput.

Overview

SAN best practices

IP network best practices

vSphere network best practices



Design Guide

For example, in the VMware virtualization environment, use two physical NICs per vSwitch and uplink the physical NICs to separate physical switches.

When configuring the NIC teaming settings, it is a best practice to select No for the NIC teaming failback option. If there is some intermittent behavior in the network, this will prevent the NIC cards from flip-flopping.

When setting up VMware High Availability, also set the following ESX server timeouts and settings under the ESX Server advanced setting tab:

NFS.HeartbeatFrequency = 12

NFS.HeartbeatTimeout = 5

NFS.HeartbeatMaxFailures = 10

To access the NFS advanced options, follow these steps:

1. Log into the VMware vSphere Client.

2. Select the ESXi/ESX host.

3. From the Configuration tab, select Advanced Settings >NFS.

Refer to Best Practices for Running VMware vSphere on Network Attached Storage (NAS) for more NIC teaming best practices for VMware vSphere.

Refer to the VSPEX Proven Infrastructure Guide for other best practices in network design for the VSPEX Proven Infrastructure.

Oracle direct NFS or dNFS, which was introduced in Oracle 11g R2, allows the NFS operations to be moved to the database layer instead of the operating system kernel. Oracle strongly recommends configuring the Oracle database to use the Oracle dNFS Client ODM disk libraries. This is a one-time operation. After logging into the Linux system, type the following commands as an Oracle user to enable the dNFS Client ODM library:

[oracle@small ~] cd $ORACLE_HOME/rdbms/lib

[oracle@small ~] make –f ins_rdbms.mk dnfs_on

rm –f /u01/app/oracle/lib/libodm12.so; cp

/u01/app/oracle/lib/libnfsodm12.so /u01/app/oracle/lib/libodm12.so

Refer to the EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide for other best practices in network design for the VSPEX Proven Infrastructure.

Designing the storage layout

This section provides guidelines for planning storage for various business requirements in Oracle Database 12c environments.

Note: Your customers must select the deployment architecture that best fits their specific needs. EMC unified storage provides flexibility and manageability for a storage infrastructure that supports either file or block protocol.

Oracle settings for NFS-specific considerations

Overview



Figure 3 shows an example of the high-level architecture between the Oracle Database 12c components and the storage elements validated in the VSPEX Proven Infrastructure for virtual Oracle Database server on a VMware vSphere virtualization platform.

Figure 3. Example: Storage layout of Oracle 12c Database on VMware

Note: Figure 3 shows the use of Oracle Direct NFS (dNFS) over Internet protocol (IP) with Oracle. All of the Oracle Server volumes can also be stored in Virtual Machine Disk (VMDK) format in a VMware virtualized environment.

EMC recommends that you use different storage pools to store OS and Oracle Database data in addition to the VSPEX private cloud pool for virtual machines. Table 14 provides an example.

Table 14. Oracle Sever storage pools example

Storage pool name Purpose Recommendation

VSPEX private cloud pool

The private cloud pool where all the virtual machines reside.

Mixed pool including RAID 5 with SAS disks and RAID 1/0 with FAST VP SSD

Oracle data pool

(FAST Cache optional )

The Oracle data pool to serve the datafile LUNs and volumes for user databases

RAID 5 with SAS disks, or a mixed pool

Oracle redo pool The Oracle redo pool to serve the logfile LUNs and volumes for user databases

RAID 1/0 with SAS disks

Storage layout and design considerations



Design Guide

Consider the following best practices for storage and layout design in the VSPEX Proven Infrastructure for virtualized Oracle Database 12c solution.

Oracle Database data pool

Use SAS disks with RAID 5 (4+1) protection for the Oracle datafile and temp file systems. This combination of RAID protection and disk type provides high capacity use and good I/O performance at a low cost, while simultaneously ensuring data availability if a drive failure occurs.

Oracle Database redo log pool

In this solution, we configured the file system for redo logs on the different pools protected by RAID10 on SAS disks. For highly write-intensive workloads, consider a separate pool for redo file systems on physically separate disks.

Customization

EMC recommends that you work with the customer to estimate the capacity and IOPS requirements for the storage layout. Consider future growth when laying out the storage, and include projected growth as input to the VSPEX sizing tool.

Additional performance requirements for FAST Suite

The EMC FAST Suite—FAST VP and FAST Cache—provides two key technologies available on the VNX series that enable extreme performance in an automated fashion. See the VSPEX Proven Infrastructure website for more information on FAST Suite for VSPEX Proven Infrastructures.

FAST Cache can improve overall system performance for drive-related problems with Oracle applications. We recommend first using available flash drives for FAST Cache, which can globally benefit all data LUNs in the storage system. Then supplement performance as needed with additional flash drives in storage pool tiers using FAST VP.

Enabling FAST Cache is a transparent operation to Oracle Database 12c and no reconfiguration or downtime to the database is necessary. EMC recommends that you only use FAST Cache on the storage pool or LUNs that require it. FAST Cache is best for light random I/O where there is an uneven data distribution.

Users also can create blended storage pools composed of various disk types (SSD, SAS, and NL_SAS). The data migration in this highly consolidated virtualized environment produces the highest storage efficiency, from a performance and capacity perspective.

If you enable FAST Suite technology on the Oracle Database 12c, the response times, read/write throughput, and latencies will improve the Oracle Database 12c user experience. FAST Suite technology eases the burden on storage and database administrators by determining the most efficient storage layout for their customers.

This section describes the VNXe storage layouts in this VSPEX Proven Infrastructure for virtualized Oracle Database 12c based on the VSPEX private cloud. This example follows the best practice and design considerations, as previously discussed.

Storage best practices

VSPEX storage layout example




Figure 4 shows a storage layout dedicated to Oracle Database pools.

Figure 4. Storage layout example: Oracle Server for VNX series with FAST VP/Cache enabled

Note: This is only one example of a storage layout. To plan and design your own storage layouts for an Oracle database on an EMC VSPEX stack, follow the guidance for the VSPEX Sizing Tool and the best practices described in Designing the storage layout.

Configuring FAST Cache for Oracle

FAST Cache uses enterprise flash drives to add an extra layer of cache between dynamic random access memory (DRAM) cache and rotating disk drives, creating a faster medium for storing frequently accessed data. FAST Cache is an extendable, read/write cache that boosts application performance by ensuring that the most active data is served from high-performing flash drives, and can reside on the faster medium for as long as necessary.

FAST Cache tracks data activity at a granularity of 64 KB and promotes data that is frequently accessed into FAST Cache by copying it from the hard disk drives to the flash drives assigned to FAST Cache. Subsequent I/O access to that data is handled by the flash drives and is serviced at flash drive response times: this ensures very low latency for the data. As data ages and becomes less active, it is flushed from FAST Cache to be replaced by more active data.

A few flash drives implemented as FAST Cache provides a greater application performance increase than many short-stroked hard disk drives.

FAST Cache is particularly suited to applications that randomly access storage with high frequency, such as Oracle OLTP databases. Also, OLTP databases have inherent locality of reference with varied I/O patterns. Applications with these characteristics benefit most from deploying FAST Cache. The optimal use of FAST Cache occurs when the working dataset can fit within the FAST Cache.

Overview



Design Guide

EMC recommends the following best practices:

Only enable FAST Cache on pool LUNs that require it

Size FAST Cache appropriately based on the application's active dataset

Disable FAST Cache on pool LUNs where Oracle online redo logs reside

Never enable FAST Cache on archive logs because these files are never overwritten and are rarely read back (unless you need to recover the database)

Spread FAST Cache SSDs across all available buses

Use the available FAST Cache SSDs first for FAST Cache and then supplement performance as needed with additional flash drives in the storage pool tiers

EMC recommends that you enable FAST Cache for the Oracle datafiles only. Oracle archive files and redo log files have a predictable workload composed mainly of sequential writes. The array's write cache and assigned hard disk drives can efficiently handle these archive files and redo log files. Enabling FAST Cache on these files is neither beneficial nor cost effective.

Enabling FAST Cache on a running system FAST Cache can improve overall system performance if the current bottleneck is drive-related. However, boosting the IOPS will result in an increase in CPU utilization on the VNX SPs. Systems should be sized so that the maximum sustained utilization is 70 percent.

Use Unisphere to check the SP CPU utilization and then proceed as follows:

SP CPU utilization less than 60 percent—Enable groups of LUNs or one pool at a time until they are equalized in the cache. Ensure that the SP CPU utilization is still acceptable before turning on FAST Cache for more LUNs/pools.

SP CPU utilization of 60 to 80 percent—Scale in carefully. Enable FAST Cache on one or two LUNs at a time, and verify that the SP CPU utilization does not exceed 80 percent.

SP CPU utilization greater than 80 percent—Do not activate FAST Cache.

Configuring FAST VP for Oracle

Like FAST Cache, FAST VP works best on datasets that exhibit a high degree of skew. FAST VP is very flexible and supports several tiered configurations, such as single tiered, multi-tiered, with or without a flash tier, and FAST Cache support. Adding a flash tier can locate “hot data” on flash storage in 256 MB slices.

For VNXe, EMC recommends that you first enable FAST VP on the Oracle datafile storage pool to make the best use of either of the FAST technologies.

Aggressive reduction of TCO and an increase in performance can be achieved with FAST VP. A common strategy is to use FAST VP to gain TCO benefits while using FAST Cache to boost overall system performance. This paper discusses considerations for an optimal deployment of these technologies.

FAST Cache best practices

Overview



For further information on FAST VP algorithm and policies, see EMC FAST VP for Unified Storage Systems.

EMC recommends the following best practices:

Spread FAST VP SSDs across all available buses.

Enable FAST VP on a pool, even if the pool has only one tier, to provide ongoing load balancing of LUNs across available drives.

Maintain some unallocated capacity within the pool to help with relocation schedules when using FAST VP.

Schedule relocations for off-hours, so that the primary workload does not contend with the relocation activity

Designing the virtualization layer

Oracle Database 12c is fully supported when deployed in a virtual environment with VMware vSphere ESXi technology. The following sections describe the best practices and design considerations for Oracle Database 12c virtualization.

In this VSPEX Proven Infrastructure for Oracle database, EMC recommends that you consider implementing best practices for managing the following resources in your virtualization design:

Compute resources

Network resources

VMware features

Compute resources

EMC recommends implementing the following compute resource best practices:

Enable hyper-threading. Hyper-threading technology allows a single physical processor to execute multiple independent threads simultaneously. ESXi is designed to make use of hyper-threading by controlling the placement of logical processors on the same core and intelligently managing processor time to guarantee that load is spread evenly across all physical cores in the system.

Use Hardware-Assisted MMU Virtualization (Intel EPT and AMD RVI) to reduce memory consumption and to speed up workloads that cause guest operating systems to modify page tables too frequently.

Use Non-Uniform Memory Access (NUMA), a computer architecture in which memory located closer to a particular processor is accessed with less delay than memory located farther from that processor.

Allocate Virtual machine memory (vRAM) in a virtual machine to be less than the local memory accessed by NUMA node (processor).

Schedule the vCPU to use the fewest sockets required using the virtual machine parameter numa.vcpu.preferHT=TRUE.

FAST VP best practices

Overview

Virtualization best practices



Design Guide

Install VMware Tools that include utilities that enhance the performance of the virtual machine's guest operating system and improve the ability to manage the virtual machine.

Network resource

EMC recommends that you implement the following network resource best practices:

Use the most recent paravirtualized virtual network device from VMware: VMXNET Generation 3 (VMXNET3), which supports 10 GbE.

Use vLANs to separate vSphere infrastructure traffic from virtual machine traffic for security and isolation.

Enable and configure Jumbo frames throughout the virtual and physical stack for the vMotion and the IP storage networks.

Use an In-guest NFS mount from an Oracle dNFS client within the virtual machine rather than VMDK on an NFS datastore.

VMware features

EMC recommends that you implement the following VMware features:

vSphere HA—This feature uses multiple ESXi hosts, configured as a cluster, to provide rapid recovery from outages and provides cost-effective high availability (HA) systems for applications running in virtual machines. vSphere HA protects applications against:

Server failure by restarting the virtual machines on other ESXi servers within the cluster

Application failure by continuously monitoring a virtual machine and resetting it if a guest OS fails

VMware DRS—This feature automatically balances the workload between the hosts using the vMotion function when migrating virtual machines. When Oracle database workloads increase, DRS automatically moves a bottlenecked virtual machine to another host with more resources available, without downtime.

DRS Affinity rules—This feature controls the placement of virtual machines on hosts within a cluster. DRS provides two types of affinity rules:

A virtual machine host affinity rule, which specifies an affinity relationship between a group of virtual machines and a group of hosts

A virtual machine virtual machine affinity rule, which specifies whether particular virtual machines will run on the same host or remain on separate hosts

VMware templates

VMware templates are virtual machine master copies used to create and provision virtual machines. Use a template to deploy a virtual machine with a guest OS installed and applications and users configured. The guest OS is ready for use with minimal user intervention.



Templates minimize the deployment time and automate installation and configuration tasks required for each virtual machine. Virtual machine deployment is simplified by customizations stored in vCenter. A deployment wizard, automation tool, or script can use templates to automatically create or edit server settings before deploying the new virtual machine.

Monitoring the VSPEX Proven Infrastructure regularly

Monitor the performance of the VSPEX Proven infrastructure regularly. Performance monitoring happens not only at the virtual machine level, but also at the hypervisor level. With an ESXi hypervisor, you can use performance monitoring within the Oracle Database machine to ensure that the virtual machine or Oracle Database performs as expected. You can also use monitoring tools such as esxtop to observe host performance at the hypervisor level.

Designing the Oracle Database 12c implementation

One of the purposes of this solution is to detail how two storage networking technologies are used with Oracle: Oracle Automatic Storage Management (ASM) over Fibre Channel(FC) and Oracle Direct NFS(dNFS) over Internet Protocol(IP).

There are many design considerations for ASM and dNFS in the context of an online transaction processing (OLTP) workload from Oracle Database 12c. This section provides guidelines for the most common and important design considerations and best practices to follow.

Since the Oracle Database 10g Release 2, database storage management and provisioning has been simplified by using Automatic Storage Management(ASM). ASM provides file system and volume management tasks, such as creating or laying out database and diskspace management.

With ASM over FC, all database objects , including datafiles, online redo files, and control files, are stored on ASM disk groups and managed by Oracle ASM. ASM provides out-of-box enablement of redundancy and optimal performance. However, the following items should be considered to increase either performance or availability, or both:

Implement multiple access paths to the storage array using two or more HBAs or initiators

Deploy multi-pathing software over these multiple HBAs to provide I/O load-balancing and failover capabilities

Use ASM disk groups with similarly sized and performing disks.

The Oracle dNFS client is a standard feature with Oracle Database 12c and provides improved performance and resilience over the OS native NFS. dNFS also performs asynchronous IOPS, which allows processing to continue while the I/O request is submitted and processed.

With dNFS over IP, all database objects are accessible through an NFS mount. Datafiles, online redo log files and other database files are accessed using dNFS over

Overview

Best practices for ASM and database deployment

Best practices for dNFS configuration



Design Guide

IP protocol. EMC recommends configuring the Oracle database to use the Oracle dNFS client ODM disk libraries. This is a one-time operation and after it is set the database uses the Oracle-optimized native Oracle dNFS client instead of the native NFS client.

Automatic Shared Memory Management (ASMM) is a standard method of dynamically managing memory in the Oracle database and has been available since Oracle Database 10g. EMC recommends that you implement ASMM to automate the management of the following shared memory structures:

DB_CACHE_SIZE

SHARED_POOL_SIZE

LARGE_POOL_SIZE

JAVA_POOL_SIZE

STREAMS_POOL_SIZE

Set the following initialization parameters to implement this feature:

SGA_TARGET set to a nonzero value

STATISTICS_LEVEL=TYPICAL (or ALL)

Note: Do not use Oracle Automatic Memory Management (AMM) because AMM is incompatible with HugePages. If you want to use Linux HugePages, ensure that both MEMORY_TARGET / and MEMORY_MAX_TARGET initialization parameters are not set.

ASMM versus AMM and LINUX x86-64 HugePages Support (Doc ID 1134002.1) on My Oracle Support provides more information about HugePages.

HugePages is crucial for faster Oracle database performance on Linux if you have a large RAM and SGA. If the combined database SGAs are large (more than 8 GB), you need to configure HugePages. The size of the SGA matters.

The advantages of enabling HugePages include:

Larger page size and fewer pages

Better overall memory performance

No swapping

Use Large Pages to Enable HugePages (Doc ID 1392497.1) on My Oracle Support provides more information about enabling and tuning HugePages.

Set DISK_ASYNCH_IO= true. Asynchronous I/O is now recommended on all the storage protocols. The default value in Oracle 12c is true .

Set FILESYSTEMIO_OPTIONS=SETALL. This setting enables both direct I/O and asynchronous I/O. With asynchronous I/O, processing continues while the I/O request is submitted and processed.

Direct NFS does not depend on the value of FILESYSTEMIO_OPTIONS. Direct NFS always uses asynchronous and direct I/O, as it does not depend on OS support.

Manage automatic shared memory

Enable the HugePages setting

Configure I/O operations for file system files



However, you can always fall back to the OS NFS client if there is misconfiguration. As a precaution, set the filesystemio_options parameter to SETALL if the OS supports it.

Init.ora Parameter “FILESYSTEMIO_OPTIONS” Reference Note (Doc ID 120697.1) on My Oracle Support provides more information.

EMC recommends that you create the different storage pools for different Oracle data types, such as datafiles and online redo log files. You can also enable FAST Cache when appropriate, depending on the data type. The exact layout requirements will change for each deployment, based on each customer’s requirements.

Table 15 shows a database layout example in which FAST Cache is enabled on the datafile pool only.

Table 15. Database layout for a consolidated Oracle environment example

Redo pool Data pool FRA pool (optional)

Application

Data Type Redo Logs Data files FRA files

FAST Suite policies

FAST Cache No Yes No

FAST Policies No Auto Tier No

Note: Oracle recommends using FRA (Fast Recovery Area) to enable storing all backup files in one place. If the FRA is enabled, the sizing logic for Oracle archived log and backup files is different from the Oracle datafiles and redo log files. For detailed support, consult your EMC representative.

Oracle licensing considerations

In this VSPEX Proven Infrastructure for virtualized Oracle Server, EMC recommends that you consider the Oracle Server licensing models to achieve better cost savings.

The Oracle processor licensing option is based on the interaction between the software and the hardware. For Oracle Enterprise Edition (Oracle EE), the licensing is based on the number of physical cores that are available to the installed Oracle software. For Oracle SE, the licensing is based on the number of processor sockets that are available to the installed Oracle software. Oracle does not permit the soft partitioning of CPUs as a means to calculate or limit the number of software licenses required for a physical server. Oracle regards VMware vSphere technology as soft partitioning. In a vSphere environment, you must license all hosts where the Oracle executable files are either installed or running or both.

This means that the design and size of the vSphere ESXi cluster, together with the placement and movement of virtual machines that host the Oracle executable files, are essential to minimize Oracle licensing costs. Understanding Oracle Certification Support and Licensing for VMware Environments White Paper provides details.

Configure the database data type layout



Design Guide

EMC Data Protection design considerations

All VSPEX solutions are sized and tested with EMC Data Protection products, including EMC Avamar and EMC Data Domain. If the solution includes EMC data protection components, refer to EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2 Design and Implementation Guide for detailed information on implementing these options into your VSPEX solution.

Chapter 6: Solution Verification Methodologies


Design Guide

Chapter 6 Solution Verification Methodologies


Overview .................................................................................................................. 46

Verifying the solution ............................................................................................... 46



Overview

This chapter describes the verification methodologies required for the hardware, application, and data protection aspects of the solution. Hardware consists of the computer's physical resources such as processors, memory, and storage. Hardware also includes the physical network components such as NICs, cables, switches, routers, and hardware load balancers. You can avoid many performance and capacity issues by using the correct hardware for the VSPEX for virtualized Oracle Server solution. Conversely, a single misapplication of a hardware resource, such as insufficient memory on a server, can affect the performance of the Oracle server.

Verifying the solution

EMC recommends that you test the new VSPEX for Oracle Database 12c application overlay proven architecture before deploying it to a production environment. This test of the proven overlay architecture confirms that the design achieves the required performance and capacity targets, and it will also identify and optimize potential bottlenecks before they affect users in a live deployment. This section gives a summary description of the high-level steps that we performed when verifying this solution.

Before verifying the Oracle Database 12c performance in the VSPEX Proven Infrastructure, ensure that Oracle Database 12c is deployed in the VSPEX Proven Infrastructure based on the EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide.

Table 16 describes the high-level steps that you need to complete before you can implement the Oracle Database 12c environment in production.

Table 16. High-level steps for application verification

Step Description Step

1 Understand key metrics of the Oracle database environment to achieve the performance and capacity that your business requires.

Understand key metrics

2 Use the VSPEX Sizing Tool for Oracle Database to determine the architecture and resources required by the VSPEX Proven Infrastructure implementation

EMC VSPEX website

3 Design and build the Oracle database solution on a VSPEX Proven Infrastructure. Run the tests, analyze the results, and optimize the VSPEX architecture

VSPEX Implementation Guides

Understanding the goal of the Oracle server testing helps to decide which metrics to capture and what thresholds must be met for each metric when running the Oracle server validation tests. We considered the key metrics shown in Table 17 to validate the VSPEX for virtualized Oracle server solution.

Understand key metrics

http://www.emc.com/campaign/vspex/index.htm



Design Guide

Table 17. Key metrics

Metric Threshold

Average CPU utilization (%) Less than 80%

Average wait time for User I/O Less than 20 milliseconds

Average wait time for Commit Less than 15 milliseconds

Use the VSPEX Sizing Tool

Use the VSPEX Sizing Tool to understand the basic metrics and thresholds to meet your customer’s business requirements.

For more information about using the VSPEX sizing tool, refer to VSPEX Sizing Tool for Oracle Database 12c available on the EMC VSPEX website.

Follow VSPEX Implementation Guides

After you have designed the VSPEX infrastructure, refer to the EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide for information on how to implement the solution.

In the VSPEX for Virtualized Oracle Database 12c solution, we ran the tests using an application such as TPC-C to validate the Oracle server performance. We recommend that you do the following:

Evaluate the workload and I/O pattern. If it is acceptable and the real workload is similar, you can use the test results as a reference. However, customers need to consider the potential risks.

Build a test environment first, and then copy and restore the production database to test the real workload and to verify the Oracle server performance if the real application workload types are different from what we validated in our test environment.

The EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide provides detailed configuration information.

http://www.emc.com/campaign/vspex/index.htm

Chapter 7: Reference Documentation


Design Guide

Chapter 7 Reference Documentation


EMC documentation ................................................................................................. 50

Other documentation ............................................................................................... 51



EMC documentation

The following documents, available from the EMC Online Support or EMC.com websites, provide additional information. If you do not have access to a document, contact your EMC representative.

EMC VNXe3200 Installation Guide

Proven Infrastructure Guide for EMC VSPEX Private Cloud with VMware vSphere 5.5 for up to 200 virtual machines

EMC VSPEX Proven Infrastructure for Virtualized Oracle

Data Protection for EMC VSPEX Proven Infrastructure White Paper

EMC VNXe Series Configuration Worksheet

Using a VNXe System with VMware NFS or VMware VMFS

Using a VNXe System with NFS File Systems

EMC Host Connectivity Guide for VMware ESX Server

https://support.emc.com/

http://www.emc.com/resource-library/resource-library.htm



Design Guide

Other documentation

The following Oracle documents are relevant to this solution:

Oracle Edition Comparisons

Oracle Software Investment Guide

Oracle Database Licensing

Oracle Processor Core Factor Table

The following VMware documents are relevant to this solution:

Understanding Oracle Certification Support and Licensing for VMware Environments White Paper

Oracle Databases on VMware Best Practices Guide

Best Practices for running VMware vSphere on Network Attached Storage White Paper

Performance Best Practices for VMware vSphere 5.5

Oracle documentation

VMware documentation

http://www.oracle.com/us/products/database/enterprise-edition/comparisons/index.html

http://www.oracle.com/us/corporate/pricing/sig-070616.pdf

http://www.oracle.com/us/corporate/pricing/databaselicensing-070584.pdf

http://www.oracle.com/us/corporate/contracts/processor-core-factor-table-070634.pdf

http://www.vmware.com/files/pdf/techpaper/vmw-understanding-oracle-certification-supportlicensing-environments.pdf

http://www.vmware.com/files/pdf/techpaper/vmw-understanding-oracle-certification-supportlicensing-environments.pdf

http://www.vmware.com/files/pdf/partners/oracle/Oracle_Databases_on_VMware_-_Best_Practices_Guide.pdf

http://www.vmware.com/files/pdf/VMware_NFS_BestPractices_WP_EN.pdf

http://www.vmware.com/files/pdf/VMware_NFS_BestPractices_WP_EN.pdf

http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.5.pdf

Appendix A: Qualification Worksheet


Design Guide

Appendix A Qualification Worksheet

This appendix presents the following topics:

VSPEX for Virtualized Oracle OLTP qualification worksheet ..................................... 54

Gather the information from the customer’s Oracle databases example .................. 54

Printing the qualification worksheet ........................................................................ 57



VSPEX for Virtualized Oracle OLTP qualification worksheet

Before sizing the VSPEX solution, gather the information from the customer’s Oracle databases using the qualification worksheet shown in Table 18. This worksheet is appropriate for qualifying multiple databases.

Table 18. Qualification worksheet for virtualized Oracle OLTP databases

Question Answer

Do you have an existing Oracle server database that you would like to size for in the environment?

Yes or No

How many databases do you want to deploy?

What is the size of the user database (GB)?

What is the annual growth rate (%)?

Do you intend to use FAST Cache? Yes or No

Do you intend to use FAST VP? Yes or No

What is the maximum number of IOPS (read/write)?

(Optional) What is the maximum number of concurrent users at peak loads?

You can use the Oracle Automatic Workload Repository to obtain this information as described in the Oracle Database Performance Tuning Guide 12c Release 1 (12.1).

Gather the information from the customer’s Oracle databases example

The Automatic Workload Repository (AWR) provides more of the information required to populate the EMC Oracle qualification worksheet from each Oracle database. The AWR provides key statistics on database performance, load, and resources (both internal and external). You can access this data using standard Oracle supplied scripts. You can obtain the remaining information from the customer or by using the simple queries that are provided in this appendix.

Use the init.ora Parameters section of the AWR report to calculate the System Global Area (SGA) and Program Global Area (PGA) values as shown in Figure 5.

Database memory settings

http://docs.oracle.com/database/121/TGDBA/toc.htm



Design Guide

Figure 5. init.ora Parameters from the AWR Report

You can type the following SQL query to confirm the maximum number of users who can connect to the database concurrently.

SQL> select SESSIONS_CURRENT, SESSIONS_HIGHWATER from v$license;

SESSIONS_CURRENT SESSIONS_HIGHWATER

----------------------------- ----------------------------------

5 249

1 row selected.

You can use the data and temporary file sizes to populate the DB Size (MB) column and calculate the total as follows:

SQL> select ltrim(to_char(sum(bytes)/(1024*1024))) as “Total size

(M)”

from (

select sum(bytes) as bytes from v$datafile

union

select bytes from v$tempfile);

Total size (M)

----------------------------------------

256000

1 row selected.

You can find the READ IOPS, WRITE IOPS, and Change Rate (MB/s) values in the IOStat by Function summary section of the AWR report. Figure 6 shows these values.

Find the number of concurrent users

Calculate database size

Find the datafile IOPS and change rate for the Redo logs



Figure 6. IOStat by function summary from the AWR Report

The following Oracle wait events, as shown in Figure 7, provide key response time statistics for the Oracle database.

Use db file sequential read to populate the User I/O column. Oracle recommends that this value be under 20 ms.

Use log file sync to populate the Commit column. Oracle recommends that this value be under 15 ms.

Figure 7. Foreground Wait Event from the AWR report

How to Tell if the IO of the Database is Slow (ID 1275596.1) on My Oracle Support lists typical I/O response times.

You can find the value used to populate the TPS column of the worksheet from transactions in the Load Profile of the AWR report, as shown in Figure 8.

Calculate user I/O time and commit time

Calculate transactions in the Load Profile



Design Guide

Figure 8. Transactions in a load profile from the AWR report

Printing the qualification worksheet

A stand-alone copy of the qualification worksheet is attached to this document in PDF format. To view and print the worksheet:

1. In Adobe Reader, open the Attachments panel as follows:

Select View > Show/Hide > Navigation Panes > Attachments.

or

Click the Attachments icon as shown in Figure 9.

Figure 9. Printable qualification worksheet

2. In the Attachments panel, double-click the attached file to open and print the qualification worksheet.

Appendix B: High-level Oracle Database Server Sizing Logic and Methodology


Design Guide

Appendix B High-level Oracle Database Server Sizing Logic and Methodology

This appendix presents the following topics:

Overview .................................................................................................................. 60

Resource considerations .......................................................................................... 60

Sizing considerations .............................................................................................. 60



Overview

The infrastructure supporting OLTP, including vCPU, memory, and the storage layout for Oracle server databases, must provide a robust, powerful, and flexible solution. Sizing the Oracle server depends on multiple factors, such as disk type, protection type, and cache. These resources must be included as part of the Oracle server sizing method.

Note: These manual sizing instructions may be used to provide an approximate single application sizing, if the VSPEX Sizing Tool is not available. The VSPEX Sizing Tool, with its multi application, multi instance capability, is recommended as the preferred sizing approach.

Resource considerations

To satisfy the performance requirements for Oracle Server databases, sufficient resources, including the compute and disk subsystem, must be ensured. This section defines the required resources for an Oracle server in a virtualized environment to provide predictable performance.

Sufficient disk utilization—Design the sizing tool to use the sufficient disk resources and to leave capacity for any possible peak disk activities.

Sufficient memory utilization—Design building blocks with sufficient system memory to support the designed workload with anticipated peak load activities.

Sufficient processor utilization—Design building blocks to have sufficient vCPU resources to support the designed workload and to anticipate peak load activities.

Sizing considerations

This section provides detailed sizing methods and recommendations for sizing each Oracle server instance:

Determine the virtual machine resources:

vCPU resources

Memory resources

Determine the OS capacity resources and IOPS

Determine the I/O workload and the disk type and number

Consider the advanced storage features

Select the VSPEX Proven Infrastructure



Design Guide

Table 19 shows an example of three different sized databases and the recommended user input.

Table 19. Example of user input for multiple Oracle databases

Database profile Maximum database size (GB)

Maximum database performance (R/W IOPS)

Maximum users for each database

Database 1 500 165/110 200

Database 2 500 990/660 1200

Database 3 1000 3960/2640 8000

Taking the user count for three different size databases in Table 20, and referencing Table 6, we can obtain the size of the virtual machine required.

Table 20. Required CPU and RAM for different size databases

Database profile Database model

Recommended CPU Recommended RAM (GB)

Small-sized database Small virtual machine for up to 200 users

2 8

Medium-sized database

Medium virtual machine for up to 1200 users

12 38

Large-sized database Large virtual machine for more than 1200 users

32 98

We recommend setting the capacity for each Oracle server to 100 GB and setting the OS IOPS to 50/100 IOPS for each small, medium, or large server OS volume. The EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide provides more details.

Use the following design methods to determine the I/O workload and the disk type, number, and capacity in the VSPEX Proven infrastructure.

Calculate the correct number of drives for IOPS first, and then calculate the size of the databases on the disk layout. Consolidate the calculation results for both IOPS and the capacity.

Include the additional disk requirements such as annual growth rate.

Set the IOPS for different disk types based on the actual test result.

Determine virtual machine resources

OS capacity resources and IOPS

Determine the I/O workload and the disk type, number, and capacity



Calculate the number of disks based on the following formula:

Required back-end IOPS=read IOPS +(write IOPS x RAID write penalty)

Disk number = required back-end IOPS / IOPS per disk

If you use EMC FAST suite technology, determine the active data set size to match the FAST Cache/VP size.

This section describes how to calculate the resources required in a VSPEX Proven Infrastructure for Oracle Database 12c OLTP. We used three working examples that show how the Oracle manual sizing method works. You can apply the methodology used in these examples to homogeneous provisioning pools with or without configuring FAST VP or FAST Cache.

The database entry medium-size Oracle database example, as shown in Table 19, has the following storage profile:

Database size of 500 GB

Database performance with 990 read and 660 write IOPS

Five percent annual growth gives a three-year capacity of 580 GB

The redo change rate is 2.5 MB/s

As described in Designing the storage layout, when calculating the storage requirement for a database, consider both I/O performance and capacity. First determine the RAID type of the pool and the drive-group size. In this solution, all datafiles reside on RAID 5 storage and the Oracle redo files reside on RAID 10.

Calculate the storage requirements for the database using the values collected in the Qualification Worksheet and use the recommendations in Table 21 and Table 22.

Table 21. RAID type, write penalty, and capacity utilization

RAID Capacity utilization

Multiple of Write penalty

Active drives

Parity drives

RAID 5 (4+1) 0.8 5 4 4 1

RAID 10(2+2) 0.5 4 2 4 N/A

Table 22. Random disk IOPS and bandwidth by drive type

Drive Type IOPS Bandwidth (R/W MB/s)

15 K SAS 180 30/25

10 K SAS 150 25/20

Near-line serial-attached SCSI (NL-SAS)

90 15/10

SAS Flash 3,500 90/75

Note: For the maximum supported IOPS for VNXe, consult an EMC representative.

Example 1: Homogenous pools without FAST suite



Design Guide

In this example, we calculated the drive load for the example medium-size database as follows:

Total I/O for RAID 5 = 990 + 4* 660 = 3630

The IOPS-based calculation has a higher request of spindles, including the higher speed of spindles and the larger quantity of spindles compared to a capacity-based calculation in a generic OLTP application. In the example in Table 23, a 500 GB OLTP database can have more than 3600 IOPS, which needs 25 of the 10 K rpm SAS or FC spindles to support the IOPS request. However, two 600 GB mirrored SAS/FC spindles can satisfy the capacity request completely. For this example, we recommend using 25 SAS/FC spindles as in Table 23.

Table 23. Storage pool calculation example without FAST Cache

Storage pool No. of drives Total capacity (GB)

Oracle data pool RAID 5

25 SAS drives

24 = 3,630/150

Round up to a multiple of 5 to allow for RAID 5 (4 +1) = 25 drives

600 GB x 25 x 0.8 = 120,000 GB

Oracle redo pool RAID 10

4 SAS drives

1 = (2.5 MB/s * 4) / 20 MB/s

Round up to a multiple of 4 to allow for RAID 10 (2+2) = 4 drives

600 GB x 4 x 0.5 = 1200 GB

Note: The data pool uses a 600 GB 10 K SAS drive with a random read/write workload. The redo pool also uses a 600 GB 10 K SAS drive with a sequential write workload. A conservative value of 20 MB/s per drive is used for sustained writes.

Consider the advanced storage features

If the FAST Suite (FAST Cache or FAST VP) feature is enabled, the sizing logic for VNXe is different from that for storage that does not use FAST Suite.

When calculating for performance with FAST Suite selected, the flash tier needs to serve the maximum number of I/Os; therefore, it has a higher calculation priority. We used a total back-end IOPS minus FAST Cache SSDs supported IOPS and then calculated the number of SAS/FC disks for this example.

Note: We recommend enabling the FAST Suite feature when each component of the storage array is running at reasonable levels, such as keeping the storage processor use at less than 70 percent.



The example in Table 25 sizes a workload for a data pool that is supplemented by FAST Cache. To configure FAST Cache, complete the following five steps.

Step 1: Determine the workload

The example in Table 25 uses the data from the database entry medium-size Oracle database as shown in Table 19.

Step 2: Size the FAST Cache

FAST Cache is best for small, random I/O where a small percentage of the data (active data set) produces the majority of the IO in the environment. EMC recommends sizing FAST Cache as follows:

Match the FAST Cache size to the size of active data set.

If the active dataset size is unknown, size FAST Cache to be 5 percent of your capacity.

Avoid enabling FAST Cache for Oracle log files for a small-block sequential workload

Note: Contact an EMC representative for help in determining the active data set size for customers.

In the example in Table 25, we implemented a two 200 GB flash drive FAST Cache as RAID 1. For the purposes of the calculation, we used a more conservative 50 percent cache hit rate, which means that there is a 50 percent chance that FAST Cache will handle the I/O.

This estimate is for cache that has been “warmed up” by being in place and running for a while. Initially, before the cache has a chance to “warm up,” the hit rate is lower.

Adjust the host IOPS served by mechanical drives by reducing IOPS by the FAST Cache hit rate percentage. Table 24 summarizes the calculations performed.

Table 24. FAST Cache hit rate and workload calculations

Database profile

Database Size(GB)

Host IOPS FAST Cache hit rate percent

Host IOPS after FAST Cache

Medium-sized Oracle

500 990+660=1650 50 495+330=825

Next, calculate the number of drives needed for the RAID levels to serve the required IOPS. To apply the performance calculation on a flash drive, use the random disk IOPS, as shown in Table 22, and the bandwidth by drive type to calculate the number of IOPS for the flash drive:

Flash as FAST cache = (495+330*2)/maximum supported IOPS for VNXe = ~2 aligned to 2 disks (RAID 1)

RAID 1 (1+1): (495+330*2) /3,500 = 2 drive total

Example 2: Sizing with FAST Cache



Design Guide

In this example, RAID 1 (1+1) is a default RAID type for FAST Cache, with two flash drives.

Step 3: Determine the required I/O load for non-flash drives

Apply the following formula to calculate the required IOPS served by non-flash drives:

Non-flash IOPS = total IOPS – FAST Cache IOPS

For example, calculate the drive load for database in the following way:

1650 – 825 = 825 IOPS

Table 25. Storage pool calculation example with FAST Cache

Storage pool Number of drives Total capacity (GB)

Oracle data pool RAID 5 (with FAST Cache enabled)

15 SAS drives

12 = (495+330*4)/150


600 GB x 15 x 0.8 = 7,200

2 EFD drives

1= (495+330*2)/3,500


200 GB x 2 x 0.5 = 200


4 drives

1 = 2.5 MB/ 20 MB/s

Round up to a multiple of 4 to allow for RAID 10 (2+2) =4 drives

600 GB x 4 x 0.5 = 1200

Determine the capacity The number of drives in a storage system is determined by the needs of both performance and capacity. The method described in Table 25 calculates the minimum number of drives needed to meet performance requirements. The number of drives also needs to be resolved to meet the storage capacity requirement.

Unless a database is unusually large, the storage requirement for performance usually determines the number of data drivers. In this example, the storage solution using the least number of drives from the previous calculation meets the capacity requirements.

Plan for future growth by having enough storage capacity and performance available to satisfy the workload’s near-future requirements. In this example, we included a 5 percent annual growth rate.



This example uses the steps in Example 3: Sizing with FAST VP to create a virtual provisioning pool using the FAST VP feature to calculate the number of drives required to meet both the performance and capacity requirements.

To create tiered pools, enable FAST VP, and then follow these steps:

Step 1: Determine the workload.

The example in Table 25 uses the data from the database entry medium-size Oracle database as shown in Table 19.

Step 2: Determine the required I/O load of the top tier and drive estimate

Allocate the pool resources. The capacity required for each FAST VP tier depends on the locality of the active data. As a starting point, use a capacity per tier of 5 percent flash, 20 percent SAS, and 75 percent NL-SAS. This capacity is sufficient because less than 25 percent of the used capacity will be active and relocations from the lowest tier will occur infrequently.

Note: Locality depends on the statistical distribution between the most frequently accessed and infrequently accessed capacity. The most frequently accessed data is stored on the highest performing storage devices. Occasionally, less frequently accessed data is read or written. This I/O has a longer host response time than more frequently accessed data located on higher performing storage devices.

In this example, as shown in Table 26, we use a two-tiered FAST VP pool, which includes flash and SAS drives. This provides consistently good performance using a-tier pool comprised of flash and SAS. NL-SAS can be added later when capacity growth and aged data are required. We assumed a locality of 10 percent, which means that about 10 percent of the data is used most frequently. In this example, RAID 1 is a default RAID type for FAST VP.

Table 26. Example workload for a two-tier FAST VP pool capacity

Database profile

Database size (GB)

Percent locality Top tier capacity (GB)

Other capacity (GB)

Medium-sized Oracle database

500 10 50 450

Note: During the calculation, we considered the ideal maximum IOPS of flash disks and the real run-test values. For example, the ideal maximum IOPS of a flash disk can be 3,500, but when a flash disk such as FAST VP (upper tier) is used, the usable space for an application is limited. The most frequently accessed data can be larger than the usable space in flash (for example when using 2 x 200 GB FAST Cache SSDs to serve a 500 GB OLTP user database with 300 GB most frequently accessed data). To keep other disks, such as SAS/FC/SATA, storing the less frequently accessed data or used as the lower tier in FAST VP, the actual supported IOPS of the flash disk may be less than 3,000.

Example 3: Sizing with FAST VP



Design Guide

+Step 3: Determine the required I/O load of the other tiers and drive estimate

Not all host IOPS require the highest tier. The lower tier has a performance margin that can also be productive. Using the workload captured in Table 19, subtract the FAST VP top tier to get an estimate of the IOPS load on the lower tier using the following calculation:

(990+ (660*4))*0.1= 363 IOPS

Always check that the lower tier can sustain the required IOPS load. In this case, a RAID 5 SAS group has about 363 IOPS, which reduces the host response times for data on the lower tier.

Step 4: Determine the capacity of FAST VP pool

Calculate the FAST VP pool storage capacity by adding the capacity of the usable drive space of all the data drives in the tiers. In this example, we used 600 GB capacity SAS drives and 200 GB flash drives. You can substitute the other capacity or speed drives in the calculation.

Note: Database redo logs have a predictable sequential write workload, and this type of activity does not benefit from relocating to flash. EMC recommends that you pin them to their existing tier.

Table 27. Storage pool calculation Example with FAST VP

Storage pool Number of drives Total capacity (GB)

Oracle data pool RAID 5 (with FAST VP enabled)

5 SAS drives

3 = (990+660*4)*0.1/150


600 GB x 5 x 0.8 = 2400

2 EFD drives

1= (990+660*2)/3,500


200 GB x 2 x 0.5 = 200


4 drives

1 = 2.5 MB/ 20 MB/s

Round up to a multiple of 4 to allow for RAID 10 (2+2) =4 drives

600 GB x 4 x 0.5 = 1200

The storage solution with the fewest number of drives that meets the performance and capacity requirement is usually the best solution. However, a more reasonable solution matches the additional workload and expected annual data growth. In this example, flash drives with RAID 1 are a better solution that fits the requirements and uses fewer drives. Also, the lower tier performance, which includes NL_SAS(RAID 6), is compatible with extended future loads.



Select the right VSPEX Proven Infrastructure

After sizing the application and determining the resources required and the recommended disk storage layouts, use the following steps to select the right VSPEX Proven Infrastructure based on the calculated results:

1. If the customer wants to deploy other applications in the same VSPEX Proven

Infrastructure, refer to the VSPEX Design Guides for those applications to determine the total resources required and the recommended storage layouts for the combined workload.

2. Aggregate the required virtual machine resources (number of disks, total IOPS, and so on) for all applications.

3. Discuss with your customer the virtualization platform they want to use to meet their business requirements.

4. Refer to the EMC VSPEX Proven Infrastructure Guide and calculate the number of disks required for the VSPEX private cloud pool.

5. Aggregate the total number of disks required, including the number of disks for the combined applications, for the VSPEX private cloud pool and for hot spares.

6. Calculate the total number of virtual machines based on the compute and storage resources of the combined applications workload. Refer to the EMC VSPEX Proven Infrastructure Guide for guidance.

Note: Ensure that the selected VSPEX Proven Infrastructure supports the total number of disks required for the combined applications and the private cloud. If it does not, you may need to upgrade to the next VSPEX Proven Infrastructure model. For detailed support, consult your EMC representative.

emc vspex for virtualized oracle database 12c oltp - dell emc · oracle database 12c ... emc vspex...

Documents