mongodb & emc xtremio: validation of consistent performance

White Paper

EMC E-LabTM Vertical Solutions Group

Abstract

This white paper provides technical validation for deploying MongoDB on the EMC XtremIO all-flash array and explains the key solution benefits.

May 2015

MongoDB & EMC XtremIO Validation of Consistent Performance, On-Demand Scale-Out Capacity, Inline Data Reduction & Agile Snapshot Services

MongoDB & EMC XtremIO Validation of Consistent Performance, On-Demand Scale-Out Capacity,

Inline Data Reduction & Agile Snapshot Services

2

Copyright 2015 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

All trademarks used herein are the property of their respective owners.

Part Number H14226

3 MongoDB & EMC XtremIO Validation of Consistent Performance, On-Demand Scale-Out Capacity,


Table of contents

Executive Summary .............................................................................................................................. 4

Introduction.......................................................................................................................................... 5 MongoDB ......................................................................................................................................... 5 Business Case ................................................................................................................................. 5

MongoDB & XtremIO Solution Benefits ................................................................................................ 6 Consistent Performance ................................................................................................................... 7

Performance Tests for MongoDB Database Operation in XtremIO ................................................. 7 Test Summary ............................................................................................................................. 7 Test Details ................................................................................................................................. 7

On-Demand Scale-Out Capacity and Performance .......................................................................... 11 Cost Efficient Inline Data Reduction ............................................................................................... 11

Overall Efficiency for Reduced Total Cost of MongoDB Infrastructure ......................................... 11 Agile Snapshot Services ................................................................................................................. 12

Clone MongoDB Database in Minutes with XtremIO Snapshot ................................................... 13 Simplicity of Management.............................................................................................................. 14 Advanced Data Protection & High Availability ................................................................................ 15 Free Data at Rest Encryption ........................................................................................................... 16

About the Testing environment .......................................................................................................... 17 Server Configuration ...................................................................................................................... 17 Software Configuration .................................................................................................................. 17

Conclusion ......................................................................................................................................... 18

References.......................................................................................................................................... 19



4

Executive Summary This document provides technical validation for deploying MongoDB 2.6.7 on the EMC XtremIO flash array.

This white paper examines the solution benefits and provides an introduction for deploying an XtremIO storage array in a MongoDB environment, including:

Consistent performance with



Introduction

MongoDB is one of the leading next-generation databases that helps businesses transform their industries by harnessing the power of data. The worlds most sophisticated organizations, from cutting-edge startups to the largest companies, use MongoDB to create applications never before possible at a fraction of the cost of legacy databases. MongoDB is the fastest-growing database ecosystem, with over 9 million downloads, thousands of customers, and over 750 technology and service partners.

Most MongoDB deployments use Direct Attached Storage (DAS), primarily because DAS is initially seen as cost effective, simple, and quick. Nearly all MongoDB environments start out small with a few nodes and basic DB Operations/sec requirements. The challenges begin as companies look to scale.

Difficulties arise as these next generation applications gain wider adoption, resulting in growth and increased performance requirements of the underlying infrastructure. As the environment scales, issues with DAS can arise around the complexity and inefficiency in managing performance, capacity, and availability across islands of DAS instances, increasing risk and cost to the organization.

For these reasons, many IT organizations are consolidating their MongoDB DAS architectures onto XtremIO AFAs. XtremIO is the leading all-flash array that transforms MongoDB workloads through a consolidated all-flash architecture which delivers scale-out, on-demand performance and capacity with consistent sub-millisecond application latency.

A rich set of inline, always-on data services including deduplication, compression, and space-efficient snapshot services ensure XtremIO can be remarkably cost-effective when compared to DAS architecture for MongoDB environments. Unlike DAS, XtremIO also offers advanced data-at-rest encryption and the richest availability services including non-disruptive upgradesall with no performance penalties, keeping MongoDB environments available and protected. And the ability to consolidate non production and production environments coupled with XtremIOs in-memory cloning enables new capabilities for agile application development.

MongoDB

Business Case



6

MongoDB & XtremIO Solution Benefits EMC XtremIO all-flash array is the most optimal choice for customers who want easy to use, cost effective, and consistently performing scale-out storage with the additional benefits of data reduction and data services that DAS can not offer. The benefits for deploying MongoDB on XtremIO are the following:

Consistent Performance

On-Demand Scale-Out Capacity and Performance

Cost Effective Inline Data Reduction

Agile Snapshot Services

Simple Management and Administration

Advanced Data Protection and High Availability

Free Data at Rest Encryption

In this paper we discuss each of these advantages in detail.



XtremIO is the ideal storage platform for accelerating MongoDB databases, providing outstanding transactional speed with consistently low latency. XtremIOs automatic balanced data distribution enables consistent performance across the entire lifecycle of the volumes, regardless of I/O patterns, data placement, or cluster size. With the benefits of flash-optimized algorithms and active-active controllers, XtremIO provides consistent performance even under mixed workloads. Furthermore, this performance stays completely balanced, and latency remains under a millisecond even at scale.

Performance Tests for MongoDB Database Operation in XtremIO

The tests below illustrate the consistent performance of XtremIO using a two X-Brick cluster using different query and write operations in a MongoDB environment. We performed 2 test scenarios, utilizing MongoPerf to simulate different MongoDB operations on an XtremIO volume.

MongoPerf is a utility to check storage I/O performance independently of MongoDB. It records and presents tests of random I/O.

Test Summary

Test Scenario

Details Storage IOPS Latency DB Operations/s

Test 1 Simulate MongoDB database write operations on 1TB test file

50,000 (50% R, 50% W)

< 0.25 ms 25,000

Test2 Simulate MongoDB mixed (query and write)operation on 1TB test file

55,000 (60% R, 40% W)

< 0.25 ms 45,000

Note: The table above illustrates the performance that can be expected from a single MongoDB node connected to an XtremIO X-Brick cluster. As additional nodes are added, a dual X-Brick cluster (like the one used in this testing) is capable of servicing substantially higher storage IOPS and bandwidth.

Test Details

Test scenario 1: Simulate Write MongoDB Database Operations by MongoPerf

MongoPerf runs with the following parameters on MongoDB LUN:

nThreads:16,fileSizeMB:1000000, mmf:true, r:false, w:true

Consistent Performance



8

Inflating Test File (100% write operations)

Volume Bandwidth in MB/s during inflating the 1000GB test file

The graph shows that XtremIO was easily able scale to nearly 2000MB/s of write bandwidth while inflating a 1TB test file due to its multiple active/active controller architecture.

Simulating DB Operations on the Testing file

Storage IOPS during the DB Operations



Storage Latency remained constant in microseconds during the DB Operations

Test scenario 2: Simulate Mixed MongoDB Database Operations by MongoPerf MongoPerf runs with the following parameters on MongoDB LUN:

nThreads:16,fileSizeMB:1000000, mmf:true, r:true, w:true

Inflating Test File



10

Volume Bandwidth in MB/s during inflating the 1000GB test file

Simulating DB Operations on the Testing file

Storage IOPS during the DB Operations

Storage Latency in microseconds during the DB Operations



MongoDB scales easily to deliver fast, real-time performance to meet your business needs. XtremIO is the only AFA that can scale out to multiple nodes with linear increases in IOPS, bandwidth, and of course capacity all with the same consistent sub-millisecond latency. As your MongoDB application needs grow, XtremIO will scale IOPS and capacity linearly with consistent latency in parallelon demand, all while online.

The XtremIO storage system is based on a scale-out architecture. The system begins with a single building block, called an X-Brick. When additional performance and capacity are required, the system scales out by adding X-Bricks. Performance scales linearly, ensuring that two X-Bricks supply twice the IOPS, four X-Bricks supply four times and so on, of the single X-Brick configuration. Latency remains consistently low as the system scales out.

Expected Performance 2 X-Bricks 4 X-Bricks 6 X-Bricks 8 X-Bricks

Typical I/O Profile range (10% - 40% Writes)

IOPS 220K 300K

440K 600K

660K 900K

880K 1.2Million

Latency < 1ms < 1ms < 1ms < 1ms

Aligning with the performance features of MongoDB, flash becomes a vital component of the underlying architecture; however managing the economics of flash in a traditional DAS deployment becomes the challenge. XtremIOs data reduction features consisting of compression deduplication, thin provisioning, and space saving snapshot services drive extraordinary data efficiencies, resulting in a reduced total cost of ownership. Unlike other all-flash arrays, all data services are always-on and inline with no performance impact.

Overall Efficiency for Reduced Total Cost of MongoDB Infrastructure

The XtremIO storage array that was tested, leveraging the benefits of thin provisioning and inline data reduction, achieved an overall capacity efficiency of 22:1. With 27.083TB used volume capacity, XtremIO only consumed 3.98TB physical capacity. It is worth noting that about 7x the storage capacity requirement was reduced inline by always-on deduplication and compression.

On-Demand Scalable Capacity and Performance

Cost Efficient Inline Data Reduction



12

For more information on Data Reduction, please refer INTRODUCTION TO THE EMC XTREMIO STORAGE ARRAY (Ver. 3.0) on http://support.emc.com

MongoDB provides developers with the ability to significantly increase productivity and deliver faster time to market in both agile DevOps and traditional test /dev workflows. Traditional storage copy operations not only impact performance, but are also incredibly slow, painful, and wasteful usually requiring hours or days to create brute force copies, hindering Mongos ability to accelerate agile development cycles.

XtremIOs revolutionary snapshot services advance agile processes and enable new capabilities that transform your business. Utilizing unique in-memory instantaneous, full performance snapshots, XtremIO can consolidate test/dev and production environments onto one platform. This results in providing developers with parallel environments, generating as many copies as they want without impacting production SLAs. Consolidating these workloads and snapshots improves TCO, accelerates workflow, and even enables dev copies to be tested in the actual production environment, effectively making the code better.

For more information on XtremIO Snapshot, please refer INTRODUCTION TO THE EMC XTREMIO STORAGE ARRAY (Ver. 3.0) on http://support.emc.com

Agile Snapshot & Clone Services

http://support.emc.com/



Clone MongoDB Database in Minutes with XtremIO Snapshot

Customers can utilize XtremIO to clone the database for backup, test/dev, and production copies. The performance of the clone will be identical to that of the production volume.

Copying a database from one node to another can be done in the 4 steps below.

(Note: To obtain an application consistent state, all necessary operations needs to be done in a database and operating system, before taking the copy)

Step 1: Take and name a copy on the volume(s) hosting the database

Step 2: Connect the copy to destination host by 4 clicks



14

Step 3: Scan and mount the snapshot on the destination host. For example, in Linux you can use

# rescan-scsi-bus.sh -i -r

Edit /etc/fstab add the correct content

/dev/mapper/mpathb1 /mongodb ext4 discard,noatime 1 2

#mount /mongodb

Step 4: Start the database pointing dbpath to the snapshot Mongod --dbpath /mongodb/data --logpath /mongodb/log/config.log --port 27017 --fork

A MongoDB clone is created. Customers can utilize this MongoDB clone for developing and testing paralleled with production.

MongoDB needs to be run on a storage platform that stands up to its ease of use. Large scale DAS deployments result in islands of storage that are inefficient and difficult to manage. Every time a server is added to the environment in order to keep up with heightened application requirements, management complexity is made worse since each server must be administered separately. Unlike DAS, XtremIO can consolidate all mongo clusters and applications onto one simple platform so you never have to worry about overseeing countless storage silos.

Similar to MongoDBs ease of use, XtremIO is incredibly easy to configure and manage, with no RAID configuration, no tiering, and no need for manual performance tuning and optimization settings. XtremIO requires no storage skills and little training and MongoDB admins can monitor all volumes down to each MongoDB instance providing granular reporting.

With XtremIO, managing a scale-out MongoDB environment has never been easier. Easily track your effective vs physical capacity growth to make planning your expansion simple. As you expand your cluster size and add more X-Bricks, your dataset automatically rebalances across the cluster online and all arrays can be managed under a single GUI. XtremIOs tag and search features align storage management with business needs, allowing you to tag volumes with labels for test/dev or a specific mongo instance, and then give you the ability to filter your reports by tag.

Simplicity of Management



Figure below shows the GUI's Dashboard, enabling the user to monitor the system's storage, performance, and alerts.

Figure below shows the GUIs Dashboard enabling the user to monitor the hardware status

MongoDB environments are increasingly mission-critical and require architectural HA and business continuity services to match. XtremIO is N-way active-active with no single point of failure. XtremIO is field measured in excess of 99.999% availability and when paired with EMC solutions to enable active/active data centers in excess of 99.9999% availability. It also offers best-of-breed EMC replication and data mobility solutions. Additionally, XtremIOs full performance and zero overhead snapshot services can also be used for backup and logical corruption protection. Lastly, XtremIOs data protection scheme, XDP, leverages the random access nature of flash to deliver much lower capacity overhead,

Advanced Data Protection & High Availability



16

superior data protection, and much better flash endurance and performance than any RAID algorithm.

For more information on Advanced Data Protection and High Availability, please refer INTRODUCTION TO THE EMC XTREMIO STORAGE ARRAY (Ver. 3.0)on http://support.emc.com

With customers adopting MongoDB across all areas of the business, the sensitivity of data becomes a key priority. XtremIO offers free, inline array based data-at-rest encryption, allowing customers to meet compliance and regulatory needs without added cost and complexity. This eliminates the need for any 3rd party encryption software.

Data at Rest Encryption (DARE) provides inline encryption to ensure that all data stored on the SSDs is unusable if the SSD media is removed. This prevents unauthorized access in the event of theft or loss during transport, and makes it possible to return/replace failed components containing sensitive data.

For more information on Data at Rest Encryption, please refer INTRODUCTION TO THE EMC XTREMIO STORAGE ARRAY (Ver. 3.0)on http://support.emc.com

Free Data at Rest Encryption

http://support.emc.com/http://support.emc.com/



About the Testing environment The diagram below represents the hardware configuration in this paper

One Cisco UCS C460 server was used for testing

Component Description Processor 4 intel XeonRE A E704830 @ 2.13 GHz Memory 128 GB HBA 8 Gbps 1 port FC HBA x 4

Component Description EMC XtremIO Release 3.0.1 MongoDB Release 2.6.7 Operating System CentOS Release 7.0 x86_64

For detail host configurations please refer Host Configuration Guide For XtremIO Versions 2.2.x - 3.0.1 at http://support.emc.com

Server Configuration

Software Configuration

http://support.emc.com/



18

Conclusion XtremIO transforms storage infrastructure for your MongoDB environments. XtremIO provides a revolutionary scale-out platform that consolidates and accelerates dev/test and production workflows with incredible data reduction and snapshot services. Developers, engineers, and administrators will benefit from a consolidated and vastly simplified infrastructure to boost MongoDB performance, capacity planning, and availability, all while reducing and controlling storage costs.

Consistent performance with



References The documents referred in this paper are listed below

INTRODUCTION TO THE EMC XTREMIO STORAGE ARRAY (Ver. 3.0)

Host Configuration Guide for XtremIO Versions 2.2.x - 3.0.1

MONGODB MANUAL 2.6

Executive SummaryIntroductionMongoDBBusiness Case

MongoDB & XtremIO Solution BenefitsConsistent PerformancePerformance Tests for MongoDB Database Operation in XtremIOTest SummaryTest DetailsTest scenario 1:Simulate Write MongoDB Database Operations by MongoPerfInflating Test File (100% write operations)Simulating DB Operations on the Testing fileTest scenario 2:Simulate Mixed MongoDB Database Operations by MongoPerfInflating Test FileSimulating DB Operations on the Testing file

On-Demand Scalable Capacity and PerformanceCost Efficient Inline Data ReductionOverall Efficiency for Reduced Total Cost of MongoDB Infrastructure

Agile Snapshot & Clone ServicesStep 1: Take and name a copy on the volume(s) hosting the databaseStep 2: Connect the copy to destination host by 4 clicksStep 3: Scan and mount the snapshot on the destination host.Step 4: Start the database pointing dbpath to the snapshot

Simplicity of ManagementAdvanced Data Protection & High AvailabilityFree Data at Rest Encryption

About the Testing environmentServer ConfigurationSoftware Configuration

ConclusionReferences

mongodb & emc xtremio: validation of consistent performance

Documents