two-tier storage strategy: a gigaom market landscape report...transport, which are growing in...

21
Two-Tier Storage Strategy: A GigaOm Market Landscape Report Credit: Vladimir_Timofeev By Enrico Signoretti

Upload: others

Post on 22-Aug-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

Two-Tier Storage Strategy: AGigaOm Market LandscapeReport

Credit: Vladimir_Timofeev

By Enrico Signoretti

Page 2: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

Two-Tier Storage Strategy: A GigaOmMarket Landscape Report11/08/2018

Table of Contents

Summary

Market Framework: Why a Two-Tier Storage Strategy

Maturity of Categories

Considerations for Using Two-tier Solutions

Vendor Review

Near-term Outlook

Key Takeaways

About Enrico Signoretti

About GigaOm

Copyright

1

2

3

4

5

6

7

8

9

10

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 2

Page 3: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

1 Summary

Most organizations face a growing number of data storage challenges. The number ofapplications and services IT departments support are increasing; with users accessing data fromeverywhere, at any time, and from different devices. The variety of workloads are increasing aswell, with applications competing for resources from the same infrastructure. The cost oftraditional infrastructure is incompatible with the exponential growth of unstructured data, bigdata workloads, or Internet of Things (IoT) applications.

The traditional classifications of primary and secondary data, that correlate primary withstructured data/databases and secondary with unstructured files, are no longer valid. With databecoming one of the most important assets for organizations, structured and unstructured dataare now equally important, and they should be protected and treated accordingly. A newprimary/secondary classification has emerged and is based on the value of data, with dataindexation and classification. Coupling this new classification with a two-tier storageinfrastructure can help reduce costs and simplify the process, especially if the two tiers areintegrated and data can move seamlessly between them.

Modern applications can now be divided into two families: latency-sensitive or capacity-driven.The first group needs data as close as possible to the processing engines (e.g. CPU, GPU, etc.)while the latter usually requires easily accessible data spanning multiple devices across thenetwork. New infrastructure designs must take this division into account to cope quickly withnew and ever-evolving business requirements.

In this report we analyze several aspects of the two-tier storage strategy including:

• Why a two tier-storage strategy

• Different types of tier integration

• How to manage file-based storage in a two-tier storage strategy

• Automated tiering and application-based profiling

• Security considerations about two-tier storage strategy

• Key players

• First steps for adopting a two-tier storage strategy and improving overall infrastructure TCO

Key findings include:

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 3

Page 4: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

• A two-tier storage policy is easy to adopt and saves money while optimizing infrastructureresources.

• This approach enables improved ROI on infrastructure, and makes future investmentsnecessary only where and when they are required, as opposed to months or years inadvance.

• The infrastructure layout is highly simplified and optimized to take advantage of the cloudand seamlessly integrate it with the rest of the infrastructure. This also helps to simplify andredistribute budget resources from CAPEX to OPEX.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 4

Page 5: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

2 Market Framework: Why a Two-Tier StorageStrategy

The first reason for adopting a two-tier strategy is to enable simplification and cost optimization.But, if we look deeper into the problem, we can see that this strategy becomes necessary tobuild a sustainable infrastructure over time.

New approaches to data center design have left some gaps in storage management. Forexample, most converged and hyper-converged infrastructures (CI/HCI) are focused on latency-sensitive workloads and are not designed to scale capacity while simultaneously optimizingcost. This leads end users to consider alternative solutions apart from the data center, allowingthem to meet all kinds of requirements.

Most primary storage systems currently offered have common characteristics. They all address,in one way or another, challenges concerning total cost of acquisition (TCA) and total cost ofownership (TCO), albeit with limits. The cost per Gigabyte (GB) is reasonable but not optimal inabsolute terms and, if your organization is storing huge amounts of data, overall TCA and TCOfigures will not be attractive. A second storage tier is needed to serve capacity-drivenrequirements without neglecting security, resiliency, or availability. It must be designed withthese characteristics in mind, along with reduced $/GB when compared to the first tier,sacrificing some latency but not overall throughput and efficiency, nor scalability.

The ideal architecture, designed from scratch, should only have two storage tiers, seamlesslyintegrated to satisfy the dual objectives of performance and capacity. The first tier should beblock or file storage, while the second should be based on private or public object storage.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 5

Page 6: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

The main characteristics of the performance tier, delivered on premises, include:

• Speed: Solid state drives (SSDs) have superior price/performance ($/IOPS) and latency. Inaddition, systems designed around SSDs using non-volatile memory express (NVMe)transport, which are growing in popularity, can easily provide sub-millisecond I/O responselatency at reasonable costs.

• Scalability: Even though the external storage array market is still dominated by scale-upsystems, with some being federated to mimic a scale-out infrastructure, pure scale-out is theway to go when possible. Most HCI architectures are based on this kind of design and, if it iswell implemented, scalability and forklift upgrades become problem-free while theincremental added latency introduced by the network is often acceptable.

• Ease of use and integration: Next-generation storage systems are typically feature-rich andvery easy to administer. In most cases, they can be managed directly from the hypervisor,and provide self-tuning features making the system very user-friendly.

• Advanced Analytics: Many vendors provide helpful insight into their storage systems andthe next layers up in the stack (e.g. hypervisor), helping the end user receive alerts forpotential issues, continuously and automatically optimizing storage, and making strategicdecisions about capacity planning.

The main characteristics of the capacity tier, no matter if it is delivered from the cloud or onpremises, include:

• Scale-out capacity: Hard disk drives (HDDs) have the best price per Gigabyte ratio and areusually installed in nodes connected in a scale-out fashion to ease system expansions. Somevendors already offer hybrid or all-flash systems, a trend that will accelerate with theforecasted decrease of NAND flash memory prices in the upcoming quarters.

• Data protection: Huge repositories using high-capacity HDDs require a different approachto data protection than used in the performance tier: specifically, object replication, but alsoerasure coding and multi-site distribution. For example, an object store can also be a backuptarget and, due to its nature, also provide automatic electronic vaulting.

• Geo-distribution: Disaster recovery, as well as follow-the-sun applications or mobile access,are part of the product DNA.

• Energy efficiency: Next-generation hard drives can be very power savvy. These drives maybe spun down when they are idle and therefore are optimal for establishing long-term dataarchives.

• Commodity hardware: Almost all object storage systems are software solutions which aredeployed on inexpensive, commodity hardware. Some of them are also deployable as virtualstorage appliances (VSAs).

• Automation: The life of objects stored in the system is usually regulated by policies:

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 6

Page 7: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

retention, versions, number of copies, access policy, and so on. This has both a direct andindirect impact on many aspects of the TCO.

• Multi-protocol: Object storage enables a flexible approach across many applications. Datacan be accessed through APIs, traditional file sharing protocols, modern sync & share, blockprotocols, or even through scale-out file interfaces like Hadoop distributed File System(HDFS). Many file workloads can be served through specialized gateways. These gatewayscan also have a local cache, to speed up access and metadata operations, minimizing trafficto the HDDs themselves.

• Integration: Software vendors are releasing integrations with object storage systems:content management, mail, archiving, backup servers, data collection servers, and so on.Each new application added on top of an object storage system automatically inherits all itsbenefits and becomes easier to manage.

These two lists are not definitive, but if you look at your particular environment, keeping in mindthe basic characteristics and features of both types of storage systems, you will find moreevidence of what I am saying. With these two storage tiers, adequately integrated, it is possibleto cover all workloads.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 7

Page 8: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

3 Maturity of Categories

Vendors have made several attempts to simplify storage infrastructure but, in most cases, theyhave failed – either because the technology was not up to par, or the added complexity andcosts did not deliver a better TCO.

In particular, approaches such as Hierarchical Storage Management (HSM), storagevirtualization, and automated tiering have all failed. Even though they were able to move dataacross two or more storage tiers, several drawbacks make these technologies expensive in realworld scenarios and hard to adopt.

• HSM was typically limited to files, rarely integrated at the application level, and difficult tomanage over time.

• Virtualization added an additional layer to procure, manage and maintain, and requiredcompliance with strict compatibility matrices issued by the vendors.

• Automated tiering was usually limited to the scalability of the single system, good for small tomedium enterprises but without the scalability and multi-tenancy capabilities necessary forlarger organizations.

Technology is quickly evolving towards the integration of primary storage and object/cloudstorage through the adoption of popular protocols (S3 for example). This approach bringsseveral benefits:

• Simplification: the primary storage is usually in control of data movement, based on policiesor other simpler manual mechanisms, without any additional layer to manage.

• Scalability: the scalability of the secondary tier is no longer an issue due to the architectureof hybrid or cloud object-based solutions.

• Seamless: some solutions manage block, files, and objects in a two-tier fashion with thesame tools and UI seamlessly from the application or user standpoint.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 8

Page 9: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

Different types of tier integration

Depending on applications or workloads, primary storage can be block or file. At the same time,the second tier is usually based on an object store realized in a public or private cloud.

A ‘one-size-fits-all’ strategy is not possible, but neither is a silo for a single application orworkload. A two-tier storage strategy simplifies the infrastructure and provides applications therequired characteristics at a reasonable cost. The goal is the integration of the two tiers,abstracting complexity and providing a seamless data movement between them.

By establishing business-oriented policies to move data between tiers, human intervention islimited and resources are allocated automatically according to application and user needs. Thiskind of automation simplifies storage provisioning and capacity planning. Primary storagecapacity requirements usually grow more slowly over time and are more predictable than file-and object-based storage (which is less expensive and easier to expand). For example, thecapacity needed for an ERP database is much more predictable over time than a file repositorythat stores files coming from several sources.

Another advantage of this approach comes from the cloud. When the secondary tier resides in apublic cloud, the infrastructure expenditure moves from capital expenditure (CAPEX) tooperating expense (OPEX). This helps limit the investment on primary storage (usually the mostexpensive), improving the overall TCA and TCO of on-premises infrastructure.

Integration between the two tiers is usually associated to the type of primary storage involved:

• Block storage: modern primary storage arrays have two ways of managing infrequentlyaccessed data and moving it to the cloud, namely volume snapshots and clones. These can

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 9

Page 10: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

be offloaded to an object store to save space and increase retention time, while cloudtiering creates data volumes that span on-premises arrays and a cloud back-end. Mostadvanced solutions offer interesting opportunities when it comes to asynchronous datareplication, allowing the implementation of a simple cloud-based DR solution.

• File storage: Large filers, frequently adopted for Big Data Analytics, High PerformanceCompute (HPC), and other high performance workloads, can automatically offloadinfrequently accessed files to the cloud or compatible object stores. In this case, the goal isto have new and most frequently accessed files on the performance tier, moving all others toless expensive storage. Some of these products, often difficult to back up because of theirsize, are also capable of making volume copies or syncing entire volumes or snapshots tothe cloud.

Large enterprises, who often require both block and file storage for different workloads, canadopt a similar strategy and get even better benefits. In fact, all cold data can be consolidated ina single repository and the economics of object stores improves slightly when large capacitiesare involved.

Thanks to the popularity of AWS Simple Storage Service (S3) protocol, the number of solutionsavailable in the market are increasing over time. Many applications and devices are nowcapable of writing data directly to an object store, and storage arrays are no exception. Somestorage vendors also offer support to connect their arrays to other major cloud providers suchas Google Cloud or Microsoft Azure. Moreover, an increasing number of data services offeredby these systems take advantage of external, cheaper, and reliable storage options for cold dataor secondary data copies.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 10

Page 11: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

4 Considerations for Using Two-tier Solutions

Organizations that are rethinking and modernizing their infrastructure must pay particularattention to how data is stored, accessed, and moved across different storage tiers to avoidcomplexity and proliferation of data and application silos. Solid state storage technologyintegrated with cloud is the key, but there are several other aspects to contemplate, such as:

• Workload and application classification. Applications and workloads should be classifiedon their basic characteristics regarding latency and capacity needs. This will enable insightinto the size of each tier and their composition. This classification operation should berepeated over time to gain insights about data growth and its composition for capacityplanning. The process is now simplified by management tools included in practically everymodern storage system.

• Performance requirements. Most performance analysis is executed on the first tier but, in atwo-tier storage infrastructure, the performance expectations of the capacity layer must betaken into account. For example, even though absolute latency of the second tier is of lesserimportance, integrating it with the performance layer requires a consistent and predictablelatency profile. In this regard, on-premises object stores are superior to public cloud.

• Type of storage systems to consider. While most primary storage systems could be eitherscale-up or with limited scale-out capabilities depending on performance requirements,scale-out is now the predominant choice for the capacity tier. Object storage systems areusually scale-out and based on commodity hardware, simplifying their expansion over time.Public cloud storage options could be less expensive for some types of workloads, offerflexible configuration, and minimize capital expenditure. Operational expenditure for publiccloud should be evaluated relative to private cloud in order to optimize overall expenditure.

• Integration and protocols. Integrating primary with secondary storage is straightforward.Most storage systems enable connection to secondary storage resources using the S3 API,the de facto standard for object storage. Some systems use additional APIs, increasing thefreedom of choice. At the same time, some vendors provide end-to-end solutions aimed atsimplifying infrastructure management.

• Automation. Data movement between the two tiers is usually managed by policies definedby the administrator and/or end user. Depending on the type of primary storage (block orfile), data can be moved according to its type, age, size, last access, and other specificmetadata. These rules are usually configured on the primary storage or the externalappliance that manages them, which is also manages data movement between the tiers.

• Multi-tenancy. Multi-tenancy is uncommon in primary storage systems and newtechnologies like NVMe minimize risk resource contention. On the other hand, theseparation of workloads and users is key for secondary storage systems. The object store isthe capacity tier integrated with the performance tier, but it can also be used as a repositoryfor many other applications and as persistent storage for local and remote devices.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 11

Page 12: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

Examples include: data protection solutions that use object storage to manage archives oras a backup target, cloud storage applications like sync & share, and many more. Multi-tenancy is an important feature to segregate workloads and give priority to the operationsrequested by the primary storage.

• TCA and TCO. Capacity tier storage can be provided by public services, on premisessolutions, or a mix of the two. Acquisition costs, agility, scalability and other variables(including egress costs for cloud storage, for example) should always be taken into accountbefore choosing the right solution.

How to manage file-based storage in a two-tierstorage strategyIn the previous chapter we mentioned that, in complex and large infrastructures, the primarystorage layer can be split in File and Block storage. However, for many organizations, a simplerfile-oriented approach brings greater benefits.

File services are critical for most organizations but, if they are not managed correctly, the TCOoften becomes prohibitive. With data distributed geographically in remote offices, or mobileusers accessing it from everywhere at any time, these types of services need to be redesignedto cope with new business requirements.

Primary storage and cloud/object have important roles in this picture. Instead of looking attraditional Network Attached Storage (NAS) and file servers, end users can take advantage of acloud store and use local and remote gateways to provide a seamless user experience.Common file workload analytics show that recently created files are frequently accessed in thefirst hours of their life and become cold soon after.

The approach of using gateways with cloud storage has three important benefits:

• Improved security: Remote gateways are expendable, and files are encrypted beforeleaving the gateway. Even if the remote device is stolen, or improperly accessed, the risk ofin-the-clear data exposure is vanishingly small.

• Improved data protection and DR: Object storage provides superior reliability and durabilitythan any other storage system. File/object versioning maintains several copies of files overtime without requiring expensive backup solutions. In case of a disaster, with loss of thegateway, a new virtual gateway can be instantiated anywhere in the cloud, or in anotherlocation.

• Improved infrastructure TCO: Communication between gateway and object store is usuallyoptimized through data compression and deduplication to save bandwidth. Backup and DRprocedures are practically integrated in the system and simplified, without additional

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 12

Page 13: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

software or hardware involved. Usually, physical and virtual appliances of this kind arecentrally managed, minimizing the need for local or remote operators.

The Return on Investment (ROI) of this type of architecture is superior to other approaches,considering the diminished risk of obsolescence of the solution, improved and simplifiedcapacity planning, provisioning, and additional services that can be enabled around it.

Most of the gateways available in the market can be deployed as virtual appliances, takingadvantage of a small portion of primary storage for caching frequently accessed files, while therest is moved to less expensive storage in the cloud or on premises.

These solutions are available as end-to-end options from most object storage vendors, or asstandalone products which are compatible to multiple cloud providers and object stores.

Most of these next-gen file solutions go beyond simple file sharing, adding Sync & Sharecapabilities (private DropBox-like services) with advanced analytics, security, and privacyfeatures. Contrary to traditional file storage solutions, the majority of these next-gen solutionsare focused on data management as opposed to storage-centric management, and typicallyaddress new regulations such as the European GDPR.

Compliance considerations for implementing asuccessful two-tier storage strategyWhen it comes to security and compliance, there are several aspects to consider when adoptinga strategy for data distributed in private and public cloud environments.

Most solutions encrypt data before leaving the primary storage, eliminating the risk of data leakseven if data is moved to the cloud. On the other hand, encryption adds compute cycles, andapplication performance may be impacted by it. In most countries, the law forbids savingsensitive data outside state borders or in data centers owned by companies under a differentjurisdiction. Europe, for example, has stringent rules regarding this issue.

Local or regional providers are usually less attractive than popular cloud providers such usAmazon, Google, or Microsoft. both in terms of price and infrastructure. In this case, the bestoption could be a private object store.

A public cloud service may enable economic benefits, but a local object store installed onpremises may be faster with superior latency, contributing to building a better end-to-endsolution. Many vendors (including NetApp, Hitachi Vantara, and Dell EMC) support their objectstorage solutions in conjunction with their primary storage systems.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 13

Page 14: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

All object stores in the market are now Amazon S3 compatible; this is also the API of choice forpractically all primary storage systems. End-to-end solutions can be preferable if there arespecific functionalities that make the integration more efficient but, in general terms, it is betterto choose an object store capable of serving multiple needs and workloads. This helps toconsolidate more data and workloads on the second storage, improving savings and efficiency.

The object storage market is growing and maturing at a rapid pace, with a wide range ofproducts that cover the needs of most organizations. S3 compatibility is no longer an issue formost, nor is scalability or data protection efficiency. In addition, advanced products arebeginning to provide enhanced performance; this reduces the gap with primary storage in termsof latency, making interaction between the two storage tiers quicker and more responsive.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 14

Page 15: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

5 Vendor Review

The number of two tier strategy solutions is growing. Some vendors offer end-to-end solutions,but end users can take advantage of several alternatives thanks to S3 and other very commonAPIs like those offered by Microsoft Azure Blob or Google Cloud Storage. For an in-depthanalysis on alternatives to Amazon AWS S3, please look for the S3 Alternatives Report comingsoon.

NetApp, through its Data Fabric vision, is one of the first major storage vendors to implement abroad set of features and services across most of its product line-up, connecting primary andsecondary storage seamlessly. Primary storage systems from different product families can nowconnect and move data to on-premise object stores as well as public cloud providers.

For NetApp ONTAP users, StorageGRID offers a natural, capacity-driven expansion of their on-premises infrastructure. For example, by updating systems to the latest ONTAP releases andenabling new features like FabricPool, it is possible to move inactive data quickly to the objectstore and save performance-oriented resources. Performance-optimized StorageGRIDconfigurations are now available.

Hitachi Vantara, which recently has gone through a significant transformation, has beenworking on integrating its products for quite some time. Most of its primary storage systemshave the ability to offload data to Hitachi HCP object store and/or public clouds.

Hitachi HCP object store also has an interesting ecosystem of companion solutions aimed atserving enterprise use cases for distributed file access, IoT, and Big Data. The company has alsoadded interesting products for data augmentation and big data analytics that take advantage ofthese functionalities.

Many Dell EMC products can tier data to EMC ECS object store or public cloud. Some productsdo so natively while others take advantage of an external appliance or service. All theoperations are managed through policies set up by the user and easy to use user interfaces.

In this regard, the most interesting solution in the Dell EMC spectrum comes from theintegration between Dell EMC Isilon and ECS. It is very common to see it proposed as an end-to-end solution and has been well received by end users.

IBM Spectrum Virtualize, the core of IBM’s strategy around software-defined storage, has strongautomated tiering capabilities that include object storage and cloud. This means that thisfunctionality is available across the board for the storage systems in its product line-up.

It is also important to note that Spectrum Scale (GPFS scale-out file system) has this functionality

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 15

Page 16: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

embedded, enabling customers with high performance needs for unstructured data to movedata easily and seamlessly across different tiers including IBM Cloud Object Storage and otherS3-compatible systems.

DDN has recently entered the enterprise market through the acquisition of Tintri. Tintri’s cloudconnector offloads snapshot and volumes to the cloud for longer data retention and simplifieddisaster recovery. The rest of DDN’s product line-up has utilized DDN WOS 360 object storesfor several years now thanks to automations aimed at offloading cold data from primary tosecondary storage when necessary.

Igneous has a complete data protection and management solution which also providesintegration with major scale-out NAS suppliers such as Pure Storage, Qumulo, EMC Dell Isilon,NetApp, as well as major cloud providers. This allows the end users to offload data from primaryNAS systems to Igneous, which can also manage tiering it to the cloud for storage in a long-termarchive.

The solution design from Igneous also offers data analytics features for optimal understandingof data stored in the systems, their lifecycle, and data movement capabilities for moving dataacross multi-cloud environments.

Caringo FileFly is a file management solution compatible with Windows file servers and NetAppfilers. Compatible with Caringo Swarm, as well as major cloud providers, it allows the end userto set policies and manage local retention and replicas to the cloud.

Komprise developed a data management solution aimed at moving data seamlessly betweenprimary NAS storage systems and private or public object stores, depending on user-definedrules. It also provides tools for disaster recovery to the cloud and analytics tools to monitorinfrastructure cost, efficiency, and security.

Datrium offers an end-to-end solution that spans from primary filestorage to data protection anddisaster recovery on the cloud. Its core product, DVX, is a software-defined solution; it is basedon split-provisioning, a two-tier structure that allows customers to get performance at scaleadvantages from flash memory installed on compute nodes for primary latency-sensitiveworkloads, and is capable of host-based encryption and built in backup while offloading colddata to a secondary, HDD-, flash- or cloud-based, layer for resiliency.

Datrium also offers CloudShiftTM, a SaaS solution for DR orchestration and data mobility aimedat simplifying operations in hybrid-cloud environments while lowering TCO and providingfailproof DR. Datrium DVX supports common hypervisors such as VMware vSphere and RedHatKVM as well as Docker containers.

HPE 3PAR takes advantage of the integration with HPE Recovery Manager Central (RMC)offloading snapshot or entire volumes to the cloud, which helps manage long data retention andDR operations. At the same time, thanks to a reselling agreement with Scality, HPE can provide

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 16

Page 17: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

an end-to-end solution to build a two-tier storage infrastructure on premises.

It is interesting to note that solutions primarily designed for the cloud are now adopting a similarmulti-tier approach. The example comes from Elastifile, a scale-out file system that is usuallyinstalled on virtual machines (VMs) running in public clouds serving data via standard fileprotocols for high performance workloads such as big data analytics. With the recentlyannounced v3.0 of its product, end users can now select how much storage will be providedfrom block devices in the VMs or the object store of the public cloud provider.

Practically every object storage solution in the market (including IBM COS, Scality RING,Cloudian HyperStore, Red Hat Ceph, and others) offers strong S3 compatibility and can be anideal target for primary storage systems that support this protocol. Several storagemanufacturers are partnering with one or more object storage vendors to certify their solutions,enabling customers to have more freedom of choice for the implementation of theirinfrastructure.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 17

Page 18: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

6 Near-term Outlook

Two-tier is becoming increasingly common among enterprises for on-premises, hybrid, andcloud-only solutions. Primary production storage benefits from a secondary object tier bybalancing performance and cost.

Depending on the size of the organization, and the type of workloads involved, end users havedifferent approaches. Latency remains the key parameter when it is time to evaluate the back-end solution. Local object store is more predictable, both in terms of latency and cost, while thecloud is more flexible when it comes to expanding infrastructure quickly.

End users should ask for this kind of feature in all their future infrastructure upgrades, even ifthey do not have plans today to expand their infrastructure in this direction. It will come in handylater to answer any potential and sudden data growth they might experience.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 18

Page 19: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

7 Key Takeaways

Traditional storage infrastructure is incapable of dealing with today’s data challenges, especiallynot all at the same time. Latency and IOPS on one side, with capacity and throughput on theother, are the two driving factors for all workloads. They are part of the same equation with$/GB. A low-latency storage system, integrated with on-premises or cloud capacity layer(s), isthe most viable option to optimize the balance of performance and cost.

If the goal is to drive down TCA and TCO, simplification and higher efficiency is required. Newstorage systems, capable of moving data between tiers according to user-defined policies, arestraight forward to adopt and deploy. They provide the same features and functionalities oftraditional systems but with superior overall resource management and the ability to move dataacross on-premises and public cloud locations.

With organizations of all sizes and kinds adopting ‘never-delete’ data retention corporate policy,the challenges of data growth and protection require new solutions and different approaches.This is why using an object storage tier is becoming more common for several applications anduse cases. Consolidation of infrequently accessed data on these kinds of platforms savesmoney over time while improving many aspects of data and infrastructure management,including data protection, retention, durability, and disaster recovery.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 19

Page 20: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

8 About Enrico Signoretti

Enrico has 25+ years of industry experience in technical product strategy and managementroles. He has advised mid-market and large enterprises across numerous industries andsoftware companies ranging from small ISVs to large providers.

Enrico is an internationally renowned visionary author, blogger, and speaker on the topic of datastorage. He has tracked the changes in the storage industry as a Gigaom Research Analyst,Independent Analyst and contributor to the Register.

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 20

Page 21: Two-Tier Storage Strategy: A GigaOm Market Landscape Report...transport, which are growing in popularity, can easily provide sub-millisecond I/O response latency at reasonable costs

9 About GigaOm

GigaOm provides technical, operational, and business advice for IT’s strategic digital enterprise,and business initiatives. Enterprise business leaders, CIOs, and technology organizationspartner with GigaOm for practical, actionable, strategic, and visionary advice for modernizingand transforming their business. GigaOm’s advice empowers enterprises to successfullycompete in an increasingly complicated business atmosphere that requires a solidunderstanding of constantly changing customer demands.

GigaOm works directly with enterprises both inside and outside of the IT organization to applyproven research and methodologies designed to avoid pitfalls and roadblocks while balancingrisk and innovation. Research methodologies include but are not limited to adoption andbenchmarking surveys, use cases, interviews, ROI/TCO, market landscapes, strategic trends,and technical benchmarks. Our analysts possess 20+ years of experience advising a spectrumof clients from early adopters to mainstream enterprises.

GigaOm’s perspective is that of the unbiased enterprise practitioner. Through this perspective,GigaOm connects with engaged and loyal subscribers on a deep and meaningful level.

© Knowingly, Inc. 2018. "Two-Tier Storage Strategy: A GigaOm Market Landscape Report" is atrademark of Knowingly, Inc.. For permission to reproduce this report, please [email protected].

Two-Tier Storage Strategy: A GigaOm Market Landscape Report 21